Challenges
Data is the main asset for any social network. Yet StackExchange does a wonderful job of exposing its data for exploration and analysis. Unlike other social networks, which expose their data through APIs mostly and restrict many details, StackExchange not only provides multiple channels like data dumps and data explorer apart from APIs, but it also provides access to an almost complete set of public information.
That being said, there are challenges while working with a platform such as StackExchange. The following are a few of them:
Data dumps expose the data in the form of XML files. Though there are parsers available in R for using XML data, there is an inherent limit imposed if the XML files are huge (StackOverflow's XML files amount to 30 GB). This limitation can be overcome by first loading the data into a local database such as MySQL and then working upon the required subset of data.
Data explorer has row limits imposed upon the data extracted through the explorer (current...
