Below are additional examples for my 2015 ACM/IEEE International Conference on Advances in Social Networks Analysis and Mining paper “Arguments and Interpretation in Big Social Data Analysis: A Survey of the ASONAM Community.”
B. Argumentation Theory and Research Design
The proposed framework for evaluation looks at research design, the strategic plan built by the researcher to coherently and logically organize the research process. An ideal research design provides a technical roadmap for the researcher to collect and analyze their data, and it also insures that the research addresses the problem successfully . To put this another way, the research design is the planned route to keep human errors from effecting the results. Components of most research designs include performing a literature review for other’s work on the topic, the proposal of research questions, the identification of data, a plan for collection and processing, and a method for analysis.
Big social data analysis complicates traditional notions of research design because the data exist independently of the research project and prior to the formulation of a research question. Due to this obstacle, I propose that we consider research designs as more than technical roadmaps: research designs are also arguments. By treating them as arguments, we can create standards for evaluation of components of the plan as propositions. The evaluation of research plans as arguments allows for the production of the best work possible by facilitating the explicit consideration of alternative explanations.
During the research process, there are numerous moments of interpretation where the researcher selects from a range of appropriate alternatives . In these moments, selecting the right or wrong answer over-simplifies the situation. The survey of ASONAM participants uncovers interpretive moments to evaluate them as arguments: while there is not a right or wrong answer, there are better answers that more completely or accurately address the problem space.
Argumentation theory provides a structure to understand how research designs function as arguments . Toulmin’s model for addressing formal arguments is composed of data, warrant, claim, ground, backing, and qualifier. Claims are the final conclusions, and warrants are what link data and the ground to a claim. The ground, which can often overlap with the data, is the basis for using a specific type of data. The ground is the definitions and theory where most arguments begin. The backing is additional support for an argument that bolsters unexpected or counter-intuitive claims. Finally, qualifiers condition when the claim should be accepted (e.g. “if x, then y”) or provide the strength of belief in its veracity (e.g. “sometimes x occurs”). These constituent parts can be found in big social data research designs, and by charting the arguments using this model, they can be evaluated and improved.
Fig. 1 is an example of the argumentation framework applied to the research design of Bollen, J., Mao, H., & Zeng, X. (2011). Twitter mood predicts the stock market. Journal of Computational Science, 2(1), 1-8.
The backing for the ground—that collective mood states may impact systemic decisions— is italicized because it is a proposition that only logically supports the research plan after the technical demonstration of the model. The ground emerges from behavioral economics, borrowing strength from a well-established observational discipline. Ultimately, the technical aspects are sophisticated and performed without error and the qualifier maintains reasonable expectations for the results. In this case, the research as an argument is very persuasive to the community: It has been implemented in numerous real world applications and cited over 2,400 times.
Fig. 2: Chatterjee, A., & Perrizo, W. (2015, August). Classifying stocks using P-Trees and investor sentiment. In 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) (pp. 1362-1367). IEEE.