After passing my Prospectus Defense in December, my PhD project was suddenly real: I must now actually do primary research and write a several hundred page document. The best advice I have gotten so far is to take it one step at a time. So here is a progress report on Stage 1: The Survey.
I submitted my survey text to Rensselaer’s Institutional Review Board in January and received an exemption: “45CFR46.101(b)(2): Anonymous Surveys – No Risk”. Since my survey is anonymous and does not harm any of the respondents, I am cleared for action.
My target community is big social data researchers in the United States working out of academic institutions. For the first round of invitations, I have been using the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining proceedings to solicit respondents. Everyone has been very polite and interested in my work, and I have had a solid 20% response rate. My goal is to get at least 20 responses, but I will continue collecting through the end of March.
For this initial survey, I am interested in how data scientists use interpretation to complete their projects and how they communicate their results to their audience. My survey questions focus on a few key themes. First, I was interested in how respondents understood their disciplinary role and why they became interested in big social data. Next, I asked about interpretation: how they decided on research questions and generated explanations for their results. If they changed their research questions mid-way through the analysis, I also wanted to know what steps they took to ensure accuracy. Then, I turned to technical aspects of the process, asking what steps they took and how they handled false-negative and false-positives. Finally, I asked about communicating results persuasively and to a target audience. The preliminary results look promising, and I personally find them fascinating!
In case anyone is particularly interested, here are the exact questions. The bulk of them are directed at the researcher’s specific project they submitted to the ASONAM conference.
1. What term or phrase would you use to describe your research project?
Big Data; Cultural Analytics; Big Social Data; [Other]
2. Please provide a brief description of your project:
3. How did you become interested in your project and big social data analysis?
4. How did you decide what research question(s) to ask for this project?
5. Once a pattern emerged in your data, how did you generate a theory or explanation for its cause?
6. Did you change or adapt your research questions partway through the project?
7. If you changed your research questions partway, what strategies did you use to ensure that the questions were related to your collected data?
8. What were the steps in your research process? [Select from a list]
9. If you had the option to select a conservative or liberal selection criteria, that is, design your analysis to have a preference for type I (a potential false-positive) or type II errors (a potential false-negative), which did you select?
I minimize false positives; I minimize false negatives; I use my field’s default selection criteria; Other
10. Why did you take this approach for your project’s selection criteria?
11. Who was the primary audience for your research?
12. Are there any other secondary audience(s) for your research?
13. When during your research process did you start considering how to share your results with your audience?
Before I began; After the results are finalized; I never actively considered it; Other
14. What steps or strategies did you employ to make your results persuasive and clear to your audience?
15. At what professional conference(s) did you present your research?
16. Do you respond to or follow public engagement and discussion of your research?
17. Have you ever had a memorable experience interacting with the public once your research was published?