Messy Data
January 28, 2013
From Maxwell Perspective...
Messy Data
Political scientist Colin Elman is helping change the way qualitative research is standardized, stored, and shared.
Colin Elman spearheads a set of projects aiming to standardize qualitative research in political science.
These days, a political scientist who wants to explore, say, a theory about presidential leadership has a wealth of compelling qualitative data to draw from. In the modern era, minutes, memos, speeches, even formerly secret tapes are available as never before. Whether from dusty archives or fresh interviews, rich qualitative data provide deep content you won't find in the anonymized numbers and charts of quantitative data.
However, these benefits do not come free of costs.
"Qualitative data are often unstructured and much harder to organize, store, share, and cite than their quantitative counterparts," says Colin Elman, associate professor of political science, who is co-founder of the American Political Science Association's section on qualitative and multi-method research.
He is also executive director of the Consortium for Qualitative Research Methods, based in Maxwell's Moynihan Institute. In that role he co-directs an annual training institute on qualitative and multi-method research. Partly funded by the National Science Foundation, it has trained more than 1,000 graduate students and junior faculty.
As a member of political science's qualitative and multi-method research movement, Elman is leading a team focused on data collection and analysis. The goal is to help provide the incentives and infrastructure to do "messy" qualitative research in a rigorous, transparent, and replicable way.
"As with quantitative research, qualitative evidence-based claims should be made transparently, where possible with the supporting data to show how claims are inferred," argues Elman, who received a 2012 Mid-Career Achievement Award from the American Political Science Association for his work on qualitative methodology. "New qualitative methodology is setting high standards, but the tools and infrastructure needed for researchers to meet those standards are still being developed."
To help scholars systematically store and share qualitative research, Elman and his colleagues are developing a new centralized repository of qualitative data, funded by a $600,000 National Science Foundation grant. The repository, also based at the Moynihan Institute, is a joint project with SU's School of Information Studies and has co-principal investigators from Harvard University, University of California-Irvine, and University of Chicago.
The repository will be an online archive where researchers can upload and store their primary sources and data sets and link them to their texts via "active citations." Active citations will make a scholar's research methods more transparent and easier to replicate. Scholars will have an incentive to do research in a more rigorous, scientific fashion.
“Qualitative evidence-based claims should be made transparently, where possible with the supporting data to show how claims are inferred.”
— Colin Elman
One of the first projects to be uploaded to the repository will be selections from Elizabeth Saunders' Leaders at War: How Presidents Shape Military Interventions, published last year by Cornell University Press. When the upload is complete, scholars will view Saunders' primary sources online by clicking on hyperlinks embedded in her footnotes and references. For example, clicking one footnote will reveal a foreign policy speech blue-penciled by President Eisenhower.
"Reading Saunders' activated text online will make her presentation richer and her arguments more persuasive," Elman observes. "And digital archiving will make it easier for subsequent scholars to critique her research and build on her efforts."
Elman plans to use the repository for his upcoming book, Regional Hegemony: The United States and Offensive Realism. Active citations, he says, will make his historiographical research immediately available to readers and show more clearly how cited materials support the book's inferences.
The repository will also provide search tools, a portal to other databases, and opportunities for peers to annotate texts. It will facilitate training and advice on collecting, storing, and sharing data — and, as such, it will be a remarkable teaching tool. "Now students will be able to go to the repository to see what qualitative data look like and how they are used," says Elman, who also co-edits a book series on Strategies for Social Inquiry for Cambridge University Press.
He notes that archiving qualitative data online is not without difficulties. Research subjects may have to provide informed consent, and in some cases confidentiality will have to be protected. So the repository will have access controls to separate "safe" and "toxic" data. "There will be levels — fully open data, data that are available with permission, and data that may not be accessible for some time, if at all," Elman says.
He hopes the repository will allow data to be recycled for other research projects, thus making data more durable. "Because there's no norm for storing or sharing," he explains, "qualitative data are often discarded after just one use. It's a terrible waste."
— Martin Walls