Qualitative Literacy: A Guide to Evaluating Ethnographic and Interview Research
by Mario Luis Small and Jessica McCrory Calarco
August 2022, 230 pp, University of California Press
This book is a must-read for any researcher, even those who specialize in quantitative methods.
It aims to be a textbook but achieves much more than that because it focuses on what it takes to be “literate” in qualitative data – the very thing that our stakeholders need to be more sophisticated customers of our work. Qual researchers know all too well that basic qual familiarity is lacking in most contexts, but we don’t always have the language to explain to our stakeholders how to best leverage qual data. This book helps us do that.
Imagine being a writer in a world where no one was literate. Imagine the constant struggle to train up, support, and explain your work to people who cannot read, while at the same time being told the reason they cannot understand your writing is because it isn’t good enough. This is what qual researchers often face, as we are embedded in a quantitative world. We can always improve the quality of our research, but this book’s title alone reminds us that quality alone is not the reason that our work is often misunderstood. It is the lack of qualitative literacy among our clients and stakeholders that often holds us back from making impact.
If you share that experience, this book is definitely for you. But it is also for you if you are still learning the ropes of qualitative research and need a framework to evaluate the quality of your own work. Perhaps you did not take qualitative research methods at university, or perhaps you didn’t take research methods at all. Either way, you will get value from this book.
The authors lay out a clear set of standards for what constitutes good qualitative research. They identify five key dimensions:
- Cognitive empathy: the degree to which the researcher understands the participants and their points of view
- Heterogeneity: how the diversity of the actual population is represented in the data
- Palpability: how strongly the empirical observations and quotes support the overall conclusions
- Follow-up: how well the researcher follows threads to uncover the deeper meaning of any observation or interview response
- Self-awareness: more than simply reflexivity, but deep awareness of the impact of their very presence
Students of qualitative methods will recognize similarities to Guba and Lincoln’s (1985) seminal work on validity in qualitative research; cognitive empathy has some overlaps to their concept of “trustworthiness.” But what differs in this text is the very straightforward voice of the writing, the conceptual clarity, and the useful ways the authors spell out what it means to have each of these dimensions in your qual research.
Small and Calarco offer one chapter for each dimension, with in-depth descriptions of how to achieve them. A particularly helpful technique they use is to provide a poor-quality fieldnote or interview excerpt, and progressively layer detail on the very same excerpt to demonstrate how specifically to improve.
For example, they offer fictional fieldnotes of a study of social capital of immigrants in New York and rework the very same fieldnotes several times to show how more detail helps improve the study. The fictional “Sofia” starts out as “moving to New York without papers when she was 19,” and progresses to someone who describes herself as “well-connected” and speaking in her native Spanish. This progressive improvement is in each chapter, and clearly illustrates how to improve your own fieldnotes and interviews.
Small and Calarco even help us with the dreaded “sample size” question. Your average applied qualitative researcher is asked “How many people did you talk to?” more often than any other validity-related question. They argue that sample size is not the important question, but exposure, or the number of hours a researcher spends with and talking to participants. As they sum it up: “The greater the contact, the better the data” (p. 30). Exposure is far more helpful a test for qualitative data than mere sample size. You can spend hundreds of hours with a single person and get incredibly rich data that will stand the test of conceptual clarity, even if it will not provide predictive power of the entire population. Predictive power, in other words, is simply not a good test for qual research.
This book helps you, the applied researcher, understand what makes qualitative research rigorous; but if it falls short in any way, it’s in the extension of concepts to stakeholders in applied settings. It’s unlikely your stakeholders will understand the relevance of “palpability” as a concept, even if they do understand concepts like direct evidence.
For this reason, I recommend reading Qualitative Literacy alongside Margaret Roller’s Applied Qualitative Research Design: A Total Quality Framework (2015), which will give the applied researcher very actionable indicators of success. Roller’s notion of “analyzability,” for example, is a key way I explain to stakeholders why a bulleted list of observations is not good enough to count as “qualitative data.”
But this is a small quibble. Small and Calarco’s book should be on every qual researcher’s bookshelf.
References
Lincoln, Y., & Guba, E. (1985). Naturalistic Inquiry. Beverly Hills: Sage.
Roller, M., & Lavrakas, P. (2015). Applied Qualitative Research Design: A Total Quality Framework. New York, NY: The Guilford Press.
Small, M. L., & Calarco, J. M. (2022). Qualitative Literacy: A Guide to Evaluating Ethnographic and Interview Research. Univ of California Press.
Related Resources
Ethnographers, Bearers of Bad News, Sam Ladner
EPIC Talk: Mixed Methods, Sam Ladner
EPIC Tutorial: Ethnographic Research Design, Sam Ladner
Tutorial: Fundamentals of Observational Research, Mike Youngblood
>featured image: Protozoa by Ashok Boghani via flickr
0 Comments