HeyMarvin is an EPIC2024 gold sponsor. EPIC is a nonprofit organization, conference, and community, and sponsors support programming that is developed by independent committees members and invites diverse, critical perspectives.
I’m bullish about AI’s potential — but I’m also the first to point out that it is not a replacement for human beings. Technology will never supplant the brilliance of trained researchers, but when employed mindfully, it can be a useful and complementary tool for them.
Challenges such as lack of transparency in datasets and misguided assumptions about the nature of AI demand that we reframe AI’s value, particularly when it comes to qualitative research.
A Conversation with an AI Innovation Expert
During a recent HeyMarvin event, I discussed AI Innovation and Research with Mary L. Gray, a prominent anthropologist and Microsoft Research’s Senior Principal Researcher.
Mary advocates for an AI approach that respects cultural differences and remains attuned to the nuances and complexities of human behavior. In qualitative research, transparency about data and processes used is crucial, as is understanding the context and nuances behind the data.
“Thinking of [AI] as something that is going to help me analyze material, analyze data, and at the same time give me new ways to prompt my own reflection. That, to me, is a very powerful thing about artificial intelligence,” Mary said.
Mary also highlighted the significance of participatory AI innovation, where qualitative approaches can enhance the development of AI. She argued that integrating qualitative methods can lead to higher-quality data because it allows for deeper insights and a better understanding of the context in which data is generated.
Understanding the Challenges of Working with AI
As Mary and I discussed, the integration of AI into research is not without challenges. As we continue to explore its potential, it’s crucial to understand what AI can and cannot do. In particular, we must address the lack of transparency in data sets and models and understand that AI tools do not reveal underlying, structural truths.
“Who’s shuffling the deck?”
Lack of transparency in data sets and models means we don’t know “who’s shuffling the deck”, where the models come from, and what bias we’re dealing with.
“Being able to see…who’s got the hands here, who’s shuffling the deck, that’s the critical piece…Now, we haven’t done due diligence, so we’re right now working with models like, nobody knows where it came from. Nobody knows what’s in it,” said Mary.
It is imperative to address harmful bias in training data and AI models, but as researchers, we must understand that totally UNbiased data simply doesn’t exist. Bias is inherent in all research and data, so it’s always essential for us to be rigorous about understanding what the biases are and strive to minimize them.
Misguided assumptions
There is also a misguided assumption that AI tools can almost magically reveal underlying truths that are inherent within the data and hence automate interpretation.
“I think there’s too much of the assumption that I’m finding something deep and structural in the numbers game here with qualitative analysis. It’s like, yeah, we do like to turn qualitative analysis into quantitative data,” said Mary.
Mary notes that for certain contexts and quantitative research agendas (e.g., sequencing proteins), specialized AI tools are aiding important discoveries. But human decision making is not one of them. Human behaviors, choices, and experiences are place specific, culturally specific, relational, emerge in real time, and change over time. What is more, when we attempt to generalize these experiences, we become disconnected from the utility and richness of specific times, places, and groups of people.
Qualitative researchers need to resist attempts to quantify qualitative research, and more broadly what Mary calls the “banality of scale”:
“Obsessive attachment to making something no longer connect to a specific locale, a particular need, so that you would have something that can be used all the time, by everyone, anytime. …banality of scale is thinking that my role as a researcher is really just a fact finding mission. And if I pick up enough facts, then I no longer have to ask anything. And I think that is the opposite of what we do in qualitative research.”
Once we understand these challenges, we can reframe our expectations for AI research tools, and truly make use of the powerful applications of this kind of software.
The Power and Potential of AI
Once Mary reframes AI research tools as expanding possibilities instead of giving answers, she is excited about their power and potential.
The leap forward in automating certain tasks is significant. Mary notes that with Marvin, she “can turn over tasks that, up to this point, I have felt like it was just easier for me to do it old school – code by hand.” HeyMarvin also offers AI tools to support researchers in managing large amounts of data efficiently, sharing it effectively, and integrating it into other platforms and formats.
Mary is also impressed that she can ask quantitative questions of her data much more efficiently; for example, “tell me the number of instances in this context, with these demographics…that’s pretty awesome because otherwise that’s a lot of find and replace, or a lot of control F, control F”!
Beyond time savings, what is really promising about AI research tools are the new kinds of value qualitative researchers can create with them. Mary is exploring uses of AI that prompt new spaces of possibility (a nod to Bourdieu), which she calls the bread and butter of qualitative research. AI can help us explore, “what are the range of things that could happen in relation to this group, these places, these dynamics, and be able to map all of that out, but not in a way that holds it in place.”
To do this, it is crucial to craft prompts that extend the methodological rigor of qualitative research design, rather than revert to prompts based on a quantitative or predictive approach.
Ultimately, Mary says, “no tool replaces us. We just open up new things that we add to what it is we’re doing…and that, to me, is the boon we’re about to hit. Qualitative research hasn’t had its day yet.”
Want to learn more about the role of AI in research? My team is doing great work on the topic — check us out at heyMarvin.com.
0 Comments