In this paper, I argue for the value of multi-dimensional ethnography. I explore the potential for ethnography to venture beyond sites, into different dimensions. As an example of work moving in this direction, I present a new approach, dubbed TRACES, which emphasizes the assemblages that constitute our lives, interweaving digital, embodied, and internal experiences. Various data streams and sources provide different vantage points for analysis and synthesis. I illustrate how we have used these to gain greater insight into the human lives we study, with different data sources providing different perspectives on a world, then delve into our use of tools, data sources, and methods from other traditions and other fields, which, combined, give us not only a more holistic picture, but a truer one, which refutes the false dichotomy of the digital and the real. I argue that we must continue to adapt and extend ethnography today into such spaces, and that reformulating the sites of ethnography as dimensions enables us to envision future subjects and objects of study in different ways.
INTRODUCTION: A CALL FOR MULTI-DIMENSIONAL ETHNOGRAPHY
The spaces we study are changing. At EPIC 2016, a conversation arose around the “flatness of the world” in current ethnography. Some EPIC attendees pondered whether ethnographic work is less exciting nowadays, because often our participants are staring at “little screens,” thus rendering our work less experiential, less action-filled. The assumption was that we, as ethnographers, only study the embodied interaction with such a device, with no insight into the world of that interaction happening beyond. Counter to this perspective, I argue that not only is the world more action-filled with the extensions made possible by the little screens of our digital worlds, but it makes our work that much more exciting because it adds dimensions of exploration and understanding not possible before– richer perspectives, and new opportunities for extending our practice.
The notion I present of multi-dimensional ethnography builds upon Marcus’s arguments decades ago around the need for multi-sited ethnography. Much as he argued “why this locale, rather than another” (Marcus, 1995), I argue that the embodied, physical dimension need not be the privileged one. Experiential spaces are not always physical places; they are also virtual, ephemeral. The notion of dimensions allows us to consider these spaces. And notably, while accounting for spaces is the primary way in which the word dimension is used, one of its less frequently used meanings is: “the elements or factors making up a complete personality or entity1.” So, by taking a multi-dimensional perspective, we take into consideration both a variety of experiential spaces and how those spaces are integrated into the lives of those we study. Accounting for multiple dimensions of experience and utilizing different data sources provide us with different vantage points and a richer picture. And, as I will delve into later, it allows us to explore human experience and action while also accounting for machine agency and action.
There is a rich tradition of work that explores these realms, including Hine’s (2000) research on virtual ethnography, Nardi’s (2010) digital ethnography, Geiger and Ribes’s (2011) approach vis-à-vis trace ethnography, and other work on cyberethnography, ethnomining, etc. I rely on, and aim to build upon these approaches in this work, also integrating traditions, data streams, and methods from other fields and using a variety of tools, such as Paco (Evans, 2016). I present this approach, which is manifest in a series of studies dubbed TRACES. This approach has allowed my colleagues and I to investigate our ever-evolving and creative assembly of a life through engagements with constellations of devices and apps, services and agents across different dimensions. Combining these dimensions and different streams of data has provided us with insights we could not have achieved with other methods or singular approaches. Practically, this has helped us to understand the assemblages that constitute our lives and given us a richer understanding. Philosophically, I believe it has moved us toward a reconceptualization of who and what we study.
As Boellstorff (2016) and many others have noted, both the digital and the embodied are “real”; separating them is a false dichotomy. Multi-dimensional ethnography provides us not only with a richer picture, but a truer one. Resituating subject-object across dimensions also highlights the agency of machine systems as social actors. In continuing to adapt and extend ethnography for the spaces and experiences we study today, we need to consider the significance of such actors. Reformulating the sites of ethnography to dimensions enables us to envision future subjects and objects of study in a different way. For instance, what would an ethnography focused on an AI perspective look like, as opposed to taking a human user perspective? It is not hard to imagine that in the future, we could reformulate the subjects and objects of study in ethnographic praxis in the technology industry to conduct research not just on the experiences of people interacting with systems, but on the experiences of intelligent systems interacting with people. What new insights would this approach glean? What assumptions would it expose? In the sections that follow, I provide an overview of ethnographic approaches that this work builds upon, highlight how we are thinking about and conducting this type of work in practice, and then explore the theoretical and philosophical implications of such an approach.
PRIOR APPROACHES TO INCORPORATING OTHER DIMENSIONS
The notion of exploring the digital, alongside the embodied is not new. Immersing ourselves in the media world of those we study has been done for years. Since the advent of the World Wide Web in the 1990’s, ethnographers have been looking to digital sources of data and digital aspects of people’s lives. As Hsu (2014) explores in her work on digital ethnography, there have been a whole host of researchers who have brought the digital into focus in their work. As she notes, most have either focused on the role of digital media in today’s world, conducted “virtual ethnography” in digital social environments, or considered digital methods or tools to extend participant observation. I highlight a number of approaches that have considered a focus on these areas in one way or another—either as a subject of research, a site of research, or as a tool or method of data gathering (Figure 1). These are all important topics and useful approaches, but we must continue to expand on them as we move into an age with even greater technological capabilities and pervasiveness. My aim is to build upon these approaches by focusing on dimensional qualities and illustrate how we are attempting to surface such through the TRACES methodological approach.
Figure 1. Approaches to digital ethnography over time.
In the first vein are approaches like Christine Hine’s (2000) virtual ethnography and Kozinets’ (2010) netnography work. Both emerging in the late 1990’s, these approaches were prompted by the rise of online communities and “net life.” They both focused on the need for new forms of research to understand the Internet and to explore how it fit into people’s lives. These approaches took ethnography into digital spaces and are very much still used today to understand current online media and communities. More recently, Coleman (2010) has delineated different approaches taken the ethnographic bodies of work on digital media, separating them into overlapping categories including cultural politics, vernacular cultures, and prosaics of digital media. While providing rich perspectives, these approaches are still very one-dimensional in focus within the scope of where technology and artificial intelligence (AI) are headed today. They are observing social networks or online media as the focus of study, with a particular focus on the role of those media in today’s world. For all intents and purposes, these explore a very embodied experience, not other dimensions of reality in which identities are formulated and reshaped; where actors – both human and algorithmic – interact and engage beyond the individual subject.
A second approach to virtual ethnography focuses less on the digital media as subject, and instead looks more deeply at how people engage in virtual worlds. Both Nardi’s (2010) and Boellstorff’s (2008) work fit within this area of focus. Nardi’s exploration of World of Warcraft and Boellstorff’s examination of Second Life study the lives that are lived within these virtual worlds. Compared to the virtual ethnography/netnography approaches that came before, they focus less on the role of these digital spaces in modern society, and more on the interactions and cultures formed within these worlds. These are examples of fascinating ethnographic work, but for the purposes of thinking about dimensionality, it’s important to note that the digital remains a site, unexplored from the algorithmic perspective or a broader societal perspective.
Finally, a third set of approaches has focused on the digital extension of data collection. This subset ranges from the practical use of digital tools like smartphones for capturing data to various data streams produced organically as part of the ethnographic artifacts considered. The former includes approaches like sensory ethnography. Droumeva’s sensory postcards (2015) focus on using a combination of smartphone apps as a multi-modal data-capture tool and method of data collection. The aim, much like DScout and other tools, is to capture rich data from subjects’ lives, including audio and video. These are data produced as data – through prompts, not organically captured. On the other hand, both ethnomining (Aipperspach et al., 2006) and Geiger and Ribes’s (2011) trace ethnography work fit within the latter approach of looking at organic sources of data. Trace ethnography aims to combine the “wealth of data in logs” with other rich data to “reconstruct patterns and practices of users in distributed sociotechnical systems (2011).” These data are combined to create a more holistic picture of users’ lives. This is also captured in Hsu’s (2014) work on ethnography in the digital humanities, in which she argues for moving the focus of the digital from a subject to a method of research. Each of these has a different perspective – data collection tool, data stream, methodological approach. All of them are important in taking a multi-dimensional approach, but each is only one piece of taking such a holistic approach. The aim is to incorporate all of them.
The history of digital ethnographic approaches and areas of focus is rich, and has expanded and necessarily shifted over time. The first group of approaches focuses on the digital as subject. The second focuses on digital as sites of interaction. The third focuses on digital as source or data stream. These are all useful and important approaches, but we must continue to expand upon them as technologies become ever more capable, influential, and pervasive in our lives. We must aim to address the multi-dimensional nature of today’s world. These approaches are but one part in a larger universe of action and interaction; of internal and external; of ephemeral and sustained. The aim in this push toward a multi-dimensional ethnography is to examine the combination of the internal and the embodied with digital, which together create assemblages of experiences over different temporalities, within different spaces, and between different agents. In the following section, I will explain how we are just beginning to explore these possibilities and where we are headed with our initial humble attempts in this area.
TRACES: AN APPROACH TO MULTI-DIMENSIONAL ETHNOGRAPHY
The concept of TRACES did not emerge from a philosophical perspective, but rather from the practical challenges of conducting ethnographic research that focuses in on the integration of technology in people’s lives. The aim was to get a more holistic picture of people’s lives. And beyond the complexities of traditional ethnographic praxis, the biggest challenges were to understand the boundaries and seams, the flows and the assemblages, and the multiple identities, agencies, and temporalities inherent in the use of digital technologies. We humans do not now, nor have we ever, had just one self. But with digital technologies, we have an incredible multiplicity of selves – selves that can exist and interact and shape things, even when we are not present. And as we continue to move into a machine-learned world, this will become an even more important force playing a role in our lives. These experiences are proliferating and add ever-increasing complexity to our lives. As researchers, we must adjust our research approaches, methods, and ways of sharing insights. We need grounded methodological and analytical tools to interpret things in a tractable way.
Figure 2. TRACES methods, sources, and tools/instruments.
TRACES emerged from work in which we were seeking a holistic perspective to inform the design of intelligent systems. We needed an approach that allowed for a focus on dimensionality, that let us gather and interrogate data at multiple levels of granularity, and that was scalable to at least some extent. The approach I developed includes a combination of different qualitative and quantitative data sources, a variety of data collection methods, and utilizes a range of tools and instruments (Figure 2). It also necessarily incorporates new ways of analyzing and mapping the data together and emerging techniques for assessing the broader picture and making meaning from data. It is a humble start at developing an approach aimed to probe this multi-layered, multi-faceted realm of exploration, laying the foundations for future investigations. In the sections that follow, I describe a bit about the theoretical underpinnings and motivations for TRACES, then provide an overview of how I am trying to implement this approach practically. I discuss the data streams I used and tools that teammates have built to enable this data collection and then illustrate some of the ways I and other team members have begun to analyze and synthesize the data and highlight some surprising findings that have emerged and touch on future directions, but also some limitations and caveats moving forward.
An Emergent, Activity-Oriented Approach
The motivations behind the TRACES approach are multiple. The aim was to take a deep look into people’s lives as they exist today- fragmented, yet flowing between different spaces, devices, services, temporalities. We needed a holistic picture, and knew that in order to do so, it was imperative to capture multiple nested layers of granularity of people’s lives. At a very practical level, we needed to do so to understand the various ecosystems of technologies and how they mesh (or do not) within day-to-day activities, thoughts, and desires. And at the vision level, we needed to understand how machine learning can, should, and will integrate into and impact people’s lives in the future. More simply put, if we are to develop systems that learn people’s contexts, activities, identities, networks, interactions, etc. to aid them, then we ought take a holistic, multi-dimensional ethnographic approach to understand the richness of those lives and guide development of in a humanistic, thoughtful, reflective way.
In considering all of these motivations and needs, one of the major influences in the development of this approach was Activity Theory. Activity Theory (AT) is a conceptual framework rooted in the socio-cultural tradition in Russian psychology. It provides a number of foundational concepts that are important in thinking about a multi-dimensional perspective. And it is also a pragmatic and fairly well-known perspective within technology research and design circles. Victor Kaptelinin (2013) notes that the activity-theoretical perspective “has an immediate implication for design: it suggests that the primary concern of designers of interactive systems should be supporting meaningful human activities in everyday contexts, rather than striving for logical consistency and technological sophistication.” There are five main elements of AT, as delineated by Kaptelinin, that influenced the TRACES approach: 1) subjects have needs, 2) AT is object-oriented, 3) there are hierarchical layers of activity, 4) there are internal/external and individual/social dimensions, and 5) AT focuses on context. I will touch just briefly on why each one of these was important to the aims of our research – which is technology-driven and future-focused – and thus our emerging approach.
AT emphasizes that subjects have needs. They carry out activities, interacting with objects, to meet those needs. As we are imagining interaction with future objects, which do not yet exist, this influenced us to consider higher level motives and goals rather than think about actions as task-based and therefore framed by the constraints of today. Following from this, it is also clear that AT is object-oriented. This means the world is organized around objects– which are defined as including both the physical world and social and cultural properties. Kaptelinin notes: “…[T]he subject’s activity is subordinated to properties of the object which gives rise to new activity structures; in turn, new activity structures bring about new subjective phenomena.” We are imagining future technologies, and thus future objects that give rise to new activity structures – new ways to pursue motives and goals. So we must understand those motives and goals and be able to decouple them from how they are currently achieved today.
Figure 3. The why, what, and how of Activity Theory2.
Following from this, one can break down activities into hierarchical layers, as illustrated in Figure 3. First, there is the activity itself, which is oriented toward a motive. This is the “why.” Below that are actions, which are conscious processes taken to fulfill a goal. These can be further decomposed into sub-goals and beyond. These are the “what.” And then there are operations – routine processes that address conditions and aid in reaching the goal. This is the “how,” and is also typically considered to be something people are unaware of. This structure helps us get at the issues of why, what, and how. Starting at the action layer allows a focus on qualitative methods, wherein people can report or express goals. And this analysis can be expanded both up to higher goals and motives and down to sub-goals and operations, which has been important in considering the dimensionality in how activities are accomplished.
The very notion of AT reveals a focus on both internal/external and individual/social dimensions. Activity is very intertwined between these dimensions. These divides are important to consider – moving from internal thoughts to external actions and vice versa; looking at the process of transformation of an individual activity into a socially distributed one and vice versa. Finally, AT focuses on context. It emphasizes society, culture, and development and a set of concepts for capturing the context of use. In the research we are pursuing, it is imperative to understand context and how that informs people’s motivations and actions so that we can capture a more holistic perspective and probe how an intelligent system might supply and suggest relevant information, affordances, and interactions. These principles of AT influenced the structure of the TRACES approach, and in particular the different types of data streams that might be needed to glean a richer perspective of activity across multiple dimensions.
New Tools, Data Streams, and Methods
At its core, TRACES is a multi-method approach to gather insights into the lives, practices, and needs of everyday users at multiple levels of granularity. Given all of this underlying theory, how are we attempting to implement such an approach practically? In its current state, the approach combines multiple data streams, including device logging, experience sampling and trigger-based sampling from device usage with more traditional ethnographic methods like in-home interviews, diaries, participant reflection activities, and observation and rich media capture. The goal is to combine fine-grained data with richer, qualitative data about the higher-level activity and more detailed data about the context and broader goals of the individual. As an ethnographer audience is already familiar with the more traditional ethnographic data sources, I highlight the less traditional ones – namely, the logging data, triggers, and mobile experience sampling data – which we have been able to implement and integrate into this work by using new tools, most notably, Paco (Evans, 2016), whose creator is part of this research team. First I discuss the data collection approaches – both etic and emic – then I will delve into how we are synthesizing and analyzing these streams.
An etic perspective – Utilizing log data itself is not new. It has been used in myriad studies within the human-computer interaction community and beyond. And approaches like ethnomining and trace ethnography have also brought it into the ethnographic toolkit. TRACES takes a similar approach to collection, building slightly on this by expanding the scope, but more importantly extends the ways in which it is integrated with other methods and synthesized with other data streams. First, TRACES expands the scope of log data by utilizing not just one source of logging, but many. Using Paco as well as other tools, I have logged not only mobile data, but also tablet, laptop and desktop sources in an attempt to begin to explore the ways in which different devices, both individual and shared, in different locations and with different roles and uses integrated into the lives of those who use them as they move through their typical day. And as capabilities extend, particularly on mobile devices, these log data provide not just behavioral characteristics of usage, but also contextual ones that provide insight into the physical world around the person, through sensor data. This also begins to provide a richer picture of how the embodied and digital are intertwined, enmeshing and embedding multiple identities – such as work identities, family identities, or gaming identities – in different spaces and dimensions.
Capturing more types of log data extends the scope of usage of this data stream, but TRACES has also extended upon log use as a data source by also incorporating triggers based on the logs. This is instrumented a priori, through the Paco tool. It provides the capability to choose an action, such as opening or closing an application, or taking an action within an application, which will then trigger a prompt for the person using the device to respond to. So, for instance, it enables the possibility of diving deeper into an interaction of interest like the use of a particular communication medium or a game, providing an opportunity to contextualize the actions into broader activity and, going back to the AT influence – when an individual activity becomes one that is social. In addition to asking simple text questions, Paco enables capture of rich media, so photos, screenshots, and audio responses can be used to gain deeper understanding of the moment.
Additionally, team members have developed tools for visual exploration of the mobile log data that enable a look into behaviors in real time, which are being incorporated into Paco. These visualizations inform the ethnographer’s perspective and allow an opportunity for rapid data exploration and analysis. The visualizations show things such application usage types and individual mobile usage patterns throughout the day, which have allowed both the primary researchers as well as other stakeholders in the research to develop questions and areas to probe before an interview.
These different streams of logging data, the triggers that capture even more rich data, and the tools for visualizing it, are all rich sources of data themselves, but also provide topics to probe deeper with the people being studied. So they are a source of etic, observer perspective, from the outside, looking at behaviors. But they then help inform areas where we can gather an emic perspective from the subjects themselves in interviews.
An emic perspective – In addition to the other, more traditional ethnographic approaches we employ (interviews, diaries, reflection activities) where we can get a subjective perspective, we have also used the Paco tool to help capture an emic perspective of moments and their contexts throughout the day by instrumenting experience sampling on their mobile devices. In these experience sampling moments, people respond with information about their primary and secondary activities and the context surrounding those activities in the external, physical world. This includes questions around things like the materials used, where they are, and whom they are with. But we also probe for focus on the internal world of the individual, asking about how they are feeling, what they are thinking about, what their goals are for the activities that they are conducting, and what they are planning to do next. As with the previously mentioned trigger sampling, and in addition to asking simple text questions, Paco enables capture of rich media. So photos, screenshots, and audio responses are also part of this data stream. Going back to the influence of AT, this data allows us to explore both the external and the internal worlds of the individual from their perspective. So, there is additional dimensionality there as well. Thus, these overlapping sources of data provide both emic and etic perspectives looking at multiple worlds – the embodied worlds, the digital worlds, and the internal worlds of the individual.
When we combine all of these new data streams with those from more traditional methods, we have quite a combination: in-person interviews and observations, reflection activities and visual storytelling, daily diaries, experience sampling of moments throughout the day (focused on internal and external activities), various sources of logging data from mobile phones, tablets, laptops and desktops) and, from mobile phone, additional contextual data and trigger sampling date based on certain device activities. Extending beyond prior studies, these myriad forms of data allow us to not just understand the embodied, observable lives or the subjective perspective of it. They also enable us to dive into the internal state, the external enactment, the digital dimensions of these lives – individual and social, ephemeral and sustained – and from both etic and emic perspectives. As a result, we are not only able to understand daily phenomena and how technology fits into people’s lives at a surface level, but to more deeply investigate the meanings, processes, contexts, and culture across these multiple dimensions. And in turn, I believe this will help us to better understand the people whom we are attempting to “learn” (as in, machine learn) for to develop rich interactions with intelligent agents.
Analyzing TRACES – So, we have a number of qualitative and quantitative data sources, from emic and etic perspectives. We have thick data and big data, in an individual sense. How do we begin to make sense of all of it and interpret it? I have already noted the use of some things, like visualizations. And we of course employ a number of traditional qualitative coding techniques and grounded theory explorations. In addition to more conventional analysis practices, my teammates and I have also used some other approaches to synthesizing and analyzing the data. I will highlight a few unconventional approaches, including use of semiotic mapping, community link detection algorithms, and machine learning techniques.
Semiotic analysis is not new to the ethnographic community. But where it differs in its use here is how it is utilized to map signs, their signifiers, and what is signified across multiple different types of data from these various data streams. The starting point of analysis is a reflection activity, which I conduct while interviewing each of the individual participants. Part of this includes asking them to reflect on their past week and visually tell the story of it. The anchoring foci and how the story unfolds are different for each person. But regardless of the way it unfolds, it reveals some of the more powerful symbols of their recent experience. Taking these symbols, we can then delve into lots of other data from that previous week – logs, responses to triggers, contextual data, and experiential moments that include both external and internal aspects. We are thus “mapping” that sign into multiple dimensions. This has revealed some fascinating insights into what’s salient in the data versus the human (participant) experience.
Another approach utilized analysis algorithms to understand user clusters, among other things. User clusters are focused on looking at the degree of similarities among people in terms of their technology usage patterns. This technique primarily uses log data to compute pairwise comparisons (Jaccard similarity and Cosine similarity) and then uses relational class analysis (Goldberg, 2011) to investigate the different user clusters. Based on the patterns that emerge, we can first look to the other rich sources of data to try to understand these different clusters and label them. And we can also use them iteratively with other data to analyze behaviors to understand such things as whether usage patterns are really similar across different people we have studied. This is very valuable for a number of reasons, including having new ways to segment users, rather than on basic demographics. Other algorithmic approaches, such as community link detection algorithms (Ahn et. Al, 2010), can also be used to look at things like app clusters. Analyzing recurrent and recursive app usage patterns helps us understand how applications may be used in flows of activities. And this has provided deeper insights into emergent activity and the assemblages that constitute the individual.
This research is ultimately focused informing the development of technology systems that include machine intelligence, so part of our explorations have focused on how we can combine quantitative and qualitative data in unique ways to explore how machine learning can help us understand behaviors and predict individual traits. In one exploration of this, we looked at notions of privacy. A team member experienced in machine learning (ML) techniques first looked to interview data to qualitatively categorize perspectives towards and around privacy as a concept. These categories were then analyzed quantitatively across a wide swath of participants who had experienced varying degrees of privacy breaches to understand if there was a relationship. And then ML algorithms were trained on the behavioral data from the logs and then used on a subset of features to determine if it was possible to predict the attitudes around privacy that were uncovered in the interviews. In short, the goal was to understand if we can predict a privacy breach, with the aim of trying to develop better solutions or help users of services understand this better.
These are just a few of the approaches to analysis we have engaged with thus far. They incorporate tools and techniques from other fields, but they have provided a rich, multi-dimensional perspective that enhances and extends what we are doing ethnographically. These approaches provide robust ways to dig into thick, multi-dimensional data. They are immersive, providing rich insights into the lives of the individuals we study. And they provide new lenses to interpret meaning. Our approaches have been iterative and we are still experimenting with different types of interpretive methods and how we can creatively get at research questions that require explorations reaching into these various dimensions. In the section that follows, I will highlight some of the higher-level findings from this research and broader emerging insights into research in this space.
Future Directions, and Caveats
The current state of the TRACES research program is just the beginning. We have utilized new tools, new data streams and combinations of methods for synthesis and analysis in order to make sense of these emerging areas. But as the physical and digital worlds continue to change, we must continue to adapt and refine our foci and our methods for understanding those worlds. We aim to combine different data streams and continue to experiment with new ones, and we hope to find new iterative and creative ways to explore the data.
The aim of this paper is to argue for the need for multi-dimensional ethnography. TRACES in its current state is just one initial set of attempts at getting at multi-dimensionality. We are probing multi-dimensionality through the use of a variety of data streams – through logging of multiple devices, which capture behavioral use and some level of context, through triggers that probe deeper, and through experience sampling focused on both the internal the external. All of that, combined with more traditional ethnographic techniques, has helped us make some headway. But there’s a lot more to continue to explore.
In terms of data there is a lot more we can glean. From mobile phones, for instance, we are only getting a minimal level of contextual data, namely GPS location, time, and when the phone is locked/unlocked. There is a lot more sensor data available in mobile phones, like the accelerometer, magnetometer, gyroscope, barometer, proximity sensor, and ambient light sensor, as well as understanding presence of other devices, and thus perhaps people through peer-to-peer connections etc.
But there is also a lot more in terms of device possibilities. In addition to mobile devices and other general computing devices like laptops and tablets, there’s a whole host of other devices people are beginning to interact with on a daily basis, including wearables and in-home Internet of Things (IoT) devices. The sensor data from each of these gives insight into different dimensions and at different levels. Wearables open up areas of biosensing like galvanic skin response. And in-home IoT devices move us into a new realm that considers the richness of ambient interactions. We still tend to think of screens when we think of device interaction, but as we move into a post-screen world, we must also account for other modalities, such as sound, touch, and beyond. And we must consider the interactions within them at a more algorithmic level. These spaces highlight the ephemeral nature and importance of thinking about amalgamated experiences – they are not “places” in the same sense as a website, and they are increasingly guided by machine learning, with adaptive algorithms shaping the experience. They are dimensions. And to get at these dimensions and interactions within them to understand the lives lived within those dimensions, we must continue to expand our toolkits. But we must also be cautious in doing so.
Caveats – It goes without saying that there are important concerns, sensitivities, and considerations in this space. TRACES is but a humble start at getting there. I have not gone into the detailed consideration we have given to the types of data we are getting from who and why and how we use it, but these were all incredibly important aspects of making sure this work is ethical. Ethics are a driving force in this work, not just from a research perspective, but also from a goals perspective. The TRACES work is intended to help understand models we can develop for machine learning and shape the design and development of systems with machine intelligence. And with machine intelligence (MI) having increasing agency in our lives, it is important to fully understand the ethics of what we are learning from people and how we are using it to shape their world as a result.
There are also caveats to setting things up properly. And there are cautions for interpreting them appropriately. This sort of work has immense raw materials- ranging from participant quotes to rich logs. This work requires continuous exploration, confirmation, and validation. We must continue to develop our protocols and instruments. We must continue to look to the grounded data as a source for inductive inquiry. And we must continue to consider the process of looking within and across people and populations and ensure data veracity. These are imperative as we move from data and information to knowledge to meaning.
A PHILOSOPHY OF MULTIPLE DIMENSIONS
We live in a rapidly changing age where the digital dimensions in which we engage shape us and society writ large. As Boellstorff (2016) notes: “Our era of the Anthropocene is now a ‘Digitocene’ as well.” And understanding this space has vast implications, especially as we move into a paradigm of specialized AI informing many of the digital spaces we engage with and through.
Prior work in the ethnographic community has certainly delved into the realm of the digital. Some have approached the digital as subject. Others have looked at the digital as site of interaction. And more recently, there has been more focus on digital as rich data input source or organic data stream. Each of these approaches focuses on important aspects of the digital in ethnography. These are important areas of developing focus and methodology within ethnographic praxis, but we must continue to push them forward and capture multi-dimensionality. Our research must engage with what I would call our “cyborg society.”
Digital is an important topic, a space where humans engage, a source of artifacts and data. But, as I argue, it is also an important to see the other dimensions of life – in our cyborg society – to have a holistic picture. Think for a moment of writing as a “prosthetic memory.” Yes, our cognition is distributed in such a way and has been for centuries. But it is much more complicated in a digital world. Now, humans are already cyborgs by the principle that they constantly use things like laptops and mobile phones to live their daily lives – with a reliance beyond even what Haraway (1984, 2013) conceptualized decades ago. As “cyborg anthropologist” Amber Case (2011) notes, we have “‘second selves’” that exist online and can be interacted with, even when we are not present. We are moving into a world in which AI and machine learning are changing our prosthetic capabilities. We are moving into a world that allows us as ethnographers to reflect on what we make of our objects of study.
People – human beings – never were the subject of ethnographic study. We don’t study people, we learn from them. And those learnings tell us about things like meanings, culture, power. Taking a multi-dimensional approach allows us to also consider more deeply the ways in which machines are also shaping things. This does not separate them from their human creators or users as such. But it emphasizes that, while there are beliefs and values built into algorithms a priori, they also have emergent properties that can shift this. In his work on sieves, like spam filters, Kockelman notes: “In particular, the values of the variables are usually steps ahead of the consciousness of the programmers (and certainly of users)—and thus constitute a kind of prosthetic unconsciousness with incredibly rich and wily temporal dynamics. Note, then, that when we make algorithms and then set those algorithms loose, there is often no way to know what’s going to happen next (Bill Maurer, personal communication). (2013)” This has far-reaching implications once we start to think beyond static algorithms as well.
Specialized machine intelligence has an ever-increasing presence in our lives, and taking it as subject would be revealing of our world today. And taking it as the focus of study is not new. Woolgar (1985) articulated this many years ago. What is perhaps new is the ability to get an even richer perspective over multiple dimensions. The purpose of shifting our framing to thinking of ethnography within multiple dimensions is not just to point out the variety of temporal, spatial, analog, and digital realms in which we live and in which our identities are embedded. It is also about the philosophical implications of studying the world – what does it look like from a machine perspective? And how can we bring more of a human element back into that? For those of us working in the organizations in which these technologies are being developed, it is imperative to focus on how these developments will influence and shape power, culture, and meaning in our world.
Julia Katherine Haines conducts research at the intersection of technology, innovation, and human practices. She is a User Experience Researcher at Google and a member of the ACM Future of Computing Academy. Julia received her PhD in Information and Computer Sciences from University of California, Irvine. juliahaines@google.com
NOTES
Acknowledgments – This work would not have been possible without the contributions and thoughtful feedback of many fantastic collaborators at Google. Elizabeth Churchill provided incredibly valuable guidance, insight, and encouragement throughout this research. I would also like to thank Su Baykal, Tao Dong, Bob Evans, Michael Gilbert, Krithika Jagannath, Katie O’Leary, Minsu Park, and David Ramsay for all of their tremendous contributions to the TRACES project. And thanks also to the reviewers and curators of EPIC, especially Michael Powell, for their thoughtful comments, which helped shape this paper. Please note that this work does not represent any official positions of Google, Inc. or Alphabet.
1. Definition of “dimension” from Merriam Webster.
2. Based off of illustration (author unknown) here: https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/activity-theory
REFERENCES CITED
Ahn, Yong-Yeol, James P. Bagrow, and Sune Lehmann
2010 Link communities reveal multiscale complexity in networks. Nature 466.7307: 761-764.
Aipperspach, Ryan, et al.
2006 Ethno-mining: integrating numbers and words from the ground up. Electrical Engineering and Computer Sciences, University of California at Berkeley, Technical Report No. UCB/EECS-2006–125.
Boellstorff, Tom
2008 Coming of age in Second Life: An anthropologist explores the virtually human. Princeton University Press.
Boellstorff, Tom
2016 For whom the ontology turns: Theorizing the digital real. Current Anthropology 57(4): 387-407.
Case, Amber
2011 We are all cyborgs now. Ted Talk. https://www.youtube.com/watch?v=z1KJAXM3xYA
Coleman, E. Gabriella
2010 Ethnographic approaches to digital media. Annual review of anthropology 39 (2010): 487-505.
Droumeva, Milena
2015 Sensory postcards: Using mobile media for digital ethnographies. Ethnography Matters. Retrieved from http://ethnographymatters.net/blog/2015/06/29/sensory-postcards-using-mobile-media-for-digital-ethnographies/
Droumeva, Milena
2015 Curating Everyday Life: Approaches to Documenting Everyday Soundscapes. M/C Journal, 18(4).
Evans, Bob
2016 Paco—Applying Computational Methods to Scale Qualitative Methods. Ethnographic Praxis in Industry Conference Proceedings. Vol. 2016. No. 1. 2016. https://www.epicpeople.org/applying-computational-methods/
Geiger, R. Stuart and David Ribes
2011 Trace Ethnography: Following Coordination through Documentary Practices. Proceedings of HICSS 2011.
Goldberg, Amir
2011 Mapping shared understandings using relational class analysis: the case of the cultural omnivore reexamined. American Journal of Sociology 116.5: 1397-1436.
Haraway, Donna
2013 Simians, cyborgs, and women: The reinvention of nature. Routledge.
Hine, Christine
2000 Virtual ethnography. Sage.
Hsu, Wendy F.
2014 Digital ethnography toward augmented empiricism: A new methodological framework. Journal of Digital Humanities, 3(1), 3-1.
Kaptelinin, Victor
2013 Activity Theory. Encyclopedia of Human-Computer Interaction. Chapter 16.
Kockelman, Paul
2013 The anthropology of an equation. Sieves, spam filters, agentive algorithms, and ontologies of transformation. HAU: Journal of Ethnographic Theory, 3(3), pp.33-61.
Kozinets, Robert V.
2010 Netnography: Doing ethnographic research online. Sage publications.
Marcus, George E.
1995 Ethnography in/of the world system: the emergence of multi-sited ethnography. Annual Review of Anthropology, vol. 24, pp. 95–117
Nardi, Bonnie
2010 My life as a night elf priest: An anthropological account of World of Warcraft. University of Michigan Press.
Woolgar, Steve
1985 Why not a sociology of machines? The case of sociology and artificial intelligence. Sociology 19.4 (1985): 557-572.