With user research becoming more common within organisations, there is an emerging issue of meeting demand whilst also developing the craft of research. A new profession is emerging in response – research operations. This paper will describe the current state of publicly available frameworks for research operations. These tend to deal with one aspect of scale – the people who are doing the research, not how they do the research, when, or what we do with the research. Two frameworks will be combined to create a matrix that provides the tools to identify an investment strategy for research within the context of an organisation and their strategic goals. This matrix provides a significant contribution to the field by making it possible to be strategic and proactive about developing research practices in the context of individual organisations, how and why they do research, and to better manage the tension between scale and craft.
Keywords: ResearchOps, UX, Strategy
INTRODUCTION
Qualitative research as an embedded practice in industry and in government has been emergent since at least the 1960’si, and has grown to the point of being commonplace in the world today. Modern design is largely attributed to the collision of the arts and crafts movement with the machine age.ii Alongside this emerging popularity, the practice of modern design has matured, and our understanding of art and design as an embodied experience, one worthy of replicating in applied ways, has also matured. Seen in this light, it comes as little surprise therefore, that in recent years, the attention economyiii has raised the profile of (and pressure on) human researchers and research outcomes even further, as companies attempt to squeeze out every last minute in a person’s day that can be spent on their device, in a platform, and/or watching ads. It is a bleak picture to paint, but the opposite story is there too – through human centredness/system centredness, researchers have a role in enabling industry and government to have a meaningful impact on people’s lives. Our current state with regard to the health of the planet and the people therein may cause many to want to engage in qualitative research as a part of their design work in order to have the best possible chance of effecting meaningful, ethical and human centred change. That means it may seem that everyone, from the smallest to the largest company, are employing researchers to do more and more research in less and less time.
The possibility of effecting that change is incredibly exciting, intoxicating even. In many ways, the chance to do some real, lasting change work has never been more present. The profession of applied qualitative research (commonly known as user research, as it will be referred to throughout the rest of this paper, but also including design research, UX, CX – largely a broad umbrella of qualitative, human led, conceptually post-modern research) has developed significantly in the past few decades. Alongside traditional ethnographic and anthropological or human factors research methodologies, or other disciplines such as psychology, researchers can be trained in systems thinking, and in the practice of co-design.iv It is possible to see that the field is becoming more established.
The reality of having teams of any size – whether 1, 100, or 1000v, is that the demand for research far outstrips anyone’s ability to meet that demand. While it might be traditional for a research team of one, or 5 even, to individually be running their own processes and procedures, their own contracts, panels and ways of working when it comes to research data management and sharing, it can become a huge time impost, leading to duplication, unintended replication and burnout.
The ResearchOps Community is a volunteer run online community of over 6000 (to date) individuals from 62 countries working in the field of user research and research operations, coming together with a common goal of giving shape to, and validating research operations as a profession. The common challenge facing the members of the community is doing research at scale.
As work has progressed on understanding what research operations is, so too has our understanding of how to manage the tension of delivering research at scale, whilst also maintaining rigour in research. This paper will describe the current state of frameworks (that are publicly available) for research operations, which, to date, tend to have been maps for research and research operations. Following this, the paper will bring together two frameworks that, once combined, allow one to see the terrain of research in individual contexts. This blending of frameworks, known as the Pace Layers Matrix is the result of observation and experience from 3 global research projects undertaken by the ResearchOps Communityvi (one on what research operations is, one on building a research skills framework, and the most recent on research repositories) and the author’s own work in understanding the research outcomes from those projects. Having the tools to identify the terrain of one’s own research practice in context provides a significant contribution to the field by making it possible to be more strategic and proactive about developing research practices in context and better manage the tension between scale and craft.
The Problem with Scale
By now, the industry is well and truly aware that there is a problem with managing the demand for user research. The issue is a seemingly simple one – qualitative research takes time, lots of it, and this does not scale well. Demand for research grows, and the expectation that good research can be done in months moves to weeks, and sometimes even days. At what point fast research becomes poor quality research is what is at issue. At what point the profession suffers from poor quality outcomes from overstretched, under-resourced or untrained researchers is an ever-present burden when the topic of scale emerges.vii
There are myriad ways to deal with demand – adding more and more researchers, creating hub and spoke models to have core researchers at the hub and ‘people who do research’ (PWDRs, a phrase coined by Kate Towseyviii) operating in small teams, or having a core group of researchers embedded individually across the organisation but reporting to a central research leader. But each of these models only deals with one aspect of scale – the people who are doing the research, not how they do the research, when, or what we do with the research.
Research operations has emerged from this gap – a field dedicated to:
“the people, mechanisms, and strategies that set user research in motion. It provides the roles, tools and processes needed to support researchers in delivering and scaling the impact of the craft across an organisation.”ix
Within the field of research operations (also known as ReOps, or ResearchOps), there are a group of sub-fields, all addressing slightly different issues to do with how we create the right environment for research to happen. They include (non-exhaustively):
- Making better use of existing research through the creation of a research library or repository, though these often fail to achieve the results that are hoped for.
- Research operations playbooks or ‘centres of excellence’ are one of the first ways one sees operations leaders attempting to address the ‘how’ research happens at scale. This also attempts to address the additional issue of the tension caused when trying to do a lot of research in a short time – creating efficiency, and also enabling others to do research, even if not fully trained in doing research.
- Systematising and streamlining recruitment
- Centralising budgets and managing tools, platforms and contracts centrally within large organisations
All of these responses to scale create ripples that are felt across the organisation and the broader user research industry. Indeed, the concept of the democratization of research is a hot topic, occupying whole streams at research conferences (see for example: Advancing Research 2020x) and the topic of debate in blog posts and papers.xi Interestingly, the democratization of research has long been a topic of debate in academic circles also, but it is framed instead as a feminist act, or an act of ‘research justice’,xii enabling research to be decolonised. This is not two separate disciplines using the same terms for different ends. Rather, it is a different lens on the same issue – the practice of extending research spaces to people outside the role of research. From researched to researcher, from consumer of research to doer of research. Kara states that “The term ‘democratizing research’ covers a range of emancipatory approaches to research such as activist research, feminist research, decolonizing methodologies, community-based research and participatory research”.xiii User research, being embedded within design and design principles such as co-design, co-production and othersxiv is a practice of research in, of, and sometimes with, community, and is often participatory – the tension therefore, is the same.
Complex Systems and Frameworks: Tasks vs Strategy
To understand the current state of ‘research at scale’, it is worth acknowledging the myriad frameworks that have arisen in the wake of the emergence of research operations. Given that the ResearchOps community (that really is a catalyst currently for the emergence of the profession and the development of frameworks for understanding what ResearchOps is) started with the ‘what’ of research operations, early frameworks, such as the ResearchOps Community’s ‘What is ResearchOps’xv and the Nielsen Norman Group’s ‘ResearchOps 101’xvi have focused on dividing up the tasks or things that need to be done in order for research to happen effectively. It is possible to see however, that tasks are only part of the way research happens. How it happens, with whom, and most importantly for this paper, in what contexts, are all essential in effectively delivering research at scale.
Towards a Strategy for Scale: PESTLE Models
Towards the end of 2018, a group on the board of the ResearchOps Community (Emma Boulton, Holly Cole, Tomomi Sasaki and myself) realised that the taxonomy, or the conceptual framework we’d applied to understand the data from the ‘What is ResearchOps’ project could be used to understand the relationship between research and operations. Emma Boulton took this forward with the 8 Pillars framework.xvii This model (Figure 1 below) can be seen as a typical PESTLE strategy model. The PESTLE model arose from the work of Professor Francis Aguilar following his book, Scanning the Business Environment in 1967.xviii It is a framework for understanding the political, economic, socio-cultural, technological, legal and environmental factors that are involved in managing business, with the idea being that if one is aware of the forces impacting the business, then it is possible to create a strategy for optimizing opportunities and mitigating risk.
Figure 1: 8 Pillars of User Research
In the same way, the 8 Pillars, with the focus on environment (in the PESTLE model: environment), scope (political), people (socio-cultural), organisational context (economic and political), recruitment and admin, data and knowledge management, governance (legal), and tools and infrastructure (technological) can be used as a way to generate an understanding of the factors and forces at play when research happens.
Briefly stated, the 8 Pillars, as they pertain to research, are:
- Environment: Why does research happen? Who engages with what I do?
- Scope: The nuts and bolts. Methods, processes. How and when does research happen?
- People: Research maybe done by designers or product managers. Can we create a community of practice to support and mature the craft? What does a career path look like?
- Organisational context: What is the maturity level of the organisation I work in? What are the external constraints that affect me? Things such as budget, resources, time, space.
- Recruitment and admin: How do I manage all the logistics and admin for research projects and participants?
- Data and knowledge management: Often valuable insights are lost over time as teams grow and change. How do we ensure the same studies aren’t repeated? What happens to the research findings, data and insights?
- Governance: As a researcher what are the legal and ethical considerations that affect my work?
- Tools and infrastructure: What tools and infrastructure do I need to help me with my research?
The 8 Pillars is an effective strategy framework and provides a high-level view of the concepts and things that need to be in place to make research happen. However, over time, as the community grew, it became obvious that different methods employed to do user research were also important to understand as the concepts and things that need to be in place to make research happen are dependent on the method employed. Across the industry, researchers tend to use mixed methods dependent on context – on time, resources, capability and of course, the research question and outcomes required. The 8 Pillars provides a map – but to take a step further, what’s required, is knowing the terrain. The profession of research operations has moved from a list of what is present in the map (the ‘what’ of research operations), to a 2D map of research (the 8 Pillars here can be viewed as a way of seeing the pathways present in getting us ‘down the road’). But what good is a map if it cannot tell us whether to pack a kayak or snowshoes? How big is the mountain? How steep is the road?
Complex Systems and Pace Layers
To that end, Brand’s Pace Layers framework can be used to see the terrain. In 1999, Stewart Brand wrote The Clock of the Long Now: Time and Responsibility. In it, he suggested Pace Layers could be used to understand complex systems.
“Fast learns, slow remembers. Fast proposes, slow disposes. Fast is discontinuous, slow is continuous. Fast and small instructs slow and big by accrued innovation and by occasional revolution. Slow and big controls small and fast by constraint and constancy. Fast gets all our attention, slow has all the power.”xix
This is deeply analogous to research, because Pace Layers are all about time, speed and depth. An important point about the above quote is that ‘fast learns, slow remembers’, also, noting that ‘fast gets all our attention and slow has all the power’.
In many ways, organisations with sufficient research to have need of an operations function can also be understood as complex systems. They tend to have researchers who use varying methodologies, have people who do research (PWDRs), and people who need research as well as full-time researchers. All these people influence how research gets done, why and when.
Figure 2: The Pace Layers Framework, © Brand, S, used with permission.
The Pace Layers shown above represent depth and pace. The deeper layers move or change more slowly, but conversely, also serve as a foundation. The higher the layer, the faster it moves and changes. Using Pace Layers to understand different research methods can help with strategy development as it takes a holistic approach to the ecosystem surrounding research. In this way, it is possible to move from a reactive position towards a proactive, strategic one.
The Tension We All Feel: Constructive Turbulence
To Brand, the relationships between layers are key to the health of the system. Paul Saffo, a collaborator with Brand on the Clock of the Long Now project, goes further with this idea, stating that conflicts caused by layers moving at different speeds keep things balanced and resilient. Saffo called this “constructive turbulence”.xx Managing this constructive turbulence is the key to understanding inertia in the system, the things that constrain research teams, and the opportunities to scale. Turbulence in the system that is off balance can be seen clearly in the tension noted previously between the speed demanded by business and the time taken to do contextual, generative research. A symptom of imbalance, is researchers needing to spend so much time on faster evaluative types of research that they cannot gain the time or by-in for generative research favoured by ethnographic research methodologies for example. Or researchers spending so long doing the deeper layers of research that they are unable to respond quickly or lack the skills or infrastructure to do evaluative research when it is required.
Figure 3: If research methods were Pace Layers. Pace Layers framework used with permission.
A consistent complaint of researchers who use slower and more in-depth research methods (those closer to the practice of ethnography and anthropology- closer to ‘people’ and further away from researching people in the context of ‘things’), is that they struggle with constant pressure to reduce the cadence of their research. They struggle with pressure to deliver according to the cadence of business, rather than deliver within the traditional research methodology (lots of observation and research at the start, during which almost nothing is ‘delivered’). If we use Pace Layers to understand the nature of generative research and its place within the system, this will help us to reframe the value of the slower, deeper layers, and also to see the friction between those layers.
On the other hand, the research that gets done at the top is the one that makes all the noise. There is high demand and constant pressure with regard to time and the findability of the insights generated in this layer. Researchers tend to be working within agile sprint cycles with product development teams. Researchers doing mostly evaluative research, struggle with the denigration of the value of their research. Sometimes, they have research leads or executives wanting to be more strategic or to get more from the research than is possible. The noise and speed can make it hard to drill down through the layers and get support for the slower types of research methods that can contribute to those strategic needs better than the evaluative research can.
Pace Layers and Research Operations
When research methods are viewed using Pace Layers, it is possible to see that the operational work that needs to be done to help researchers do their best work is also different. All research methods require every aspect of the 8 Pillars to be in place in order for research to occur, but the focus is different depending on the dominant method used.
Figure 4: Research methodologies with their Ops foci — aligned with Pace Layers. Pace Layers framework used with permission.
In Figure 4 above, it is possible to see that the research methods used to do evaluative research will have an operations focus of tools and platforms for research such as moderated and unmoderated usability testing. Evaluative research tends to be more frequently done by researchers or PWDRs embedded in product teams, and so communities of practice become essential in ensuring research practices are consistent and that researchers can develop their research practice as a group. Descriptive research focuses on scope and knowledge management as it tends to be desktop research. Finding research that has already been done to evaluate is crucial. Having access to that research is crucial, as is receiving help with refining the scope of the research. Causal research has a focus on time, by-in and recruitment. Causal research requires a particularly rigorous approach which means that it can take a long time. Getting buy-in for these methods, such as A/B testing, is important, because it is the first of the layers in the model that really cannot be quickly carried out without undermining the rigor of the research. A careful approach to recruitment is essential in order to ensure the validity of the research. Generative research has a focus on consent, ethics and knowledge management. Generative longitudinal is the same but also has a focus on research data management and recruitment. Ethnographic and anthropological research tend to use observational methods and require what Clifford Geertz famously described as ‘deep hanging out’.xxi This generates a tremendous need for careful research data management (as this deeper, more contextual work tends to automatically accumulate a lot of personally identifiable data that grows in complexity and risk over time) including a long view on the ethics of the research and on ensuring participant consent is genuine, informed and that the participant retains the agency to work with the researcher on the way their data is used and managed over time. Research data management is important always, regardless of the research method, as is compliance with GDPR, however, it is simply that the complexity of managing this over time only increases as research data for an individual participant builds and layers over time. Triangulation of the data can render previously de-identified research data in practice, identifiable. Effective use of this growing parcel of unit level data in turn generates a large reliance on knowledge management practices.
BRINGING THE MODELS TOGETHER: THE PACE LAYERS MATRIX
Understanding the layers and the 8 pillars as a matrix provides a tool for diagnosing strengths and weaknesses of a research practice and operations practice within an organisation, and therefore provides a path to both scale research within an organisation and also deepening research and operations capacity between the layers. To describe this effectively, two case studies are presented below. The first is an organisation with a strong research practice with researchers focusing on ethnographic (generative) research methods. The second is an organisation with a focus on utilising user research to best understand the use and effectiveness of the platform the organisation sells. Here there are a group of researchers embedded in product teams tending to work with developers through design and delivery to best evaluate the product development. The focus here is on evaluative research. The two matrices are shown side by side at first so that it is possible to see how the layers impact on the operations needed and how these vary in different contexts.
Figure 5: The Pace Layers Matrices in two organisations (explored in more detail below). The organisation on the left is case study 1, with a focus on ethnographic research practice. The organisation on the right is case study 2, with a focus on evaluative research. Pace Layers used with permission.
Case Study 1
Figure 6: The Pace Layers Matrix in an organisation with a strong ethnographic research practice. Pace Layers used with permission.
The first case study is a research team of around 50 with a strong ethnographic research practice. The team tends to focus mainly on doing generative longitudinal, generative, causal and descriptive research. The focus in the environment pillar is on communicating with stakeholders, in the scope pillar on integrating previous insights, in the people pillar on developing the career paths of researchers, in the organisational context pillar on working within business constraints, in the recruitment and admin pillar on undertaking effective panel management and participant experience, then in data and knowledge management the focus is on research data management, in governance it is on consent, ethics and to some extent, in tools and infrastructure, having spaces and systems in place to allow researchers to be in the field. Their challenges will be about continuing to show the value of what they’re doing, though they won’t need to evangelise research as such. Instead, the push back will be the time it takes, the cost, it will be managing the melody of long and slow with the needs of business. They will do that through a rigorously managed panel, good participant experience, and by building their base of research to a level that others can dip into it as needed. Their research is very manual, so there is not a lot of focus on tools.
How Might They Scale?
The team’s strength is in the depth and power of the deeper layer of research that they are creating for the organisation. If they can focus on getting the most out of that layer of research for the organisation through a research library, they can support the organisation to be able to move quickly (but with deep certainty regarding their evidence base) by giving the PWDRs a rich research asset to refer to. This will support less experienced researchers to feel more confident in their findings. They can be the foundation on which everything grows. If the research team feels the need to utilise faster, less contextual research methods, moving straight to the top layer (evaluative research methods) isn’t going to be effective, because they don’t have the tools and technology in place to do so. Instead, they can direct their efforts towards moving gradually up the layers from the bottom (moving more heavily into descriptive research), or they could bring PWDRs into their research team who may have the tools and technology in place to do evaluative research, and focus their efforts on developing their research practice until the whole ecosystem has strengths across the layers.
Case Study 2
Figure 7: Case Study 2: Pace Layer Matrix for an Organisation with a Focus on Evaluative Research. Pace Layers framework used with permission.
The second is an organisation with a focus on evaluative research. This organisation has PWDRs and researchers dispersed throughout the organisation. Their focus in the environment pillar is on being careful to break down silos caused by being embedded across different teams and on gaining support to do deeper, more strategic research. In the scope pillar, Prioritising the research throughout sprint cycles is important, and in order to get a lot done in short time frames, the researchers will likely be good at treating research as a ‘team sport’. This helps them continue to improve the buy in for more research. The design and development teams will expect insights to be delivered quickly, and their involvement in the research may generate tension around the democratisation of research. They might have descriptive and some generative research, but it will all be in support of the top, noisy layer.
Unmoderated usability testing and other methods undertaken in evaluative research requires a heavy focus on the tools and infrastructure. In the organisational context pillar, recruitment and admin, data and knowledge management and governance pillars, there will be a need for resources in the form of tools, templates, and guides for the PWDRs. In the people pillar, a community of practice could help them develop their research practice in the organisation. Given the research methods at play here, their consent is lightweight, most of the research is de-identified right from the start. They will not be thinking about a library, and if they do, it will be likely held in whatever system the developers use to track their work.
How might they scale?
This organisation will struggle to develop their research practice without employing more experienced researchers. If their next hire is a senior researcher, their role could be to mentor and train the existing PWDRs. An evaluation of the skills of the current PWDRs is likely to uncover some people with skills in statistics due to the more quantitative nature of evaluative research methods. This could assist the organisation to undertake descriptive and causal research without significant change in the structure of their research teams, investment in technology or a significant increase in operational responsibility.
The barrier for this organisation in developing their research practice to include all layers is that they will need to be careful to communicate about the slow, deep layers and how they fit in the research lifecycle or the turbulence between the layers will become too great – the pressure on the lower layers to move at the same pace as the top layers will undermine any efforts they make to increase the depth. If they scale too quickly, panel management and data management (in terms of ethics and consent) will become a problem. If this organisation has an operations function, the Ops team can focus their attention on working with the researchers on enhancing research data management and panel management. If the team begins to notice the way they conceptualise their research has changed to thinking about their research as an asset, rather than as evidence (as people tend to do with evaluative research), then this will be a good indicator that their research practice has matured across the layers, and they are well on the way to a research practice that includes each layer.
Pace Layers Matrix: Understanding the Terrain
The case studies shown above highlight that each organisation has a different context, but by bringing the 8 Pillars and Pace Layers Frameworks together, it is possible to chart the terrain within a single organisation – to identify individual strengths and weaknesses within the context of the organisation. Identifying the bumps in the road, the rivers and streams, mountains and valleys within an organisation is a difficult piece of work, best done together with all across the organisation who might be involved or have a stake in research. These people may be people who use research, people who do research, people who read research to aid their own research, operations people and people with a strategic responsibility in the organisation. Interviews using the matrix as a reference alongside a research lifecycle view can help uncover what each person perceives as the organisation’s strengths and weaknesses, what they felt the organisation needed from research, where the demand for different research methods comes from, and what capacity they have to make the turbulence between the layers more constructive.
Some responses that people within the ResearchOps Community have used include co-creation days where the participants worked with product owners in real time on their products. Others have implemented a schedule for a stakeholder to join them on an observational session once a fortnight. A common response of course, is to implement an operations function to work on getting the 8 Pillars in place across research that is happening in an organisation. There are no simple answers to managing the turbulence across the layers – it is an ongoing process of adjusting the threads, the warp and weft of the complex system, but it provides the mechanism to not only see the map, but the terrain of the system one works within to see the turbulence, and adjust the tension as needed.
NOTES
Acknowledgments – Special thanks as always to the ResearchOps Community whose passion for the craft and potential of user research drives the community to achievements that are normally unthinkable, and virtually unheard of in a volunteer run community. Together, we are influencing the profession of research and research operations.
Special thanks to collaborators who have pushed my thinking forwards about Pace Layers and the 8 Pillars, Emma Boulton, Tomomi Sasaki, Holly Cole, Mark McElhaw, Benson Low, Behzod Sirjani and Lou Rosenfeld.
Please note, the views expressed herein are the opinion of the author, and in no way represent that of my employer or educational institution.
i. Jones, & Thornley. (1963). Conference on design methods: papers presented at the Conference. Pergamon Press.
ii. Raizman, David. (2003). History of modern design: graphics and products since the industrial revolution. Laurence King Publishing, London, pp. 66.
iii. The concept of the attention economy stems from the work of Herbert Simon who stated in 1971 “A wealth of information creates a poverty of attention and a need to allocate that attention efficiently.”, Simon, H. A. (1971) “Designing Organizations for an Information-Rich World” in: Martin Greenberger, Computers, Communication, and the Public Interest, Baltimore. MD: The Johns Hopkins Press. pp. 40–41.
iv. McKercher, K. A. (2020). Beyond Sticky Notes. Sydney: Beyond Sticky Notes.
v. Sekai, Farai, (2020, September). Twitter post, retrieved from: https://twitter.com/SekaiFarai/status/1308813419173347331?s=20
vi. The findings, outputs and talks on each of these projects can be found on the community website, https://researchops.community).
vii. See here for example, Sirjani, B. (2020, July 22). Democratization is our Job. Retrieved from Yet Another Studio: https://yetanother.studio/blog/democratization-is-our-job, and Saha Mitra, S. (2020, April 22). Undemocratising User Research. UX Collective. Retrieved October 3, 2020, from UX Collective: https://uxdesign.cc/undemocratising-user-research-6897e6f4cf72
viii. Towsey, K. (2019, July). An interview with Kate Towsey, ResearchOps Manager at Atlassian. (N. Francis, Interviewer) Marvelapp. Retrieved from https://marvelapp.com/blog/kate-towsey-researchops-atlassian/
ix. ResearchOps Community. (2019, February). ResearchOps Community: About. Retrieved from ResearchOps Community: https://researchops.community/about
x. Rosenfeld Media. (2020, October 10). Advancing Research Program. Retrieved from Advancing Research 2020: https://rosenfeldmedia.com/advancing-research-2020/program/
xi. See for example Reichelt, L. (2019, November 11). Five dysfunctions of democratised’ research. Part 1 – Speed trumps validity. Medium. Retrieved October 3, 2020, from https://medium.com/@leisa/five-dysfunctions-of-democratised-research-part-1-speed-trumps-validity-1606fa63a3e8, de la Nuez, A. (2019, August 12). Democratization of UX Insights: What does this really mean? UX Matters. Retrieved from https://www.uxmatters.com/mt/archives/2019/08/democratization-of-ux-insights-what-does-this-really-mean
xii. (2018). Democratizing Research in Practice. In R. Iphofen, & M. Tolich (Eds.), The SAGE Handbook of Qualitative Research Ethics (pp. 103-113). ProQuest eBook Central: SAGE Publications.
xiii. (2018). Democratizing Research in Practice. In R. Iphofen, & M. Tolich (Eds.), The SAGE Handbook of Qualitative Research Ethics (pp. 103-113). ProQuest eBook Central: SAGE Publications, p.103.
xiv. McKercher, K. A. (2020, October 3). What is co-design? Retrieved from Beyond Sticky Notes: https://www.beyondstickynotes.com/what-is-codesign
xv. ResearchOps Community. (2019, February). ResearchOps Community: About. Retrieved from ResearchOps Community: https://researchops.community/about
xvi. Kaplan, K. (2020, August 16). ResearchOps 101. Retrieved October 3, 2020, from Nielsen Norman Group: https://www.nngroup.com/articles/research-ops-101/
xvii. Boulton, E. (2019, July 11). The Eight Pillars of User Research. ResearchOps Community (Publication). Retrieved October 3, 2020, from https://medium.com/researchops-community/the-eight-pillars-of-user-research-1bcd2820d75a
xviii. Aguilar, F.J. (1967). Scanning the business environment. New York: Macmillan.
xix. Brand, S. (1999). The Clock of the Long Now: Time and Responsibility. New York: Basic Books, p. 34.
xx. Brand, S., & Saffo, P. (02015, January 27). Pace Layers Thinking. (L. N. Foundation, Interviewer) Retrieved from https://longnow.org/seminars/02015/jan/27/pace-layers-thinking/
xxi. Geertz, C. (1998, October 22). Deep Hanging Out. Retrieved October 4, 2020, from The New York Review of Books: https://www.nybooks.com/articles/1998/10/22/deep-hanging-out/
REFERENCES CITED
Aguilar, F. J. (1967). Scanning the business environment. New York: Macmillan.
Boulton, E. (2019, July 11). The Eight Pillars of User Research. ResearchOps Community (Publication). Retrieved October 3, 2020, from https://medium.com/researchops-community/the-eight-pillars-of-user-research-1bcd2820d75a
Brand, S. (1999). The Clock of the Long Now: Time and Responsibility. New York: Basic Books.
Brand, S., & Saffo, P. (02015, January 27). Pace Layers Thinking. (L. N. Foundation, Interviewer) Retrieved from https://longnow.org/seminars/02015/jan/27/pace-layers-thinking/
de la Nuez, A. (2019, August 12). Democratization of UX Insights: What does this really mean? UX Matters. Retrieved from https://www.uxmatters.com/mt/archives/2019/08/democratization-of-ux-insights-what-does-this-really-mean
Geertz, C. (1998, October 22). Deep Hanging Out. Retrieved October 4, 2020, from The New York Review of Books: https://www.nybooks.com/articles/1998/10/22/deep-hanging-out/
Kaplan, K. (2020, August 16). ResearchOps 101. Retrieved October 3, 2020, from Nielsen Norman Group: https://www.nngroup.com/articles/research-ops-101/
Kara, H. (2018). Democratizing Research in Practice. In R. Iphofen, & M. Tolich (Eds.), The SAGE Handbook of Qualitative Research Ethics (pp. 103-113). ProQuest eBook Central: SAGE Publications.
McKercher, K. A. (2020). Beyond Sticky Notes. Sydney: Beyond Sticky Notes.
McKercher, K. A. (2020, October 3). What is co-design? Retrieved from Beyond Sticky Notes: https://www.beyondstickynotes.com/what-is-codesign
Reichelt, L. (2019, November 11). Five dysfunctions of democratised’ research. Part 1 – Speed trumps validity. Medium. Retrieved October 3, 2020, from https://medium.com/@leisa/five-dysfunctions-of-democratised-research-part-1-speed-trumps-validity-1606fa63a3e8
ResearchOps Community. (2019, February). ResearchOps Community: About. Retrieved from ResearchOps Community: https://researchops.community/about
Rosenfeld Media. (2020, October 10). Advancing Research Program. Retrieved from Advancing Research 2020: https://rosenfeldmedia.com/advancing-research-2020/program/
Saha Mitra, S. (2020, April 22). Undemocratising User Research. UX Collective. Retrieved October 3, 2020, from UX Collective: https://uxdesign.cc/undemocratising-user-research-6897e6f4cf72
Sirjani, B. (2020, July 22). Democratization is our Job. Retrieved from Yet Another Studio: https://yetanother.studio/blog/democratization-is-our-job
Towsey, K. (2019, July). An interview with Kate Towsey, ResearchOps Manager at Atlassian. (N. Francis, Interviewer) Marvelapp. Retrieved from https://marvelapp.com/blog/kate-towsey-researchops-atlassian/