Cast Study—In 2014, Kaiser Permanente began implementing a next-generation medical office model that reimagines the outpatient care experience, combining new architecture, workflow, and technology to create a more convenient experience for patients and a connected, efficient experience for staff and care teams. As the first next-gen facilities were being built, challenges emerged as teams across a variety of disciplines attempted to translate the model’s vision into reality. Teams were making design and operational decisions in parallel, without the ability to see how their decisions impacted the overall user experience.
To resolve these challenges, our innovation team at Kaiser Permanente used a hybrid make-and-observe method of prototyping and ethnography. Employing a co-creation mindset (Bødker and Grønbaek 1991), we engaged staff and patients to help us bring the future state of these next-gen clinics to life in a minimally viable way. We created a full-scale prototype experience at an existing clinic during working hours that included functional proxies of the future space; iterative techniques that simulated automation, i.e., Wizard of Oz-ing (Kelley 1984); rewritten workflows; and recast staff roles. With this model we were able to mock up the future user experience, test assumptions in the field, surface latent needs, and resolve unanticipated conflicts.
This case study provides an example of how ethnography, in combination with working prototypes of the future, can be essential tools for clarifying radical changes to new services. Using these methods, we were able to provide leaders with tangible evidence of why some elements of the initial vision needed to change in order to ensure a better user experience. Here we outline lessons learned and discuss limitations to our approach.
BACKGROUND
Kaiser Permanente is a leading health care provider in the United States, currently serving 12.2 million members in eight states and the District of Columbia. Kaiser Permanente is an unusual U.S. health care system, as we are made up of three distinct but interdependent entities — a not-for-profit health plan and hospital system, plus a network of regional medical groups — that work together to deliver care.
This case study describes a series of experiences that our innovation team at Kaiser Permanente worked on from 2015 to 2016 as the organization implemented its next-generation medical office model. This model influenced the architecture, created new workflows and roles, and leveraged new technologies to deliver an entirely different and personalized outpatient experience.
Typical in many “reimagining” projects, ethnography and design were used in the initial vision-setting phases of this multi-year initiative. Once the vision was set, functional teams (staffing, IT, architecture, facilities) worked in silos, and decisions were made by experts, without the aid of empirical, observation-based insights. However, as we implemented a new model for medical operations in a new building, these teams did not have enough experience with the new model of care described in the vision to make confident decisions, so they were open to an ethnographic approach to implementation.
Our approach involved creating full-scale, low-fidelity mock-ups of spaces, as well as rehearsing in an actual medical building during normal business hours, so that we could test real users completing real tasks and get immediate feedback. This combination of technological prototypes and ethnography in context is similar to the “Living Labs” that Pierson and Lievens describe as “to embed complex product ideas and prototypes in an environment that resembles as much as possible the context and everyday life setting.” (Pierson and Lievens 2005) We used this make-and-observe approach to understand how best to use new technology to support the envisioned user experiences. These rehearsals allowed us to observe and predict what user behaviors might be in the new spaces before expensive systems and structures were finalized. While this robust ethnographic prototyping cannot fully predict what the user experience will be in the actual space, our work led the organization toward the most user-centered and efficient way of delivering care within a vision that was already established. Our application of prototyping and ethnography successfully catalyzed the architects, builders, building owners, and tech teams who were poised and ready to act on our findings immediately.
This paper focuses not on how we used ethnography at the conceptualization phase, nor to investigate or redefine the fundamentals of health care delivery, but rather our application of ethnography to refine the implementation of a previously agreed upon vision. We present this case as an example of how a make-and-observe approach can resolve gaps in understanding and disagreements around next steps when bringing to fruition a dramatically new vision of a complex service.
SOLVING LATE-STAGE DESIGN CHALLENGES
When our innovation team engaged with the project, the first facility’s construction was nearly completed. Within the constraints of not being able to change the size and shape of the space, we identified three discrete experiences where combining ethnographic research with prototyping was essential to resolve questions about how to deliver on the patient and care team experience outlined in the initiative’s vision:
- Patient check-in: The patient declares that they have arrived for an appointment and payment is exchanged if needed.
- Patient waiting: Because appointments rarely start precisely on time, the patient is invited to enjoy the space, indoors or outside, until the nurse informs them that they are ready to bring them into the exam space.
- Care team coordination: New workflows and technologies needed to be identified and designed to enable care teams to work in the new collaborative open spaces and shared exam rooms.
A New Patient Check-in Experience: Surfacing Incorrect Assumptions
The vision of the next-gen medical office was one in which patients never had to wait in line. For our leadership team, this felt like an emblem of modern, user-centered service delivery. They imagined patients walking through the door, wandering toward an inviting setting, and then being greeted by a roaming receptionist with a tablet computer, who could check them in at the patient’s convenience. Based on this assumption, no space was allocated at the entrance for a queue.
To understand the future vision for check-in, we followed a familiar set of steps we now typically use in these projects:
- Observe the current user and worker experience.
- Bodystorm with minimal props and in a simulation space (Oulasvirta 2003).
- Transform a current, working medical facility into the format of the future buildings. We brought in new furniture, signage, and technology to the building and coached the staff to work in new ways.
Figure 1. Bodystorming in a simulation space where staff could fail without worry of being seen by customers. Photograph © Kaiser Permanente, used with permission.
In this way, we practiced new roles and explored multiple solutions before we worked up to pressure-test our best ideas with staff who had urgent jobs to do and patients who were most concerned with getting to their appointments on time.
Our testing of multiple ideas revealed a clear conclusion: When there is an urgent task at hand, such as getting to the doctor on time, patients felt more comfortable in queues. When patients entered the facility, they were primarily worried about missing their time with their provider. They wanted the efficient, fair, and orderly aspects of a queue. Instead of removing the queue, we explored ways to enhance it. In the end we defined new roles for service reps, new supporting furniture pieces, and new architecture layouts. All of these elements helped to clearly indicate to patients where check-in can happen within the new and different space.
As we delivered the findings about the new features and functions of the check-in experience, those in leadership who were planning openings for larger facilities grew concerned about scaling the designs with many more patients moving through the entrance. To address these concerns, we repeated our methodologies at three locations where patient check-in volumes were 100 patients per hour instead of per day. Through our experiments we proved that the model could scale successfully with some modifications to the facility space design, staff role assignments (how many), and the use of self-service technologies.
Untethering Patients from the Waiting Room: Identifying Hidden Barriers
An important aspect of the vision called for untethering patients from waiting rooms and engaging them with healthy activities in a common waiting area. To explore this concept, we temporarily took over a single-story clinic and offered different activities for patients to engage in. However, the untethering did not work. There was a hidden barrier preventing the patients from enjoying the full space.
Through our observations, we found that patients were uncomfortable leaving the clinic module for fear of missing their appointment. Despite our efforts to entice them away, patients stayed where they knew they would be called, expressing concern about missing the nurse. Additionally, the nurses expressed annoyance at not being able to quickly call patients from the doorway.
Figure 2. Patients sit close to the door, prioritizing waiting for the nurse’s call over enjoying the amenities of the waiting area. Photograph © Kaiser Permanente, used with permission.
One proposal from our team was to give patients a sense of their wait time, so that they could feel confident about where and how to spend that time. If we could not do that, the entire vision was at risk, because patients would feel the need to stay close to the door. However, the idea of providing patients with estimated wait times was met with immediate resistance from leadership across many departments. Medical executives worried about publicly showing this information if a clinic fell behind schedule. Providers worried about delays being associated with their names. And everyone, IT included, thought it would be impossible to accurately calculate wait times in such a dynamic, unpredictable environment.
Yet our prototyped models spoke for themselves. Leadership could see the vision was not being achieved in the current state, so they agreed to let us pursue the concept. After brainstorms and concept iterations, we built several communication techniques for wait-time notifications, and clinical experts were able to draft a way of approximating wait times based on existing provider data that we could test in a live clinic. This allowed us to watch how patients responded to the information about wait time and ask how they felt about delays. Through three days of testing we demonstrated that:
- Contrary to beliefs, it was possible to accurately calculate wait times, something that had never been done before at Kaiser Permanente.
- Patients appeared to be less agitated by any delays in appointments when the information from the prototype was available to them. When we inquired about it, patients said that they knew appointments were likely to be delayed, so the proactive communication was a pleasant surprise. We observed people stepping outside to make a phone call or visiting the bathroom without the worry of missing the nurse at the clinic door.
- Unexpectedly, providing wait times reduced the front-office staff’s workload. Said one service representative, “We didn’t get anybody asking us about how much longer until they will be called today!”
Figure 3. A patient checks her wait time on a prototype of a screen displaying an estimated time until they will be called to the clinic, for patients who have checked-in. Photograph © Kaiser Permanente, used with permission.
The ability to watch patients engage with a wait-time notification board and act differently because they knew they had more than a few minutes to wait convinced medical leadership to sign off on the concept. This was an exciting moment for us where we had built a prototype that seemed impossible and proved to leadership that patients valued it. From there, IT automated the algorithm we had created and continued prototyping to refine a full-fledged product that is now in use and standard equipment in all next-generation medical offices, enhancing the patient and staff experience.
Care Team Coordination: New Collaborative Tools
The third aspect addresses the way physicians, nurses, and other care team members work together. The new vision proposed a more efficient workflow and an open-plan back-office space, with shared exam rooms and shared support staff—a model completely different from the existing one—a less centralized model in which each physician has a dedicated office, exam rooms, and support staff. This new model was predicated on the use of technologies not yet built.
One tool we created through conversations and observations with staff was a working “outpatient dashboard” prototype. On each staff member’s tablet computer, the dashboard displayed each patient’s physical location once they were roomed in an exam room, what stage of the back-office process they were in, and how long they had been in that stage. During one field test in a modified back office, we observed the reoccurrence of an impromptu huddle where the staff referenced material on their tablet computers together and then quickly split up to tackle the work. Based on that observation and inquiry into that interaction, overnight the IT team created a larger touchscreen display of the outpatient dashboard. The next morning the care team immediately gravitated to it and conducted all their huddles at the new display, interacting and updating it as a team. This touchscreen display was not identified or defined prior to our field testing. It was created as a direct result of combining ethnographic methods with rapid prototyping and is now a standard component of all next-gen medical offices for Kaiser Permanente.
It was crucial to the development of this new technology that IT developers were on hand to witness the patient and care team needs. Otherwise this significant change in the technology roadmaps might not have happened. With first-hand observations of staff behaviors, they understood and were motivated to make rapid changes to their prototypes in order to improve the care team experience.
IMPACT AND VALUE IN A LARGE ORGANIZATION
The typical approach to implementing large-scale changes in our organization involves functionally specific teams (e.g., IT, facilities, operations) executing the vision over long periods of time. Implementing the next-gen medical office model required a different, more integrated approach, especially during the later stages, where these teams were racing to complete their deliverables on time and on budget for the opening of the new clinics. When teams did not have insights into what other teams were doing, it created blinders for each team. The impact and value of combining ethnography and prototyping not only helped remove these blinders but also enabled leaders to refocus their teams on the most critical issues. Simultaneously it accelerated adoption of new ways of working because staff could preview it in action and watch patients respond. Essentially, we moved the conversations from conjecture to observations.
Influencing Leadership Decisions
In “Design as Sociopolitical Navigation,” Clark describes the importance of action-oriented techniques that engage decision makers in order to influence leadership’s perspective on the needs, scope, and funding of a design project (Clark 2007). We carefully engaged the extended executive teams across functions at strategic times throughout the project so that they could observe the developing prototypes in action and offer input and ideas about how to evolve the idea. This was essential in informing their visions and changing their minds late in the implementation process. A senior vice president in our IT organization described the value this way: “You can make bad assumptions about what people need until you see it in action and find out you’re very wrong.”
Leaders could see what worked, what didn’t work, and what was missing. They asked questions and proposed new solutions based on the evidence we presented. We tested those new ideas so that they could see the results for themselves. And together we further refined the concepts. These observations as evidence helped these leaders shift their thinking. In this way, the value of ethnography within a prototype is not just for the researchers, designers, and engineers working through the iterative cycles of testing concepts. Perhaps more significantly, the crafted performance of real patients and in situ staff conducting real work with mock-ups of space, tools, and roles provide the ability for leaders and stakeholder teams to witness first-hand how their systems would come together in the future (Halse and Clark 2008). This helped us to deliver on our goal of shifting previous planning conversations from “subject matter expertise” to observation-based discussions of how patients and staff were responding to proposed solutions in use.
Acceptance and Adoption of Change
People respond better to changes in workflow when they are engaged in it. Our process inherently did this by involving all impacted roles in the design of their work (responsibilities, technology, spaces, workflows). By engaging providers and clinical staff in the process, they took ownership of the solution and championed the new technologies, facilities, and workflows to their peers across the region. The old way of working was no longer a viable option for those involved in our projects. An administrative leader described her amazement at the change with this statement: “The transition from ‘We can’t do it.’ to ‘Now I don’t want to give it up.’ was a great surprise.”
In addition to changing mindsets about the model, our approach also paved the way for smoother medical office building openings than ever before. After the opening of the first three new facilities, an executive director from the Facilities Planning organization remarked about the ease and reliability of the new systems: “The challenge in the past [with new facility openings was] that technology didn’t quite work as planned… Because of the field testing, it works exactly like we intended.”
Through our experiences experimenting in live clinics we understood the pain points, bottlenecks, and biggest obstacles to change. With that knowledge we were able to conduct staff trainings based on a strong understanding of the new buildings, technologies, workflows, and roles. One care team commented on their first day of operations that it felt like they had been practicing there forever because the training simulations had been so realistic.
CONCLUSION
As innovation transforms healthcare and many other industries, more and more technologies and services are being tried. However, in creating something to make a difference we often end up creating something that actually makes people’s jobs harder because we are not able to see how the product actually fits within the context of the environment. And in late-stage development, every issue and question, from high level needs to detailed needs, become equally critical to the completion of the project. We can get so focused on meeting timelines and budgets that we don’t have the opportunity to observe and assess the entire experience and discern what is important.
What this project has shown us as an organization—and as design and ethnography practitioners—is that with a growing emphasis on merging digital and physical experiences, there is also a growing need to use ethnography and prototyping to ensure user needs are front and center at every stage of facility design and implementation. These methods to study user experience are particularly important during implementation, when multiple systems are being resolved and information is needed about how they will influence each other.
We propose that teams leading capital and construction projects consider using our method of usage-focused ethnography and full-scale prototyping of future experiences in order to make critical adjustments to space, workflow, and technology as the project develops and before it is too costly to make changes.
Brady Sih is a Kaiser Permanente IT principal consultant who focuses on technology experience, design, and delivery. brady.sih@kp.org
Hillary Carey is an independent design researcher and innovation consultant focused on service design and systems change. Hillary@WinnowResearch.com
For a decade, Michael C. Lin helped Kaiser Permanente develop its health care design practice, contributing original knowledge to the field of service design. Mike is currently vice president of innovation at Centura Health. mikelin@centura.org
NOTES
We would like to acknowledge the Kaiser Permanente leaders who made this work possible. Jodie Lesh, senior vice president, National Delivery System Strategy, Planning and Design, developed the vision for the next-gen medical office, and also supported our work throughout, including our use of ethnography and prototyping in late-stage implementation. Wendy Lee, senior vice president, and Ann Sherry, executive director, Digital Corporate Services & Solutions championed the innovation team’s work—and gave us the bandwidth and resources to do it. We would also like to thank Shannon Coughlin, communications manager, for her editorial guidance on this case study. Please note: The authors of this case study represent their own perspectives and are not representing the official position of Kaiser Permanente.
REFERENCES CITED
Bødker, Susanne and Kaj Grønbæk
1991 “Design in Action: From Prototyping by Demonstration to Cooperative Prototyping.” In Design at Work: Cooperative Design of Computer Systems, edited by J. Greenbaum and M. Kyng, 197-218. Hillsdale: Lawrence Erlbaum Associates.
Clark, Brendon
2007 Design as Sociopolitical Navigation: a performative Framework for Action-Oriented Design. Mads Clausen Institute. Sønderborg, University of Southern Denmark. PhD dissertation.
Halse, Joachim and Brendon Clark
2008 “Design Rituals and Performative Ethnography.” Ethnographic Praxis in Industry Conference Proceedings 2008:128-145.
Kelley, J. F.
1984 “An Iterative Design Methodology for User-friendly Natural-language Office Information Applications.” ACM Transactions on Office Information Systems, 2 (1): 26-41.
Oulasvirta, Antti, Esko Kurvinen, and Tomi Kanjaunen
2003 “Understanding Contexts by Being There: Case Studies in Bodystorming.” Personal Ubiquitous Computing, 7: 125–134.
Pierson, Jo, and Bram Lievens
2005 “Configuring Living Labs for a “Thick” Understanding of Innovation.” Ethnographic Praxis in Industry Conference Proceedings 2005:114-127.
Suchman, Lucy, Jeanette Blomberg, Julian E. Orr, and Randall Trigg
1999 “Reconstructing Technologies as Social Practice.” The American Behavioral Scientist; 43 (3): 392-408.