The paper examines the living lab as an approach within communication studies for examining the naturalistic involvement of users in ICT design, based on ethnographic principles. First a more precise definition of the living lab is presented, indicating the epistemological background. Next the different phases of our living lab configuration are elaborated, illustrated by a research project on a handheld electronic reading device (e-paper). Finally we discuss the value of this approach for companies involved in ICT research and design. In the conclusion also the advantages on product development level and social level are indicated.
Introduction
Understanding and involving users in (broadband) innovation has become a central issue in private and public research and policy on information and communications technologies (ICT) in Europe and elsewhere. Furthering design and development of future technologies and services that are well adapted to potential users will lower the threshold for social acceptance and increase the potential for e-inclusion. In addition it can help to understand why certain innovations can be ‘disruptive’ for existing technology paths. For companies comprehending the (potential) user more thoroughly can help in minimising risks of technology introduction and possibly shorten the time-to-market.
Our basic idea is that for thoroughly understanding users:
‘technologies should be studied in situ and in use, as part of socio-technical arrangements of humans and machines joined together in action and embedded in social contexts.’ (Ratto, 2000)
This highlights the rejection of technological determinism and the naturalising influences of functionalist ideas of technologies, while seeing technology as practice. Therefore they can only be assessed in their relations to the sites of their production and use (Suchman, 1987).
Research Focus
These considerations fit in the living lab method, which entails studying underlying social processes that drive the use and development of communication artefacts. The main question in the paper is how the living lab setting, based on ethnographic principles, can be seen as an optimal way of optimally enabling users to contribute in the early stages of innovation processes.
Our approach relates to the field of Social Shaping of Technology (SST). SST research:
‘(I)nvestigates the ways in which social, institutional, economic and cultural factors have shaped: 1. the direction as well as the rate of innovation; 2. the form of technology: the content of technological artefacts and practices; 3. the outcomes of technological change for different groups in society.’ (Williams & Edge, 1996: 868)
The overall living lab approach and the different phases as presented in this paper were developed and finalised by experiences within different projects. In order to answer the research question in the paper first a more precise definition of the living lab is elaborated, indicating the epistemological background. Next the different phases of our living lab configuration are elaborated, illustrated by a case study on a handheld electronic reading device (e-Paper). Finally we discuss the value this ethnographic approach has for companies involved in ICT research and design. In the conclusion also the advantages on product development level and social level are indicated.
Defining living lab
The ‘living lab’ is a specific type of test and experimentation platform (TEP). The latter indicates all facilities and environments for (joint) innovation including testing, prototyping and confronting technology with usage situations (Ballon, Pierson, & Delaere, 2005). Living labs refer to facilities for designing, developing, testing and evaluating communication technologies and services in early stages of the innovation process by involving (early) users, in line with the SST research. These can take the form of some sort of lab (like a house), in order to test and experience (uses of) technologies. This corresponds to the smart house idea, which has been around since the eighties. However our focus is on living labs that have a broader geographical reach, covering a range of (mobile) people, several households, a specific area (e.g. campus), a community, a neighbourhood or even a town. These kinds of facilities can be set up and managed by one company. They can also be configured as open and innovation-oriented platforms that involve various technology and service providers as well as users in different stages of technology design, development and testing.
The living lab aims at using the natural user environment as much as possible, hence enabling ethnographic research on the participants in the living lab. One the first scholars to use this notion of living lab was William Mitchell from MIT (Boston), Medialab and School of Architecture and city planning.
‘Living Labs is a research methodology for sensing, prototyping, validating and refining complex solutions in multiple and evolving real life contexts.’1
In a rare systematic overview of joint innovation facilities, Niitamo describes the living labs as: ‘A broad regional development program where testing, developing and validating new products and services indicate future systemic innovation needs.’2
This kind of living lab is based on the notion of user as co-producers of ICT, which refers to the idea that the user is never an ‘end user’ but re-interprets technological artefacts within his social context once they are adopted (Bergman & Frissen, 1997). This point of view needs to be situated within the theoretical notions of ‘social construction of technology’ (Bijker, Hughes, & Pinch, 1987). The living lab is thus characterised by confronting (potential) users with (ideas, prototypes or demonstrators of) technology early on in the innovation process. Within the (new) product development and innovation management literature this is also typified as the ‘fuzzy front-end’, as dubbed by Smith and Reinertsen (1991). This refers to all activities preceding the start of formal product development process (Khurana & Rosenthal, 1997).
Epistemological background of the living lab approach
The account of a living lab in this paper is based on our disciplinary background in communication studies, as an interdisciplinary study of the relation between media, society and people. We view media and technologies as a form of social mediation (Garnham, 2000). The communication studies historically have been focussing on the content side of media. In our research we start from the vision of media as technological tools, being the basis for human culture. In alliance with authors like Silverstone, Haddon and Morley we look at ICT from a domestication perspective, which has its roots in cultural studies (Berker, Hartmann, Punie, & Ward, In press; Morley, 2003; Silverstone & Haddon, 1996; Silverstone & Hirsch, 1992). One of the basic goals is:
‘(…) to frame the analyses (…) within a broader framework of the role of various media in articulating the private and public spheres, which (hopefully) allows us to articulate these micro-analyses to broader perspectives on macro-social issues of politics, power and culture.’ (Morley, 1992: 40)
In order to clarify the epistemological background of our living lab approach within communication and technology studies, we refer to the dichotomy in sociological approaches of the technological innovation process: a factor approach – which explain social phenomena through an identification of factors – versus an actor approach – where the perception and experience of the actors is the point of departure. The division between factor and actor approach corresponds with similar epistemological classifications, like ‘objectivist’ versus ‘subjectivist’ (Willmott, 1993) or ‘positivistic’ versus ‘interpretative’ (Servaes & Frissen, 1997).3
Ethnography in general and the living lab approach in particular fit in mainly with the actor or interpretative approach. Nevertheless to operationalize living lab research we choose for a ‘multimethodological’ approach (Mingers, 1997). The prevalent qualitative part enables us to identify the meanings and experiences of the actors, while the quantitative research plays a supportive role. It is the combination of both, that will enable a more elaborated triangulation of findings, based on methodological or systematic pluralism (Ford, 1990; Roth, 1987). Before positivism reached its high point in the 1930s and 1940s this pluralism was generally accepted in sociology and social psychology (Hammersley & Atkinson, 1995: 3)
Living lab configuration
Our living lab configuration is set up as a meta-methodology for analysing the confrontation between ICT and users from an ethnographic perspective. The overall configuration and the underlying research methods – that are further elaborated in this paper – are partly founded on an analysis of other international living lab cases.4 Yet for the most part they are built upon hands-on experiences in three projects we were or are involved in: ‘Vlaanderen Interactief’5, ‘e-Paper’6 and ‘i-City’7.
The living lab research cycle presented below is in fact a methodological synopsis of our experiences in setting up and conducting living labs in these projects. The innovative character of our living lab approach does not so much stems from the individual – often familiar – (ethnographic) methods applied, but from the specific configuration of (qualitative and quantitative) methods and the way these different methodological components are geared to one another. This means that our living lab configuration is still work-in-progress. At present we identify four different phases: contextualisation, concretisation, implementation and feedback. These phases are illustrated by elements from the e-Paper project.
The e-Paper project is a case of how users can be actively involved in technology development and large-scale testing of a specific handheld electronic reading device, based on the e-ink technology applied for an online newspaper. In order to develop and implement the device, standing up to the needs and expectations of the user, 200 test devices will be distributed for a period of three month among a test sample. The socio-technological research framework is based upon the living lab configuration (from contextualisation until feedback). E-paper is public-private sponsored research project, initiated by the Interdisciplinary Institute for BroadBand Technology (IBBT).
Figure 1. Overview of living lab research cycle
Contextualisation
The contextualisation phase is an explorative phase. This phase resembles the exploration phase in grounded theory (Glaser & Strauss, 1967). A theoretical basis has to be expounded in order to (1) define the research framework and (2) to identify eligible respondents. In this phase different research methods are being applied in order to provide the required background and insights. Starting from the technological challenges as well as – if applicable – the defined use cases specific desk research is performed on two levels: technological and social. This enables a first confrontation on an abstract level of what is technologically possible with what is socially feasible and vice versa.
Contextualisation methods – In order to situate the technology or service under investigation, first a technological scan is performed. The purpose of this analysis is not only to give a(n) (quick) overview of current and future technologies but also to map the specific functionalities and characteristics related to these technologies. Second a (state-of-the-art) study is deployed in order to determine the (socio-economic) contextualisation of the research focus (research framework as well as research topic). Complementary, other methods can be applied like environmental scanning (Gordon & Glenn, 2003). This serves as input for phenomenal variation in purposeful sampling (see below).
The outcome of these different steps – which are not always strictly separated from each other – does not only help in setting out but also in the validation of the qualitative research framework. In order to identify potential users or user groups, this research phase allows us to stipulate the selection criteria and therefore the selection profile.
Selection – Crucial in our living lab configuration is the formalization of the selection of respondents. For our selection we apply sampling procedures from qualitative research. The selection is based on non-probability sampling, which means that not everyone has an equal chance of being selected. We choose our respondents in the living lab purposefully. Sandelowski discerns three kinds of purposeful sampling: maximum variation, phenomenal variation and theoretical variation (Coyne, 1997). The objective of maximum variation is analytical diversity. The kind(s) of variation need(s) to be made explicit as well as when to maximize each kind. This refers for example to socio-demographic variation in gender, age or education. Respondents that differ on these variables are useful in the living lab research because of their analytic value, not because of generalization. People of a particular age are selected because, by the virtue of their age, they can provide certain kinds of information. Phenomenal variation aims for variation of the target phenomenon under study. The decision to seek this variation is ‘often made a priori in order to have representative coverage of variables likely to be important in understanding how diverse factors configure as a whole’ (Sandelowski, 1995: 182). This is also referred to as ‘selective or criterion sampling’, where sampling decisions are made beforehand on ‘reasonable’ grounds. A third type is the theoretical variation. This is a variation on a theoretical construct that is being developed on analytic grounds in the course of the study (Golden, 2000). In this way the selection of test users is based on the evolving theoretical relevance of concepts, which ties in with the notion of ‘theoretical sampling’ typical for grounded theory studies (Strauss & Corbin, 1990).
In the living lab research cycle there are two instances of sampling. In the contextualisation phase we select the main living lab participants. In our living lab configuration we apply maximum variation and phenomenal variation for our purposeful sampling procedure. More in particular the selection criteria are based on data from the preceding technological scan and the socio-economic contextualisation study (as described above). In order to check the identified criteria and to determine the definitive selection profile or ‘screener’,8 explorative focus groups – based on these criteria – are organised. Besides finalising the selection profile, this generates a first glance at the prospective sample within the living lab configuration. Later on, in the implementation phase, we select the people that will be monitored and examined more closely by way of (ethnographic) observation and qualitative interviewing. This selection is mainly based on phenomenal variation and theoretical variation. The quantitative logging analysis serves as guideline for identifying the most information-rich cases.
Concretisation
Once the recruiting of the test-users – according to defined selection profile – has been completed, the next step is to get a thorough description of the current characteristics and everyday life behaviour and the perceptions of the selected test users regarding the research focus. Therefore an initial measurement of the sample is made before the technology or service is introduced or before the test panel becomes active in the living lab.
In this initial measurement we look at specific characteristics on the level of the user (socio-demographic and economic) as well their relation towards (the introduced) technology or service. The methodology to perform this snapshot depends on the size of the test panel. Usually a(n) (online) survey is used. Through a (semi) structured questionnaire this method enables us to gather an extensive data-stream in a quick way from a large audience. Depending on the scale of the sample, this method can be supported by qualitative methods like in-depth interviews. This measure point enables us to perform an evaluation at the end of the living lab project.
In the initial measurement we distinguish two types of components: a fixed and a variable component. The fixed component looks at data that can be applied in all living lab settings. In this component we first look at socio-demographic and economic characteristics of the user regarding his/her family profile, personal profile and professional profile. Second we look at the general media-profile (access, usage,…). The variable component exists of a number of questions that are case-specific. Depending on the research topic this could also mean a more in-depth questioning of the fixed component.
The initial measurement provides the necessary information to start with the living lab. This start up can occur in two ways: simultaneous or phased. The way of phasing depends not only on the goals and objectives of the living labs but also on the level of applications and services that has to be tested and/or is available. Depending on how the latter is offered, different categories within the sample can be introduced in the test and experimentation environment.
Implementation
The implementation phase is actually the operationally running test phase. From a user-oriented and ethnographic research viewpoint we distinguish two major research methods: direct and indirect analysis.
For the direct analysis remote data collection techniques and strategies are used (Blomberg et al., 2003). In our projects technological monitoring is implemented. On the device level (e.g. pda, mobile phone or digital television) as well as on the platform/network level (if applicable) software logging tools are employed. This tool registers, on an individual level, all the relevant user actions on the technology/device level. In order to organise this logging efficiently, we determine in advance the logging framework, including for example how detailed the logging needs to be done. The data processing based on this logging provides an insight in the usage and is triangulated with other research findings. Besides logging analysis, in this research phase an appeal is made on the basic ethnographic methods like (participatory) observation.
From here on indirect analyses are being applied to investigate the meaning and motivation for behaviour. For the indirect analysis, different research methodologies can be used like (thematically organised) focus group interviews, in-depth interviews and self-reporting techniques like diaries. The themes of the focus groups depends on the logging analysis, e.g. around some applications which are heavily used by a certain type of user.
All these direct and indirect analyses within the implementation phase are complementary and contribute in gathering information on adoption, usage, meaning, motivation and possible influence.
Feedback
The feedback phase at the end of the living lab project consist of two research steps. First an ex post measurement is conducted. Based on the questionnaire of the initial measurement a closing survey is administered on the whole test sample. For this ex post measurement the survey method is used. The goal is to check if there is any evolution in the perception and attitude towards the introduced technology or service, to assess changes over time in everyday life in relation to technology use and to detect transitions of usage over time. The results of this measurement are compared with the insights from the qualitative research in the previous phase.
The second research step is to infer technological recommendations from the analysed data, gathered during the implementation phase. This refers in the first place to the findings based on the direct and indirect analyses in the former phase. This generates among others characteristic user profiles and user patterns, elaborated in the research report. The outcome of the feedback phase can be used as the starting point for a new research cycle within a living lab project. In this way the iterative feature of our research cycle can be made operational in the living lab configuration.
Value of the living lab
Based on the process overview of our living lab configuration, we now specify the particular usefulness. For this we make a difference between living labs that offer a platform for the co-operation between different (sometimes competing) companies and living labs by one company only.
Living lab by an individual company
For an individual company the living lab offers a number of advantages in comparison to other methodologies. We identify four major benefits.
First our living lab configuration offers assistance for designing and conducting (large-scale) naturalistic multimethodological research around a particular technology, device or service, based on ethnographic principles. It gives the opportunity to embed complex product ideas and prototypes in a environment that resembles as much as possible the context and everyday life setting of the people that (could) use the object of research in real-life. This kind of ‘new product ethnography’ is helpful in structuring the ‘fuzzy front-end’ investigation of new product development (Cagan & Vogel, 2002: 183-188). The living lab is thereby especially useful in a multi-platform environment. Through digitalisation services and applications are no longer dependant on one platform or device. In the living lab setting different devices can be tested simultaneously among (potential) users in the front-end stage.
The value of this living lab approach can also be found in the particular composition of the different methodological elements. The composition includes both qualitative and quantitative methods, enabling a more diversified triangulation of findings. The objective is to enrich the description and the evolution of behaviour, motives, attitudes and knowledge of the persons involved in the living lab, during the research period. Yet these findings can to some extent be transposed to people with similar profiles, inside and outside the living lab.9
Third the living lab is uniquely positioned to involve sociality in the real-life testing of network technologies and services. Through this kind of test and experimentation environment we configure a setting where people socially interact with each other in an everyday context. This enables a thorough investigation of the mutual shaping process between sociality and the use of ICT. It is this domestication process which can generate unpredictable uses.
Finally the living lab is suitable for investigating the use of ICT in three spheres: work, home and elsewhere. Because this method often covers a larger geographical space, it is especially helpful for ethnographic research of people being mobile and people in public spaces. This refers for example to the use of mobile technologies in the bus, on the road, at the shopping mall, in a queue, on the bicycle, in public spaces, while doing sports etc. These situations have received less attention in the literature than work- or home-related ICT use.
Living lab by a cluster of companies
The living lab configuration as presented here, becomes even more valuable when it integrates different companies and stakeholders. This is a typical case of an open innovation platform in a pre-competitive setting, where different organisations join forces to test out early ideas and to see for example how sociality can form new and unexpected technologies and services. In the current techno-economic landscape these kinds of test and experimentation platforms can overcome a number of systemic failures in the innovation process (Edquist, 2001).
Test and experimentation platforms like these living labs focus on stimulating interactions, creating institutional support for innovation and accelerating the emergence of new technological systems in order to reduce innovation failures. This implies that they should be open to various business stakeholders, actively building trust, allowing business model experimentation and promoting the formation of clusters. The openness can also engender more creativity, because ideas can flow more freely due to less concern for competition misuse in this protected environment.
Conclusion
Understanding and involving users in the design and development of ICT is essential for industry. Too many technologies and services are still being developed and designed from a technological point of view. The living lab method presented in this paper offers an alternative approach. However living lab research still lacks clear procedural guidelines, which makes it hard to use for industry practices on consumer and user research. In this paper we presented a framework for conducting a living lab as a test and experimentation platform enabling (early) user involvement in technology and service innovation. More in particular our focus was on ways to establish the living lab as a ‘field’ where ICT can be tested in real life test environment, based on ethnographic principles.
The paper provided an initial impetus for formalising the living lab approach. Four general phases in the living lab configuration were identified: contextualisation, concretisation, implementation and feedback. The strength of this configuration is the integration of different (qualitative and quantitative) methods. This multi-methodological structure reinforces the triangulation, which is valuable for an ethnographic approach. In addition our communication studies approach merges views from different disciplines dealing with the involvement of users in technology design (CSCW, economics, sociology and anthropology,…). While this living configuration has been empirically tested, more research is required to additionally validate the method.
The advantages of this living lab approach are situated on the technological (system) level and on the social level. On the technological level it helps in developing more social context-specific insights on ICT use in relation to technological design. In the field of design this fits in with the shift from ‘user-centred’ design to ‘people-centred’ design. Instead of on task-centric users, the emphasis is on people in their social context as the fundamental source of innovation (Wakeford, 2004). This entails the reconfiguration of the relationship between ethnography and design from a ‘product-oriented’ model to a ‘socio-technical model’ of research. In the latter ethnography is employed as a research tool for exploring the social aspects of innovative technologies:
‘(…) the aim is to explore the sociality of novel design spaces opened up through the deployment of radical technology configurations in real world situations of use.’ (Crabtree & Rodden, 2002).
The outcome from a living lab is not in the first place aimed at user requirement specifications. This kind of socio-technical approach is mainly meant for developing and elaborating sensitizing concepts that draw attention to central characteristics of sociality implicated in ICT usage. The latter may then be further explored through continued design. In this way – underscoring the notion of mutual shaping – ICT becomes a vehicle for social research, the results of which in turn drive design.
On the social level these experiments inform us about possible conditions for stimulating the societal and economic embedding of technology. Embedding technology in real life situations also generates images of potential societal impacts of innovation (Frissen & van Lieshout, 2004). Ideally the living lab is created as an ‘experimental field’ within a socio-technological scope with specific goals and with a structure, but simultaneously dealing with the uncontrollable dynamics of everyday life.
To conclude, the relevance to people-centred innovation of test environments like living labs, arguably lies in the extent of user participation that is made possible, the establishment of an experimental setting that resembles real-life situations as closely as possible and the enabling and support of non-linear, mutual shaping innovation processes. In this way the construct of different methods that form our living lab configuration is an attempt to bridge the divide between sociological studies and technology design.
Jo Pierson is senior researcher for SMIT-IBBT, involved in innovation strategic research on design and use of (new) media technologies. Besides lecturing at the Communication Department of the Free University of Brussels (VUB), he also holds a research position on social science research of ICT.
Bram Lievens is a researcher for SMIT-IBBT at the Free University of Brussels (VUB), working on different policy and user oriented projects regarding new and emerging technologies, services and applications.
Ethnographic Praxis in Industry Conference Proceedings 2005,pp. 114–127. © American Anthropological Association, some rights reserved.
NOTES
1 http://www.sric-bi.com/LoD/meetings/2005-06-08/VPNiitamo.ppt; accessed 25 September 2005
2 http://www.sric-bi.com/LoD/meetings/2005-06-08/VPNiitamo.ppt; accessed 25 September 2005
3 In this sense Crabtree (2001: 159-172) discusses the shift in systems design from ‘human factors’ to ‘human actors’.
4 See Ballon, Pierson & Delaere (2005) for a description of some international living lab cases and other test and experimentation platforms (TEP).
5 The living lab research in the project ‘Vlaanderen Interactive’ (Flanders Interactive) looked at the design and use of interactive digital television in a domestic setting (http://www.vlaandereninteractief.be). The initiative was subsidised by the Flemish government. The project ran from October 2003 until October 2004. It included the following partners: Flemish Government, Telenet, Interkabel, VRT, VMMa, VT4. The user research was subcontracted by Telenet and financed by IWT (Institute for the Promotion of Innovation by Science and Technology in Flanders). The partners in the user research were the University of Ghent (UGent), Catholic University of Leuven (KULeuven) and Free University of Brussels (VUB). Due to NDA disclosures not all the details or results can be given at this time.
6 The e-Paper project (https://epaper.ibbt.be) is carried out by a consortium of the following companies: Philips, De Tijd, Belgacom, Hypervision and I-Merge in co-operation with the following IBBT research groups: SMIT from the Free University of Brussels, CUO and Distrinet from the Catholic University of Leuven, MICT and INTEC from the University of Ghent and IMEC. The project runs from January 2005 until June 2006. In this project SMIT is responsible for setting up the trial as a living lab configuration. Due to NDA disclosures not all the details or results can be given at this time.
7 The i-City project (http://www.i-city.be) is a wireless experimentation environment in a medium-sized Belgian city (Hasselt). i-City is a participative consortium, which officially started on July 13, 2004, consisting of Microsoft bvba, Telenet Operations NV, Siemens NV and Concentra Media NV with the support of the regional and local governments. At present, SMIT (Free University of Brussels) is advising i-City on selecting and analysing the living lab test users.
8 The ‘screener’ is the instrument designed to identify the characteristics that are appropriate for the project, based on sampling by quota (Blomberg, Burrell, & Guest, 2003: 968).
9 This is indicated as idiographic, holographic, naturalistic or analytic generalization in contrast to statistical generalization (Sandelowski, 1995) (Peters, 1995).
REFERENCES CITED
Ballon, P., Pierson, J. & Delaere, S.
2005 Test and experimentation platforms for broadband innovation: examining european practice. Paper presented at ‘16th European Regional Conference by the International Telecommunications Society (ITS)’, Porto, Portugal.
Bergman, S. & Frissen, V.
1997 ‘De eindgebruiker bestaat niet – de dynamiek van het gebruik van ICT in het leven van alledag’, in B. Tiggelaar (Ed.) I&I jaarboek – Onderzoek in nieuwe media. Amsterdam: Otto Cramwinckel, 68-74.
Berker, T., Hartmann, M., Punie, Y. & Ward, K.
In press Taming the technology? Domestication – A key concept in media and technology studies revisited. Berkshire: Open University Press.
Bijker, W. E., Hughes, T. P. & Pinch, T. J.
1987 The social construction of technological systems: new directions in the sociology and history of technology. Cambridge: The MIT Press.
Blomberg, J., Burrell, M. & Guest, G.
2003 ‘An ethnographic approach to design’, in J. A. Jacko & A. Sears (Eds.) The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications. Mahwah, N.J.: Lawrence Erlbaum, 964-986.
Cagan, J. & Vogel, C. M.
2002 Creating breakthrough products: innovation from product planning to program approval. Upper Saddle River, N.J.: Financial Times/Prentice Hall.
Coyne, I.
1997 ‘Sampling in qualitative research. Purposeful and theoretical sampling: merging or clear boundaries?’ in Journal of Advanced Nursing, 26, 623-630.
Crabtree, A.
2001 Wild sociology: ethnography and design (PhD Thesis). Lancaster University, Lancaster.
Crabtree, A. & Rodden, T.
2002 Ethnography and design? Paper presented at ‘International Workshop on ‘Interpretive’ Approaches to Information Systems and Computing Research’, London, 70-74.
Edquist, C.
2001 ‘Innovation Policy – A Systemic Approach’, in B.-Å. Lundvall & D. Archibugi (Eds.) Major Socio-Economic Trends and European Innovation Policy. Oxford: Oxford University Press.
Ford, J. E.
1990 ‘Breaking the paradigm mentality’, in Monist, 73(3): 335-350.
Frissen, V. & van Lieshout, M.
2004 To user-centred innovation processes: the role of living labs. Delft: TNO-ICT.
Garnham, N.
2000 Emancipation, the media, and modernity: arguments about the media and social theory. New York: Oxford University Press, 206.
Glaser, B. & Strauss, A.
1967 The discovery of grounded theory. Chicago: Aldine.
Golden, A.
2000 ‘What we talk about when we talk about work and family: a discourse analysis of parental accounts’, in The Electronic Journal of Communication, 10 (3 & 4).
Gordon, T. J. & Glenn, J. C.
2003 ‘Environmental scanning’, in J. C. Glenn & T. J. Gordon (Eds.) AC/UNU Millennium project – Futures Research Methodology Version 2.0. Ohio: American Council for The United Nations University.
Hammersley, M. & Atkinson, P.
1995 Ethnography: principles in practice (2nd ed.). London: Routledge, 323. Khurana, A. & Rosenthal, S. R.
1997 ‘Integrating the fuzzy front end of new product development’, in Sloan Management Review (winter), 103-120.
Mingers, J.
1997 ‘Multi-paradigm multimethodology’, in J. Mingers & A. Gill (Eds.) Multimethodology: the theory and practice of integrating management science methodologies. Chichester: John Wiley & Sons, 1-20.
Morley, D.
1992 Television, audiences and cultural studies. London: Routledge, viii, 325.
2003 ‘What’s ‘home’ got to do with it: contradictory dynamics in the domestication of technology and the dislocation of domesticity’, in European Journal of Cultural Studies, 6 (4), 435-458.
Peters, V.
1995 ‘Case-study’, in H. Hüttner & K. Renckstorf & F. Wester (Eds.) Onderzoekstypen in de communicatiewetenschap. Houten: Bohn Stafleu Van Loghum, 559-607.
Ratto, M.
2000 Producing users, using producers. Paper presented at ‘Participatory Design Conference’, 28 November 2000, New York.
Roth, P. A.
1987 Meaning and method in the social sciences: a case for methodological pluralism. Ithaca: Cornell University Press.
Sandelowski, M.
1995 ‘Focus on qualitative methods: sample size in qualitative research revisited’, in Research in nursing and health, 18:179-183.
Servaes, J. & Frissen, V. (Eds.)
1997 ’De interpretatieve benadering in de communicatiewetenschap theorie, methodologie en case-studies‘. Leuven: Acco, 303.
Silverstone, R. & Haddon, L.
1996 ‘Design and domestication of information and communication technologies: technical change and everyday life’, in R. Mansell & R. Silverstone (Eds.) Communication by design: the politics of information and communication technologies. Oxford: Oxford University Press, 44-74.
Silverstone, R. & Hirsch, E. (Eds.)
1992 ’Consuming technologies: media and information in domestic spaces’. London: Routledge.
Smith, P. G. & Reinertsen, D. G.
1991 Developing products in half the time. New York: Van Nostrand Reinhold.
Strauss, A. & Corbin, J.
1990 Basics of Qualitative Research. Grounded Theory Procedures and Techniques. Newbury Park: Sage.
Suchman, L. A.
1987 Plans and situated actions: the problem of human-machine communication. Cambridge: Cambridge University Press.
Wakeford, N.
2004 Innovation through people-centred design: lessons from the USA. Guildford: University of Surrey.
Williams, R. & Edge, D.
1996 ‘The social shaping of technology’, in Research Policy (25), 865-899.
Willmott, H.
1993 ‘Breaking the paradigm mentality’, in Organization Studies, 14 (5), 681-719.
Web resources
http://www.sric-bi.com/LoD/meetings/2005-06-08/VPNiitamo.ppt, accessed 25 September 2005.
https://epaper.ibbt.be, accessed 5 November 2005.