This case study examines agency within monitoring and evaluation (M&E) schemes for international development projects. Specifically, it evaluates a sensor to measure fuel consumption of clean cookstoves as a method of maintaining accountability and soliciting data on stove performance. Despite trends of increasingly automated M&E, the decisions of choosing, analyzing, and translating outcomes and indicators are influenced by stakeholder input. Through various rapid ethnographic methods including surveys and interviews with government agencies, non-profits, and clean stove users, in addition to participant observation and focal follow of stove users in Central America and Uganda, the interactions and inputs of various agents throughout the project lifetime are assessed. Further, it is discussed that while not all actors were equitably engaged throughout the entirety of the project, sometimes as a result of misaligned goals, M&E can be leveraged as a communication mechanism between stakeholders to enable increased engagement and goal alignment.
INTRODUCTION
The international development sector spends billions of dollars each year on projects attempting to reduce global poverty and increase quality of life (OECD 2017). Clean cookstoves are just one technology that have been incorporated into development strategies targeting the nearly 3 billion people who still rely on biomass fuels (e.g. wood, charcoal, dung, crop residues) to cook and heat their homes (UNDP 2009). To improve health and reduce environmental harm from traditional cooking methods (e.g. three-stone fires), clean cookstoves were designed with the goal of reducing smoke emissions and fuel consumption by increasing both combustion and thermal efficiencies. To achieve this goal, the Clean Cooking Alliance, formerly the Global Alliance for Clean Cookstoves, aims to transition 100 million households to cleaner cookstoves and fuels by 2020 (Global Alliance for Clean Cookstoves 2015). An essential component of this program is monitoring and evaluation (M&E) to determine to what extent these goals are being met and which has been an increasingly critical element for international development projects. If done properly, M&E can increase user agency in international development projects and improve project success and sustainability.
This case study investigates the development of a sensor as an M&E tool for cookstove usage and fuel consumption in Guatemala, Honduras, and Uganda, and the influence of various stakeholders (users, designers, and development practitioners) in choosing what outcomes and indicators to measure, how to analyze them, and how to translate analyzed data into actions that meet the needs of end-users. The existing power structure within which this case study was conducted is discussed and alternative frameworks suggested. As the researchers came to find, the international development system has the potential to propagate unintentional misalignment of goals between the complex network of people involved, but it also has the potential to use M&E as an avenue for user agency and multi-sectoral partnership. To foster closer alignment and recognition of user agency the role of ethnographic methods throughout the automated M&E process is discussed. Although this case study is specific to development projects, the process and challenges are generalizable to most companies functioning under a general user-designer-stakeholder model with development work exhibiting extreme characteristics due to cultural divides that can lead to potential disconnections between donor demands, designer-proposed solutions, and actual user needs.
BACKGROUND
International Development Paradigms
Starting with an overview of the structural underpinnings of the international development sector can aid in understanding how it currently operates. The field of international development originated after World War II, during European efforts to rebuild their cities. Following the initial rebuild, some European countries sought to invest their additional funds in revitalizing lower-income countries to increase international trade and thereby gain more control, according to post-development theorists (Rapley 2007). Regardless of the intentions, European involvement and control in outside countries’ affairs created an inherently top-down structure, as they were choosing the desired outcomes and development strategies. During the second half of the 20th century, local and international NGOs began to develop around this movement of humanitarian assistance and development. These NGOs traditionally sourced their funding from governments and aid agencies who often set internal agendas, further perpetuating the top-down nature of the sector.
Today, the sector has evolved into a complex web of stakeholders, including governments, government aid agencies, private donors, NGOs/nonprofits, academic institutions, and end beneficiaries. Power hierarchies within this web have led to conflicting priorities in pursuit of the intertwined goals of poverty relief and sustainable development. In addition, a lack of viable mechanisms for communicating across cultures and in areas of low or no connectivity has deepened this divide. What can unintentionally result from this disconnected structure is project failure due to a lack of user input and agency. There are countless instances of initiatives that have fallen short of delivering their promised impact. Some examples include the LifeStraw (Boisson et al. 2009), PlayPump (McGrath 2011), One Laptop Per Child (Keating 2009), and India’s National Programme on Improved Chulha [stoves] (Hanbar and Karve 2002).
Clean Cooking & Fuels
Despite past difficulties with clean cookstove projects, they remain a prevalent technology in the development sector. The goal of distributing 100 million cleaner cookstoves set by the Clean Cooking Alliance stems in part from understanding the risks associated with traditional cooking methods, which have been linked to a wide array of health, environmental, and economic problems. To illustrate, picture a scene observed by the research team during fieldwork of a woman using a traditional open fire in rural Uganda. Several times a week, she sends her daughter deep into the bush to search for growingly scarce firewood, which takes several hours per trip. She spends three to six hours a day by the fire cooking for her family, continually inhaling the smoke that becomes a constant cloud in her poorly ventilated kitchen. The smoke that eventually escapes the kitchen will permeate the atmosphere and contribute to carbon emissions. The smoke inhaled by the woman and any of her children will be the catalyst of future lower respiratory infection. Perhaps, with a cleaner stove, the time spent on cooking could be allocated towards something else, like education or an income-generating job. Perhaps the fuel saved from a more efficient stove could help to limit forest degradation. Observations of the potential for positive impacts drive the Clean Cooking Alliance’s mission. Ultimately, these impacts are nuanced and context-specific.
Traditional open cooking fires are timeless to human history and deeply ingrained in culture. Cooking and preparation methods, food flavors, and meal textures depend on these existing technologies. If new stove models do not completely consider these cultural contexts, they often go unadopted (Thacker, Barger, and Mattson 2017). However, stoves that better accommodate cultural and/or economic preferences can sacrifice efficiency and sustainability, resulting in similar amounts of smoke and fuel use to their traditional counterparts. Because of such varying social, cultural, and economic constraints and tradeoffs, it is extremely challenging to design a stove that optimizes usability, affordability, and technical performance. For technical performance specifically, it is difficult to predict in-home fuel consumption and emissions from lab-based testing. Monitoring and evaluation (M&E) is one method to help measure, evaluate, and ultimately address the balance of achieving various impacts on the household and community level.
CHOOSING OUTCOMES & INDICATORS
Current automated M&E tools to assess cookstove performance and impact include temperature and emissions sensors to measure indicators including cookstove usage, ambient emissions, and indoor air pollution. While M&E methods in this sector are advancing, there are still gaps. During a twelve-day fieldwork course in Guatemala, two of the lead researchers from Oregon State University (OSU) worked with a clean cookstove nonprofit on the manufacturing and distribution of their stoves into a local community. A few months after the field course, conversations with the nonprofit revealed the need for new M&E methods to report impacts to donors. Included in these selected outcomes were changes in children’s school attendance, fuel savings, and time savings. Hearing this, the lead researchers started brainstorming possible ways to measure these donor-driven impact metrics.
Visualizing the practice of cooking from start to finish using field observations from Guatemala, the team decided to focus on firewood, and the idea for a sensor to weigh fuel over time, called the Fuel Use Electronic Logger (FUEL), was conceived. Measurement of fuel consumption is an important metric for donor evaluation for several reasons. First, cleaner cookstoves are intended to reduce the amount of fuel used, requiring both baseline and post-intervention usage data. As previously stated, metrics obtained from lab testing on cleaner cookstoves historically vary significantly from actual field performance (Lombardi 2017). As such, obtaining accurate field data is important to validate the assertion that the new stove saves fuel. Additionally, cookstove usage patterns are far from uniform. Stove stacking, or using multiple stoves (cleaner and traditional), to regularly cook, is common, reducing the effectiveness of stove interventions in terms of both adoption and technical performance (Masera 2000; Masera 2005; Pine 2011). One of the main motivations for the team was, therefore, to contribute to transparency and accountability mechanisms within the sector, ensuring that the proposed performance outcomes were being met. By understanding stove adoption and fuel consumption patterns and the reasons behind these trends, the cookstove sector can begin to move towards more integrative stove designs that more comprehensively meet end-user needs.
NEEDS EVALUATION
Stakeholder Needs Assessment
Creating an economically viable sensor was critical to ensure product sustainability and impact. To understand the proposed sensor’s economic viability, the team participated in the Oregon State University Advantage Accelerator Program, the National Science Foundation Innovation Corps, and VentureWell Student E-Teams. Through these programs, the team conducted semi-structured interviews with over 50 stakeholders in the cookstove community with the aim of understanding current gaps in the sector and whether/how the sensor could help fill those gaps. One result of these interviews was a better understanding of the challenges NGOs faced when trying to form a sustainable business model, in addition to other financial constraints that shape the sector.
Using the compiled interview data, the research team constructed a non-profit ‘needs pyramid’ to depict how funds were being allocated and prioritized in the current system. The needs pyramid was rooted in Maslow’s Hierarchy of Needs theory (Maslow 1943), which ranks a set of needs from vital to least vital and posits that if base-level needs are not met, less essential needs cannot be addressed. From this, the team realized that the success of the FUEL sensor required basic non-profit needs (i.e. distribution and logistics, marketing, and qualitative user feedback) to be met before considering quantitative monitoring and evaluation, as shown in Figure 1. Budgets in international development are typically constrained, and therefore most resources are often allocated towards cookstove distribution, the most direct and quantifiable achievement for a cookstove nonprofit. Because of limited funding and a lack of regulatory policy or incentives, rigorous M&E is sometimes not feasible within the current system. When NGOs did conduct M&E, it was often at the direct request of their donor and written into the funding budget (approximately 5-10% of total, if any).
Figure 1. Hierarchy of NGO needs
The resulting hierarchy of needs led to several areas of consideration for the team concerning the purpose of the FUEL-derived data. If used, and once collected, would these data be used to inform donors of project success? If so, why was the metric of measuring fuel chosen as an important indicator and what level of input did end-users have in this decision process? If needed, what would be the NGO’s capacity to effect change? In addition, while the metrics FUEL can report are constant, how important fuel consumption is to different stakeholders is variable. How could the FUEL be used to elicit, align and reconcile end-user and donor needs? These questions helped to highlight the differences between end-users and other stakeholders.
Figure 2 shows a representation of the various levels of relative institutional power and the different groups engaged within the development scheme throughout the project lifetime. Power rankings were based on interview data and tracing who made the key decisions for projects, often characterized in terms of money flow. Notably, despite being the focus of this intervention, the end-user often has the least institutional power and, sometimes, least say in the project outcomes. The researchers were faced with the question of how this system could be equilibrated and work to be more empowering and collaborative for underrepresented populations.
Figure 2. Levels of institutional power throughout the project lifecycle
Alignment of Various Stakeholder Needs
Illustrative of the institutional power structure outlined in Figure 3, needs may be unintentionally prescribed to end-users by other stakeholders in development projects. For example, studies have shown low demand for clean cookstove technologies with other needs prioritized above reducing indoor air pollution (Mobarak et al. 2012). Although there is clearly still a place for promoting cleaner technologies and longer-scale healthier behaviors even if they are not the first choice or primary concern of the user, practitioners should still consider what other technologies and/or metrics might be more aligned with user needs. These considerations also raise the ethical question of how to identify and prioritize the objectives of multiple stakeholders and how these initiatives can be designed in a way that does not place the majority of the burden to adapt on the end-user. These ethical factors were further contemplated and evaluated as the team began to develop and test the physical product.
PRODUCT DEVELOPMENT & TESTING
The idea to develop a fuel weight sensor was chosen as it represented an improvement from the standard method of manually measuring fuel at set intervals over a period of time (typically 4-7 days) – a task that is both time consuming and sometimes unreliable (Bailis et al. 2018). Based on these findings, the Fuel Usage Electronic Logger (FUEL) sensor was conceived and designed in partnership with Waltech Systems and Climate Solutions Consulting.
OSU researchers provided initial specifications based on field observations, including a maximum weight capacity, resolution, and data logging rate. The final design utilized a wireless sensor to autonomously log fuel weight over time and a temperature sensor to corroborate cooking events, providing both firewood consumption data and cookstove usage patterns. Figure 3 shows the system installed in a household kitchen. The intent is that a household cook stores a portion of their fuel in the supplied holder and removes fuel as needed for cooking. The load cell logs these mass changes and stores them in its internal memory. An algorithm developed by the OSU research team then integrates these changes in mass to calculate fuel use over time. To test, field studies in Honduras and Uganda were conducted with five and then 100 sensors, respectively, between April 2017 and July 2018. Additional information on the design of the FUEL system is outlined in (Ventrella, Zhang, and MacCarty 2019) and the algorithm and preliminary results from a study in Uganda are described in (Ventrella and MacCarty 2019).
Figure 3. FUEL system in household kitchen (Ventrella, 2018)
ANALYZING RESULTS
After the studies were completed, OSU researchers developed an algorithm to analyze sensor data in the lab. Raw outputs from the sensor provided a time-stamped log of the cookstove temperature and weight of the fuel in the holder. These raw data were then translated into donor and NGO-driven metrics including fuel consumption, stove usage, and projected tons of CO2 mitigated. More detail on raw and extrapolated outputs can be viewed in a previous paper (Ventrella and MacCarty 2019).
Upon further analysis, some of the data from the study in Uganda showed peculiar trends. Large spikes in the weight of fuelwood would appear sporadically, and the researchers originally planned to remove these data points assuming they were just noise (e.g. accidental jostling of the fuel holder). Comparing these spikes with insights from a focal follow of firewood collection and participant observation helped to contextualize the data, as described in a previous EPIC paper (Zhang, Zhao, and Ventrella 2018). The researchers eventually determined that this noise resulted from intentional actions of the cook. From observations, it was not uncommon for women either directly before cooking or during mid-meal preparation to split additional wood and add it to the fire. From the FUEL training, they were told that they could only cook with fuel that came from the holder. As such, women would add and then immediately remove the firewood from the holder to cook, resulting in unexpected spikes in the data. Despite the automated nature of the FUEL, human input, both from researchers and end-users, was essential to interpret the data.
Using this new understanding of how the sensor was being used in practice, an algorithm was created to filter actual noise from these intentional spikes. A later study conducted by the research team showed that applying this cleaning technique increased the goodness-of-fit of the FUEL to manual measurement results from 0.6 to 0.8 (Ventrella, LeFebvre, and MacCarty 2019). Comparison of the FUEL to manual firewood measurements confirmed that the sensor worked and was generally well accepted by stove users. The researchers then had to decide what to do with results from the FUEL.
TRANSLATING RESULTS
Using the analyzed results, relevant metrics were conveyed to the NGO partners indicating that the FUEL sensor was successful in its goal of measuring firewood consumption and cookstove usage patterns. Additionally, the FUEL could enable NGOs to collect more quantitative data on fuel consumption, better understand cookstove usage patterns, and take more informed steps with their project moving forward and to ensure longer-term impacts.
In addition to communicating results to the NGO partners, one priority of the researchers was to communicate the results back to those who had participated in the studies. Although Institutional Review Board (IRB) oversight requires specific actions to be taken before and during the study, there is little to no guidance as to the ethical requirements of reporting data back to participants after the study if the research is not directly health-related. To address this missing feedback mechanism, the lead researcher worked with International Lifeline Fund, a local non-profit in Uganda, to generate both a script and visuals to convey the results of the study back to participants. Figure 4 shows an example of results that will be conveyed to participants, both verbally and using a visual scale. In this case, the amount of fuel used for each combination of stove type(s) used in the study location is depicted in a manner intended to be more accessible for areas with low literacy rates. A previously published paper provides further detail and images of each stove type (Ventrella and MacCarty 2019).
Figure 4. Example of visual results for research participants
Results of fuel consumption measurements from this initial study were positive, confirming that use of the cleaner stove reduced fuel usage. A question that remained was what decisions would need to be made if the results proved otherwise and who would be involved in these decisions. For example, if results showed that there was no decrease in fuel usage, there was low adoption of cleaner cookstoves, short cookstove lifetime, or stove stacking was a major problem, what would be done? The project could be abandoned or the results not addressed, or, in contrast, these results could be considered and lead to positive changes.
There are several examples from the broader sector that demonstrate some of these positive initiatives following qualitative feedback from end-users and sensor-based data. Following user-voiced concerns about cookstove durability, designers have started extensive research on materials and durability to improve the lifetime of cleaner cookstoves (Brady et al. 2017). Cleaner cookstoves have had value added by including thermo-electric generators that power lights and charge cell phones (Wilson et al. 2018), something that may make investing in cleaner cookstoves more appealing to men who typically control household spending but aren’t subjected to the daily detriments of cooking with a traditional stove. Others have made context-specific design changes to facilitate the usability of their stove models. These examples represent positive changes that have been made following M&E of projects that embrace user agency.
FUTURE STEPS: INCREASING AGENCY
As discussed throughout the paper, M&E that engages users throughout the project lifecycle has the potential to elicit user needs and align stakeholder objectives. When done improperly, the solely top-down power structure that is sometimes present in development projects can be perpetuated. However, designers and researchers can play a role in transcending these communication barriers by actively engaging with end-users and helping to create a space where their insights are included in the decision-making process. While top-down approaches insofar as they include the role of governments, donors, and NGOs are still relevant, they must be integrated with bottom-up, local knowledge.
One step taken to accomplish this on a technical level for the FUEL was the development of a wireless data collector to replace SD cards and help resolve usability issues faster. For example, field staff can now identify issues and work with end-users to solicit feedback, reduce data bias, and brainstorm the next steps to be taken almost immediately. Additionally, the wireless data collector can now be fully integrated with a suite of sensors to monitor fuel use, cookstove usage patterns, and emissions, expanding the number of metrics assessed and allowing for more holistic performance assessment. However, power dynamics are still at play here as the monitoring metrics are not necessarily prescribed by the user. Stronger mechanisms for translating user and sensor-based feedback into useful programmatic strategies are needed. The method of triangulating qualitative and quantitative ethnographic data can certainly help to assess current progress and gaps in development projects, and more resources should be allocated towards this end.
In an ideal world, the stakeholder hierarchy of needs would look quite different, as portrayed in Figure 5. The priority and large allocation of funds would go towards understanding user needs first and engaging with local entities. Once understood, the end-user, designers, and other stakeholders would work together to identify relevant quantitative metrics. These initial data would be informed qualitatively by the end-user. A combination of these qualitative and quantitative data would then inform more effective marketing campaigns, thus reducing the need for late-stage M&E, with researchers acting as cultural brokers to ensure all stakeholder needs and knowledge are incorporated into action-based, long-term project steps. Once a successful market was established, more resources could then be allocated to the distribution of cookstoves and logistics. The FUEL can fit into either hierarchy depending on the use case. For example, if fuel consumption is an identified issue in the area, the FUEL can be implemented early on in the process to determine how much in-home fuel is being decreased and inform marketing strategies, as opposed to the traditional hierarchy where it would only be used at the end of the process mainly for donor reporting.
More organizations are moving toward this market-based structure as they strive towards establishing a sustainable market of which understanding user needs is a requirement. Recently, there has been a trend towards better user engagement in market-based approaches. First steps for this endeavor include engaging with users to better understand current knowledge, attitudes, and practices around cooking using a combination of qualitative and quantitative methods. This partnership-based approach brings together international and local governments, universities, NGOs, and the end-user, with M&E as one mode of communication between each group. For this project, and others like it, the M&E tools deployed over time may need to be updated to more explicitly track user-defined needs. Additional complexity is added as some situations may call for a hybrid approach in which donor-driven and market-based initiatives are combined in the pursuit of a strong future market, with the M&E methods evolving throughout the project lifespan.
Figure 5. Ideal hierarchy of needs
CONCLUSION
As the researchers learned during this case study, human feedback informed every step of the automated M&E process. Human-based ethnographic data was essential to selecting outcomes and indicators, designing the algorithm, and interpreting quantitative data. Although the algorithm developed in this study was based on quantitative, non-human collected values, its assessment and true value is derived from human inputs.
While autonomous technologies are often assumed to lead to accuracy, neutrality, objectivity, and transparency that is especially important in the field of M&E, this case study demonstrates that the integration of human inputs enhances, rather than compromises, accurate and meaningful interpretation of data as well as transparency and accountability in M&E.
The demand for rigorous M&E in the international development sector is becoming more ubiquitous. Researchers need to remain conscious of what metrics are chosen to be monitored and allocate resources effectively and efficiently towards those initiatives. This is especially important given the dichotomy between constrained organizational operating budgets and the broad and pressing nature of the issues these organizations must address. M&E that combines automated quantitative and qualitative data and brings together the inputs of multiple actors can help create an avenue for those at the bottom of the institutional power structure to more actively participate.
Jennifer Ventrella is a PhD student in public and urban policy at the New School. She recently earned a dual Master’s in mechanical engineering and applied anthropology from the humanitarian engineering program at Oregon State University. Her email address is ventj804@newschool.edu
Erin Peiffer is a PhD student in mechanical engineering with a minor in applied anthropology in the humanitarian engineering program at Oregon State University. Her email address is peiffere@oregonstate.edu
Shaozeng Zhang conducts research in applied anthropology and Science and Technology Studies (STS). He received his PhD in anthropology from the University of California, Irvine in 2014. He is currently an Assistant Professor of anthropology at Oregon State University. His email address is shaozeng.zhang@oregonstate.edu
Nordica MacCarty is an Assistant Professor of mechanical engineering at Oregon State University. Her research as part of the humanitarian engineering program is focused on the design and evaluation of clean water and energy projects with particular attention to user considerations. Her email address is nordica.maccarty@oregonstate.edu
NOTES
Acknowledgements – We are grateful for the support of the International Lifeline Fund and Stove Team International teams for hosting field studies, as well as the families in Guatemala, Honduras, and Uganda who welcomed us into their home. We would also like thank the National Science Foundation I-Corps grant and VentureWell Student E-Teams for financial and professional support.
REFERENCES CITED
Bailis, Rob, Ryan Thompson, Nicholas Lam, Victor Berrueta, Godfrey Muhwezi, and Esther Adams. 2018. “Kitchen Performance Test (KPT) 4.0.” Clean Cooking Alliance website. Accessed [October 19, 2019]. https://www.cleancookingalliance.org/binary-data/DOCUMENT/file/000/000/604-1.pdf.
Boisson, Sophie, Wolf-Peter Schmidt, Tsegahiwot Berhanu, Henock Gezahegn, and Thomas Clasen. 2009. “Randomized controlled trial in rural Ethiopia to assess a portable water treatment device.” Environ Sci Technol 43 (15): 5934-9. doi: 10.1021/es9000664.
Brady, Michael P., Kelly Banta, John Mizia, Nathan Lorenze, Donovan N. Leonard, Yukinori Yamamoto, Morgan DeFoort, and James R. Keiser. 2017. “Alloy Corrosion Considerations in Low-Cost, Clean Biomass Cookstoves for the Developing World.” Energy for Sustainable Development 37: 20-32. doi:10.1016/j.esd.2016.12.002.
Global Alliance for Clean Cookstoves Report. 2017. “Haiti: Action Plan for the Transformation of the Cookstoves and Fuels Market.” Global Alliance for Clean Cookstoves.
Global Alliance for Clean Cookstove Report. 2015. “Five Years of Impact: 2010-2015.” Global Alliance for Clean Cookstoves.
Hanbar, R.D. and Priyadarshini Karve. 2002. “National Programme on Improved Chulha (NPIC) of the Government of India: an overview.” Energy for Sustainable Development. 6 (2): 49-55. doi:10.1016/S0973-0826(08)60313-0.
Keating, Joshua. 2009. “Why did One Laptop Per Child Fail?” Foreign Policy website, September 9. Accessed [October 19, 2019]. https://foreignpolicy.com/2009/09/09/why-did-one-laptop-per-child-fail.
Lombardi, Francesco, Fabior Riva, Giorgio Bonamini, Jacopo Barbieri, and Emanuela Colombo. 2017 “Laboratory protocols for testing of Improved Cooking Stoves (ICSs): A review of state-of-the-art and further developments.” Biomass and Bioenergy 98: 321-335. doi:10.1016/j.biombioe.2017.02.005.
Masera, Omar, Barbara Saatkamp, and Daniel Kammen. 2000. “From Linear Fuel Switching to Multiple Cooking Strategies: A Critique and Alternative to the Energy Ladder Model.” World Development 28 (12): 2083-2103. doi:10.1016/S0305-750X(00)00076-0.
Masera Omar, Rodolfo Díaz, and Victor Berrueta. 2005. “From cookstoves to cooking systems: The integrated program on sustainable household energy use in Mexico.” Energy Sustainable Dev 9: 25–36.
Maslow, Abraham. 1943. “A Theory of Human Motivation.” Psychological Review 50 (4): 370-396. doi:10.1037/h0054346.
McGrath, Rita Gunther. 2011. “Failing and Learning with the PlayPump.” Harvard Business Review website. March 30. Accessed [October 19, 2019]. https://hbr.org/2011/03/failing-and-learning-with-the.html.
Mobarak, Ahmed Mushfiq, Puneet Dwivedi, Robert Bailis, Lynn Hildemann, and Grant Miller. 2012. “Low demand for nontraditional cookstove technologies.” PNAS 109 (27): 10815-10820.
Organisation for Economic Co-operation and Development (OECD). 2017. “Official Development Assistance (ODA) flows for DAC and non-DAC members, recipients, and regions.” OECD website. Accessed [October 19, 2019]. http://www.oecd.org/dac/financing-sustainable-development/development-finance-data/aid-at-a-glance.htm.
Pine, Kathleen, Rufus Edwards, Omar Masera, Astrid Schilmann, Adriana Marrón-Mares, Horacio Riojas-Rodríguez. 2011. “Adoption and use of improved biomass stoves in Rural Mexico.” Energy Sustainable Dev 15: 176–183. doi:10.1016/j.esd.2011.04.001.
Rapley, John. 2007. Understanding development: Theory and practice in the third world. Boulder, CO: Lynne Rienner Publishers.
Thacker, Kendall, K. McCall Barger, and Christopher Mattson. 2016. “Balancing technical and user objectives in the redesign of a Peruvian cookstove.” Development Engineering 2: 12-19. doi:10.1016/j.deveng.2016.05.001.
United Nations Development Programme (UNDP). 2009. “The Energy Access Situation in Developing Countries: A Review Focusing on the Least Developed Countries in Sub-Saharan Africa.” UNDP website. Accessed [October 19, 2019]. https://www.undp.org/content/dam/undp/library/Environment%20and%20Energy/Sustainable%20Energy/energy-access-situation-in-developing-countries.pdf
Ventrella, Jennifer. 2018. “FUEL system installed in Apac, Uganda.” Zenodo. doi:10.5281/zenodo.1286578#.Wx2N_ydHegg.mendeley
Ventrella, Jennifer, and Nordica MacCarty. 2019. “Monitoring impacts of clean cookstoves and fuels with the Fuel Use Electronic Logger (FUEL): Results of pilot testing.” Energy for Sustainable Development 52 :82-95. doi:10.1016/j.esd.2019.06.004
Ventrella, Jennifer, Shaozeng Zhang, and Nordica MacCarty. 2019. “An international, multi-site, longitudinal case study of the design of a sensor-based system for monitoring impacts of clean energy technologies.” Design Studies (Pending publication).
Ventrella, Jennifer, Olivier LeFebvre, and Nordica MacCarty. 2019. “Techno-economic comparison of the FUEL sensor and the Kitchen Performance Test to quantify in-field cookstove consumption.” Submitted to Development Engineering.
Wilson, D.L., Monga, M., Sakesena, A., Kumar, A. and Gadgil, A., 2018, Effects of USB port access on advanced cookstove adoption. Development Engineering.
Zhang, Shaozeng, Bo Zhao, and Jennifer Ventrella. 2018. “Towards an archaeological-ethnographic approach to big data: rethinking data veracity.” Ethnographic Praxis in Industry Conference Proceedings (EPIC).