Connecting with other people is at the heart of ethnographic research – understanding their perspectives, preferences, and behaviors helps organizations create and align offerings with consumers. Research relies on clear communication to optimize participant experience and develop meaningful insights from research results. Yet not all communication is created equal – especially when working in multiple languages. Translation by machine or an inexperienced translator often lacks cultural nuance and can miss the mark, resulting in a poor participant experience, study attrition, less than optimal interpretation, and ultimately insufficient research outcomes.
To illustrate the differences in output between human and machine translation, we set up several experiments – first pitting human vs. machine and then pitting two experienced translators against each other. Read on to explore the results.
Part 1: Human vs Machine
Will Skynet gain consciousness? Who will win in the epic battle of man versus machine? While we’re not quite ready to throw out our predictions for the coming AI-pocalypse, we’re gearing up for a face-off between human and machine translation in UX and market research arenas.
On the surface, demographic questions can seem straightforward and simple to translate. Yet cultural and linguistic differences can result in challenges in translation and can muddy data without the proper framing. Consider a common question surrounding annual household income. Did you know some countries in Asia and Latin America typically reference income on a monthly rather than yearly basis? Can machine translation take into account the typical income breaks used in each region so participants are less likely to mis-select? The reality is human-powered translation is necessary to capture the full intent and nuance of market research questions, as you’ll see from the examples below.
| Original | Machine Translation | Human Translation |
| What is your total annual household income? A. Less than $10,000 B. $10,000 to $24,999 C. $25,000 to $49,999 D. $50,000 to $74,999 E. $75,000 to $99,999 F. $100,000 and greater G. Prefer not to answer |
あなたの年間総世帯収入はいくらですか？ A. 10,000ドル未満 B. $ 10,000〜 $ 24,999 C. $ 25,000から$ 49,999 D. $ 50,000から$ 74,999 E. $ 75,000から$ 99,999 F. $ 100,000以上 G.答えたくない |
合計世帯年収を教えてください。 A. 100万円未満 B. 100万円以上200万円未満 C. 200万円以上300万円未満 D. 300万円以上400万円未満 E. 400万円以上500万円未満 F. 500万円以上600万円未満 G. 600万円以上700万円未満 H. 700万円以上800万円未満 I. 800万円以上900万円未満 J. 900万円以上1000万円未満 K. 1000万円以上 L. 答えたくない |
Explanation: In this example, the human translator’s first step was to convert the US dollars to Japanese yen. $1 is about ¥109 today, but they rounded it to ¥100 for readability. In Japan, it is more common to reference figures such as “over 1 million yen to under 2 million yen“ versus the US English style of ending breaking points with 9’s such as “1,999,999 yen.” The linguist also localized the income range to fit the way it is usually expressed in Japan.
In the next example, we translated a standard race and ethnicity question – something that might occur when trying to reach non-English speaking households in the US.
| Original | Machine Translation | Human Translation |
| Please specify your ethnicity (please select all that apply): A. Caucasian B. African American C. Latino or Hispanic D. Asian E. Native American F. Native Hawaiian or Pacific Islander G. Some other race (please specify) H. Prefer not to say |
あなたの民族を指定してください（該当するものをすべて選択してください）： A.白人 B.アフリカ系アメリカ人 C.ラティーノまたはヒスパニック D.アジア人 E.ネイティブアメリカン F.ハワイ先住民または太平洋諸島民 G.その他の人種（具体的に記入してください） H.言わない方がいい |
あなたの民族的背景を選択してください（当てはまるものをすべて選択してください）。 A. 白人 B. アフリカ系アメリカ人 C. ラティーノまたはヒスパニック D. アジア人 E. ネイティブアメリカン F. ハワイ先住民または太平洋諸島民 G. その他：________ H. 答えたくない |
Explanation: In this instance, the machine translation essentially translates as “please point out your ethnicity” which is not considered a natural or thoughtful request. In response H, the machine translation reads, “it’s better not to say,” whereas a more culturally appropriate human interpretation would land closer to “I would like to not answer.”
For quick internal needs or straightforward translations, automated translation apps might suffice, but for something like a research screener, interview discussion guide, or group discussion transcription, you need more.
Experienced researchers understand the value of a long-term language solutions partner to deliver high-quality translations on time and on budget. In certain circumstances, automated translation can get you close – but when quality matters, humans matter.
For the next set of translations, we examined a set of instructions for a deprivation market research study where clear participant instructions directly impact the quality of the data.
| Original | Machine Translation | Human Translation |
| For the next 2 weeks, we need you to unplug all electronics from 5pm to 8am – no T.V., no phones, no screens. Each morning at 8am, make a video recording on how you spent your evening and who you spent it with. | 「次の2週間は、午後5時から午前8時まで、すべての電子機器のプラグを抜く必要があります。テレビ、電話、画面はありません。毎朝午前8時に、夜の過ごし方と一緒に過ごした人のビデオを録画してください。 「」 | 「これから2週間、午後5時から午前8時まで、テレビや電話機、モニターなど、すべての家電のコンセントを抜いてください。毎朝午前8時に、前日の晩の過ごし方、一緒に過ごした人について説明する動画を撮影してください。」 |
Explanation: Understanding the nuance of the Japanese language, the human translator was able to translate “we need you to” to the imperative mood – which makes the verb into a command. In Japanese, this would be a more natural phrase and easier to understand as an instruction. Additionally, the human translator rephrased “no T.V.” to “including T.V” for additional clarity.
The machine output used the literal translation of “unplug.” Whereas the human instructs the participant to “unplug from all outlets” – which is more natural in Japanese. Clear participant instructions increase compliance and improve study conversion.
For our next exercise, we sought to understand feedback from a group discussion. In this example, we translated the following sequences from Japanese to English via machine translation and human translation to illuminate the different outcomes from each approach.
| Original | Machine Translation | Human Translation |
| 「また使うときがくるかもしれない」と思って、削除するのは気が引けます。削除はしたくないけど、非表示にしておきたいアプリが、ホーム画面の半分くらいを占めています。そうした「あまり使わないアプリ」の対処方法について、考えてみました。 | I’m reluctant to delete it because I think it may be time to use it again. Apps that I don’t want to delete but want to hide occupy about half of the home screen. I thought about how to deal with such “apps that I rarely use.” | I hesitate because I think, “There may come a time when I’ll use them again.” About half of the applications on my home screen are ones that I don’t want to delete but also don’t want to have constantly displayed, either. So I thought about how to manage those “applications that I don’t frequently use.” |
Explanation: The machine, focusing on direct translation, doesn’t fully understand the text and is unable to reword statements based on intent. For example, “It may be time to use it again” would more accurately be translated to, “there may come a time when I will use them again.” Most of the machine translation is technically accurate, but it fails to recognize the nuance in certain areas. While “occupy” is the correct wording in Japanese (“occupy about half of the home screen”), in English, it would be more direct and simpler to say, “half of the applications on my home screen….” And for overall readability, the human translation still sounds more natural – like something a person would actually say.
While we can’t say for sure our robot overlords will never gain self-awareness, we can agree when it comes to translations, humans win this round. The computer-generated language lacks the ability to convey subtleties and intent. When organizations are seeking to understand their customers, the human touch helps translate data into insights.
Part 2: 2 Translators
Next up we’re pitting human translator versus human translator. When you need to select a translation agency for a research project, how do you know the agency you’ve selected or the translators working on your materials have the right expertise?
They might be native speakers of the language, and even professional translators, but do they truly understand and have the necessary experience for your project? When it comes to the instructions for a deprivation study or focus group responses, deep cultural understanding is needed to get your meaning across.
For our next example, we wanted to demonstrate that even two professional and experienced translators don’t always arrive at the same destination. The text references a diary entry initially provided in Spanish by a participant in a deprivation research study.
| Original | Translation 1 | Translation 2 |
| “Este es mi primer registro en video sobre mi experiencia de desconexión. En lugar de ver la televisión o mi teléfono, se supone que debemos encontrar otra actividad que hacer. Ayer fue mi primera noche del experimento. La mayoría de las noches, después de la cena, yo organizo las loncheras para el día siguiente, mi marido aprovecha para lavar los servicios mientras mi hija juega con su computadora y mi hijo ve televisión. Anoche, en lugar de ver la televisión, nuestra familia jugó Nervioso después de la cena. Fue lindo tener a todos reunidos en un solo lugar, aunque mi hija, que no tiene pelos en la lengua, nos dijo que se moría del aburrimiento sin su celular.” | “This is my first video entry about my unplugged experience. Instead of watching TV or scrolling on my phone, we’re supposed to find another activity to do. Last night was my first night of the experience. Most nights, after dinner, I prepare lunch boxes for the next day. My husband takes advantage of that time to wash the dishes while my daughter uses his computer, and my son watches TV. Last night, instead of watching TV, our family played Nervioso, a card game, after dinner. It was nice to have us all gathered in one place, although my daughter, who doesn’t mince her words, told us she was totally bored without her phone.” | “This is my first video record about my disconnection experience. Instead of watching TV or my phone, we’re supposed to find another activity to do. Yesterday was my first night of the experiment. Most nights, after dinner, I organize the lunch boxes for the next day, my husband takes the opportunity to wash the dishes while my daughter plays on her computer and my son watches TV. Last night, instead of watching TV, our family played Nervioso after dinner. It was nice to have everyone gathered in one place, although my daughter, who does not mince her words, told us that she was bored to death without her cell phone.” |
Explanation: Independently, both translations are accurate and easy to understand. However, when insights are at stake, accuracy isn’t the only metric that matters – overall quality matters too. When the focal point is the original intent of the Spanish speaker, the slight difference between the two translations matter.
In Translation 1, the linguist uses the phrase “unplugged,” which matches the language given in the instructions. Translation 2 uses the phrase “disconnection experience.” While the second translation is technically correct, it could cause confusion during data analysis since the language is inconsistent with the word choice of the original instructions.
As for the activities the family is abstaining from, the first translation uses the statement, “scrolling on my phone” while the second simply uses “my phone.” Though both phrases involve phone usage, by using the verb “scrolling” the first translation implies social media usage (e.g. scrolling the social media feed). The second statement is more vague and could refer to any phone usage (e.g. calls, texts, emails).
Toward the middle of the entry, the translators also show a discrepancy in pronoun usage. Translation 1 uses “his” while Translation 2 uses “her.” The first translation implies the daughter is using the father’s computer or that the daughter uses he/his for pronouns. The second translation clearly indicates that she is using her own computer. While small, these extra details can make a difference when trying to understand human experience. This is also seen in the different ways the linguists describe Nervioso. Translator 1 specifies that it is a card game, while Translator 2 uses italics to imply it is a title. While a quick web search would reveal what Nervioso is, Translator 1 saves the researcher a little time.
Lastly, it’s the linguist’s job to communicate impact, which requires judgement. When explaining the daughter’s feelings about being without her phone, Translator 1 uses the phrase “totally bored” while Translator 2 uses “bored to death.” Both communicate the intent, but the statement “bored to death” delivers impact.
When the Insights Matter, Humans Matter
Whether human vs. machine or translator vs. translator, nuance makes all the difference. Data collection through research requires deep understanding – even within the same language. When adding the complexity of translating data from one language to another, it’s best to work with an expert who understands language, culture and the research industry.
When done well, research allows brands, products and services to explore the thoughts, experiences and opinions of their audiences and users – so be thoughtful about translation services so you don’t risk missing the mark. Creating a translation that accurately conveys your voice and the voice of the participant and that creates the same emotional impact requires human skill and experience. Look for that right balance when looking for an insight translation partner. A proven track record of work within the research industry can make all the difference in bringing the voice of the customer to life for greater business impact.
Multilingual Connections is an EPIC2021 Sponsor. Sponsor support enables our unique annual conference program that is curated by independent committees and invites diverse, critical perspectives.
Image: “Language” by Mac McCreery via flickr https://flic.kr/p/2mncQTf (CC BY-NC 2.0)
Related Articles
Why does Nuance Matter when Translating and Transcribing Multilingual Content?, Jill Bishop
Found in Translation: Pro Tips for Translating and Transcribing Multilingual Content, Jill Bishop
Great Interpreters Inspire Insights: The Gifts around Language Dialogues, Stuart Henshall
0 Comments