Lisa Matthews from Right to Remain talks about her research into translating the Right to Remain toolkit. She shares with us what she has learnt from being part of the Explore programme and invites us all to think about next steps.
The Explore programme
Right to Remain was recently one of the organisation’s lucky enough to be part of Refugee Action’s Explore programme - an 8 week programme on which charities are supported on a digital journey. Taking place during another challenging Covid lockdown, the twice weekly zoom calls and conversations on Slack were a great chance to connect to other organisations, learn about their work and their digital journeys and share ideas and challenges, and have some productive, informal human interaction!
The Right to Remain project
On the Explore programme, I was looking at the issue of website translation. A major part of Right to Remain’s work is providing accessible legal resources: the Right to Remain Toolkit (a step-by-step guide to the asylum and immigration system); our Legal Updates blog includes our Covid updates webpage; our asylum navigation board and more.
We produce these resources so that people who are navigating the asylum and immigration system (and those supporting them to do so) can understand the legal process, what is required at each stage of the process, what their rights are, and what they can do to be in the best situation. The asylum and immigration system is very complicated, and it’s very important that people have knowledge of their own cases, how that fits into the legal process, and have some agency over what is happening. This is important on an ethical level, but also means that people are more likely to be able to access justice and make proactive, positive decisions about their legal cases.
Our users
Our priority users (and potential users) mostly do not speak or read English as their first language. Our resources are predominantly in English. We have translated sections of the Toolkit, and a series of multilingual videos made with Refugee Info Bus. Formerly translating all of our resources (which are updated often on a weekly basis, due to the rapidly changing legal context) would be an ambitious task. Indeed, the English-language resources are already a translation, with complicated legal language simplified as much as possible. Our compromise has been – in addition to getting key sections translated when possible – adding in a Google Translate bar into the Toolkit website, meaning people can translate the content of the page without having to go to another browser and open up Google Translate. We had never tested out with users or potential users, however, whether this worked for them.
Google Translate bar – a ‘good enough’ fix?
Over the last 18 months, Right to Remain has been conducting user research on digital behaviour with members of the asylum and migrant support groups we work with across the UK. It has been a fascinating learning process – and a baptism of fire on user research and usability testing! - thankfully mainly completed before the first Covid lockdown began.
How testing went
In this user research, the issue of language barriers had come up in almost all of the conversations. The participants had a wide range of digital literacy but the vast majority were comfortable using Google Translate as a tool, and largely aware of the drawbacks (some languages not available; inexact translation of some words or phrases). I’d also recently been talking to Translators Without Borders, who have a huge amount of understanding of Google Translate and other machine translation. They had pointed out that for some languages (eg French), Google Translate was nearly as good as human translation. Other languages are nowhere near this.
During the Explore programme, I did specific testing of how comfortable people (who are currently navigating the asylum process) were with automated machine translation. I tested whether people could spot the translation function on a website when it was the standard Google Translate format; and when it had been designed to have a different look.
100% of the people I spoke to could easily spot the Google Translate feature when it was the standard design (a design that is familiar to them from interacting with Google Translate in other ways online). There was also unanimity on preferring a drop-down list of languages rather than, for example, images of flags.
The research suggests that the Google Translate in-site bar, in standard design, was therefore a good “good enough” solution.
How do people google?
Having set myself a discrete problem and solved it very satisfactorily, I then foolishly allowed much bigger questions to plague my thoughts.
My usability testing had shown that people were comfortable with the style of automated translation on our website, when they were on the site - but how do people know to go there?
The vast majority of our Toolkit website users (more than 80%) come to it via a google search. More than 90% of the visitors to the Toolkit website have their web browser set to English, indicating either fluency or a high level of proficiency in the English language. Are we missing out on potential users who interact online not in English, could benefit from our online information, but don’t know they could access it auto-translated because they never make it to our site?
Googling in different languages
When I tested what happens when I googled in French (“demandeur d'asile uk”), almost all of the google search results were French-language pages, including Right to Remain’s translated French resources. You can then switch to English-results only, but would someone not confident in reading English do this? If someone is googling a term that in English is covered in our resources, but in a language for which we don’t have formerly translated resources, it seems they might not be directed to Right to Remain at all.
I then fell down a bit of a black-hole of trying to research multilingual google search behaviour. Google has attempted to address the challenges of this in its own algorithms, but what does google digital behaviour mean for how we prioritise translation?
I also explored the issue in further user research. Everyone I spoke to googled in English (even if they used voice notes, or the help of family members, to do so), therefore were unable to shed light on the question. My participants, were of course, somewhat self-selecting anyway as people more confident in English were more likely to say yes to having an English conversation about digital behaviour.
Some unanswered questions
I’ve been left with some fascinating but unanswered questions. What language do people less confident in English google in? What consequences does this have for reaching people via google searches for those of us producing online resources?
Another question is the value of auto-translation itself. One of the other organisations on the Explore programme, Bristol Refugee Rights, spoke to people about translation. The responses suggest that people are more confident in auto-translation for basic information, but less trustful of it for more complex information.
Bespoke professional translation (such as that provided for the multilingual information on the Scottish Refugee Council’s website) would presumably always be preferable … but would require huge resources (time and funds) for comprehensive resources that are updated on a regular basis.
When language barriers are frequently cited as obstacles to getting vital information, and when the majority of the people we are trying to reach will not have English as a mother tongue, translation is a big access issue. How best should we address this in our resource-limited context?
