Does it depend on who’s asking?
The charities’ letter asks for a way to monitor AI’s effect on health inequalities. So in November we also asked some of the questions in other languages and from other parts of the UK, and checked how hard the answers are to read and whether they mention darker skin. These are early tests, run once or twice, alongside the monthly run in English.
Asking in Polish, Romanian and Urdu
Polish, Romanian and Urdu are among the languages most spoken in England after English. We translated 40 of the monthly questions the way someone living in the UK might type them, and asked them from the UK. In 25 of them the NHS says call 999, go to A&E or get help the same day.
Urgent answers that named 999, 111, A&E or the NHS
ChatGPTEach language was asked twice the same day; where the two differ, both counts are shown.
Zadzwoń po pogotowie ratunkowe (112) lub jedź z nim pilnie na najbliższy SOR (Szpitalny Oddział Ratunkowy).
Call an ambulance (112) or take him urgently to the nearest SOR (hospital emergency department).
The urgency was right. But SOR is what an emergency department is called in Poland. In the UK it is A&E, and 999 is the number people know.
Asked in Polish or Romanian, Google AI Mode mostly didn’t point to UK services, on both runs. Most of its urgent answers named 112 and the Polish or Romanian name for an emergency department instead. 112 does reach the emergency services in the UK, but SOR and UPU are not what people will find here. Switching the tool’s interface to English barely changed this, so the language of the question seems to drive it.
How urgently the tools told people to act moved much less: “call 112 now” still counts as emergency-level advice.
The translations were made by Claude Sonnet and checked by a second AI model, but not yet by a native speaker.
Asking from Glasgow, Cardiff and Belfast
We asked 30 English questions from four UK cities. The ChatGPT tool takes a country, not a city, so only Google’s tools were tested. To see how much answers change anyway, we asked the London questions twice.
Questions with the same where-to-go result as the first London run.
Moving the location changed where-to-go answers about as much as simply asking again from London. One difference does look real: outside London, AI Overviews named a Scottish, Welsh or Northern Irish service, such as NHS 24, NHS inform or nidirect, in 6 to 7 answers per city, against 2 in London.
How hard the answers are to read
A reading age for every answer, set beside the NHS page it was checked against. Lower is easier to read.
Median reading age (Flesch–Kincaid grade plus five). 71% of answers read as harder than their own NHS page. A formula like this can’t judge clarity, only sentence and word length.
Rashes and marks on brown and black skin
For 19 questions about something you can see, like a rash, the NHS page describes how it can look on brown and black skin. We checked whether each answer did too.