Live subtitling errors on TV are something deaf people have had to grow very used to.
They turn live news or sports programmes into a frustrating guessing game. Way back in 2008, I wrote this diary of a week watching TV to show how many mistakes crop up on screen, and I’m sad to report that subtitles haven’t improved in that time. (To get an idea of just how many errors there are, take a look at this site dedicated to subtitling mistakes.)
But if there is an upside, it’s that every now and then hilarious mistakes crop up.
Back in January, I spotted one such gem on BBC Breakfast. During an item about breeding pigs, a roving reporter explained to the presenter that pigs “love to nibble anything that comes into the shed, like our wellies”. Unfortunately, the subtitles came up as pigs “love to nibble anything that comes into the shed, like our willies” instead. After bring uploaded to Twitter, it even made it into the Guardian.
Last week, the issue came up again, with both the Telegraph and The Mail having a lot of fun (along with doing a bit of ahem, borrowing from yours truly) featuring subtitling errors like the Archbishop of Canterbury being called ‘the arch bitch of Canterbury’ and Ed Milliband being called ‘Ed Miller Band.’ The topic also made it onto the extended version of Have I Got News For You on Sunday.
Interestingly, both articles laid the blame for the subtitling errors at the door of the BBC.
No doubt, the BBC, as a public broadcaster, should be seen to uphold the highest standards when it comes to subtitles, but most deaf people will tell you that mistakes are not limited to Auntie’s shows. They happen on whatever channel live shows are being broadcast on (check out ITV News, Channel Four News or Sky News, and you’ll get the idea).
Additionally, subtitles across the broadcasters (the BBC included) aren’t created in-house. They’re provided by a handful of companies who tender to provide the work for different channels. Companies like Red Bee Media, IMS Media and ITFC, several of whom I spoke to when I wrote my subtitle diary.
The bigger issue is the method by which subtitles are created.
They’re made either by a stenographer typing what’s been said onto a phonetic keyboard as they listen to a show, or by speech recognition, where dialogue is repeated more clearly into a microphone during a programme, with a computer automatically recognising the words.
It’s speech recognition that seems to lead to the most mistakes, as the computer throws up words that sound similar to the right one. ‘Bitch’ instead of Bishop, and so on.
So aiming complaints at the BBC is to be missing the point. What would have been more useful for deaf people would have been for those papers to take a detailed look at subtitle providers and discover which companies (irrespective of channel or broadcaster) provide the most accurate live subtitles. Or look at which method, of stenography or voice recognition, was the most accurate, and to what degree.
That would have helped deaf people out, and in turn held those companies to account.
But it also would have taken up a lot of time and effort. And potential headlines like ‘Deaf viewers complain about standard of random company you’ve never heard of’s live subtitles’ wouldn’t have been nearly as catchy as simply blaming the BBC.
Update: A stenographer wrote this comment on my Facebook wall:
THANK YOU CHARLIE! If I read another “live subtitlers are lazy feckless arses who can’t spell”-type article I think I’ll scream! The problems are inherent in the system: use of respeakers, low pay which means stenographers are not attracted to this type of work unless they have to work at home for personal reasons, and a lack of information for live subititlers which means that when I sit down to subtitle a programme, I know as much about it as the viewing public! No people, we don’t get scripts! But at its most basic, this is the point: live subtitles are produced by people, and people make mistakes. If I had written an absolutely perfect hour, which included the error “Ed Miller Band”, would people still talk about it? Probably. There is no better way of making live subtitles at the moment, so accept the limitations, laugh at the errors and look forward to the future.
Charlie Swinbourne 
I get the point, but not one I necessarily entirely agree with. Deaf people are the end consumers, your contract or service provider is the BBC / ITV or whomever, and they are the ones who are airing the subtitles.
The BBC has purchasing power, and enters into a contract with subtitling providers. They need to use their bargaining position / contractual or puchasing power, to demand standards.
In the same way, if you buy a laptop and it is not up to standard – say there’s an issue with the motherboard. You as the end consumer don’t go and seek out manufacturers of the motherboard, and get into technicalities over motherboards, start petitioning motherboard manufacturers. Your focus stays with the end product as it ships.
Subtitling is part of a programme package. Hearing people would not be expected to find out the brand of microphones the BBC happens to use, contact the said manufacturers etc. They’d just contact the BBC and say there’s an issue with the sound and be done with it.
Why do should deaf people to live their lives knowing the ins and outs of whatever, and have a different consumer experience?
Sure – but if we take your analogy about the motherboards, what would be happening is that motherboards across a range of laptop manufacturers would be showing faults, yet the media are deciding only to highlight problems with one manufacturer’s motherboard (!)
In this situation, the standard of the BBC’s live subtitles are roughly equal to the standard of live subtitles across a range of other channels and broadcasters so it doesn’t make sense for the focus to be on just one broadcaster.
If the media could understand the issue (of how live subtitles are created) and if it were identified that a certain company’s standard was lower, what I’d hope would happen as a result is that the BBC and all the other broadcasters would demand improvements from that company, or take their business elsewhere.
Btw, at no point was I saying that the consumer/deaf person should be looking into the providers – I was saying that the media could take a more detailed look at it rather than using it as a stick to beat the BBC with.
With mass access comes lower standards, its sods law. Part of the problem is the Americanisation of captioning and titles, yankee software ? So often the spelling errors are down to that. Perhaps we shouldn’t have let them have independence but give them spelling lessons… I’ve got so used to the many errors I can fill in the gaps myself fairly easily. It could be harder for the deaf signer, but he or she would have English issues anyway. As was said it isn’t just the BBC, Sky is dire as well. Their news is un-watchable because they flood the screen with text all over the place and have scrolling titles as well. I can understand why they then decided sign was a non-start as well…..
Although it took the hearing lobby to get that off. During the sports I turn titles off, because they obscure the picture, I’d like to see positioning changed. ITV when the rugby was on, the text was slap in the middle of my screen I HAD to turn it off to see the action, sport isn’t really an issue because I prefer to watch what is going on, not read the inanities of the presenter.
Who would have thought the deaf would be the champions of plainer English lol…. To an degree we must accept an percentage of errors. It’s still 90% better than trying to lip-read them !
I worked with deaf people in social work for many years, I sympathise with the people who are working on subtitles especially with regard to the comment that “subtitlers do not have specific knowlegde on every subject under the sun” Sign language interpreters have very similar problems, when signing/interpreting a lecture or a talk at some function. The speaker generally has never worked with nor understands the compexities of interpreting spoken language into sign and the associated difficulties. This places both the interpreter and the target deaf audience in very dificult situations.
Confusing post Robert ! Subtitlers just have to type word for word what is being said, they need only that skill, not an in-depth knowledge of what is talked about. As regards to sign, most sign is intelligible via terps on screen and given the wide variations regionally. I cannot name anyone who can follow presenters on See HEAR at all without captioning…. The problem there is hearing translators using ‘artistic licence’ instead of of translating exactly what is being said or done. Deaf ones resort to local concepts… For my money there is nothing complex if they use Signed English, and not whatever they ‘think’ BSL is. Contrary to popular spin, deaf can follow SE just as well as their local signs…. It’s more about deaf politics (Sign), than communication.
I’m afraid your two postings exhibit a lack of how captioning systems operate and under what limitations. When you refer to the Americanization of captioning, the first effort to caption unscripted programming simultaneous with its occurrance was actually a British Palantype based stenographic system used by BBC to caption the Royal Wedding (Charles and Diana.) It was indeed a disaster. The first truly successful system that followed was a stenotype based one used initially in the United States to caption the unscripted portions of the 1982 Academy Awards Presentations on ABC Television. I am totally familiar with the development of that and subsequent systems, as I was part of the team that developed that system and was the stenotypist that captioned that first program, as well as many other such programs. I also played a significant role in establishing the first such system to be used to caption news programming on Canadian English language television, and subsequently was one of the owners of the largest captioning company in the USA.
The BBC programming I view in the USA is all scripted and captioned in the USA, so it is difficult for me to comment on the errors that occur within BBC’s live (we call it “real time”) captioning. But I can tell you that whether the system is stenographic based or voicewriter (repeater) based, errors will occur during the translation process. The number of those errors will vary based on a variety of factors, the most significant of which is the skill of the operator of the system and the nature of the program. It has nothing to do with whether they are speaking British, Canadian, Australian, or American English, because the operator of a stenographic system controls the dictionary creation process and speech recognition software has different packages for different English language dialects. I have seen programs captioned with 99.5% verbatim accuracy in the USA, but I’ve also seen some captioned at under 95.0% verbatim accuracy. Some public affairs programs have a host and several guests that constantly interrupt and speak over each other, at extraordinarily high rates of speed (words per minute) who are virtually impossible to understand, let alone caption. Even then, a highly skilled captioner, with the proper training, will produce more readable captions than a rank amateur.
I can go on further, but space does not permit me to. I will sum it up by pointing out that if BBC is not equaling the levels of verbatim captioning on its American network counterparts, the fault has nothing to do with the system, whether of American or British design and manufacture, but rather it lies with the BBC and the skills of its captioners. But again, whatever the system that is used, consistent verbatim perfection will never be achieved, and there will always be an occasional faux pas kicked out; which in the end, gives us something to laugh about.
Thanks for this Martin, no doubt you’ll get a reply from MM shortly.
I just wondered how the American system is measured? When you say 95% accuracy, is that referring to individual words or to lines of text?
It’d be useful in terms of looking at the standards in the UK.
Depends on if the operator is captioning or the software is doing it on its own. Some BBC output is speech to text-wise with all its dodgy variations. THAT software is often American. I refuse to use american dictionary software personally. As I pointed out the odd 5% does NOT bother me, 100% is not really feasible. I’m more concerned with the positioning of text. I attended the BBC (Wales), captioning services so I do see how they operate.
American 95% accuracy is based on american spellings to american viewers, (And probably ignores the fact there are 4 major other languages there !), but this would be lower in the UK who at least know where the vowels do go, how english grammar works. Reading the text on TV can improve an deaf person’s English knowledge, it’s really more important an aspect for them given they will never get signed access on par.
Canadian English language spelling falls somewhere between British and American English. The bulk of the realtime captioning in the U.S. and Canada is still produced by stenotypists. Canadian stenotype captioners use the same American developed systems as the British captioners and they output the Canadian (British-like) spelling. Unlike the original captioning systems which utilized what were called universal dictionaries, modern day captioners fully control their translation dictionaries. If they put “labour” into the dictionary as the English output equivalent of every one of their machine shorthand combinations for that word, they should never get “labor” as the translation output — never. I think one of the problems is there might be some American real time captioners who are working in the U.K. that have not replaced “labor” with “labour”, and so forth.
Answering another question, the 95% verbatim accuracy is based on the totality of the words in a specific section of text. On the National Court Reporters Association examinations, where most tests are five minutes in duration, it is 95% accuracy over the course of the entire five minutes of dictated text. At VITAC, when I was one of the owners of the company, we measured verbatim accuracy as a percentage of an entire broadcast. While I think most companies measure it that way, there’s a possibility that some measure it based on a segment of a broadcast, or some other standard. Incidentally, not to confuse matters worse, but there is a substantial amount of Spanish language realtime captioning in the U.S., in addition to English. There are multiple Spanish language networks and many local stations, as well.
Television delivery is still somewhat different in North America from in the U.K., though at least the technical standards are becoming world-wide. In my home our television delivery is over cable and we have a choice of over 400 network (national) and local programs. A small amount of that is only music or text delivery, but the bulk of it is actual programming (some of which is pure trash in any case.) I can watch football (soccer) from the U.K. almost 24 hours a day, whether it is live or a repeat, and it is almost 100% captioned, by law. There’s an awful lot of realtime captioning being performed in the States compared to when there was just little old me in the beginning. Unfortunately, as in other places, the quality varies quite a bit. Most of the prime time network and major cable company captioning is very good, but the real time captioning, other than on the networks, CNN, and the Weather Channel, during non-prime time hours, is all over the board in terms of being verbatim and free of mistranslations and untranslates.
A brilliant post that I’m sure gives a lot of people an insight into how the subtitling software we use today works. It must be very frustrating for users, and in the next few years I expect some major developments.
Just to pick up on a point here, when respeaking live programmes one of the most important factors in producing accurate subtitles is familiarity with the material. (I’m not referring to pre-prepared typed subtitles). For example, if you love football and you follow it religiously, you will inevitably recognise all the nuances of the commentary and analysis much more readily than if you were subtitling a more obscure sport which receives less coverage. I can’t stress how important familiarity with the material is. Imagine you are watching a film you have seen before. You kind of know what’s coming next, don’t you? Maybe not the exact lines in the script, but you know roughly where the story is going. It requires less concentration to follow the story and you will often notice other aspects that you didn’t the first time you saw the film. The same applies with subtitling live programmes. If you know what to expect it requires less focus on following the theme of the programme and so you can concentrate more closely on the quality of the subtitles. When a subtitler is unfamiliar with the subject material of a programme there will inevitably be a trade-off between your concentration on the material you are hearing and the output you are creating.
Re: complaints about American software. If you don’t like it, don’t use it.
Thank you for the sensible critique. Me and my neighbor were just preparing to do some research on this. We got a grab a book from our area library but I think I learned more clear from this post. I’m very glad to see such wonderful information being shared freely out there.
Charlie, first of all apologies to all readers for my English. I am not a native. Then, thanks for focusing somebody’s attention to live subtitles. I have written extensively on live subtitles through speech recognition (respeaking) as a linguist and a professional and can confirm that mistakes are there. This is a quite evident fact. Other facts are in the replies of your readers who try to explain why mistake happen (low costs, lack of interested good stenographers – who also make less funny mistakes – etc.). Something which is not that stressed here, however, is that live subtitlers do a crazy job, whose results cannot be measured in terms of accuracy (95% seems good but it is not, because it means that a word out of 20 is misspelled. And if speech is calculated to be at 180 wpm, this means 9 mistakes a minute!!), but in terms of usefulness. Because live subtitling is a process and a product at the same time, like impromptu speech. And in a conversation many mistakes are made, but the message can still be guessed. In simultaneous interpreting the same happens. The interpreter also makes mistakes (some of them are evident like grammar mistakes or mispronunciations, but others are not like mistranslations), but nobody blames the interpreter because of comprehensible, “difficult to highlight” mistakes. This happens because we are used to mistakes in an oral discourse. On the contrary we are not used to mistakes in the written form. That is why when faced to a mistake in the written form we more fiercely react. And respeaking is much more an oral than a written process (because word is simultaneously and orally translated into the written word by means of an operator interacting with a machine which translates speech into text). Its problem is that the final result is written. By this I don’t want to say that one should not care of mistakes, but that they should be seen in a more comprehensive way and evaluated in communicational terms not after a comparative analysis of the “a posteriori” transcription of two oral texts. Sure, respeaking can be done more properly (for example by adding an editor working alongside the respeaker, or by reformulating, or by introducing an antenna delay which allows editors to edit the text produced by the respeaker and to broadcast it perfectly in synch with the text to be subtitles, or again by investing more in speech recognition technology, etc.) but this costs lots of money and lobbies to accept compromises. Moreover, it does not automatically brings good perfect results. If in the future a solution will be found and there will be ways to avoid such mistakes, much better. But for now it is the best solution to a problem, that of deafness, which prevents people from getting a message and from being included in the society. So live subtitles DO deaf people a favour, even a ver important favour, those of access to information, entertainment, TV in general and consequently that of social inclusion…
Excellent point. However, I doubt it will be any time soon. We were experimenting with voice recognition over ten years ago and it’s hardly progressed.
Clearly MM has no understanding of the process of subtitling as well as a weak grasp of the grammar and spelling himself. There is no major difference in quality between US and UK subtitles, just different ideas of what the viewer wants. And US companies do tend to use more bodies rather than the voice recognition technology the BBC and other UK companies are employing. The problem is that this technology has troulble understanding a news presenter speaking clear English, let alone a Liverpool dock worker and his Glaswegian girlfriend yelling at each other on the Jerry Springer/Jeremy Vine show while the audience whoops. It will be years, if ever, before the technology is capable of getting high-quality live subtitling. However it is a lot cheaper than paying someone and only marginally worse. To keep costs down, as all media companies are under intense financial pressure, it’s simple econonomics to invest in the new technology. And even when real human beings are used, there are many other issues, from familiarity with subject matter to the standard of equipment being used. Even state of the art technology can’t do anything about background noises and poor sound quality. Even the switch from analogue to digital about 15 years ago played a big part in subtitling errors. Digital is by its very nature inferior to the range in pitch and tone that analogue offers, For pre-recorded material, when we switched to using servers instead of using tape, we had enormous issues with sound quality. They were so bad that we’d go and listen to the original recording on Betamax to get much clearer sound. To be clear there is no simple way to produce decent live subtitles, because we don’t all speak like BBC or CNN presenters and we have enough trouble understanding each other without entrusting it to computers.
The current process is NOT the best way of subtitling live TV. You just need a properly trained person and a keyboard. Years ago when I was in the RAF I was trained to sit with my headset on and type what I heard at the speed it was being spoken. I had scores of colleagues who did the same. We were expected to produce 100 percent accuracy. Using an automatic system is a nonsense and unnecessary. It is also an insult to those who need subtitles.