Quite a lot of our work these days involves correcting machine transcription work (AI transcription), as it is not yet sufficiently accurate for most qualitative research work. Enquiries from researchers looking for assistance tend to include queries on cost, particularly if they have already got a transcript from their use of the AI transcription service.

We regularly conduct studies to check on the progress of AI transcription, and this is the most recent test we have undertaken, which relates to a piece of qualitative research.

Recent AI Transcription Test

We took a recent recording and sent it off to one of our competitors who offers AI transcription services. One of our senior human transcribers a) completed the transcription from scratch and b) proof read and adjusted the machine version.

We only used a short recording and the paragraph below is an extract from that recording.

AI (Machine Transcription) version of sample paragraph

Because I would always cheque the cost of something, because I know these things can be expensive, but even if there had been a cost of up to $150, we have income that we would have been able to wanted to pay for that. It did surprise me whenever it was up to 18 years. And I thought, well, I think that makes sense. I think that’s really lovely that they do that for me, for the [blanked for confidentiality] , the size it was, I didn’t think it would cause them a lot of well, I mean, yes, but such a small amount of matter. So, yeah, I was slightly surprised, but more surprised that they do it for kids up to 18. Yeah. Not so surprised for a piece because it’s such a small and they were lovely. They really made sure we had all the information we needed, the paperwork for us.

Human version of sample paragraph

PAR2: ..because I would always check the cost of something, yes – because I know these things can be expensive, but even if there had been a cost of up to £150 or £100, I would have still have paid it because we have income that we could have, you know, we would have been able to, wanted to pay for that. It did surprise me when I read that it was up to 18 years, and I thought, well I think that sort of makes sense. I think that’s really lovely that they do that. For me I didn’t fe- yes, for the [blanked for confidentiality] the size it was, I didn’t think it would cause them a lot of- Well the admin, yes, but such a small amount of matter. So, yes, I was slightly surprised but more surprised that they do it for kids up to 18. Not so surprised for a [blank for confidentiality] because it’s such a small – yes and they were lovely, they really made sure we had all the information we needed, did the paperwork for us, on our behalf.

Our transcriber had the following comments to make about the machine transcription version:

“I’ve proofed this one and changed it to our format. I’ve left the timestamps in, as if the client wanted timestamps on change of speaker. I have called the speakers PAR1 and PAR2 for ease of reference. The machine transcription software picked up three voices for some reason; there were only two people speaking. The machine also added a ‘Yeah’ or ‘Okay’ that speaker PAR1 said in the middle of PAR2’s speech, so it couldn’t clearly differentiate between the two speakers. It took me around 35 mins to change it over to our format and to proof (but if we were changing to adding in [our formatting] and deleting timestamps I’d say about 40 mins.”

Our transcriber then typed out the audio recording from scratch, without referring to the machine version.

“I had a go at typing up the [ ] file and it took me 25 mins to type it up (including double checking spelling) and then giving it a proper proof afterwards. Without the proof it took around 16-18 mins to type up.”

Conclusion

For accurate qualitative research interviews, it is still faster, cheaper and more efficient to use human transcribers. It took almost half the time to manually transcribe the file rather than correct all the errors in the machine version and then produce an accurate transcript.

The machine transcription was not at all useable and we think that when transcription companies (including ourselves) advertise ‘machine transcription polishing services’, they are likely to be offering a completely fresh transcription service and ignoring the AI version.