How Hard is it to add Closed Captioning to Internet Video?

There are hundreds of thousands of great videos on the internet, from Google to YouTube to Netflix, TV station websites with television shows, documentaries, web series, instructional videos and more.

THe vast majority of these are inaccessible to the deaf – they don’t have any form of Closed Captions or Subtitles.
Some companies only have a portion of their videos Closed Captioned, such as Apple’s iTunes & Netflix.

New FCC regulations will require all of these to have Closed Captions, by Sept 2013 at the latest (in the US).

Why Close Caption videos on the internet if only 15-20% of the population is deaf? Because deaf people aren’t the only ones who use Closed Captions. Young children who have no hearing disability, who watch TV with Closed Captions turned on learn to read faster, and get better at reading. They can see and hear the words being spoken.

People learning to speak English also benefit from Captions, as they can see and hear the words being spoken, and learn pronounciation and usage.

People watch videos on their phones while traveling, on the subway, bus, train and more, and often the background noise drowns out the sound. Closed Captions are a way for them to follow the video without having to turn the sound up to ear shattering volumes, or have sound drowned out completely.

So I decided to try an experiment. How hard is it to add Closed Captions to a video? I’ve never tried captioning a video before, and was curious to see what was involved.

Obviously, I didn’t want to spend hundreds of dollars on software just for an experiment, so I tried World Caption, for the mac.
It’s not a bad program, for a free one. You drag and drop a Quicktime .mov file into the movie window, and drag and drop the transcribed text file onto the Track section.

I picked an episode of Fresh Hell, downloading the video titled “Billionaire“, and the transcript for that episode.

After a few quick tries, I figured out what World Caption could, and couldn’t do. One of its biggest limitations is you cannot edit the content of a caption – you can only adjust the timing for when it’s displayed. You can’t add, delete, split or merge individual captions. You have to do all that in the text file you’re importing before you add it to World Caption.

Below are screen captures of the original Transcript I copied and pasted from the website on the left, and the edited version on the right.

Editing the trenscript was a two stage process. Stage One: remove anything that isn’t dialog (speakers names, visual cues etc), and changing hard returns to Carriage Returns. ‘Find and Replace’ did a lot of the work very quickly for me.

Stage two was breaking some lines of dialog into two or three lines. If a line of captioning is too long, it won’t all fit on screen.

So now I was ready to import files and start Captioning. The editing process took about 10 minutes.

After importing the file several times, I finally figured out I ought to set the preferences first, then import the text file. The text file has to be ascii, UTF-8 format in order to import properly.

Never go with a serif face for Closed Captions. Serif fonts are easier to read on paper, but on videos, especially small sized ones, they can be very difficult to read.

I turned off “Ignore Line Breaks” – this way World Caption uses Carriage Returns to separate each line of the text. Otherwise it breaks them up after X words. (whatever you set for Words per Caption).

So now I had something that looks like this to work with:

World Caption uses an algorithm to space the timing of each line of caption. One word lines have less time between them and the next line of dialogue. Long lines are given extra time before the next line is displayed. And lastly, none of the timing on the captions matches when it is heard on the video. That’s my job to fix.

World Caption has only one way to adjust the timing on a line of captions. Adjust Key Forward, and Adjust Key Backward. There’s keyboard shortcuts for both. And an incredibly annoying beep when you use it – there’s no way to turn off the beep or lower its volume without affecting the rest of the sound.

Using the Adjust Key function not only shifts the timing of the selected caption, but it also re-adjusts the timing of captions between Keyed captions. This has its drawbacks, and advantages.

The advantage of the Adjust Key function is that it will adjust all un-Keyed captions in between. The imported text sets the last line of dialogue at 11:22.71. In the video, that’s right at the end of the credits for the show, and about 1:40 past the last line of dialogue spoken. So by adjusting the first line, and last line, it also adjusts all the rest of the captions in between. Once a specific line is Keyed, its timing won’t be adjusted when you Key other lines. This is handy, but can also lead to accidentally keying a line to display after the next line (if it’s keyed), instead of before.

Once that was done, I started keying about every 10th line of dialogue, adjusting it to display when it was spoken in the video.

When you double click a line of dialogue, World Caption plays about 5 seconds of video, starting from the time displayed, shown to the left of the line of captions.

I’m going to bring up a couple of disadvantages of World Caption. I’m not knocking it – it’s free, so I wasn’t expecting it to do what some of the $200 – $1,400 Closed Captioning programs do.

The first disadvantage is that you can only adjust the timing of a caption by a full second ([00:00:01.000]). Most times, you want to adjust it as little as 1/24 or 1/25th of a second, depending on the frame rate of the video.

Second, there’s no way to set it so that it displays multiple lines at once. This is handy when several characters speak very quickly. You’d set the first speaker at the top line of captions, the second speaker below that. Otherwise, the captions will appear and disappear too quickly to read.

Third – there’s no way to set how long a caption should be displayed. Subrip (.srt), and SCC (Scenarist Closed Caption, .scc) both have start, and end times for a line of captions.

So once I had keyed every tenth line or so, I went back to the top, and started keying other lines in between those tenth lines, adjusting the timing of the captions. Sometimes I only had to key one or two lines, sometimes nearly all of them, depending on delivery of dialogue, and pauses between the spoken lines.

This whole process took about 40 minutes, for what is nearly 10 minutes of dialog – adjusting the timing, learning the program, playing back the video preview with captions etc.

One thing that made the job easy is that I had a transcription to start with. If I’d had to transcribe everything from the video, it would have taken at least an hour and a half, and would have errors. I’m deaf, and some things definitely would have been incorrectly transcribed.

On the other hand, most film, TV and web shows all have a script saved, which can be edited for creating closed captions.

Here’s a screen capture of the video with Closed Captions showing, and the little “CC” toggle in the lower right:

My next problem was figuring out how to set up the video so that you could toggle the captions on and off.

I found a Quicktime CC sprite movie online, copied and pasted it into the captioned movie, then saved the file. Clicking on the “CC” image in the video turns captions on and off.

If I convert the Text Track to Scenarist Closed Caption format, the Quicktime Video Controller should display a built-in “CC” button to toggle captions on and off.

As far as movies and television shows go – they’re all already Closed Captioned for television broadcast (it’s mandatory by FCC law), and subtitled or Closed Caption (often both) for DVD and Blueray, so there’s no excuse for not having them captioned on the internet – it’s sheer unwillingness and laziness on the part of film studios, networks and websites that won’t include Closed Captions.

Once the project was done, I exported it from World Caption, opened the video in Quicktime, and hit play. The timing of the captions aren’t perfect due to limitations of the World Caption software, but they work.

In the end, it’s neither difficult, nor expensive to Close Caption videos on the internet.

Below is a 26-second clip showing the Closed Captions, and the ability to toggle it on and off (requires Quicktime Plugin). The on-screen CC toggle works in Firefox, and Safari on Snow Leopard. If you’re using OX 10.5 (Leopard), you have to control-click the video and select “Load Quicktime” – it defaults to html5 video to play it, which screws it up (Thanks Safari 5.06).

I’m not posting the entire video with all the Closed Captions, as that violates all sorts of copyright laws. A short except falls under fair use for education purposes.