THe vast majority of these are inaccessible to the deaf – they don’t have any form of Closed Captions or Subtitles.
Some companies only have a portion of their videos Closed Captioned, such as Apple’s iTunes & Netflix.

New FCC regulations will require all of these to have Closed Captions, by Sept 2013 at the latest (in the US).

Why Close Caption videos on the internet if only 15-20% of the population is deaf? Because deaf people aren’t the only ones who use Closed Captions. Young children who have no hearing disability, who watch TV with Closed Captions turned on learn to read faster, and get better at reading. They can see and hear the words being spoken.

People learning to speak English also benefit from Captions, as they can see and hear the words being spoken, and learn pronounciation and usage.

People watch videos on their phones while traveling, on the subway, bus, train and more, and often the background noise drowns out the sound. Closed Captions are a way for them to follow the video without having to turn the sound up to ear shattering volumes, or have sound drowned out completely.

So I decided to try an experiment. How hard is it to add Closed Captions to a video? I’ve never tried captioning a video before, and was curious to see what was involved.

Obviously, I didn’t want to spend hundreds of dollars on software just for an experiment, so I tried World Caption, for the mac.
It’s not a bad program, for a free one. You drag and drop a Quicktime .mov file into the movie window, and drag and drop the transcribed text file onto the Track section.

I picked an episode of Fresh Hell, downloading the video titled “Billionaire“, and the transcript for that episode.

After a few quick tries, I figured out what World Caption could, and couldn’t do. One of its biggest limitations is you cannot edit the content of a caption – you can only adjust the timing for when it’s displayed. You can’t add, delete, split or merge individual captions. You have to do all that in the text file you’re importing before you add it to World Caption.

Below are screen captures of the original Transcript I copied and pasted from the website on the left, and the edited version on the right.

Editing the Transcript for importing into World Caption.

Editing the trenscript was a two stage process. Stage One: remove anything that isn’t dialog (speakers names, visual cues etc), and changing hard returns to Carriage Returns. ‘Find and Replace’ did a lot of the work very quickly for me.

Stage two was breaking some lines of dialog into two or three lines. If a line of captioning is too long, it won’t all fit on screen.

So now I was ready to import files and start Captioning. The editing process took about 10 minutes.

After importing the file several times, I finally figured out I ought to set the preferences first, then import the text file. The text file has to be ascii, UTF-8 format in order to import properly.

World Caption preferences settings.

Never go with a serif face for Closed Captions. Serif fonts are easier to read on paper, but on videos, especially small sized ones, they can be very difficult to read.

I turned off “Ignore Line Breaks” – this way World Caption uses Carriage Returns to separate each line of the text. Otherwise it breaks them up after X words. (whatever you set for Words per Caption).

So now I had something that looks like this to work with:

World Caption window, after importing movie file & text

World Caption uses an algorithm to space the timing of each line of caption. One word lines have less time between them and the next line of dialogue. Long lines are given extra time before the next line is displayed. And lastly, none of the timing on the captions matches when it is heard on the video. That’s my job to fix.

World Caption has only one way to adjust the timing on a line of captions. Adjust Key Forward, and Adjust Key Backward. There’s keyboard shortcuts for both. And an incredibly annoying beep when you use it – there’s no way to turn off the beep or lower its volume without affecting the rest of the sound.

Using the Adjust Key function not only shifts the timing of the selected caption, but it also re-adjusts the timing of captions between Keyed captions. This has its drawbacks, and advantages.

The advantage of the Adjust Key function is that it will adjust all un-Keyed captions in between. The imported text sets the last line of dialogue at 11:22.71. In the video, that’s right at the end of the credits for the show, and about 1:40 past the last line of dialogue spoken. So by adjusting the first line, and last line, it also adjusts all the rest of the captions in between. Once a specific line is Keyed, its timing won’t be adjusted when you Key other lines. This is handy, but can also lead to accidentally keying a line to display after the next line (if it’s keyed), instead of before.

Once that was done, I started keying about every 10th line of dialogue, adjusting it to display when it was spoken in the video.

When you double click a line of dialogue, World Caption plays about 5 seconds of video, starting from the time displayed, shown to the left of the line of captions.

I’m going to bring up a couple of disadvantages of World Caption. I’m not knocking it – it’s free, so I wasn’t expecting it to do what some of the $200 – $1,400 Closed Captioning programs do.

The first disadvantage is that you can only adjust the timing of a caption by a full second ([00:00:01.000]). Most times, you want to adjust it as little as 1/24 or 1/25th of a second, depending on the frame rate of the video.

Second, there’s no way to set it so that it displays multiple lines at once. This is handy when several characters speak very quickly. You’d set the first speaker at the top line of captions, the second speaker below that. Otherwise, the captions will appear and disappear too quickly to read.

Third – there’s no way to set how long a caption should be displayed. Subrip (.srt), and SCC (Scenarist Closed Caption, .scc) both have start, and end times for a line of captions.

So once I had keyed every tenth line or so, I went back to the top, and started keying other lines in between those tenth lines, adjusting the timing of the captions. Sometimes I only had to key one or two lines, sometimes nearly all of them, depending on delivery of dialogue, and pauses between the spoken lines.

Sometimes you only need to Key a few lines of captions.....

...sometimes I had to Key almost every line of captions.

This whole process took about 40 minutes, for what is nearly 10 minutes of dialog – adjusting the timing, learning the program, playing back the video preview with captions etc.

One thing that made the job easy is that I had a transcription to start with. If I’d had to transcribe everything from the video, it would have taken at least an hour and a half, and would have errors. I’m deaf, and some things definitely would have been incorrectly transcribed.

On the other hand, most film, TV and web shows all have a script saved, which can be edited for creating closed captions.

Here’s a screen capture of the video with Closed Captions showing, and the little “CC” toggle in the lower right:

Quicktime video with "CC" toggle for turning Closed Captions on and off.

My next problem was figuring out how to set up the video so that you could toggle the captions on and off.

I found a Quicktime CC sprite movie online, copied and pasted it into the captioned movie, then saved the file. Clicking on the “CC” image in the video turns captions on and off.

If I convert the Text Track to Scenarist Closed Caption format, the Quicktime Video Controller should display a built-in “CC” button to toggle captions on and off.

As far as movies and television shows go – they’re all already Closed Captioned for television broadcast (it’s mandatory by FCC law), and subtitled or Closed Caption (often both) for DVD and Blueray, so there’s no excuse for not having them captioned on the internet – it’s sheer unwillingness and laziness on the part of film studios, networks and websites that won’t include Closed Captions.

Once the project was done, I exported it from World Caption, opened the video in Quicktime, and hit play. The timing of the captions aren’t perfect due to limitations of the World Caption software, but they work.

In the end, it’s neither difficult, nor expensive to Close Caption videos on the internet.

Below is a 26-second clip showing the Closed Captions, and the ability to toggle it on and off (requires Quicktime Plugin). The on-screen CC toggle works in Firefox, and Safari on Snow Leopard. If you’re using OX 10.5 (Leopard), you have to control-click the video and select “Load Quicktime” – it defaults to html5 video to play it, which screws it up (Thanks Safari 5.06).

I’m not posting the entire video with all the Closed Captions, as that violates all sorts of copyright laws. A short except falls under fair use for education purposes.

In your settings, under General, select 'Restrictions' to find these two settings.

If you have kids, and want to save yourself money and aggravation, turn on the two settings highlighted in red in the image to the left. You’ll find them in Settings -> General -> Restrictions.

The default setting for In-App Purchases is ON, and Require Password is 15 minutes.

This means if your child your iPhone, iPad, or iPod they can inadvertently purchase new apps without requiring your password for up to 15 minutes. They can also make indefinite, and unlimited In-App Purchases. Sometimes this has caused people to receive bills of hundreds, or even thousands of dollars.

Some people want to blame Apple, or Developers for this, but since it’s your device, it’s up to you to make sure you know about these settings.

Save yourself money, time, and aggravation, by setting In-App Purchases to OFF, and Require Password to Immediately.

OPlayer – there’s three different versions:

• OPlayer lite, a free version that has small ads above the movie.
• OPlayer – the paid version for iPhone & iPod
• OPlayer HD – the iPad version.

These all support smi and srt subtitles. You transfer the movie and subtitle via iTunes, then play. It is a little finicky about subtitle formating – it will fail to display any subtitles with <i> or </i> in the coding (usually used for italic text).

More information about the three flavors of OPlayer

ServeToMe & StreamToMe

StreamToMe is a video player that also plays srt subtitles. This app requires  the server app, ServeToMe to be running on your mac. ServeToMe streams your movies and subtitles, while StreamToMe plays them on your iOS device. This lets you to watch movies from your mac wirelessly, without having to transfer them to your iPad, iPod or iPhone. Great for saving space on your device.

More information about StreamToMe & ServeToMe

EyeTV

I bought this several years ago when I was looking for something to record tv shows on my mac. I was getting tired of VCRs and videocassette and wanted something better. This was the answer. One advantage it had over its competitors was the it recorded Closed Captions. You could turn them on or off, and if you export to iPhone/iPod/iPad, it also exported the Closed Captions, which could also be turned on or off on your iOS device. However, their iOS app…. don’t buy it.

More information about recording television shows with EyeTV

EyeTV app

The  EyeTV app allows wireless streaming of live or recorded TV shows from your mac to your iOS device. The biggest problem with this is that it doesn’t stream Closed Captions.

We recently switch from Shaw Cable to Telus’ Optik TV, and have a PVR and a digital box for our old TV. I discovered a workaround for EyeTV’s lack of Closed Caption Streaming. Normally your television set decodes the line 21 information that contains the Closed Captions. Telus digital boxes hardcode captions into the video feed before it reaches your TV. The TV set doesn’t do the line 21 Closed Caption decoding – it’s already done.

So once I hooked up our macbook & the EyeTV USB transcoder to the Telus Digital box, “Closed Captions” are streamed to my iPod or iPad. I use “Closed Captions” with quotes, because in this case it’s more like hard subtitles – if you turn off Closed Captions on the digital box, you lose them on your iOS device. If you have a digital box, then this iOS app *might* work for you. Otherwise, it’s a waste of $5 due to lack of Closed Caption support.

More information on the EyeTV iOS app.

You can download version 1.3.2 here.

The first change – the program is much faster. It takes more time to import a .srt file (about 10 seconds), but that’s now where the bulk of the work is being done. The actual conversion to dfxp is about five times faster, so it takes less time if you need a few tries to adjust the subtitle timing.

The next change – adjusting the timing now shows in the ‘Imported .srt subtitle file’ window. It shows ‘Time Start’ and ‘Time End’ for subtitles, and when you adjust the timing, it’s updated immediately.

The third change is an option to save modified .srt subtitles if you want to use it to adjust subtitle timing only for .srt subtitles.

The fourth change – I added another button – “23.976 to 25 fps” Some movies are not converted  to 23.976 seconds from 25 fps – this would mean dropping frames, causing a ‘stuttering’ effect on the video. So they’re re-encoded as 23.976 – meaning the movie runs longer than it would if it were converted. If subtitles are going further and futher out of sync, appearing earlier and earlier than characters speak as the movie goes on, you may want to try the convert ”23.976 to 25 fps” button. It will adjust the subtitles accordingly.

I found this while trying to watch “Friday the 13th part III” (yes, I like horror movies ). I was getting frustrated with subs that kept going further out of sync as the movie played. So I started researching, and experimenting.

Once I finally figured it out and implemented it, the netflix movie & subs worked like a charm. Below you can see the last line of subtitles highlighted in blue, just as I click the  ”23.976 to 25 fps” button.

Before converting '23.976 to 25 fps'

After, you can see the result – it’s adjusted all the subs, as you can see below:

After pressing the "23.976 to 25 fps" button

Now these subtitles almost sync up with Netflix’s “Friday the 13th Part III” streaming movie. It’s still off a little bit. I use the ‘Adjust Subtitle Timing’ to fix this. You can either type the amount you want to adjust the subtitle timing by, or use the up/down arrows. Before is below:

Before 'Adjust Subtitle Timing'

After, you can see the times have had 1.5 seconds added to them:

After 'Adjust Subtitle Timing'

And last, converting to dfxp format takes just a couple of seconds, instead of 20-30 seconds.

.srt file is now converted to Netflix' .dfxp format

All you have to do is save the file. Get your Netflix movie started, pause the movie, and load the Custom DFXP file to enjoy subtitles with your netflix movie.

You can download SRT to DFXP Converter version 1.3 here.

You can find version 1.2 of srt to dfxp Converter here.

Odds are, you can find a good subrip file (.srt) using a search engine. However, Netflix, via Microsoft’s Silverlight only accepts captions in .dfxp format. There’s a selection of Netflix ready subtitles on this website, but it’s a lot of work to build a catalog of movie subtitles when Netflix has over 17,000 movies and TV shows.

If you’re on a PC with windows, there are programs that will convert .srt files to .dfxp format for use with streaming Netflix videos. You can find one on Josh Erickson’s website, called Subvert. Until now, if you were using a mac with OS X, you had to use a Python script in the Terminal.

Netflix has added Canada to streaming video, but again, few subtitles are available. So over the weekend I wrote an OS X program to convert .srt files to .dfxp format, and tested it out with the movie, “Lifeforce” on Netflix, streaming to my wife’s MacBook. I also tested the .dfxp file on a PC as well, and it works on both, as you can see in the image below:

My SRT to DFXP converter for mac OS X also has the ability to delay subtitles in case a movie or TV show has an intro – you’ll have to time how long the intro is, and adjust accordingly.

Once you’ve created your .dfxp file, to access the subtitles, start your movie on your mac, pause it, and hit Control-Option-Shift-M to access a pop-up menu. Select “Load Custom DFXP,” and select your converted dfxp file. Continue playing the movie and enjoy having subtitles playing on your Netflix streaming movie.

Not all .srt files are the same – different files may have different timing indexes, which means the longer a movie plays, the further off the subtitles will be from what they are saying, so you may have to try converting several different .srt files that you find.

You can find SRT to DFXP Converter here.

So until Netflix adds proper Closed Captions to their movies, you now have a work-around to watch a movie with subtitles.

Special thanks to @Codeman38 on twitter for all his help and information that made this program possible.

If you have any problems, feel free to contact me through this website, or through my twitter profiles, @Sweeney_Games or @MarkSweeney

To start with, you’ll need an Apple iPhone Developer License – this is $99 US.

You’ll also need Snow Leopard and the Apple Software Developers Kit, or SDK for short. There’s no way around these two – you’ll require a an Intel Mac that runs Snow Leopard in order to compile your apps to run on an iPhone, iPod, or iPad, and you’ll need the Developer’s License to be allowed to install games on your hardware for testing, and to upload to the App Store.

For the game programming, I used Unity iPhone 1.7, and Unity 3.0 iPhone Beta 3.04, in conjunction with the latest Apple SDK to build, test and install apps on my ipod (iOS 4.1).

To create the 3D art, I used three different software packages. For modeling and texturing, I used Strata 3D 6.12. For UV mapping of 3D meshes created by Strata 3D, I used Silo, from NeverCenter. Finally, I exported the textured meshes from Strata 3D into Wings 3D as an OBJ file. In Wings 3D, I’m able to separate the component mesh into separate elements, and export as an .fbx file which imports into Unity 3D iPhone 1.7 & 3.0 seamlessly.

Lastly to create the art to apply to 3D meshes and textures, I used both Illustrator CS3 and Photoshop CS3. Illustrator is great for vector art. While you can’t apply vector art to 3D meshes, you can import them into Photoshop at any size you want and still have the images/textures show up sharp, clean and crisp, no matter what the resolution or size. If you do them solely in Photoshop, you can only size images down. If you need to size them up, you’re going to end up with ugly artifacts from interpolation.

This way I can create art in Illustrator, and size it up or down as much as I need, without losing quality. So if I decide to make a game for the iPad as well, which has a screen size of 1024-by-768 pixels (vs 480-by-320 pixels for the iPod/iPhone), I don’t have to recreate art from scratch – I can resize, and re-import it into Photoshop ready to go.