Mark Sweeney
Author Archives: Mark Sweeney

Subrip to iTunes Text Track (.srt to iTT) converter

This is one of a few accessibility programs I’ve been working on.

This application imports subrip files and converts them to iTunes Text Tracks for YouTube.

Advantages it has over Final Cut Pro X is it directly imports subrip files, and the exported iTT file works properly on YouTube.

Currently Final Cut Pro X exports iTT captions with a default cellResolution of 32:15, a holdover from CEA-608 Closed Captions which had a limit of 32 characters per line, and a maximum of four lines of text.

Two side by side images of video displaying similar captions on YouTube. The one on the left shows the closed captions centered in the video, while on the right captions are on the left side of the video.

One the left is how captions are supposed to display in YouTube, on the right is how captions with lines longer than 32 characters show on YouTube.

Currently, YouTube shunts captions to the left side of the video if any line of captions is has more than 32 characters, as shown below, on the right.

This program sets the limit to 48 characters per line, and warns you when a caption exceeds that limit so you can avoid having YouTube screw up the positioning.

It also allows you to assign color styles to entire lines of captions, which can be handy when you have several people talking – colored text can help deaf and hard of hearing viewers understand who is speaking.

In the preferences is an option to add a name tag to the beginning of captions to also help identify the speaker.

Preferences showing four settings: AutoLoad next Caption. Show Speakers name where possible when exporting captions. Maximum characters captions can have with speaker's name tag. Caption Background Opacity.

It’s still beta, so expect bugs/crashes. This version will only work until the end of December 2018. I’ll be updating (and extending the trial period) it as I finish it up and polish it off.

You can download it here: Subrip to iTT

Quick Look plugin for WebVTT (.vtt)

macOS screenshot demonstrating Quick Look preview for WebVTT files.

On macOS I got tired of dragging webvtt files onto TextEdit to view them, so I wrote a Quick Look plugin so you can just select the .vtt file and press the space bar to preview them.

You’ll need to download the WebVTT Quick Look plugin and install it in the ~/Library/QuickLook folder. To view the Library folder from the finder, select the Go menu in the menubar, and hold the Option key (⌥). Drag the uncompressed WebVTT Quick Look plugin file to QuickLook folder.

You can either restart your Mac, or use the Terminal and enter the command: qlmanage -r
Using the terminal command above means you don’t have to restart your Mac.

Here’s where I got the information to write this plugin: Creating a Quick Look plugin.

Testing WebVTT Closed Captions

Testing WebVTT captions some more so I can compare them to how YouTube renders the captions vs Quicktime and browsers.

Html 5 and Quicktime render a little differently (not shown here). Quicktime with WebVTT captions embedded in the video has a built-in ‘safe-area’ where it displays Closed Captions, so captions can’t be displayed right at the edges of the video.

Youtube has some quirks. Left align does not work, vertical position is uneven, and right align only works if you add a space after the colon in the markup (i.e. “align: right” works, “align:right” does not.)

mp4 video below with sidecar WebVTT Closed Captions:

Youtube video with the same WebVTT Closed Captions file showing inconsistencies in vertical spacing and left alignment that doesn’t work.

The code used in Unity3d Gaming Subtitle Options Web App.

Screenshot ofWebGL app showing what the minimum subtitle options games should have.

Below is all the code it took to create the subtitle options in the Subtitle Options WebGl App in the previous post, Minimum Gaming Subtitle Options.

Note: this does not include the code for free version of uGUI Color Picker from the Unity Asset Store I used in the options menu. It’s not mine, so I’m not posting it.

But it should illustrate just how easy it was to create all the options shown in the Subtitle Options Web Gl App in the Minimum Gaming Subtitle Options post.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using TMPro;
using UnityEngine.UI;
using uCPf;
using System.Text.RegularExpressions;
using System.Linq;

public class ResizeCaptions : MonoBehaviour
{
  public Image captionBackground;

  public TextMeshProUGUI fontBox, shadowbox;

  public Slider fontSizeSlider, fontLineSpacingSlider, fontOpacitySlider, bgOpacitySlider;

  public Button optionButton;

  public ColorPicker font_Color_Picker, background_Color_Picker;

  public Toggle outlineToggle, shadowToggle;

  public Dropdown fontList;

  public float sidePadding = 3;  

  public TMP_FontAsset[] plainFonts, outlineFonts, outlineshadowFonts, shadowFonts;

  public float[] charSize = {10f,12.5f,10.5f};

  public float[] lineSize = {36f,62f,86f};

  [Range(1.5f,4.5f)]
  public float shadowDistance = 1.5f;

  public GameObject optionMenu;



  private int font_No;

  private GameObject shadowFontBox;

  private string[] captionSplit;
  private string myCaption;

  private RectTransform imageboxRT, shadowBoxRT;

  private Color newFontColor = new Color(1,1,1), newBackgroundColor = new Color(0,0,0), newFontColorOpacity, newBackgroundColorOpacity;

  private bool fontUpdateRequired = false, backgroudUpdateRequired = false, hasOutline = true, hasShadow = false;

  private Vector2 shadowOffset;

  private float minFontSize = 20.15f, maxCharacters, scaleUp, currentFontSize, newWidth, newHeight, fontOpacity;
  private float shadowOffsetX = 0f, shadowOffsetY = -333.1f, fontscale = 72f;

  private float[] fontsizeMinMax = { 72f, 58f, 68f };


  // Use this for initialization
  void Start ()
  {
    shadowFontBox = shadowbox.gameObject;     //sets gameobject so it can be turned on or off.

    shadowBoxRT = shadowbox.rectTransform;      //used to position dropshadow offset which is affected by font size.
    imageboxRT = captionBackground.rectTransform; //used to change size of subtitle background

    font_Color_Picker.color = newFontColor;           //sets color for color font color picker

    background_Color_Picker.color = newBackgroundColor;     //sets color for background font color

    UpdateColorPickers();                   //Late Update updates colors set on Color Pickers

    outlineToggle.onValueChanged.AddListener(Change_Outline); //Executes code when toggle is changed
    shadowToggle.onValueChanged.AddListener(Change_Shadow);   //Executes code when toggle is changed
    fontList.onValueChanged.AddListener(Change_Font);     //Executes code when dropdown is changed
    optionButton.onClick.AddListener(Check_Options);      //Executes code when button is pressed

    Change_Outline(hasOutline);                 //Turns Outline font on
    Change_Shadow(hasShadow);                 //Turns Drop Shadow off

    scaleUp = currentFontSize/minFontSize;            //gets scale of font to set background box size and drop shadow offset
    resizeBackgroundImage();                  //sets up background box size
  }


  //turns options menu on and off.
  private void Check_Options ()
  {
    if(optionMenu.activeInHierarchy == false) { optionMenu.SetActive(true); }
    else { optionMenu.SetActive(false); }
  }


  //changes font when new one is selected in Dropdown menu
  private void Change_Font (int fontNumber)
  {
    font_No = fontNumber;

    currentFontSize = fontBox.fontSize = shadowbox.fontSize = fontSizeSlider.value * fontsizeMinMax[font_No]/fontscale;

    resizeBackgroundImage();

    Change_Outline(hasOutline);   //here's where the subtitle font gets changed.
    Change_Shadow(hasShadow);   //here's where the dropshadow font gets changed.
  }


  //turns outlines on and off on subtitle
  private void Change_Outline (bool is_it_On)
  {
    if(is_it_On)
    {
      fontBox.font = outlineFonts[font_No];       //changes subtitle font
      shadowbox.font = outlineshadowFonts[font_No];   //changes dropshadow font

      fontBox.characterSpacing = shadowbox.characterSpacing = 0.25f;    //increases character spacing if outlines are on for easier readding.

      hasOutline = true;
    }
    else
    {
      fontBox.font = plainFonts[font_No];         //changes subtitle font
      shadowbox.font = shadowFonts[font_No];        //changes dropshadow font

      fontBox.characterSpacing = shadowbox.characterSpacing = 0f;     //removes extra character spacing if outlines are off.

      hasOutline = false;
    }
  }


  //turns Shadows on and off on dropshadow font.
  private void Change_Shadow (bool is_it_On)
  {
    if(is_it_On)
    {
      if(hasOutline) { shadowbox.font = outlineshadowFonts[font_No]; }
      else { shadowbox.font = shadowFonts[font_No]; }

      Fix_ShadowOffset();

      shadowFontBox.SetActive(true);
      hasShadow = true;
    }
    else
    {
      if(hasOutline) { shadowbox.font = outlineshadowFonts[font_No]; }
      else
      {
        shadowbox.font = shadowFonts[font_No];
      }

      shadowFontBox.SetActive(false);
      hasShadow = false;
    }
  }


  //changes drop shadow offset amount when font size changes
  private void Fix_ShadowOffset ()
  {
    shadowOffset = new Vector2(shadowOffsetX + shadowDistance * scaleUp, shadowOffsetY - shadowDistance * scaleUp);
    shadowBoxRT.anchoredPosition = shadowOffset;
  }


  //Updates Color Picker UI
  public void UpdateColorPickers()
  {
    fontUpdateRequired = true;
    backgroudUpdateRequired = true;
  }


  //changes subtitle font color
  public void FontColorChanged (Color passed_Color)
  {
    newFontColorOpacity = passed_Color;
    newFontColorOpacity.a = fontOpacitySlider.value;
    fontBox.color = newFontColorOpacity;
    fontUpdateRequired = true;
  }


  //changes background color
  public void BackgroundColorChanged (Color passed_Color)
  {
    newBackgroundColorOpacity = passed_Color;
    newBackgroundColorOpacity.a = bgOpacitySlider.value;
    captionBackground.color = newBackgroundColorOpacity;
    backgroudUpdateRequired = true;
  }


  //changes subtitle font opacity
  public void FontOpacityChanged ()
  {
    newFontColorOpacity.a = fontOpacitySlider.value;
    fontBox.color = newFontColorOpacity;
  }


  //changes background opacity
  public void BackgroundOpacityChanged ()
  {
    newBackgroundColorOpacity.a = bgOpacitySlider.value;
    captionBackground.color = newBackgroundColorOpacity;
  }


  //Updates Color Pickers
  void LateUpdate()
  {
    if (fontUpdateRequired) { font_Color_Picker.UpdateUI (); fontUpdateRequired = false; }
    if (backgroudUpdateRequired) { background_Color_Picker.UpdateUI (); backgroudUpdateRequired = false; }
  }


  //changes Line Spacing in subtitle and and dropshadows
  public void Change_LineSpacing ()
  {
    fontBox.lineSpacing = shadowbox.lineSpacing = fontLineSpacingSlider.value;
    resizeBackgroundImage ();
  }


  public void resizeBackgroundImage ()
  {
    myCaption = fontBox.text;     //Gets Caption Text

    //sets new font size
    currentFontSize = fontBox.fontSize = shadowbox.fontSize = fontSizeSlider.value * fontsizeMinMax[font_No]/fontscale;

    //gets scale of font to set background box size and drop shadow offset
    scaleUp = currentFontSize/minFontSize;

    if(myCaption == null)           //checks to make sure there is a caption.
    {
      //Debug.Log ("It's null");
      captionBackground.enabled = false;    //turns off caption backgrund if there's no image.
      return;                 //returns if there is no caption.
    }

    if(!captionBackground.enabled) { captionBackground.enabled = true; }  //turns it back on if a caption replaces a blank caption.
     
    captionSplit = myCaption.Split(new string[]{ "\n" }, System.StringSplitOptions.RemoveEmptyEntries); //splits caption into separate lines.
    maxCharacters = 0f;

    for (int i = 0; i < captionSplit.Length; i++)   //we need to get the length of the longest ling of captions to set background box size
    {
      if(captionSplit[i].Length > maxCharacters)
      {
        maxCharacters = captionSplit[i].Length;   //gets length of longest line for setting box width.
      }
    }

    maxCharacters += sidePadding;               //adds padding at side for background box.

    newWidth = maxCharacters * charSize[font_No] * scaleUp;   //gets new width based on font parameters for background box.

    //gets new height based on font parameters for background box.
    newHeight = (lineSize[captionSplit.Length-1] * scaleUp) + (fontLineSpacingSlider.value/110 * lineSize[captionSplit.Length-1]);

    imageboxRT.sizeDelta = new Vector2(newWidth, newHeight);  //sets size of background box.

    Fix_ShadowOffset();                     //Fixes size of dropshadow offset for new font size.
  }
}

Telus finally adds Closed Captions to Optik TV Go app and online.

690 Days.
That’s how long it took Telus to “work on adding closed captions to their Optik TV Go app” and Optik on the Go online, according to their tweet.

The app was 6 years old before they finally added Closed Captions to it.

Note – while the Optik TV Go app on iOS looks good and uses Apple’s AVKit ( you have control over how the captions look via iOS settings), however using a Mac/PC to watch online, the captions are crap.

Watching OptikTV on your Mac/PC requires Flash video. The closed captions in Flash video do not use AVKit, are not customizable, and are poorly formatted. It also does not remember settings (Channel Types -> Playable on the device).

The first time you have to turn on the closed captions they work properly. However, when you return to the site and watch a video, the CC button indicates Close Captions are on, but they don’t work. You have to turn them off and back on to get them to display.

Then there’s the formatting of captions – they don’t follow CRTC standards. So many times captions end with a single word on the next line by as illustrated in the following 3 images:

Image from Telus Optik TV on computer showing poorly formatted captions, where the first one is split into two lines, with only one word on the second line. It reads, "I mean, I don't want to" on the top line, "say" on the second line, and "It's like black and white." on the third line.

Image from Telus Optik TV on computer showing poorly formatted captions. It reads, "I mean, it's shapin' up to" on the first line, "be" on the second line, one of my most favorite" on the third line, and "houses." on the fourth line

Image from Telus Optik TV on computer showing poorly formatted captions. It reads, "Okay. It looks good. I like" on the first line, "it." on the second line, and "-Right here." on the third line

The caption formatting seems to limit captions to 26 characters per line which forces the last word to a new line of captions. Current standards dictate a maximum of 42-45 characters per line of captions for  CE 708 Closed Captions.CE  608 (aka Line 21) Closed Captions has a 32 character limit, which was the standard for 4:3 aspect ratio tvs (the old CRT televisions).

Unfortunately, unlike the US Americans with Disabilities Act, Canada has no Disability Act which sets standards for online Closed Captioning, and Telus is following no standards at all for their OptikTV Go online Closed Captions.

There’s also the similar UI problem as on the television with Optik TV – the Telus UI obscures the Closed Captions, making them difficult to read. While this is worse on the television, it’s still poor design. The UI should not be covering or obscuring Closed Captions, especially since it remains on screen for several seconds, require views to rewind several seconds in order to see the Closed Captions when resuming play.

Image of Optik TV from computer where a dark UI gradient goes from transparent to black on top of the closed captions, with a progress bar at the bottom, the channel and station name on the left "306 HGTVCh", and the name of the show on the right "Home Town."

So, after 6 years, TELUS has half-assed captions that don’t follow proper Closed Captions formats or styles. I guess this puts them slightly ahead of YouTube’s Auto-Craptions.

 

 

 

 

 

Minimum Gaming Subtitle Options

Screenshot ofWebGL app showing what the minimum subtitle options games should have.

After talking with a number of people on Twitter, and given the awful, or limited gaming subtitle options for PC and console games, I decided to set up an app with the minimum subtitle options games should have.

I used Unity3d to create the WebGL app. You may need to install the Unity3d Web Player plugin available for Windows and Mac browsers.

The background image and subtitle is from Wolfenstein: The New Order. The starting subtitle size is approximately the same size as it appears in game, but I’ve split the subtitle into two lines. The original was on one line, breaking one of the primary closed captioning/subtitle guidelines of no more than 42-45 characters per line.

If there had been options to increase the size of the subtitles in the game, you wouldn’t have been able to make them much bigger without the caption getting cut off on either side of the screen. Splitting the caption to proper lengths gives you room to make subtitles much bigger.

Some people may think subtitles cover up too much of a game screen, but that’s irrelevant. The whole point of subtitles is communication. If players can’t read them, they will miss vital gameplay information and story.

Making subtitles readable to a wide range of players is what’s key with subtitle options. Some people don’t need subtitles at all. some like small subtitles. Others need high contrast between subtitles and backgrounds. Some have low vision and need them as large as possible.

Game companies need to expand the range of options available for subtitles. I’ve set what I think should be the minimum standard for subtitle options for games.

One other important thing about subtitle options – the player should be able to see exactly what they’re getting in the subtitle options with a full screen sample. Having a small sample image with subtitles on them to illustrate is not enough. The sample should be full screen right in the options menu so players can set subtitles up exactly the way they need to so they can read them as easily as possible.

Options include, font size, sans-serif, serif, and monospace fonts, background, font transparency, background transparency, font color, background color, outline, and drop shadow.

To start, click the Options button in the top right corner in the WebGl app below.

For a fantastic article on gaming subtitles by Max Deryagin, check out on “What Video Game Subtitling Got Wrong In 2017

Timing Close Captions with Aegisub

Timing Closed Captions with Aegisub

If you’ve ever needed to add closed captions to YouTube or FaceBook, Aegisub is a great, free program for creating closed caption Subrip files (.srt). Subrip files are what you upload to FaceBook or YouTube to add Closed Captions to videos.

Aegisub is available from Aegisub.org, and it’s free.

Here’s a tutorial on how to time closed captions from a transcript using Aegisub.

How to Add ‘Apply Changes to Prefab’ to right-click menu in Unity3D

 

It’s extra steps and work to keep going to the GameObject window in Unity3d to select ‘Apply Changes to Prefab’ after selecting the prefab. Now you can do it just by right-clicking the prefab and selecting ‘Apply Changes to Prefab’ from the context menu. Just download the editor file here and add it to a folder named Editor (as seen in project folder in the video).

Apple ‘Clips’ app

 

 

It looks like this app will create ‘open captions’ using iOS’s speech recognition. Open captions are captions which are ‘burned’ into the video, which would be a great for adding captions to videos on Twitter. Twitter currently has no Closed Caption api to add, or display closed captions in videos embedded in tweets. (Note – video above has no audio).

So Clips could be good for embedding open captioned videos to tweets, making them more accessible to deaf and hard of hearing. This would have been great for Vine. RIP.

This feature will also upload clips to Twitter, FaceBook, Youtube and Instagram, and requires iOS 10.3, which will be coming soon as well.

Via Apple:
“Live Titles let you easily create animated captions — just by talking as you record. Choose from a variety of styles, all perfectly timed to the sound of your voice.”

More on the Clips App from Apple.

N.B – ‘open captions’ can’t be turned off, they’re ‘burned’ into the video frames of videos, whereas ‘closed captions’ can be turned on or off.

1 2 3