2013-09-02 16:34:02

This isn't a game, so I'm posting it here in the Off-topic room.

I was recently asked if I could make a program to read movie sub titles using different screen readers.  Building something to actually play the movie files would have been a pain, since I'd have to deal with all of the different video codecs being used these days, so I went with a little trick to simplify the problem.  You select a movie file using the program, and it extracts the sound track into an MP3 file which is then played along with the sub titles.

This is just one of my little utility applications that I toss out there just to see if it will be of use to anyone.  If it works for you, then excellent, and if not, then I apologize in advance.

- Tab changes through screen reader options.  This uses the same style that I used in Swamp.
- Space pauses and unpauses your video.  This uses the same style that i used in Castaways and Lunimals (complete with authentic pause and unpause sounds!)
- The Left and Right arrows will move you forward and backward by 1 second.  Hold shift at the same time to move 10 seconds, Alt for 30 seconds, and Control is 1 minute.
- When paused, using the up and down arrows to quickly hear any sub titles you would like repeated.
- The plus and minus keys adjust the Sapi rate.

I'm sure there will end up being a few bugs.  The person who requested this program hasn't been online yet this morning to hunt for them for me, hehe.  smile

www.kaldobsky.com/audiogames/wademachine.zip
(Download size is under 5 MB)

- Aprone
Please try out my games and programs:
Aprone's software

2013-09-02 19:03:24

Lol strange, I never thought about that. I wil try it when I finish my exams.

Sorry for my english :D

2013-09-02 19:10:52

I have never actually used subtitles for any movie or what you will oweing to me not knowing how to get it working,or even what I needed to try it out, this will be handy, once I figure all the stuff out. big_smile
So,do I need to select the video file or the subtitle file? I think I need to read upon all the subtitle stuff.
Grryfindore.

Of all sad words of tongue or pen, the saddest are these, ‘It might have been.
Follow me on twitter

2013-09-02 19:29:23

The person who requested this program downloads shows that are not in his native language.  Traditionally that is what sub titles are for, but he couldn't get them to read properly with his screen reader.  I guess the programs just weren't made to be accessible.

If your video has sub titles, it will come with a separate file that has the same name as the video, but it has the file extension S R T.  Wade machine automatically loads the SRT file when you choose the video (well MP3 I mean).

- Aprone
Please try out my games and programs:
Aprone's software

2013-09-03 00:09:08

I was going to ask if it only did external subtitles, but I guess that's really the only one that makes sense since having a screenreader read hard-coded subtitles would virtually be impossible as they are part of the video itself?

Discord: clemchowder633

2013-09-03 02:34:56

Well it would be possible, but very very difficult.

I've just posted an update to this, because it was buggy if the SRT file was arranged in a way I didn't plan for.  It's working really well for me now.

- Aprone
Please try out my games and programs:
Aprone's software

2013-09-03 04:46:30

This is totally amazing!


I actually started developing something simmilar, but had to stop because I was finishing college. I will try the program for sure!

That is not dead which can eternal lie, yet with stranger aeons, even Death may die.

2013-09-03 06:13:08

hi.
I know you can't tell me exactly where to get movies with subtitles but could you give me some movies with them in that I could look on the internet to download?
Thanks.

I'm gone for real :)

2013-09-03 06:32:47

cool! but u know? there is a player named the k m player it does the job of reading subtitles by sapi5 tts

meat me on facebook azz
Yor's Animeshh Ahilya

2013-09-03 06:54:52

The km player, however ,sin't fully accessible. And Brad... any movie that isn't in english will most likely have them... any movie at all. Even movies with English usually have optional English SRT files. It all depends on the package you get from the internet.

Discord: clemchowder633

2013-09-03 07:55:15

I think something similar has been made, but we all know your program is better. big_smile I'll give this a try as soon as I'm home. I think I have a few movies with sub titles, at least I've got some CSI episodes... Guess that would work fine.

To see a world in a grain of sand, and a heaven in a wild flower.
Hold infinity in the palm of your hand, and eternity in an hour.
William Blake - Auguries of Innocence, line 1 to 4

2013-09-03 08:06:58

thanks for your wonderfull program aprone. I am using subtitles because English is not my native language and all English movies have them. this is very usefull to me. thanks..
until now I was using v player which could read subtitles with jaws. but it wasn't so accessible. this is far better.

“Get busy living or get busy dying.”
Stephen King

2013-09-03 12:12:01

Hmm, I'm confused. What are you extracting the subs from? My efforts to make DVD overlay subtitles useful has been a miserable failure owing to the fact that you need to OCR the bitmaps. Does this have anything to do with this?

Just myself, as usual.

2013-09-03 12:22:25

You could probably take this a step further, if this thing is reading subtitles, why not make subtitles that have things described instead of translate.

Just something to throw out there.

I brought this up on another forum ages ago, and a majority of responses preferred human speech describing instead, while some say this method would probably be cheaper.

You could get more things described this way, and not have to wait for someone to read and mix the content.

The drawback would mean the quality wouldn't be as good as the professionals that do these things, but that's to be expected.

2013-09-03 16:18:02

Prince, if there is already a program that does this same thing, then that's actually good to know.  If this program doesn't work well for people, K M player might be worth a look as well.  I'm certainly not setting out to compete with other programs that do this same thing, I just tossed this one together to fill a specific need someone had.  smile  Ideally we would have a list of sub title reading programs, so that people could find out which one they prefer.  Nin mentioned V player also.

Sebby, I'm just pulling the sub title information from the included SRT file.  Most movie files that you download come with a separate file that contains the sub title data.  Other movies embed the text into the actual picture of the movie, and I'm completely avoiding those types.  You are correct though, some form of OCR would be required to turn the picture of text into something the computer could speak.  I've done such things in the past, but they are always a fair amount of work, and it would be buggy since each movie that does that probably used different font sizes and styles.

I'm not big into watching downloaded movies, so I only have 2 SRT files to experiment with for this.  Some of you may know more about this so I'll post the questions here.  Some of the entries will have special markings, and I am not sure how (or if) they should change how the screen reader reads them.  Here are 3 examples:

{\a6}Please do not stream this episode

I've found a few entries that begin with this weird {\a6} mark.

- There's an extra player.

Some lines begin with a dash, and I haven't figured out what makes them different from ones that don't.

<i>Sorry, I'm late.</i>

These are the italicized tags for web pages so I get the idea that this text should be italicized visually on the screen for sighted viewers.  How should I handle this with screen readers?  Should it begin by just saying "Italicized"?  Or should I just ignore the tags?

Kook_turk, I once had the same idea years ago, and I had wanted to do that with the Stargate TV series.  The usual SRT subtitle files are pretty much meant for sighted people who are only needing a little help understanding what is being said.  There is nothing stopping us from developing our own format that is actually designed with the blind in mind.  It rhymes, Yay!  big_smile  My plan was still going to just use screen reader text, but the same thing could be done using MP3s recorded by other people that would replace the text.

I don't know how much of this is what you were already thinking, but for the Stargate thing I was thinking of having these categories of data:

- The scene
- The crowd
- History
- Fun facts

I didn't have any plans for actual subtitles as the people spoke, because I assumed viewers would be watching the show in their native language.  Stargate has a rich storyline, so the last 2 categories might not fit as well into some shows and movies.

Before I continue, this was just an idea I had so it probably has some major holes in it.  This also isn't a planned project or anything, it is just me sharing since we are on the topic.  If this helps inspire others to jump in and polish up the idea then that's great, and if it inspires someone (even me I suppose) to build this once the idea is finished, then that's even more excellent!

Imagine as you watched the show (or listened to an MP3 version of it) you would hear the screen reader chime in to tell you the scene as it changed during the show.  That would be a short summary such as "Control room of ancient ship".  Press a key and it reads off a description (assuming of course that one exists).  The description could be as vague or as detailed as the author wanted it to be, because the viewer could pause the show to hear it if they wished.  Also those short summaries could be disabled with a key in case they were distracting from the show itself.

At any time during the show you could press a key to hear the history.  Once again this would only show up if an entry existed for this portion of the show, but it would explain any history related to the current situation.  In Stargate, if the main characters were just talking about Replicators boarding the ship, the history entry could explain a summary of what they are and any major events that they have played a role in, in previous episodes.  Think of that as a wiki page, and it could probably just be fed a web address and have the program read the web page's contents when you want to hear the history.

Fun facts would work very much the same way.  Now that I'm typing it out I realize it is so similar to history that it could probably be combined.  I'm also feeling like something is missing with the current idea, so perhaps others can fix it.

This was planned out especially for Stargate, but the same idea would have worked well in most series and large-scale movies.  StarTrek, Doctor Who, Lord of the Rings, Harry Potter, and those types of stories could probably benefit from having history and facts at the viewer's fingertips to help catch them up to speed in case they are forgetting something from the past that's important now.


Back on topic with the actual Wade Machine...  The current version is giving him a runtime error when he opens his MP3, but I can't get mine to error even when I use the same file he is opening.  Is anyone else getting any errors, or is anyone else having it work perfectly?  I'm going to add in extra debug commands to help narrow down the source of the bug.

- Aprone
Please try out my games and programs:
Aprone's software

2013-09-03 19:33:56

Ah, OK. A bit of research says that SRT is text-based.

DVD subtitles can be extracted separately from the video stream, but they're still bitmaps and can use different fonts which it would appear most OCR engines don't know about or require manual correction for. What an awful, inaccessible design. Thanks a lot, movie industry.

Just myself, as usual.

2013-09-03 19:38:48

The good part is, we can get audio described movies now in mp3 files,  and open the subtitle with it. Then, it is perfect \o/

Aprone, just let me suggest one little thing:
Would be it possible  to create a command < and > to  shift the  subttitle in time? I downloaded two different movies and tried some srt files, but they are speaking the subtitle  one or two  second later of the real speech. A < and a >, to shift the subttitle 0.5 seconds in time to the left or right would be amazing.

Since I am not an English native speaker too, another feature, which I think that is not that difficult to implement as well, would be to disable and enable the automatic  subtitle reading. I like to try to hear the movie in English, and when I do not understand something, I would read what is on the subtitle, and for that, we would need to be able to disable the automatic subtitle reading.

I know that you are a busy guy, and if you had developed this in c, c++ or python, I offer myself to code this changes  if you give me some guidance on how to proceed over the things that you have already done.

That is not dead which can eternal lie, yet with stranger aeons, even Death may die.

2013-09-03 20:12:46

Yeah Aprone!!!
Cool thing. Can't test it out ...
Still sounds like a good start.

Follow me on twitch
And
Subscribe to my youtube
Leave a thumbs up if you like what I write.

2013-09-03 21:30:11

I've updated it so now you can use the < and > keys to shift the subtitles.  Hopefully that works for you lucas.  smile  Good suggestion!

- Aprone
Please try out my games and programs:
Aprone's software

2013-09-03 21:39:35

That's actually an amazing idea, but the problem with that is the descriptions would have to be written out first, assuming you want more than just location names. I do like the idea though.
lucasradaelli, you have a point as well. Just remember that often... those movies in mp3 format are illegal. lol But as for the subtitles, if they were movies nececitating subtitles, most of them would already have the subtitles spoken by the narrators. Unless you were watching a movie simply in English. big_smile

Discord: clemchowder633

2013-09-03 23:12:16

@assault_freak

That's what happens in my case. The movie is in English, but I do not understand everything, so using the subtitle is a huge help for me.

@aprone

Thank you very much, Aprone!
just to know, the feature of  disabling automatic speech is too hard to code as well? I thought that this one would be  easier than the subtitle shifting.

Anyway, I am going to give it a try right now smile

That is not dead which can eternal lie, yet with stranger aeons, even Death may die.

2013-09-04 02:34:08

Oops, I forgot about a quick way to disable the subtitles.
I'm also uploading again because the latest update has a typo that can cause it to crash as it loads MP3s.

- Aprone
Please try out my games and programs:
Aprone's software

2013-09-04 03:57:12

Good!

Now it works fine.
Post then latter please what will be the command to disable automatic subtitle reading smile

thank you

That is not dead which can eternal lie, yet with stranger aeons, even Death may die.

2013-09-04 05:01:47

wow! cool!
gonna tryout rightaway!

friends:
come and join my
facebook group!

2013-09-04 05:25:47

Holy crap I hope something like this could work. Maybe just maybe, a software with some updates like this could read english subtitled anime.