This is a pretty interesting project. Hats off for the hours spent scratching your head at the online articles that never seem to give you the piece of information to make everything click inside your head. I've definitely been there. I'd love to see how far you are able to get with this. I gave the entries a quick read, but I'll probably have more questions once I'm able to sit down and wrap my head around everything.

Honestly I'm just glad your not dead.

Oh far from dead, just not doing much of anything relating to audio games so haven't had a lot of relevant things to post on here. Strictly speaking this is not really relevant to games either which is why I put it in the off topic room, but I figured some people might find it fun to follow along since I'm sure I'm not the only one interested in speech synthesis around here.

Kind regards,

Philip Bennefall

Can you post a link to the git repo? I would like to look at the source code. I think this is also an interesting project.

I hope this project works out for you

I just had another hacking session and managed to get semi-voiced sounds working, as well as a short fade at the beginning and at the end to get rid of clicks. There is a new diary entry with audio examples (see post 1 for the link).

Kind regards,

Philip Bennefall

Questions:
1. Wouldn't the formant/bandwidth values for voiced sounds slightly vary depending on the voice itself? (my point being, if a different person recorded the stories, would your formant values be different?)
2. So, since you ended up needing two sets of filters for voicing, are you running the same signal through both sets of filters in parallel?
3. How long does it take to generate an audio file for a short sentence like the ones you've been using for testing?

@jack
For something that started out being done by hand, the synth sounds incredible so far to me. Coming from information gathered from the web and a few estimates, it's definitely farther than I'd be able to take it. The processing power needed for wavenet is insane. I don't remember exactly what it was, but the amount of time required to generate one second of audio, even with the top-notch resources that a company like Google has, took an extremely long time. (at least at the time the article I read a while back was posted)

This is super interesting. Will it be easy (once you get the synth up and running), to make it bilingual? If, for example, it could support Swedish as well? (Although that'd be a headache in itself because of the different rhythms of the languages...)

The text processing part of the system was trained by way of machine learning by the folks who made Festival/Festvox/Flite, so technically I am using machine learning - just not in the synthesis backend code.

ESpeak is similar to what I want to do for sure, but I am personally not very fond of its output so I wanted to see if I could achieve something different.

@BoundTo:

1. The formant frequencies and bandwidths do vary for different speakers, especially between men, women and children. It is definitely possible to derive a new formant table to add a new voice, though some other tweaks would be needed as well such as defining the average pitch etc.

2. I generate two completely separate signals, one with white noise and one with the pulse; a sawtooth in this case. The noise runs through the unvoiced filters and the sawtooth runs through the voiced ones. They are then combined using envelopes to smoothly turn the two sources on and off as appropriate.

3. Rendering the slow version of the "visual roses" sentence which comes out to 3.91 seconds, took 31 milliseconds on my laptop. This is generating the whole thing in one go, however, which you would not do in a streaming application - you could generate as little as 5 or 10 milliseconds of audio at a time in many cases. Note that I have not done any work on optimizing the code; I'm sure I could speed it up significantly down the road.

@Jack Festvox is a suite of voice building tools, you don't actually use the code in Festvox in the final synthesizer. Since I am using Flite for text processing, I'm still very much making use of Festvox. But you're absolutely right in thinking that all the synthesis code is being done by hand.

@harrylst Yes, you could definitely make a Swedish voice using the Festvox tools as a starting point. You would train models for duration and fundamental frequency prediction based on natural speech, and you would train a letter to sound rule set by analyzing a large pronunciation dictionary. All this is possible in Festvox, and the output can be converted to Flite which means I could get phones, durations and a pitch contour for a sentence. But it would take some tweaking and quite a bit of trial and error to get the various models right, not to mention getting formant settings for all the sounds that differ from English.

Thanks so much for all the positive feedback, everyone!

Kind regards,

Philip Bennefall

or an android tts voice?

This sounds really cool! I'm looking forward to learning more about this!

Personally I absolutely cannot stand Espeak, so I would love to see something better on the open-source market as it were.