I just read up on the grapheme based language models in Festvox and they seem awesome when you are doing a language for which you don't have a lot of linguistic data. But in the case of Swedish, if I ever get far enough where I begin looking at making a Swedish model I think I will do it in the slower way, with letter to sound rules trained from a large lexicon. I found a huge lexicon for Swedish which contains over 700,000 words and is in the public domain, so I figured that might be a good starting point. Also, Swedish is my native language so mapping the formants for the various sounds shouldn't be too difficult. Actually, I'm rather looking forward to it so I hope I can get the actual synthesis part up to scratch. If I can, adding new languages and voices will be a blast.

Kind regards,

Philip Bennefall

Well that would be cool. This is kinda fun following its development because I've always wondered how synthesizers were developed.
If you ever want to branch into other languages, I'm sure you'd have many helpers here.

I would've liked to respond to this sooner, but was having issues logging into the site. Thankfully it's sorted out.

Your research into this is really wild to me. I'm no expert in these things, but I am really interested in this stuff. I only wish I could code so that I could either try to build something on my own or contribute to yours .

One thing I meant to say earlier is that, at least given the examples on your site, I think that some of the transitions as you said are a little messy. One thing that particularly stuck out to me was the transition from voiced to unvoiced. It sounded too long to me, as though the synth were drunk. Lol. Of course the lengths of these transitions could change with different speaking rates, but as a bass line value, it sounds too long in proportion to vowels to me. Not sure if your most recent question will tie into that, but thought I'd mention it anyway.

As to the transition between phones, if I am understanding you correctly, you want to know if a static transition time between any two phones is most suitable? My layman understanding of synthesis tells me that it would be fine, at least as a starting point, to use the same transition time between consonants and vowels. With diphthongs, or two adjacent vowels, I'm not so sure but this would be a good opportunity for me to give it some thought.

A long long time ago, when I was a young teenager with nothing better to do, I did play around with this stuff in Dectalk, and I also had an old program called Flex Voice (which btw if anyone has Flex Voice, please get in touch). I wish I'd kept my singing synth stuff around but sadly I either lost most of it or deleted it.

In both synths, when singing or doing complex manual phoneme work, you specify phonemes and durations. The phonemes are exactly like those in the phone list in your post, though obviously the codes were not the same, but it's the same principal. You even specified duration in milliseconds, which is mainly why I never continued my singing efforts because syncing it with music would be difficult, not to mention there is at least one bug I know of in Dectalk that messes with the millisecond values of certain phonemes. Anyway, I found that setting consonants like b, d, f, g, h, k etc. to 60 milliseconds generally worked well if I remember right, and I don't remember having to change them. Now for words like grass, that have multiple consonants at the beginning, I can't remember what I did. I also don't remember, in a musical setting, whether I wanted to put the consonant on the beat, or the vowel on the beat. Something tells me the latter is more appropriate. For your speech synth efforts, that obviously doesn't matter since the transition will happen in either case, but it's an interesting question when you're trying to do singing.

As for diphthong transitions, they were always controlled by the synthesizer, which for a geek like me does get a little annoying when it transitions in a way you don't particularly want in the context of what you're doing. For example, let's say I want to make the word ice, with a long duration. . Some synths will stretch the duration from ah to ih. The stretch will be proportional, meaning that if you make it 5 seconds long, you'll get a very slow transition. If I remember right, Dectalk has a maximum length for this transition and will just hang on the ih sound at the end for the rest of the phoneme. I suspect other synths will do the inverse, that is, prolong the ah and do the transition at the end, which sounds somewhat commical. As to which one is best, I can't really say, especially for normal speech.

All of this reflection on speech synths reminds me of my own efforts to explore the concept, but being the techy/musical person I am, I like putting things in a musical context, as you can probably tell from this post.

I don't know if you've ever heard of a machine called the Voder. The Voder is an old machine from the late 1930s iirc. It was never commercially used but was more an educational contraption built to show that with the cutting-edge technology of the time, it was possible to synthesize speech. I think the research that went into synthesizing speech with the Voder also helped in building the vocoder concept. Anyway the Voder supposedly had a complex control board to adjust analog circuitry (filters, oscillators etc) to make speech sounds. Of course it wasn't an automatic tts. To make any sort of speech you had to learn precisely how to move the controllers to produce different phonemes, and this required many months of training to master.

One day I was exceptionally bored and started a script for a musical instrument player called Sforzando with the goal of making a singing synth, and its usage is, roughly, the same idea as the Voder. To make things complicated, each formant filter's frequency is assigned to a different control knob, mainly because in Sforzando there is almost no way I could do any complex transitions automatically so it has to be done manually. As for unvoiced sounds, I haven't at all decided how to tackle that. So this thing would never be useful to anyone unless they wanted to experiment, but it's cool that we are approaching speech synthesis from totally different angles. I am already learning a lot from reading about your efforts. The difference between parallel and cascade filter setups is really interesting to me, as well as your take on synthesizing unvoiced consonants. I fully relate to how tricky it can be to EQ noise in such a way as to creat consonants and am wondering how you did it. Will be especially interested to see how you make f and th distinguishable, but the better formant synths make valliant attempts (at least Eloquence does, and that's the one whose sound I know best).

Well that's enough rambling from me. Hope it was at least a fun read anyway.

ah. Flight. K, well I guess that's a bit more compatible with your phoneme tables . It mentions a repository, where can I find it and what do I need to make it run?

That is indeed a cool set of old speech synth examples. If you want a cool overview of the Voder, you can check out this video.

That's pretty cool. i've always been looking for good vocoders that are intelligible and clear-sounding (never was a fan of a lot of traditional analog vocoders because the speech is somewhat disguised).

If we want to start cringes, I could bring up PB Vocoder, which was actually pretty cool. Only odd thing about it for me was the sort of springy sound it had, which I'm guessing comes from the types of filters it used for band splitting. From what I've heard, most things use what are known as minimum phase filters. I don't know how it works, but I do know that such filters tend to produce that sort of sound because i've played with EQs that offer a minimum phase or linear phase option. The springy sound gets more pronounced with minimum phase when band splitting, especially if you're using a lot of bands. Linear phase sounds more clean but is more resource intensive and is probably a lot harder to implement. So most things don't use them, unless there is a particular reason why they are more appropriate.

Pb-vocoder sounded cool. It's sad, it was killed off.

The audio examples I posted above are not of PB Vocoder, but of a new implementation that I wrote a few weeks ago. I'd be curious to hear what you all think of the sound. I haven't yet decided whether to release it or not, but I might clean it up and publish it at some point if people like the examples.

Kind regards,

Philip Bennefall
P.S. I wasn't able to hear the examples on that page because apparently I don't have the Java runtime. Ah well.

This implementation sounds really good. I remember using PB Vocoder (despite the demo beeps) and even that didn't sound half bad. Would be interesting to try this thing out with Goldwave if it were released as a vst. Probably the way to go these days since this stuff is common plugin material, what with the amount of reverb plugins you see floating around for example.

Such a program isn't necessary if it means extra code. A vst is like a program anyway what with the interface and stuff like that, just without having to worry about drawing a window. After all, if Abby Road Reverb is simply a vst/vst3, then a vocoder has that same potential, including an accessible ui.

hi philip.
this is a quite nice project. keep up the good work.

I do like how intelligible the speech is with your vocoder. It's crisp and direct and to me at least it has some unique flavor, probably because it kinda has that springy sound I find odd, like the voice is going through a rubber tunnel or something. But many vocoders have that and I think a lot of people like that.

One of my favorite vocoders is the MDA Talk Box. It's a super old VST with limitations but its clarity can almost not be beat. It sound is admittedly digital and perhaps robotic/tinny, which some people might not like. It sounds like it's using some LPC techniques which produce a totally diffrent sound than a conventional bandsplitter. But I've always been looking for that sort of Daft Punk sound, and MDA Talk Box is the closest to that I have found.

Philip, you're vocoder sounds very good, on par with Mda TalkBox but it has those extra settings.

Best regards
T-m