2019-12-10 19:34:38 (edited by BaldSavant 2019-12-10 20:05:28)

Hi there folks,

I'm a designer looking to learn some things from this community of experts on audio games. My specific area of interest for my current job is voice input games, things like Hey You Pikachu or Tom Clancy's End War. While I've been researching, I quickly realized that games designed around voice input are few, but there's a large number of players who use voice input as their control method for regular games and that led me here.

I found the Voiced Gaming youtube, but I'm looking for more resources, and some personal stories.

So if you interact with games via a voice input device of any kind, I'd love to hear your thoughts. What are some types of things that are very difficult for you? What are the things that work really well and you think are super fun? What are some settings you've found you always have to use to get the voice input to work right? What are some things you've never seen people do with voice input that you wish someone would cater to?

Additional bonus points if you can weigh in on any of this related to a phone and other inputs as well. So if you use assistants like Siri or Alexa to interact with your tech, I'd love to hear about that.

2019-12-10 20:47:11

Interesting question.

Most people on this forum are blind, and (somewhat contrary to popular opinion), most blind people interact with pc's, phones, tablets etc  with conventional input methods like touch screen and keyboard (most blind people touch type). Of course most blind people use screen reading software where the computer/phone whatever speaks, however the input is still usually via typing, swiping on screen etc.

That being said, obviously the advantage to speaking games  that if the game itself is speaking to the player, there is absolutely no problem with the interface, which makes the game completely accessible.

So, I personally don't tend to use voice input for Pc, and confine my use of voice on the Phone to simply ordering siri to do generic things like send texts, however I do own an Amazon Echo and have played a great many games with that, particularly in cooperation with my wife who is a big fan of word and trivia games.

the irritations with Alexa usually come with either misunderstandings or cases where the game simply fails to recognize a given phrase, indeed flexible parzer seems to be a must for voice activated games, even for multiple choice efforts.
For example, in the hunt the wumpus varient caves under thorn hill, in most cases you can just say the number of the cave you need to move to, however if you say "five" or "nine" Alexa interprets it as "by" and closes the game, so I've had to get used to saying "move to cave 5" instead.

Likewise, a lot of basic audio design ideas in terms of interups, menus etc could be helpful, for example, instead of having to say "help" then listen through a long help tutorial, be able to just say "settings" to set given values.

Myself, I particularly like voice activated games that require the player to use their voice in a unique way and have outputs to match. Vortex was I think one of the best things I've seen using voice input with the dynamic way the robots you saw understood you.

I have heard good things about starlanes and six swords, but haven't got far with either, indeed in the case of six swords I find lack of direction in the game a miner problem, and often wish I could do many of the things found in conventional rpgs, EG examine given monsters or my team, or have more conventional map objectives.

Hope some of this makes sense, and I'll be interested to here what your thinking in design terms.

With our dreaming and singing, Ceaseless and sorrowless we! The glory about us clinging Of the glorious futures we see,
Our souls with high music ringing; O men! It must ever be
That we dwell in our dreaming and singing, A little apart from ye. (Arthur O'Shaughnessy 1873.)

2019-12-11 01:35:09

So I'm curious in that you mention you don't tend to use voice input for PC, but you do use it for phone input. Is there a particular reason why you avoid pc voice input but don't mind using it on a phone?

I'm very familiar with the misunderstandings that happen with Alexa and I laughed out loud when you mentioned it hearing "Nine" as "by" because I've spent a lot of time over the last month just debugging a voice conversation in which we asked for the users age and learned that google's speech recognition doesn't know how to handle numbers. I had another fun (and I use that term as sarcastically as possible) moment where I was trying to respond to the word "raps" but kept getting "rats" from the speech recognition. Fortunately, some of my coworkers are working on a solution to that problem.

May I ask for some examples from Vortex that you felt were particularly interesting?

and so in summary, your frustrations seem to come from being misunderstood and a general inability to do what you intend to do. You want games like Six Swords to offer you the standard functionality found in text-based RPGs, and you want their menus to be short and easy to listen to, while also accepting input at any time. Would you say that's an accurate summary?

2019-12-11 06:41:16 (edited by RTT entertainment 2019-12-11 06:49:17)

Hi BaldSavant.
Welcome to the forum.
I've never had that much experience with voice input, but i use seerie quite alot.
I've noticed that it gets quite a lot of words wrong while using dictation, for example the word advertisements will be replaced with the word advertise mints.
For reference, i'm from South Africa and i'm a native english speaker. I don't consider my accent hard to understand as many people will automatically assume that i'm from the UK.
They'll ask me questions like "Which part of the UK are you from?"
While others will ask "Are you from the UK or South Africa?"
This mistake isn't limited to my fellow South Africans either.
Anyway, that's just my thaughts.
O-T end war? Was this a game about controlling a fighter plane with your voice?
I seem to recall a game like this which my grandmother had baught me for my eighth birthday. We could never get it to actually work and accept my voice commands as it would always say no weapon when trying to instruct the plane.
Where would I find the game?
Is it accessible?

Hopefully, we'll get a fully accessible open world game someday.

2019-12-11 09:19:13

One thing voice command hasn't quite been able to do yet is know when I'm trying to say xyzzy, because people pronounce it in several ways. Things like that are why I prefer typing over voice commands mostly. I'll never play mud with voice command if it involves fast pace interaction, it's just easier to write aliases and names you aren't exactly sure how to pronounce. Still, I'm not entirely against it: I've had some good experiences with alexa games, but they've always been pretty played back experiences for when I have time to myself and can afford to play those games where you need to listen to descriptions and stats an stuff.

2019-12-11 10:15:45

@BaldSavant, the reason I don't use voice input for pc, is firstly setup and secondly ease of use.

To take a simple example, if I am reading this forum, with my screen reading program H and shift H will jump me forward and back between the different headings for people's posts, whilst hitting down arrow will simply go down a few lines.
So to read replies since my last post, I just go to the bottom of the page, hit shift h until I get to my name last, down  past all the "post time, location" etc and read, then repeat this for the next few posts. I am also at a point now where this sort of navigation takes me less time than it would to speak, indeed in general my typing speed is pretty fast.

I doubt with voice input I could skip around the page half that quickly, even assuming voice input had efficient web reading commands the way my screen reader does.
Plus of course, most voice input systems require you to read off the screen to setup anyway, which obviously could not be done with a screen reader, I am also not entirely sure how you'd navigate around a complex interface such as a webpage with many headings and buttons and links etc and be able to know which was which.

So, that I think is why most blind people type or use a touch screen as standard input method.

What I have used voice for as I said, was specifically designed environments such as Alexa, though even there there are a few issues. Misunderstanding is one, whilst not being able to quickly skip between information is another, EG if I were playing a conventional text rpg I could get information about my party's equipment by clicking or by typing "examine sword" or the like, where  in six swords you need to get your full party stats at the end of  turn by saying "more information."

Another example is the game space express from Volley FM, which is a rather fun if simple collect and map game, however, the game reports all of your stats and what your holding everytime you land on a planet, meaning that you have to constantly interupt to fly in a different location, and if you just want to know if you have a certain item in your cargo hold, you need to wait past the voice saying "you are currently on planet so and so, there is a such and such and a such and such here", whereas it'd be nice to just be able to say "cargo" and know what you're carrying.
To go with this problem, I also find it mildly irritating that Alexa's voice is entirely impossible to customise in terms of speed. My screen reading program I have at a pretty high speed allowing me to receive and process information quickly, whereas Alexa often feels slow and somewhat cumbersome.

The reason I particularly enjoyed Vortex, is that vortex used the voice in a very unique way, with the different alexa voices used to portray robot characters you were stuck on the ship with, and the commands you gave couched in a way as to make it feel as if you were giving commands to sentient robots, EG "Alpha, search the storage lockers," which made the game feel much more immersive.
I also liked the fact that even though practically speaking vortex was essentially a "go x to get y so you can unlock z" type of game, you always had a lot of choice about what areas you went to and what you would do, even as far as giving you four different robots to play with.

Indeed, at the moment I haven't really seen many fully immersive and complex games on Alexa by voice control.  Most seem to default to either moderately liniar gamebooks with a basic cyoa interface or basic question and answer games.
There's nothing wrong with this, indeed my lady and I play geopardy and the volley games every day as a matter of course, which are great fun to do with two people, however it'd be interesting to see a voice control game which gave you much more choice about what you interacted with and what paths you took.
Six swords was a really good go at this, but I always found objectives in that game a bit too unclear to work completely, since your just dumped off in a map and essentially just run around whacking monsters with little description or major choice about what your party does, or at least, that was how it was when I last played, which I admit is a while ago.

With our dreaming and singing, Ceaseless and sorrowless we! The glory about us clinging Of the glorious futures we see,
Our souls with high music ringing; O men! It must ever be
That we dwell in our dreaming and singing, A little apart from ye. (Arthur O'Shaughnessy 1873.)

2019-12-11 11:14:26

To chime in for a sec here, Alexa can now speak faster or slower.

I forget what you say to it, I know David Woodbridge did a podcast demonstrating the feature here, so I'm not sure if it's available for all regions.

2019-12-11 12:30:37

Hi BaldSavant.
Welcome to the forum. Interesting questions.
In most cases, blind people does not use voice commands to interact with technoligy if there are other options available. Why? Because of the same reason for you with normal sight. On the pc: Would you use the build in voice recognition in Windows 10 to interact with your computer, when the mouse and keyboard is right in front of you? Are you using Siri or Google Assistance on your phone and are you dictating text messages instead of typing?
Regarding using voice commands in gaming:
I have some experience with games on Alexa, Google Assistance and a few mainstream games on consoles. I would never play a game with my voice if I'm traveling by train or buss. I would only play such game when I'm alone. I prefer to interact with games in other ways simply to make sure the game understand my command. English is not my main language, and because of that, the game sometimes misunderstand my commands, because you in some cases doesn't have to confirm the command.
You mention a few mainstream console commands which also offers voice commands. I have looked at End war. This game is not accessible to blind people at all. You need to see what happens on the screen, and react with voice commands based on what's going on. There is not enough audio or spoken feedback on what's going on for blind people to be able to play the game. So, said in an other way: Offering voice commands in a game does not make it accessible to blind people, if the gameplay itself isn't accessible.
I have also played a very old game for pf2 called Lifeline. This game is very interesting because it can only be played by voice commands. This game is playable if you are following a walkthrough. It is not accessible, because you, as a blind gamer, does not always get enough information by audio to know what's going on. Especially in battles. In the battles, if you don't know what kind of enemy you are phasing and you don't know what parts of the body the enemy have and what those parts are named, then you won't be able to defeat the enemy.

Best regards SLJ.
Feel free to contact me privately if you have something in mind. If you do so, then please send me a mail instead of using the private message on the forum, since I don't check those very often.
Facebook: https://facebook.com/sorenjensen1988
Twitter: https://twitter.com/soerenjensen

2019-12-11 13:27:25

Interesting kool_turk, when I last checked Alexa settings you couldn't alter speed at all, I'll have to investigate that one.

@Slj, interesting that there are voice activated games on consoles now, though again the problems as you said.

I do remember a few years ago we had a developer advertising a race game for Ios where you had to scream every time your character approached a cliff, with the length of your character's jump dictated by how much you screamed. The problem was as you said though, there was absolutely zero audio indication of what was going on at all, making the game %100 inaccessible.

The developer got a wee bit huffy I recall when we pointed out this fact big_smile.

With our dreaming and singing, Ceaseless and sorrowless we! The glory about us clinging Of the glorious futures we see,
Our souls with high music ringing; O men! It must ever be
That we dwell in our dreaming and singing, A little apart from ye. (Arthur O'Shaughnessy 1873.)

2019-12-11 14:05:31

@dark, no need to fiddle about in the settings, just say speak faster or slower.

To quickly go back to the normal rate, just say, speak at normal rate or speak at default rate.

I have the Alexa app on my phone so some of the features aren't available to me, but this one definitely is.

Oh and if a developer gets all huffy when you're giving them feedback, then I'm sorry, maybe they're just not cut out for that line of work.

If you can't handle the heat, then stay out of the kitchen.

Of course constructive feedback goes a long way, but if you're already being constructive and they're reacting like that, you did your part.

2019-12-11 17:49:03

I don't use voice commands for anything but quick actions, things I could do my self but would be slower and require me to not be using my hands at the time, such as checking battery status, time, weather, recent notifications, or turning on do not disturb.
So I find that voice commands in games only really work for more simplistic actions and get tiring quickly if I need to use the same ones again and again.
I find they work well in multiple choice  narratives and trivia games quite well for instance, and badly in games with allot of combat or complex puzzling or minute exploration of environmental features such as in object heavy interactive fiction.
I think that verbosity settings such as (brief mode) and speech rate settings would help, as well as optional short hand for often repeated commands, as long as the syntax is consistent and can be looked up at will.
The ability to skip sections of dialogue and pause/stop speech is also important.

2019-12-11 19:36:25

A little bit off topic, but the other day I had this convo with google: me, "hey google, cancel my alarm." Google: "you can do that in the app." *opens the clock app.* Me, "You useless piece of technology! If I wanted to do it myself, I would have done so!"

Point is, sometimes, commands that you think are there just aren't.

2019-12-11 20:24:36 (edited by BaldSavant 2019-12-11 20:28:10)

So it's interesting. All of the complaints/concerns you folks are voicing essentially boil down to "We just want good design" and it amazes me that this seems so infrequently offered. I'll try my very best to be better. I hope I am able to succeed.

To the concerns about accents and misunderstandings, this is certainly a major hurdle we're trying to jump in our work at my studio. We have people devoted to figuring out how to at least improve the responsiveness, so hopefully we can make that better, but I do suspect that's simply going to be a matter of slow iteration over time as people like us refine the process. My role here is to try and ensure that we're asking the right questions. Hopefully by designing the inputs to be more distinct, it can relieve some of the pain involved. I suppose we'll have to see.

@RTT entertainment, I don't think End War is designed for play by the blind, as SLJ seems to agree with. I think it's just a cheap gimmick to use a new technology. I've never personally even played the game, I just know it as one of very few AAA games that was designed with voice input as a central mechanic. Much like Hey You Pikachu, which was also not the best game on earth.

@Dark that cliff game you mention is a lot of what I'm seeing in my research. Most voice games designed by sighted people seem to be gimmicks that offer a simple interaction usually involving yelling more and less loudly at a device. I think they enjoy the social deviousness of those interactions, but I'm not seeing very much that offers a more substantive experience.

It's also worth my mentioning that what I'm working on is going to most closely resemble an Alexa game. We're in the business of conversational interactions, so mostly what we're making will be formatted like a skype call. I realize this is not specifically catered to you all, since it's a video, but I believe it should generally be accessible to you. The exception is in places where we present something visually on the screen for you to discuss with our character. It's those moments that I'm here trying to understand better how to design.

From the sound of it, I need to make it audibly clear what is happening on the screen, and what the options are for the player. I need to be sure that if I want someone to interact with an object or a piece of information, that I have a way to express that interaction vocally. You are all making it very clear that Alexa games often offer too much information or too little and that the input methods are frequently unclear.

One question I'm curious about is the speed of reading a couple of you mention. I would expect a conversation with someone to feel most natural at normal speeds, but it seems like some of you mention turning up speeds even on Alexa games where things are more conversational. I imagine the increased speed is for the obvious reasons, but would any of you care to expand on that a bit? Do you also listen to youtube videos (or similar) at higher speeds, for instance? Do you find yourself more comfortable with speaking faster even in regular conversations? Is there a type of thing you prefer to experience at high speed, but another kind of thing you prefer at slower speeds? Any other thoughts you might have on the topic?

2019-12-12 00:53:11 (edited by defender 2019-12-12 01:00:09)

Okay so first of all this Skype call style seems like a really neat idea, it reminds me of a voice version of those "Lifeline" mobile games where your linked to a person in some way and you advise them.
They try to make it like a realistic chat messaging experience in some titles (such as "A normal lost phone") and that's what this reminds me of.


I am a bit confused as to weather you will be using the system voice or a recorded voice actor to read lines to the player.  If it's the latter, than no, most people wouldn't want to speed that up unless it was particularly slow because it messes with the natural flow of things, (though I know some people who like speeding up audio books).
In that case, it may also be worth doing an "audio description" type setup for visual moments, such as we have for movies and TV shows, something that you may even be able to use the system voice to achieve.
here's an example of what I mean, though you don't have to be quite as verbose as long as your still getting the important points across, and you don't need to repeat already known information
https://www.youtube.com/watch?v=jT5AsjzgIC4


But if it's the former than yes, customization options are important.
It's a bit hard to explain this, but essentially, think of the difference between skim reading a textbook VS listening to a professor.  If you are able to take in the info, than logically both would be equal, but for many, listening to someone explain it is allot more engaging.
The reason allot of us have our speech rate high is because it blends into our own internal thoughts.  Much as a proficient reader can let their eyes drift across each line and comprehend the words so quickly that the story flows naturally in their mind's eye and the individual characters tend to fade away, we do the same with electronic speech.
This is why most of us also prefer more computerized rather than human voices (though that is changing as of late do to improvements in human like speech) because we are so used to it and it is so generic that there is no personality, tone, or accent to color the input.
I my self often slow down my computer speech when reading something I want to fully comprehend, such as a complex idea in a textbook or a fiction novel so that I have time to really think about each statement I'm hearing, but I have it up to twice as fast for normal stuff like navigating my computer and the internet, writing, sound editing, playing games that are light on dialogue ETC.
I know of people that don't do this, or have their normal speech rate twice that of mine even, but I can't do so personally, which is why options are great.


When how ever, someone is talking naturally, or reading a story in a dramatic way, we tend to just listen to it as you would enjoy a play or poem, because that's what we want in that type of medium.

2019-12-12 02:03:16

@defender, that makes a lot of sense, and is pretty much how I would approach reading speeds were I to be using screen readers or  the like. I already skim long articles just out of tedium anyway, so I can't imagine I'd suddenly want to sit and listen patiently as someone read it to me if I suddenly lost my sight.

To your confusions, we'll have a voice actor reading lines and it'll be set up like a conversation and timing will be relevant because we're expecting a natural conversational flow for the whole thing, meaning that we'll need to not allow speed adjustments. Honestly, I just wanted to hear an answer to that as a usability concern just so I know if there's some nuance I didn't anticipate. I'm sighted, so I don't want to make any assumptions without at least asking for confirmation, ya know?

I'm glad you like the sound of the project, the "linked with some person and advising them" thing is the first thing we thought of to try and make. I'm sure you'll see a bit of that from us when we ship, but I also want to include some other types of toys and interactions. I think it'd be fun to have to solve puzzles and play games with these characters while you're talking to them.