2017-03-22 22:10:45

Hi, this is a tech tangent to the Madden thread I started http://forum.audiogames.net/viewtopic.p … 91&p=4

I'm trying to figure out how to make console games more accessible, and this would like to discuss options on how to handle sticks and button presses, especially button presses like quick time events.

I had a musician coworker suggested using the "rule of 4ths" to make tones that can be distinguished when they overlap. It would be the following notes:
C F B-flat E-flat A-flat D-flat G-flat B E A D G

The sequence repeats itself for higher/lower octaves.

So, assuming an Xbox controller, and using X Y A B buttons that have icons that appear at different times, each one would be assigned to its own note. Surround sound can be used to indicate which side of the screen the icon appeared. The notes may or may not overlap.

Would that work as a way to distinguish multiple overlapping events?

2017-03-22 22:25:13

Possibly, as long as there's a tutorial/practice mode, with text to speech telling you which buttons to press, before trying the real thing. The trouble is though, the ps4 does not currently have text to speech outside the US. The xbox one is winning in this regard. So, for everyone, I think a website would have to be develoepd with audio or video tutorials with audio in them, to take into account everyone.
Going slightly off topic though, I really wish one day though that the ps4 will get text to speech everywhere and an API like the xbox one. Everyone should have acess to whatever console they wish. WHile we learn menus and things, the text to speech stuff definitely makes it easier.

2017-03-22 22:39:50 (edited by ianhamilton_ 2017-03-22 22:42:39)

There has been a fair bit of discussion on this here already right? From what I remember it was just a straightforward literal instruction... if a QTE tells you to press square, just say 'square', if A, say 'A'.

Nice and simple to do with TTS if that's available, and saves having to learn a system.

2017-03-22 23:01:03

Wouldn't it be confusing if the buttons were said in words, and overlap? Also, they are time sensitive, and it takes a bit to say "triangle".

Mods, would it be possible to move this thread to general discussion? Ian had a good point offline that this is more of a general topic than a coding topic.

2017-03-22 23:30:17

Probably depends how it's implemented, if through TTS the speed is usually cranked up pretty high. Trying to remember where I saw the conversations about it, I think it may have been a thread regarding the QTEs in telltale's games, and another thread about the QTEs in the cutscenes in mortal kombat's story mode?

2017-03-23 00:07:25

Hello folks! The musical notes seems fair in my opinion, and to avoid text to speak application it can be used as a default sequence. For example, the buttons are placed like a cross, so the higher pitch would match triangle, the lower the cross button, and so on. If necessary, other notes could be generated for the back buttons r1, r2, l1 and l2 too.
Best regards, Haramir.

The true blind is the one who refuses to see.

2017-03-23 01:37:28

I think I remember someone talking about using stereo separation for left/right in conjunction with pitch for up/down, I.e. on an Xbox controller x/b being the same pitch but left/right in stereo

2017-03-23 05:09:40

The truth is, I'm a musician, and I still wonder how well the option of using a different pitch for each button would work. It's one thing if I know that when I hear some pitch, I need to hit some button. But to be able to hear a pitch, or even multiple pitches, in a vacuum and be able to identify which pitches they are? I don't know if it's reasonable to expect people to have that good of a sense of relative pitch. Maybe it would work better than I'm assuming—and to be fair, I don't have a better option. I just know if I know D4 is the square button and G 4 is the triangle button and you play me a D4, I might know what button to press if I have a starting reference point, but if you play me the note without any lead-up, I don't know if I'd be able to know which one it is. Does that make sense?

2017-03-23 05:23:04

OK, I must provide my 2 cents here, and here they are.

I agree with a bit of what everyone is saying. First, both methods of button indication are valid, but that is very dependent on the game, I think. In a game like Madden, you're not actually dealing with QTE's, you're dealing with realtime events that you must react to accordingly. For that reason, I feel that having the buttons spoken by TTS or recorded voice would be a mistake in Madden, but perfect for QTE sequences in other games.

Onto the topic of pitch for this idea of button indication. The idea personally works for me, but i think another question the developers should ask is whether there are ever overlaps for the same button. If a button is used for something, then at a different time that same button is used for something else, perhaps it should play a different sound. The uniformity here would be the pitches. Each pitch is always the same for each button, but the specific types of sounds that are played at that pitch are different depending on the action. One's a beep, one's a kind of ping sound... Just a thought.

Regarding the post about not necessarily being able to tell which one to press without a leadup, I feel that, especially if you are a musician, that kind of thing would come with practice, which is how anyone gets better at a game. And i think that once you have an idea of how, for instance, the plays in Madden actually work, you may have some idea of which buttons you'll be needing to press from the moment you call the play. Of course, Madden being dynamic, things could change, but I think by that point you may know how to deal with those changes.

Don't get me wrong, I could be completely off. I'm not trying to assume anything, just provide ideas and my own input. I hope it helps.

2017-03-23 06:20:27

I like the pitch idea using channel separation. an example would be, C and G on left channel would correspond to square x on ps4, D and A on right channel to triangle circle. shoulder buttons could be completely different tone. There are so many ways of doing this. you could map buttons to particular chords. so minor chords for left side, major right side. or number the buttons 1 through 8. I can see that working quite well. 10 if you must use R3 and L3. if joystick movements are needed, you could also have special sounds for that, or vibration patterns. I'm just thinking out loud here. any more thoughts guys?

2017-03-23 11:15:29

This is an interesting discussion. I do feel though that having beeps kind of gives a sort of simon feel which I don't think is desirable if we want to integrate the game better into mainstream consciousness with accessibility in mind.

A way to counter it could be to use game-centric sounds with a small tutorial, or abandoning the system of logic to just have rising cues, depending on which one it is, without the need for stereo separation.

As for the topic of using a TTS as part of this,  You can actually get around the issue of taking too long to say words buy utilising a numeric system as is mostly done in the fighting game community (FGC).
1=square/x
2=Triangle/y
3=cross(x)/a
4=Circle/B

For needing to press more than one button at once, a rarity in my experience, you'd theoretically just combine numbers.  so if you need to press x and y working with Xbox terminology, the tts would say 12 instead of 1 and 2 simultaneously.

Regards,
Sightless Kombat.
***If you wish to refer to me in @replies, use Sightless***

2017-03-23 13:53:31 (edited by Lirin 2017-03-23 13:59:25)

Hello,

This is what I will use for my game probably as I'm developing something in style of Telltale games and this is much better than something like voice telling you the name of the button you need to press. Only one difference is, I don't want to use musical tones as it can comfuse many people so to do it easier I will use two sounds in the center (lower and higher, an octave probably) to indicate A (X) and Y (triangle) and beeps playing on the left and right so... X (square) and B (triangle.) I like the idea but what do you mean by surround sound? many of us are playing with heatphones so only one option I could see is 3d audio (HRTF)

Twitter
Add me on Steam
Discord: lirintheblindguy
BattleTAG: Lirin#21759

2017-03-23 16:37:11 (edited by JaceK 2017-03-23 16:40:37)

1. Shenmue 1 had beeping for its QTEs on the Dreamcast aaaaaages ago so it's not a new thing at all, but that had a sting in the tail in one particular QTE. (Spoiler: You had not hit anything to pass said QTE, completely going against what you had been doing the whole game), and the less time you had in a QTE the more it beeped faste.

2. Just thowing this out there but wouldn't the TTS/beeping/etc have to be overlayed over any and all music or turn down the music volume to hear the TTS? Just thinking aloud here but if I couldn't hear the TTS over the music it would sort of defeat the point of it. That being said....patten recognition.

For a music based QTE for example, have each button assigned to a chord squenece and work those sequences in

Or...do what the horrifically bad Rambo rail shooter did, and give an upgrade to auto complete QTEs. Well.....that's one way of getting around the issue honestly.

EDIT: Just got this pointed out to me:

Why not pair up the audio with a rumble feature and give each buttom a specific rumble? That way even if the music is too loud there's a failsafe option

Warning: Grumpy post above
Also on Linux natively

Jace's EA PGA Tour guide for blind golfers

2017-03-23 17:14:26

no no no. auto QTE is the last thing we want. we wanna play the game, not skip most of it. what's the point then? This is the reason I don't buy stuff in games to make life easier. like SF V characters. or MKX crypt. I'd rather unlock all that myself through gameplay than pay for it if given an option, which is what I'm doing. plus this is about being included, not excluded. then there's things like trophies/achievements. how would you handle that if there's a trophy that requires one to perform a button sequence? not give it to us cause we're blind and enabled accessibility options, thus game considers auto QTE option a cheat modifier? yeah, not good. haha. what I'm trying to say here is, as a blind gamer, I want to play games like everyone else, but without sight. not have it all chopped up. so no auto QTEs, slowing things down, easier bosses etc. I want to discuss the games like my sighted counterpart. "oh man! that boss fight was so difficult! Took me 3 hours!" this is the reason I love fighting games, because we're 100% included. When I did that very last street fighter IV trial a few years ago and the platinum finally popped, I felt a sense of accomplishment, because some of them were  insanely difficult and required tight execution. my friends couldn't do many of them so it was fun to brag. ahaha

2017-03-23 18:19:22

I don't like the numbers idea proposed above, it makes you think outside the terms of the controller that you're playing on, Madden already has a ton of controls to get used to, and you don't need another layer of obfuscation making things harder for you. The fact of the matter is using TTS will be slower than just tones. I think tones could work given sufficient practice. My proposal is this, have a single click or short tone sound and position it using HRTF. We already know our controllers and our own way to find our way around them. I think of it as a baseball diamond where on Xbox B is first base, Y is 2nd, X is third and A is home plate. But, the fact is, each of us has found his or her own way to mentally map out the controls. So using HRTF to position the click or tone in the sound stage corresponding to its position on the diamond, I think that could work.

one issue I can think of is make the click audible above the rest of the game, I think a higher pitched short tone or click could do this, but not too high or you're libal to give people headaches, maybe around 800 to 900 Hz. for shoulder buttons, you could slightly alter the tone or click, or use a rumble inn either the left or right side to indicate this. A quick impulse rumble on the left or right could be LB or L1 on XB and PS4 respectively, and one on the right could be RB and R1. A double impulse quick rumble could indicate that you are meant to press LT or RT rather than the bumpers or L1 or R1.

I think a majority of people playing these games will be doing it either on headphones, if in private, or on the TV speakers if with  friends or family. Knowing this, I think a headphone-centric approach would be best. I think it would be challenging to play over the TV speakers. Maybe you could have a mode where these audio cues go over the headest if is is plugged into the controller, but the rest of the audio goes to the TV. I think this is possible with PS4, no telling about XB. I am more familiar with PS4.

Facts with Tom MacDonald, Adam Calhoun, and Dax
End racism
End division
Become united

2017-03-23 18:37:27

@15 and other respondants
I'm going to play devil's advocate here.

*Baseball isn't a universal sport*
As much as I know what a baseball diamond is, there are people who might not only know nothing about that, but also might start in a different order.  For example, I'd argue it's better like this for consistency:
first=x
second=y
third=A
Fourth=B

*Breaking imersion*

The general idea of having additional, non-immersive tones/clicks/sounds just seems really frustrating to me.  If I'm playing the game either alongside or as part of a session with sighted counterparts, I want things that I'll notice but won't be saying "ooh look.  This player's using accessibility features."  I know that NRS games break this idea, but it's less frustrating because interactible objects can be turned off and the tones are just soft enough that you know where they are, but they don't stand out too much unless you have the music off and even then it's not that bad.

*Use of HRTF*
HRTF wouldn't be reliable on consoles considering the technology is tricky enough to get working on PC.  From what I know, which admittedly isn't much, getting it to work on a multi-platform title would be tricky as well.

*Narrator/TTS VS Audio levels*

Narrator, in its current form at least, auto-ducks the audio of the game.  Yes, I would like that to be tweakable, but it would automatically solve the problem of the TTS being too low in volume.

The only other way I could see of not using obfiscation would be to simply call X Y A or B.

Auto-QTE wouldn't be that great, I agree.

I'll keep an eye on this topic but I think the only way we could give reasonable feedback would be to see a tech demo and work from that.

Regards,
Sightless Kombat.
***If you wish to refer to me in @replies, use Sightless***

2017-03-24 01:43:42

As I said about the baseball diamond, everyone finds their own way, I'm not suggesting everyone use that method. I just think TTS is too slow even if sped up. Plus, you have no way to know what people are using speed wise, someone could have it set really slow in which case triangle might take nearly a second to say, no way could you do a QTE that way. As for immersion, no way to avoid it, just get used to it because no matter what's done it will break immersion. The good thing here is you decide what breaks immersion for you, its a personal thing, a mental balance, so if you decide that little clicks aren't going to break immersion for you then they won't. A TTS engine saying square square triangle LB RB etc. is not immersion breaking? I don't know much about HRTF and how to use it but I have heard and demos of it.

Facts with Tom MacDonald, Adam Calhoun, and Dax
End racism
End division
Become united

2017-03-24 04:24:44

This idea might work, but it is also obvious that there are some questions about it. How will this work in the context of other sounds? If tones are played simultaneously, will players be able to untangle the tones quickly enough, decipher which buttons to press, and do that quickly? Maybe? I do not know if this is an option, but this might be a situation where testing a prototype with some users might help. Perhaps something as simple as playing the sounds randomly and having the player use the corresponding contrl--while the program records information on which controls were used and how long it took before a control was used after the sound played.  It might be possible to prototype that as a Windows app or maybe a web app.

2017-03-24 06:18:23

I think tones is the best option. TTS is unreliable on PS4, and so I wouldn't even base that in our discussions for the moment, if we're talking about cross platform.

I've discussed this with Musicman several times regarding the Rock Band highway, and I think the same tone idea could work here as well. I think no matter what option is used, there *will* be some overlap.

Tones for face buttons is all well and good, but we're dancing around a topic that Karen brought up in her original post, stick motions.

What about the QTEs that make you use a face button and a stick motion? Double wammy overlap!

2017-03-24 09:11:04

Hi.
What an interesting topic. I also think the tones would be the fastest way to hear what buttons to press.
Regarding rumble, it's already used in many games where you can feel if you have done it successfully. In some of the Telltale games I have played, I have used the rumble to make sure I did it correctly.
A controller can't vibrate in directions, only heavy and light vibrations, and the vibration volume.

Best regards SLJ.
Feel free to contact me privately if you have something in mind. If you do so, then please send me a mail instead of using the private message on the forum, since I don't check those very often.
Facebook: https://facebook.com/sorenjensen1988
Twitter: https://twitter.com/soerenjensen

2017-03-24 14:34:42

Just a heads up, I've moved this topic into general game discussion so hopefully more people will find it.

2017-03-24 17:50:38

@20 I wasn't suggesting a direction, I was suggesting the rumble motor in either the left side or the right side of the controller.

Facts with Tom MacDonald, Adam Calhoun, and Dax
End racism
End division
Become united

2017-03-24 19:55:00

ps4 controllers have a small and large rumble.
xb1 controllers have left/right rumble.

When attempting to support cross platform, both specifics are generally lost, so very few things use either.

The reason I'm asking about tones is to support 1 to 5 pass icons in Madden. A button icon appears above the player's head the first instant the game considers the player open to catch. Each tone would play once, but they can be in any order, and can overlap. I was considering using surround sound to help identify which side of the field each tone originated.

So this isn't really a QTE, but it helps to have a QTE discussion as a lead-in, and also because I'l like to have as many blind gaming use cases figured out as possible, for any game.

2017-03-24 22:35:26

Yeah that might work if it can be done through headphones.

Facts with Tom MacDonald, Adam Calhoun, and Dax
End racism
End division
Become united

2017-03-28 11:56:03 (edited by ianhamilton_ 2017-03-28 12:02:47)

"Tones for face buttons is all well and good, but we're dancing around a topic that Karen brought up in her original post, stick motions."

A range of different button actions as well as stick motions, for example:

- tap a button immediately when prompted
- tap a button a precise time after being prompted, not too early or too late
- tap a button repeatedly and quickly until prompted again
- hold a button until prompted again
- rotate a stick in a single circle
- rotate a stick from one specific direction to another specific direction
- rotate both sticks simultaneously at the same speed with the same start positions but in opposite directions, until prompted again
- push both sticks apart
- push a stick in a specific direction
- push both sticks in the specific direction
- waggle a stick from side to side
- use a stick to move a reticle to a specific coordinate

And  'buttons' can mean all the face buttons, shoulders, triggers, l3/r3

So for example Asura's Wrath, Tales From The Borderlands, Shadows of Mordor, MKX and Madden (the above list of actions comes from this small selection of games) might all need quite different approaches to each other, the best approach for one won't necessarily be the best approach for another.