2020-01-15 18:12:24

Hey folks, wanted to go semi-public about this project I've been working on for a while now.

Godot is an open source game engine in the spirit of something like Unity, by which I mean you create games in a graphical environment. So it's a bit more complicated than just editing scripts, but it has some key advantages:

  • Godot has a bunch of ready-to-go classes for just about any game development task you might imagine. No need to hunt up and integrate some external library. 3-D audio? Check. Multi-player networking? Check. There's even upcoming Web RTC support for chat and multi-player browser-based games if you like.

  • Multi-platform. Godot won't make your games playable on a touchscreen if you've developed for keyboards/joysticks. But you can at least run your code on Linux, Windows (desktop and universal), MacOS, Android, iOS, and the web. And UWP support means games should run on the Xbox One. Ports to other consoles are also possible if you're willing to hand off code to a third party with the necessary dev kits and signed NDAs.

  • A solid community with lots of documentation and helpful folks. Unfortunately, due to Godot being graphical, many of the tutorials contain screenshots. But at least they exist, and are accessible with help.

Anyhow, I've built a screen reader in Godot's built-in scripting language. I've managed to get enough of the editor accessible that I'm able to build games on my own, albeit with the help of external tools like VSCode to make up for the inaccessible built-in editor. The same screen reader also runs inside of Godot games, and makes their menus and other UI components accessible. Basic touchscreen accessibility also works, and text-to-speech is confirmed working under Linux, HTML 5, and Android and likely works under Windows as well, but I can't test this because I only have a VM and can't get hardware-accelerated GL working. I've exported the little Asteroids-like shooter I've made to my desktop, the web, and my Android phone, and while it isn't yet playable under Android until I build an accessible touchscreen joystick, it shouldn't be hugely difficult. But the game and GUI do work on the web, which I thought was pretty damned impressive given that all I had to do was create JavaScript implementations of my TTS functions.

Before I drop links, I'll note that this isn't yet a polished experience, and I'm not going to pour lots of support time into helping folks jumping ship from BGT and thinking this is their next alternative. As an example of what I mean, I often use the editor to create scenes to get started, but they're just INI files with a slightly less human-friendly format. Eventually I hope to make the editor more usable, but if you aren't capable of looking at an INI file and reasoning your way through how to set some values, this isn't for you. This isn't the newest, greatest way to build games. I've simply decided that the extra hoops and added complications are an acceptable trade-off for the possibility of easily releasing audio games on a handful of platforms, including the Xbox. But if you're looking for a fun and experimental new way to build audio games, and don't mind rolling up your sleeves and helping out, give this a look.

The easiest way to get up and running is probably via the accessible starter. Cloning that with --recursive gives you a basic starter game with the accessibility addon in the correct directory. You'll need to download the latest godot-tts builds as per the README to get Linux/Windows binaries, but once done, running "godot" in the checkout should speak "Hello, world." and give you a blank starter game. Otherwise, the plugin repository gets you directly to the addon without the starter template. It's probably a good idea to read the README there, as it contains a list of known gotchas and rough edges.

Enjoy, and I'm looking forward to finding collaborators to help me improve this. I'm also planning on launching a few games based on this work, and would really like to build it out into a tool that empowers blind and sighted game developers to collaborate on building even better accessible games.

Thanks for reading.

2020-01-15 19:29:44

This sounds amazing.
For now, I'll probably wait for the games. If it's any consolation, I actually really like the idea of playing an asteroids-type release, as we, honestly, don't really have one. Torrent never got released, and USA asteroids didn't become a thing because... well... I think we all know what happened there, and even after all this time, it stings. That is: unfortunately Thomas Ward passed away.

2020-01-15 21:04:59

Hi.
Just took a quick glance at the starter after setting it up and it looks very good. I myself discovered godot about 6 months ago (I think, or was it 3-4 idk) and found it's lag of accessibility disappointing.
Very good job with this, I'm looking forward to see what happens with your project.
And now for some small feedback points:

Your readme does not state, that you specifically require godot 64 bit, so after downloading godot 32 bit (out of habit) I tried and was very close to quitting because nothing was accessible (like normal), just by sheer luck I tried the 64 bit version and everything worked just fine.

If you like what I do, Feel free to check me out on GitHub, or follow me on Twitter

2020-01-15 21:23:22

I actually want to check this out. Seems neat.

Much less active on this forum than in the past.

Check out my live streams: http://lerven.me
follow me on Twitter: http://twitter.com/liamerven

2020-01-15 21:23:43

@2 Heh, I was the guy who wrote Torrent. Real life unfortunately got in the way, and I didn't finish it off. My current game, Onslaught, is my attempt at that after another decade or so of life experience. Hopefully I'll pull it off this time.

@3 Thanks for the feedback. I'll add that to the README. Unfortunately I can't get godot-tts building under 32-bit Windows. If anyone wants to take a crack at it, the repo includes my .gitlab-ci.yml containing the commands I use to build it. I have a Windows CI server, so if you can add the commands necessary to get a 32-bit build working, I'll give it a shot. IIRC, even after adding 32-bit Rust components, the builds were still referencing 64-bit libraries/DLLs somewhere and I'm not sure where. I'm a Linux person, so Windows tooling is a bit above my paygrade. smile

2020-01-15 22:35:14

I remember starting this hole accessibility thing -- I think it was me who started the discussion anyway, I can't remember and don't want to dig through ridiculously long message threads to find out. smile (Yes, the thread about this is looooooooooooong.) I would love to contribute to this but I know pretty much nothing about Godot, though I distinctly remember me trying to get TTS in at least one of the parts and failing at it. But I did try. :-) I've been watching the accessibility thread and am happy to see all our collaborative (well, sort of) efforts going somewhere. It would be nice if Godot just used normal GUI widgets instead of using ultra-fancy, 3D/custom GUI widgets (WTH is the insentive behind that, BTW?), but alas...

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-01-15 23:07:50

@6 I hear 'ya. Unfortunately, I don't know what sort of standard GUI widgets would work well on Windows, Linux, iOS, Android, UWP, macOS, etc. Possibly QT, but then you've got licensing to think of, and I don't know that QT is a great fit for games.

As annoying as this work is, I do like that my current game seems to work mostly unmodified everywhere. Touchscreen accessibility works on my Linux desktop, even though AT-SPI doesn't support touch events. The GUI works fine in HTML 5, even though the standard there is to just render GTK/QT to a canvas and pretty much kick accessibility to the curb. It's been a huge amount of work to get this functional, and there's more to do, but all that's needed to make this work on a new platform is to port the TTS addon, either by writing a bit of Rust or adding additional GDScript. When I built my game for HTML 5 and it just worked once I wrapped the JavaScript TTS API, I kind of became a convert. smile

2020-01-15 23:16:37

Can I use Godot to write games in C#?

If you want to contact me, do not use the forum PM. I respond once a year or two, when I need to write a PM myself. I apologize for the inconvenience.
Telegram: Nuno69a
E-Mail: nuno69a (at) gmail (dot) com

2020-01-15 23:24:25

@8 Yes. I've never tested this with my addon but don't see why it wouldn't work. You'll need the build that includes Mono, and I don't think your platform support is quite as broad (no Android for instance, though I think they're working on it.)

But GDScript isn't too bad. Looks a lot like a simpler Python with added constructs for game development (I.e. signals for event-handling.) You can also use C/C++.

To be clear, you're still using the graphical editor, though. Essentially, you lay out a tree of nodes in the graphical editor, then attach scripts to those whose behavior you want more control over. The C# support means you can write these scripts in a language other than the built-in Python-like scripting language. It may be possible to write a game entirely from scratch without the editor, but at that point you're better served by something like Monogame.

2020-01-16 01:16:44

@7, the fancy widgets and custom stuff could be used in games. But to use them in the editor too just seems like nonsense to me. And I'm quite surprised ATSPI doesn't support touch events. Sounds like it either needs an update or a new accessibility framework needs to come along to conker ATSPI and all its complications and make something that's actually modern. The problem with that, of course, is that then you'd need to somehow get all these other apps and GUI frameworks to use it.
When you say you "became a convert", did you mean you "became a convert" to JS or something else? You lost me there.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-01-16 01:36:42

@10 Convert to Godot's use of its own GUI widgets. I mean, in an ideal world we'd have a great, accessible, cross-platform UI toolkit. But failing that, we could do much worse than a GUI toolkit with bunches of hooks and easy plugability.

2020-01-16 02:21:45 (edited by Ethin 2020-01-16 02:26:27)

@11, true that. And perhaps that GUI toolkit can be made into an easy, 100-percent accessible GUI toolkit. And its already cross-platform too! smile Is there any way to "extract" the GUI widgets and toolkit that Godot uses into its own independent project? If I can learn how the code works I'd be happy to implement (say) ATK support for it at least. I don't know about MacOS accessibility though, since Apple uses Objective-C and I don't know how to call Objective-C code from C/C++, but Windows UIA/IAccessible2/MSAA is definitely possible. Same for the other platforms it supports too.
Edit: 11, sent you a PM.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2020-01-16 05:50:59

thanks! works like a charm! smile

so, now I have to learn the sections of the godot and keystrokes to move faster on the UI, but is a very nise job provide accesibility to the editor.

And works very well.

thanks

2020-01-16 09:13:11

Great thing your doing here Nolan!
Aside from where ever it may go, I really appreciate the effort.

2020-01-16 09:18:49

This looks amazing, I have never heard of Godot, but if you can do so much with it, then there is hope!
Have you considered adding an element to the DOM that is an aria-live region? This would allow you to interface with any screen reader that supports aria on the web exported games.

2020-01-16 17:20:21

@12 Extracting it would be tough, but fortunately the UI controls are essentially descendants of the Control node, so they're segmented off into their own tree of the project. You're probably better off working in the platform/ subtree, though, and implementing the accessibility support there. I don't think there's much utility in separating off the GUI entirely--if all you want is a completely accessible cross-platform toolkit, it's probably better to fill out accessibility implementations for WX/QT. Happy to be proven wrong, though.

@15 What would be the purpose of the live region? I suppose it could be a fallback for any browsers not supporting the JavaScript TTS API, but the browser platform also needs Web Assembly, and I suspect the intersection of browsers supporting wasm and not the TTS API is slim to none. But maybe I'm missing something?

Thanks for all the nice comments so far. Can anyone confirm that this works under Windows 64-bit? I'd like to start pushing out games in a few weeks, and Windows is a target I unfortunately can't test.

2020-01-16 18:23:35

I just tried your plugin out on Windows X64 following your readme. It came up talking right away, though using SAPI rather than a screen reader, which is what I would prefer. Other than this it seems to be working and the hello world example also spoke its message

On the topic of the web speech API and live regions, they should definitely be an option, partly because most people prefer to hear their speech via their screen reader, and partly because the web speech API implementation isn't very consistent, especially on mobile. Safari on iOS worked sometimes, but only if headphones were plugged in, and didn't seem to work at all on Chrome/Android/ So the live regions should be available as a fallback.

<Insert passage from "The Book Of Chrome" here>

2020-01-16 22:05:59

@17 Got it. I may have to accept community contributions for that one. I don't want to abandon the API entirely, but live regions are going to step on its toes, so we'll probably need folks for whom it fails to work to make it work for them and contribute back fixes. Fortunately the web is only one of half a dozen supported platforms, so I'll probably just leave it be for now.

Thanks for confirming that things work under Windows.

2020-01-17 04:05:06

If you made a javascript wrapper for the Web Speech API, then you should have access to the dom. All you would need to do is insert the following element into the dom before any speech is said, so onload:
<p aria-live="polite" aria-atomic="true" id="spk"></p>
Then the speak function would just set the innerHTML to be the text that is spoken. This interfaces with pretty much all screen readers out of the box.
The web is a fantastic platform, as you can test it on Linux and know that it works on any other platform with a browser. If they have a progressive web app option, then people can install the game from the website and it will act just like an installed app.
I, for one, would play your game, without a second thought, if all I needed to do was click a link and not need to install anything. I could just wait for the game to load, and begin playing. If I liked it, then I could install it, either through the progressive web app, or the executable.

2020-01-17 05:25:16

@19 To be clear, the issue isn't that I don't know how to do an ARIA live region. This isn't meant to make screen readers work with Godot. It's meant to make Godot accessible, full stop. JavaScript's speechSynthesis API may have cross-device complications, but I'd argue that screen readers bring their own complications. I don't necessarily want to prevent someone from using their screen reader, but I also don't want to require it either, potentially ruling out an entire class of games because they don't work when a screen reader is running.

As an example of what I mean, I'd like to build a touchscreen roguelike at some point. I'd like to support both explore-by-touch, where you can actually feel individual map tiles, and swipe navigation, where you swipe up/down to select between classes of visible objects, then right/left to place focus on them. I have no idea whether that sort of thing will work with NVDA running, whether it will also work with JAWS...and foregoing speechSynthesis to adopt a screen-reader-only solution means I'd have to figure that out. With things as they are now, I can simply tell everyone to disable their screen readers and have a mostly identical experience. If someone doesn't like that, well, there are other games and I can't please everyone. smile

2020-01-20 15:10:48

Hey,
your tts-plugin hasn't builded artifacts anymore. I want to try this out. How I could build this for myself? Or can you restart a build?

sorry for my english, I hope you have understood me.

2020-01-21 08:42:07

From what I understand, the development environment is different than the compiled code, so the two should have different behaviors. If that's not the case, and Godot allows you to build projects in the browser, I'm going to be very impressed.
I would have a simple spk function made available to the user that either sends the message to the screen reader, or to the TTS:
spk(text: string, screenReader: bool, voiceObject: object)
That way, you can have a game that uses TTS only, a game that uses the screen reader only, or a game that uses both.
In my experience, players really prefer their own screen reader if they get the chance.

2020-01-21 17:56:14

@21 Ack, yeah, I see that. Give me a few days to figure out a solution. My current system is pretty brittle and I knew that going in. I may be able to use GitLab Releases, which hopefully have been better integrated into their CI offering since last I looked. I'll update this post when I manage something, but for now it's fairly easy to rebuild with a Rust environment, Clang 9.0, and following the commands from .gitlab-ci.yml.

@22 I hear the concern, but we can't support screen reader use on all platforms. So there needs to be a solution that respects the screen reader where available, uses TTS where not, and doesn't just fail because the game's author decided everyone should use a screen reader and that's that. My hands are pretty full both building out this support and making games with it, so if you or someone else can design such a system within my requirements that it works everywhere, I'll accept the PR. In the meantime, I don't think the Unity plugin leverages screen readers at all, and I certainly don't see studios using it grinding to a halt because players prefer screen readers and won't touch accessible games otherwise. So I'm personally focused on the pretty high-hanging fruit of just working everywhere with no fuss. smile

2020-01-23 14:49:58

@23:
yeah, I tried to follow the commands in the gitlab.ci file. I had installed rust and clang 9.0.
But I get errors, when I try to build the plugin:

I have pasted the console output in pastebin:
https://pastebin.com/ENvEfeFr

It looks like a failure of tolk-sys, some file is not found, but which?
Any Idea?

sorry for my english, I hope you have understood me.

2020-01-23 15:14:43

@24 Is clang in your path? Maybe run it with --verbose to get more details? I'm not really a Windows person so am not sure.