2019-09-10 22:27:38 (edited by Ethin 2019-09-10 22:37:26)

Hey all,
I have some (hopefully) excellent news regarding the accessibility of computers running UEFI-based firmware. On August 11 I joined the EDK2 RFC list, which is a list where people can propose changes to the EFI Development Kit 2 (EDK2). EDK2 is pretty much the reference implementation of UEFI, and is fully open-source. That same day I submitted an email to open discussions on implementing accessibility features as a mandate in the specification. Today (at 11:25 AM CDT) I got an email back from Leif Lindholm from Linaro describing exactly what EDK2 can and can't do regarding the UEFI specification. Additionally, he CC'd the response (and I have CC'd those same people) as well to get suggestions and ideas from them; those other people include Rafael Machado (who did a PhD on firmware accessibility, including a prototype implementing an audio driver for EDK2), Andrew Fish from Apple, Laszlo Ersek from RedHat, and Michael Kinney and Cetola Stephano from Intel. In summary, the eventual goal is to provide a reference implementation of an accessibility subsystem in EDK2, which would take "a pretty spectacularly incompetent product owner not to incorporate it in their products", according to Leif.
If all goes to plan, we'll be taking it by stages and not implementing everything at once. (That includes the fact that this first iteration will not include support for braille displays.) In the first stage, we hope to:

  • Implement protocols for communicating with audio outputs;

  • Implement protocols/hooks/events that can be intercepted in the Human Interaction Interface subsystem (HII) by a foreground/background UEFI driver/application; and

  • Implement some kind of method to allow the accessibility protocols to read what is drawn to the text when navigating around.

The third item on this stages agenda will prevent the accessibility subsystem from doing a "say all" every time something new pops up, which would get annoying. I would like your guys's opinions on this progress and what you would like to see in the future. Do remember that this is not an operating system. This is a preboot environment that is only used when your computer is starting and not while an operating system is active. Therefore, please keep your suggestions to "reasonable" ones; i.e.: we don't need speech recog or natural language processing in UEFI.
You can find the full discussion as is at https://edk2.groups.io/g/rfc/topic/32841464

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.

Thumbs up +1

2019-09-10 23:20:02

Impressive work man, getting all these people on the same page by having all these connections is exactly what we need if we are ever to get this done.
Have you considered also roping in some motherboard manufacturers, people from Microsoft, or media sources that may be interested in such a thing?

Buy the worlds greatest machine gun. It's about 50 inches in length, It's huge! & it can shoot really far, &, it's cheap!
Lets demonstrate this: stand still Thom...
Okay ma.aaa!
Oh! crap!

Thumbs up

2019-09-10 23:58:10

@2, I have not. I reached out to the EDK2 developers mailing list after I was referred to it, and so haven't been able to yet. I do need to include someone from MS to balance out the chain I have already established though. I never considered taking this to the windows logo program... I thought that was discontinued...

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.

Thumbs up

2019-09-11 01:06:32 (edited by Chris 2019-09-11 01:10:36)

It's nice someone is finally trying to solve this issue. Here are my thoughts in no particular order. Also keep in mind I'm not a developer, so I don't know how hard some of this would be, and I don't understand most of the jargon thrown around.

I assume this is only for new systems released in the future? Would existing manufacturers have to release updates if they used UEFI in their machines? All options should be spoken and the navigation to those items should always be possible using the keyboard. As far as speech goes, I don't care what is used, because any speech in those screens would be better than nothing at all. Would eSpeak work?

Regarding audio, perhaps including the generic USB drivers would help? Even if sound couldn't be heard from the built-in card, it should be possible to connect a USB headset or other audio device and get audio. Then again, wouldn't the manufacturer be required to properly support audio anyway? I always thought the firmware was specific to its machine, so this shouldn't be an issue unless there's something I'm missing.

That's about all I can say on the subject. Good luck. This has the potential to open more employment opportunities for blind IT professionals. Oh, please get Microsoft involved. If a company like Microsoft is mandating something like this, maybe people will take it seriously. I don't know who you would contact specifically, but maybe try @msftenable on Twitter.

edit

I don't know if this is the right place for something like this, but perhaps it might help if you also posted to the Microsoft Accessibility Uservoice forum. https://microsoftaccessibility.uservoic … y-feedback

Grab my Adventure at C: stages Right here.
You may access my NVDA Remote, Three-D Velocity, Sound RTS, and Road to Rage servers by using the address christopherw.me. Road to Rage uses the default 6789 port.

Thumbs up

2019-09-11 01:13:23

This is some exciting stuff, Ethin. I shared this discussion over Twitter and sent it to Microsoft's accessibility team to look over.

I'm the only adventure at c: master hahahaha I have unlocked just about everything!

Thumbs up

2019-09-11 01:45:08

@4, in order:
1) Probably yes. Updating existing machines would require a firmware update, and updating firmware is a nontrivial (and unsafe) task. It is also unlikely that this would be standardized in UEFI 2.9 or 3.0. 2.9 is definitely unlikely, 3.0 is possible but still unlikely (it depends on how fast things go).
2) ESpeak would be hard to port to that environment. The major hurtle is getting it to compile under non-VS 2005/2008 environments, which is what its windows binary targets. The UEFI specification requires that UEFI applications/drivers use the PE executable format, but we'd need an up-to-date VS implementation to build for it (VS 2017 and beyond is required, I think). I don't think VS2015 supports UEFI.
3) Yes, every firmware implementation is unique to the machine/manufacturer. One manufacturer may have a master implementation that they use as a "generic" implementation, however they would need to modify that implementation whenever they make a new motherboard.
@jack, thanks for submitting that to MS -- I have no idea who to go to on an issue like this, but getting them on board and making their bootloader accessible would be a massive leap forward. If we have to, then we'll take one component at a time.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.

Thumbs up

2019-09-11 01:51:37

Update: so I've just given those guys over at the EDK2 RFC mailing list a link to this topic. If they choose to join then we'll have some excellent individuals joining this discussion before long.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.

Thumbs up +1

2019-09-11 03:25:28

@Ethin No problem.
@Chris: UEFI is entirely controlled by keyboard actually, so that's already taken care of. It's just the speech that's going to be hard. Especially when there's no serial port equivalent that is simply analog, i.e usb is not one of those, so having a hardware synth option isn't a good solution notwithstanding the act hardly anyone still has those (I still have my Dectalk Express) but with soundcards no longer having built-in speech processors, we now have to get a speech system to compile and run under UEFI.
@Ethin: Somewhat related to this is a fellow that I know on blind-bst with a talking preboot environment that he uses for his custom build machines. I can ask him if he'd be willing to make contributions.

I'm the only adventure at c: master hahahaha I have unlocked just about everything!

Thumbs up +1

2019-09-11 03:28:58

@8, The more the better. I'm curious how he managed that.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.

Thumbs up

2019-09-11 03:38:24

This is fabulous stuff, Ethin. Thank you much. Will be keeping tabs on this thread and the links you've posted.

The Beast adopted new raiment and studied the ways of Time and Space and Light and the Flow of energy through the Universe. From its studies, the Beast fashioned new structures from oxidised metal and proclaimed their glories. And the Beast’s followers rejoiced, finding renewed purpose in these teachings.
from The Book of Mozilla, 11:14

Thumbs up