1 (edited by realnc 2017-11-02 22:13:30)

Hello.

I am trying to make an Interactive Fiction interpreter (you could call it an "engine" if you will) accessible with JAWS. The application uses Qt 5. For the curious, the application in question is QTads (http://qtads.sf.net).

Accessibility in Qt is provided though the IAccessible2 interface, wrapped in Qt-specific classes, and I have implemented the required callbacks in a way that allows me to see what the screen reader is calling. When running NVDA, it works fine. NVDA queries the role of the output window, for which I specify QAccessible::EditableText, and it then requests a QAccessible::TextInterface, for which I return a QAccessibleTextInterface object (which translates into an IAccessibleText object). NVDA will then call characterCount() on the returned object (this is the wrapper for nCharacters() in IAccessible2) and then the various text extraction methods, like text(), textAtOffset(), offsetAtPoint() (when moving the mouse over the text), etc. It then speaks the text I return in those methods.

JAWS, however, doesn't want to play along. After it queries the object role, it performs a few calls to characterCount(), selection() and attributes(), but calls to text extraction methods never happen. QAccessibleTextInterface::text(), textAtOffset(), etc, are never called and thus the text is never spoken by JAWS. When the window is activated, JAWS speaks this out:

"Alt tab, game text, edit read only, insert f1 help, type and text." (Not sure about the "type and text"; it's kind of hard to tell what it says exactly.)

("game text" is the text I return when the QAccessible::Name text is requested though the QAccessibleInterface::text() function.)

I do not know where to go from here. Is this a Qt issue? Something I'm doing wrong? I can't tell.

My implementation looks like this:

I have a QWidget subclass, called DispWidget, that paints the text on screen, driven by the game engine's text layout engine. To make DispWidget accessible, I implement an AccessibleDispWidget class that inherits from QAccessibleWidget, QAccessibleTextInterface and QAccessibleEditableTextInterface:

class AccessibleDispWidget: public QAccessibleWidget,
                            public QAccessibleTextInterface,
                            public QAccessibleEditableTextInterface
{
public:
    explicit AccessibleDispWidget(QWidget* w)
        : QAccessibleWidget(w, QAccessible::EditableText)
    { /* ... */ }

    // ...
    // Overrides of the virtuals here -- see below.
    // ...
}

I then implement or override the appropriate virtuals:

// =========================================================
// QAccessibleInterface
// =========================================================
QString
text(QAccessible::Text txt) const override
{
    switch (txt) {
    case QAccessible::Name:
        return "Game text.";
    case QAccessible::Description:
        return "";
    case QAccessible::Value:
        return "";
    case QAccessible::Help:
        return "Help.";
    case QAccessible::Accelerator:
        return "";
    default:
        return QAccessibleWidget::text(txt);
    }
}

void*
interface_cast(QAccessible::InterfaceType t) override
{
    if (t == QAccessible::TextInterface) {
        return static_cast<QAccessibleTextInterface*>(this);
    }
    if (t == QAccessible::EditableTextInterface) {
        return static_cast<QAccessibleEditableTextInterface*>(this);
    }
    return QAccessibleWidget::interface_cast(t);
}

QAccessible::State
state() const override
{
    auto s = QAccessibleWidget::state();
    s.multiLine = true;
    s.readOnly = true;
    s.selectableText = false;
    s.editable = false;
    return s;
}

// =========================================================
// QAccessibleTextInterface
// =========================================================
// never called
void
addSelection(int startOffset, int endOffset) override;

QString
attributes(int offset, int* startOffset, int* endOffset) const override
{
    // Simple implementation for now as to not complicate things;
    // all text uses the same attributes.
    *startOffset = 0;
    *endOffset = characterCount();
    return QLatin1String("font-family:\"Open Sans\";font-size:14pt;font-style:normal;font-weight:normal;language:en-US;");
}

int
characterCount() const override
{ return length_of_text; }

// never called
QRect
characterRect(int offset) const override;

// never called
int
cursorPosition() const override;

// never called
int
offsetAtPoint(const QPoint& point) const override;

// never called
void
removeSelection(int selectionIndex) override;

// never called
void
scrollToSubstring(int startIndex, int endIndex) override;

void
selection(int selectionIndex, int* startOffset, int* endOffset) const override
{ *startOffset = *endOffset = 0; }

// never called
int
selectionCount() const override;

// never called
void
setCursorPosition(int position) override;

// never called
void
setSelection(int selectionIndex, int startOffset, int endOffset) override;

// never called
QString
text(int startOffset, int endOffset) const override;

// never called
QString
textAfterOffset(int offset, QAccessible::TextBoundaryType boundaryType, int* startOffset,
                int* endOffset) const override;

// never called
QString
textAtOffset(int offset, QAccessible::TextBoundaryType boundaryType, int* startOffset,
             int* endOffset) const override;

// never called
QString
textBeforeOffset(int offset, QAccessible::TextBoundaryType boundaryType, int* startOffset,
                 int* endOffset) const override;

// =========================================================
// QAccessibleEditableTextInterface
// =========================================================
// never called
void
deleteText(int startOffset, int endOffset) override;

// never called
void
insertText(int offset, const QString& text) override;

// never called
void
replaceText(int startOffset, int endOffset, const QString& text) override;

So all appropriate functions are implemented, and as mentioned previously, NVDA makes use of them and extracts the text just fine. Functions that are never called by JAWS are marked as such with  "// never called", so at this point their implementations are irrelevant and omitted to keep the above code short.

And this is where I'm at, with no idea on why JAWS isn't using the provided interface functions or how to proceed from here.

Thumbs up

2

No, this is a JAWS issue. And People say JAWS is so amazing when it doesn't read QT aps, and QT is where we're headed... Anyway, your nothing doing wrong. t's JAWS that's your problem.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.

Thumbs up

3

balls to NVDA. the only thing its good for are audiogames, and the windows 10 ocr. jaws is best with basicly everything else.

Thumbs up

4 (edited by Ethin 2017-11-03 18:18:00)

@3, Oh, really then? Then why does JAWS not work with any apps written in qT but NVDA does? Why are there ways to get NVDA to work with GNU VIM/GVIM but there is no way to get that to work with JAWS? Must I go on? (And if you want proof that NVDA works with VIM/GVIM, go look here.)

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.

Thumbs up

5

It would be nice if this would not turn into a JAWS vs NVDA battle.

Thumbs up

6

@5, things like this always do. However, as I said, it's nothing wrong with your code or your interface -- it's just the fact that JAWS can't interface properly with IA2 and most likely won't for a while.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.

Thumbs up

7

NVDA is a jaws knock off someone made because they were too cheep to pay for jaws. Sorry but that's the truth.

I am the blind jedi, I use the force to see. I am the only blind jedi.

Thumbs up

8 (edited by pitermach 2017-11-04 19:03:31)

I don't think you're doing anything wrong. JAWS is recognising the class correctly so whatever the issue is, it's something internal to JAWS itself. Because it's not an open-source program, you can't really do much investigative work other than what you just did. What I would now do is contact Freedom Scientific and echo what you said here, perhaps also providing an example program with your message. There is a quality feedback form on their contact page. They also recently opened a GitHub issue tracker for JAWS, but afaik that only pertains to its web support. But if you don't get a good responce from the feedback form it probably won't hurt filing a ticket there.

In the meantime, you could fix this in 2 ways. One would be to manually script your program with JAWS's scripting language. But that will probably end up being quite time consuming and require you to learn a new language just for JAWS scripting. So the other thing you could do is use a library like TOLK to directly speak text with screen readers. This would actually benefit not just JAWS as this way you could, for instance have new text automatically get spoken after you enter a command so you don't have to keep reviewing it. And for JAWS, you could implement functions that would speak the current line or character if someone presses arrow keys in the review text field. This won't be a perfect solution as it won't work as well with Braille displays, but should be OK for most people and is probably the easiest thing you can do without FS fixing things from their end.

The twins of Mammon quarrelled. Their warring plunged the world into a new darkness, and the beast abhorred the darkness. So it began to move swiftly, and grew more powerful, and went forth and multiplied. And the beasts brought fire and light to the darkness. - from The Book of Mozilla, 15:1

Thumbs up

9

pitermach wrote:

So the other thing you could do is use a library like TOLK to directly speak text with screen readers.

I didn't even know this existed. Thanks!

I'll do some testing with this before trying anything else and see whether this gives good enough results.

Thumbs up