So now there are other ways to access GPT 4 without needing to subscribe to plus?

@29 So what is your secondary language? Or the one you speak well enough to get image descriptions in?

Can I get a gpt 4 key from someone? It doesn't work for me with my free key.

I know it mentions ability to read graphs, and I have used be my AI to describe graphs on phone before, but I'm wondering how accurate the descriptions are, like can you use this to pass a math test with inaccessible graphs or is it too risky due to hallucinations?

I've been blown away by the support and use cases people have been coming up with, thank you!

To address a few questions:

This was the intention, yes. It was supposed to also take pictures in lower resolution and then compress them to cut down on upload time. I'm still considering how that piece should actually work because unsurprisingly, snapping a screenshot, modifying the resolution, and asking the API to do the same doesn't actually result in any performance improvements--sometimes the opposite even.

The thought has certainly crossed my mind, and once I can ensure the rest of the features are stable, I might work on it. Problem is, unlike any of the other APIs, vision preview doesn't seem to remember the conversation. Which means I'll need to upload each message again whenever the user asks a question. This would get crazy expensive... and fast.
To be honest I was kinda holding out hope that Sama and team would address this limitation but given the shitshow going on at OpenAI right now, I've gone from bullish to just glad developers haven't lost the ability to build on the models in a weekend.

Easily. One caveat to be aware of: the NVDA settings dialog appears to intersept the enter, control+enter, and shift+enter keys. I'll have to find a way to overwrite this behavior, but it'll probably be a part of the next release.

Good catch! It's already doing this when you snap an object from the menu, but other recognitions are indeed performed on the main thread. This will be done for the next release as well.

Speaking of the menu, there is in fact a fairly annoying bug when we try to describe focus or navigator objects without using their defined keystrokes. It occurs because popping up the menu changes both the focus and navigator positions, and somehow simply caching and setting them again is insufficient. I'll continue to play around with some of the more obscure pieces of NVDA to figure out why this is happening. It's 100% possible, just a matter of finding and reciting the right incantations.

As a tip, if you add "for some one who is blind" to the end of the prompt, it appears to add greater detail to the explanations - undoubtedly a byproduct of the collab with Be my Eyes.

@Defender

Thanks. It's been a while, I hope you are well. Out of curiosity, what frontend? Is it the LLaVA demo that's been making it's rounds recently? Or MiniGPT4?

It's cool, but i often get an error saying the read operation timed out. What's wrong?

Incredible job! When Be My AI first started, I thought it was cool but wondered when someone was going to release something similar for Windows. A dream come true. Thanks a ton for this, Carter!
DJWolfy literally has no clue what they're talking about here. Yeah, the conversational part of the model does tend to perform worse in minor languages but it still gets the point across, even if with worse grammar sometimes. Czech, which is my native language, was only 0.07 % of the total training data used, and it still works. I have noticed no difference at all between prompting it to describe an image in Czech or English, the level of detail and accuracy is generally the same. I'm no researcher, true, but I've done enough practical testing to claim this with certainty. I still prefer English for chats, but I use both languages interchangeably for image descriptions with equal results. Hallucinations have nothing to do with the language used at all, ever.
Lukas

@40: That's awesome! Are you a ChatGPT user? If so, have you tried voice chatting with it in Czech? Their tts voices are really good at speaking Spanish, though they do have a slight American accent. I'm always curious to hear about how well they perform with other languages.
I think as of earlier today, all ChatGPT users can voice chat, though if you're not a plus subscriber you'll be limited to speaking to GPT-3.5. The tts engine should be identical though.

As much as I want to try this thing, Open AI reject my card for some reasons. So I can't top up my account.

oh yep. neither of them are working for me either!
guess we will be having a funny version number tomorrow after all! haha

So far, loving this thing. Awesome work! The one issue I have though is, if you change your prompt to ask for as detailed of a description as possible, the response will get cut off part way through. Is this something that can be fixed? Or is it a limitation of the platform that I, lacking knowledge of how this stuff actually works just don't know about.
As an example of what I mean,