I'm going to have to go with BGT, just because it does handle a lot of the complicated stuff for you as a beginner, and is tailored especially to making audiogames.
As has been said, in the long run there would be other languages that will allow you to do things that BGT is not meant for, but I wouldn't let that distract you from the question you've asked. The easiest to learn, and get audio games up and running, is probably BGT. When you've learned a lot more, and eventually need to do projects not well suited to BGT, that is the time to learn something new. People get a little ahead of themselves trying to plan for something that may be years away, and it can complicate things in the here and now.
I've never told anyone to avoid playing Castaways because it is nothing like playing Swamp. Someone plays Castaways, learns the controls, and spends time playing the game. When they want to move on to a new game, they learn the new controls. With games no one worries that the current game doesn't have the same controls as future games you'll play, because we all know that learning the controls just comes along with learning each new game. Programming languages, in my opinion, should be treated the same way. If you're able to learn how 1 language works, you can learn the next one even faster, and the next one and next one. Take them 1 at a time and life is much easier than trying to find the perfect one that has the "same controls" as future languages you're possibly going to use years from now.
Just my opinion.
- Aprone
Please try out my games and programs:
Aprone's software