2017-03-06 08:42:33

OK so, this will take some explaining.
As you may or may not know, in bgt, you can have arrays with up to 4 dimensions. That in itself is fine.
What I'm curious to know is if bgt alone is really terrible when resizing anything above two dimensions?
This is even more noticeable when using string based arrays.
Are other languages in general really bad at this or is bgt just really horrible with resizing arrays? In truth, I'm going to say it's probably bgt, but I don't have enough experience with anything other than bgt to say.
A bit of background. I am attempting to use arrays for a map system which includes full 3d support.
The most I have been able to successfully resize this array to was about 90 elements before RAM usage skyrocketed, or bgt produced an out of memory error, and usually both at the same time. If I did the maths, that's 90 per dimension, right?
So, 90 and 90 and 90, gives me 270 units to play with. That might seem like a lot, though in reality, it gives me 90 up, and 90 for playing with for the rest of the 2d navigation.
I know a lot of bgt devs are going to say, hey, use dictionaries! But in truth those are designed extremely poorly.
So much so that referencing them causes concern for the (very real) possibility of crashes.
And when a bgt game crashes, it often crashes spectacularly.
Can anyone else confirm if this is just an issue of bgt, and most other programming languages don't do this and it is time for me to move on for real this time? smile
Thanks for reading if you got this far.

2017-03-06 09:52:49 (edited by Hijacker 2017-03-07 15:09:17)

Hi,

I've to tell you that this time it is absolutely you who causes this problem, at least your calculation is wrong.
If you have some array, the way of calculation for you isn't addition, but rather multiplication. If you have some 90 x 90 array, you will calculate the amount of actual entries by calculating 90 x 90, or 90^2. If you have three dimensions, you'll calculate 90 x 90 x 90, or 90^3, which results in 729000 entries in this array.
At least that's only the amount of entries in this array, meaning, you'll need the data type of this array to calculate the actual size of this array. Let's say you're using int[][][] some_array and default the size to 90 per dimension, you'll probably get the size of this array by using 90^3 x 32, since a normal int should be something around 32 bits long. This results in a total size of 23328000 bits for this array, calculating it through 8 and 1024 results in 2847 kilobytes in-memory for this array, which is something about 2 Megabytes.
The following code executes fine for me and shows a total of 4212 Kilobytes in-memory (taskmanager) after creating the whole array, which results out of the actual program which requires memory itself and produces some overhead.

void main(){
  int[][][] test;
  uint i,j,k;
  test.resize(90);
  for(i=0;i<test.length();i++)
  {
    test[i].resize(90);
    for(j=0;j<test[i].length();j++)
    {
      test[i].resize(90);
      for(k=0;k<test[i][j].length();k++)
      {
        test[i][j][k]=0;
      }
    }
  }
  alert("debug", "finished");
}

Increasing the array to a forth dimension would mean the basic size grows to 90^4 x 32 bits integer size divided by 8 and 1024 and, because the result is big anyway, divided by 1024 again, results in around 250 Megabytes, which then is quite large, which is why I would like to try and evade using such arrays, three dimensions should more or less be enough in most cases.

Best regards.
hijacker

2017-03-06 10:33:47

Oh man. I didn't realize my math was that far off. smile
That would explain this entire thing then.
So I'm curious to know if you can explain the instability of dictionaries in bgt? I've tried using them for this map system, it does work, though is not very stable and has tendencies to crash.
Thanks!

2017-03-06 10:57:42

Hi,
in this case you need to explain your actions further too us, since all things i'm doing with dictionaries do work just fine. Over all I think all errors BGT produces are quite logical and all crashes happen on purpose, just as expected, so it's almost always the user behind the code who is responsible for these crashes. So, if you don't mind explaining the actual steps you're taking to implement a map system using dictionaries, i'll be glad to assist you fixing those errors.
Best regards.
Hijacker

2017-03-06 11:24:23

So, initially, I have player x, player y, and if I'm going full 3d, player z.
I have a dictionary map, who's keys are actually coordinates on the map.
These keys are set to different values which are surfaces "rock", "grass", "concrete", etc.
When I'm done setting up the map (usually by way of forloop), the main loop takes over.
When the player moves in any direction, I check with the dictionary to see what sound file I should play, in accordance to the surface they are standing on, based on the coordinates (key) I retrieve from the dictionary at the player's position.
I also check to see if they are about to run into a wall, to play echo steps at the wall's position.
Walls are, of course a surface as part of the map dictionary.
The game works and the map system works, as I stated though, using dictionaries it has tendencies to crash.
Hope this helps.

2017-03-06 11:43:40

Well yeah, this sounds all fine, even if I wouldn't do it like this, an array should fit this situation better I think. Anyway, without the exact error message bgt throws when crashing there's no way to help you, since you didn't even mention any code here. The system you're describing is fine, though improvable, so it must be your code which is wrong.
Best regards.
Hijacker

2017-03-06 12:25:20

It unfortunately just freezes with no error messages. The last sound(s) that were playing continue to play, although a very short chunk.

2017-03-06 13:49:26

In your dictionary, you use strings like "rock". As far as I understand, using string in this particular situation is a bad idea. Firstly, each letter takes one byte. So a string like "hardwood" would take 8 bytes, while if you use a int8 or uint8 and assign a number (between 0 to 256 for uint8 and -128 to 127 for int8) for hardwood, it would only take 1 byte. Secondly, when you want to compare two tiles, the engine has to compare character by character and in a big game it will slow down the execution. Arrays of uint8 is the best in my humble opinion. that 90x90x90 array you wanted, if you make it as uint8 there won't be any problem, it'll only take 711 kb. I doubt you would have more than 256 tiles for your game, and if you have, use a uint16.

Cheers.

2017-03-06 14:06:39

Yeah, I have been told to do that several times, use ints instead of strings. I'm glad you guys put this into perspective for me, I really did not know I was the one destroying blast bay game toolkit.
Thanks again guys!

2017-03-06 15:10:21

Yep, that's actually one thing which can be improved here, and this isn't just related to BGT, but will be this way in any low-level programming language where you're able to handle data types yourself, such as BGT, C or C++ and those languages. Python e.g. doesn't need this level of abstraction, since most scripting languages will handle the different data types automatically for you, which therefore results in less performant, but much simpler code.
The freeze in your code might be related to some really hard mass-comparisons of strings inside your code, but honestly, those operations should just increase the load time drastically, but not make the program freeze completely. Anyway, try to avoid dictonaries for such simple things like maps and such stuff, since arrays are much more performant and simpler for those purposes.
Best Regards.
Hijacker

2017-03-07 01:47:54

@8, each character on the string is 8 bytes.

2017-03-07 03:44:04

I believe a character is 1 byte which is composed of 8 bits.

If you have a dictionary of 8100 strings, your programming shouldn't be freezing.

The reason your math was off: think of a multidimensional array as a list of lists. This is simplified out to a grid so you have the first element at (0, 0). Which is essentially saying the first element of the first list of the array.

The initialisation code above I think is wrong, it should be:

void main(){
  int[][][] test;
  test.resize(90);
  //No need to declare the variables before the for loop, you can declare them on the same line.
  for(uint i=0;i<test.length();i++)
  {
    test[i].resize(90); //The original code was just resizing the first dimension repeatedly instead of using the array at the index i.
    for(uint j=0;j<test[i].length();j++)
    {
      test[i][j].resize(90);
      for(uint k=0;k<test[i][j].length();k++)
      {
        test[i][j][k]=0;
      }
    }
  }
  alert("debug", "finished");
}
Deep in the human unconscious is a pervasive need for a logical universe that makes sense. But the real universe is always one step beyond logic.

2017-03-07 04:27:22

Use uint8 instead if you wish to preserve memory.

2017-03-07 15:06:12

Hi,
you're right, I missed the index there. Thanks for clarifying this, I'll update this bug above so people searching this forum for some response won't get bothered with wrong answers.

Yes, you can define variables inlined, but it's better to define the variables at function beginning, so other people don't have to mess around and search where variables get actually defined or declared. It's simply one part of writing good code. Not mentioning that someone has this way of coding in his blood if he actually migrated from the C world, where you have to define your variables before the actual function body starts.
Best regards.
Hijacker

2017-03-07 19:11:23

@Hijacker, can you clarify your last statement about not being able to inline declare variables in C? I strictly recall this being possible. In Ada, it's like you said: you have to define (or, at least, predefine them and the types they hold) all of your variables before the function body starts. However, you certainly can do:
int x() {
int w=5, y=3, z=12;
return w;
}
The lack of using those three defined vars will raise a warning but is perfectly okay.
In C++, you can even do:
template <typename T> // Not possible in BGT. Sorry, BGT lovers!
T x (T y) {
T a, b, c, d, e, f, g;
// code...
}
Or, in D:
pure @nogc @safe abs (T) (T val) {
// Code...
}
I can go on, and on, and on... hell, in Go, you can return functions, and write functions as parameters, something you definitely can't do in BGT! Man, I could write a hole list of cons why BGT is a bad scripting language for anything serious, and the cons would outweigh the pros. big_smile I hope this doesn't start a debate, but why, exactly, is BGT so popular when there are programming languages out there that can do far more than BGT ever could do? It's not even being maintained, meaning that your not going to be using the latest programming technology, which, in turn, is going to isolate you from the rest of the programmer community, and you most certainly won't be able to integrate with one.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2017-03-08 01:08:49 (edited by Omar Alvarado 2017-03-08 01:13:33)

Hi Ethin,
I myself use bgt because I don't quite have the knowledge or patience to, gather all the billions of dependencies it takes to create a decent audio system, not forgetting most people end up writing there own take on sound positioning of course, keyboard input, mouse if it's needed, speech, networking if needed which most do for update checking, file parcing if your game has save data, and file encryption/decryption so people don't mess with your save data.
And don't forget, you also get to have fun putting it all together for a test release build, or a final release so your users can play your game without requiring the language of choice, and therefore revealing your source.
I did try python and was actually liking it, really I was. But the frustrating compilation issues related to dependencies and the fact I learn slowly made me go back to the drawing board.
I applaud those who have the patience for this kind of thing, but I, am not one of them. I would have probably had it if I wouldn't have started out learning bgt several years ago.
I'm a slow learning man, and only recently have I begun playing with pure basic, you might find my topic on the forums about that recently.
Again, I applaud those of you who can actually program in mainstream languages like c#, c++, python etc. My respect and hat off goes to you.

2017-03-08 02:28:40

In C89 you have to declare all variables at the start of the function. Also doing something like

for (int c=0;c < 5;c++) {}

will just not work. C99 and C++ changed this.

Deep in the human unconscious is a pervasive need for a logical universe that makes sense. But the real universe is always one step beyond logic.

2017-03-08 10:05:47 (edited by Hijacker 2017-03-08 10:08:41)

That's exactly it, thanks stewie. This works, though:

int foo() {
  int a=1,b=2,c=3;
  return a;
}

But declaring variables after executing some statement won't work, for example:

int foo() {
  int a=1;
  bar();
  int b=3;
  for(int c=0;c<bar();c++)
  {
    laugh();
  }
  return c;
}

This would crash even two times, because b was defined after executing bar() and because c gets defined inline in the for loop head. That's the C89 standard, as stewie already said, and to maintain as much backwards compatibility it's recommended to do so in even later C standards. That's why it developed to be "good code" to declare variables at the beginning of the corresponding code, at least in team-environments, so people don't have to bother finding the actual variable, checking it's type and actually setting breakpoints to debug them correctly. and all that other stuff you can imagine to do with variables at all.

Best Regards.
Hijacker

2017-03-10 00:13:04

re: 7, I don't know if dictionaries have anything to do with it, but it sounds like you're aving a similar problem to the one I keep running into. That is, at seemingly random moments, bgt will hang, looping the last second or so of sound. Weirdly, if I have background music playing, it is unaffected, and keeps playing normally. The program eventually triggers the Windows "This poogram is not responding" message. I seem to get this at least once a day, as of late. I think I've seen it more than once today. I can think of nothing in particular that these situations have in common, other than that I can't remember any in menus or scenes, only during gameplay. But I could be mistaken (the majority of the time will be in gameplay, after all.)

看過來!
"If you want utopia but reality gives you Lovecraft, you don't give up, you carve your utopia out of the corpses of dead gods."
MaxAngor wrote:
    George... Don't do that.

2017-03-10 05:31:17

Hello.

I suspect also that the "get" method of the dictionary object can crash BGT. This seems to be related to the data type. If you try to use the "get" method with a string data and a double variable, BGT crashes. So you have to check always if the variable type is the same that the data. It's probably safer to use always strings in a dictionary and to use "string_to_number" function to convert into double.

2017-03-10 17:51:18

yes, i agree with @20

If that's helpfull, why don't you press Thumbs up?

2017-03-16 00:44:43

Just my $0.02:
BGT is a bit of a love/hate thing for me.
BGT is fine with memory usage, but it's shortcomings really show when it comes to other resource usage. BGT is slow for a number of reasons:
First, There is clear evidence that BGT is processing window events from the operating system in the same thread as the virtual machine is running. As far as I'm concerned, communicating with the operating system should be done in the program's main thread, and the game engine should be running in a second thread. This adds a bit of code complexity when the game engine tries to update the window, since you can't modify a window from a thread other than the one that created it you have to queue events for the main thread to process.
This would result in a fair performance improvement as window events can be time consuming.
Secondly, since BGT hasn't been updated in some time it's built on quite a dated version of AngelScript. AngelScript is a really kool product, but it's major struggle over the years has been surrounding array access overhead. This has been improved in recent updates, but BGT can't take advantage of the improvements unless it's updated.
I've been slowly but surely working on what is essentially a BGT clone that will run much faster, run on more platforms than just windows, use the same script APIs as BGT so that porting games will be a walk in the park and offer a few more features which audio games really need. Unfortunately time constraints have kept things moving at a crawl, but I can already tell you that a lot of scripts run several times faster in my work in progress engine than they do in BGT. and since my engine will run on Linux I can use the wonderful Valgrind to detect a lot of stability problems before they are ever experienced by the public. (BGT used to hang and crash a lot, but to be fair I haven't seen it hard crash once in easily over a thousand combined hours of Manamon and Paladin playing).
Back to the topic at hand:
I'm sure your Z dimention doesn't need to be nearly as big as your X and Y dimentions. Even if you're portraying outdoor areas where there's virtually infinite space above the player's head, you shouldn't need to represent any more of the Z dimention than what the player can jump or climb to? This might be a great way to get memory consumption down.
You definitely want to use integer constants instead of strings to represent tile types. Doing this much string comparison is going to result in a huge performance issue as soon as there are more objects than just the player moving around.
and I would definitely stick to arrays when it comes to maps. Dictionaries are designed for fast lookups by name, but nothing can outperform true random access offered by a contiguous block of memory.

Official server host for vgstorm.com and developer of the Manamon 2 netplay server.
PSA: sending unsolicited PMs or emails to people you don't know asking them to buy you stuff is disrespectful. You'll just be ignored, so don't waste your time.

2017-03-17 17:42:54

@Trajectory, will this engine be open-source? If so, I'd love to help!

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2017-03-20 07:16:23

@Ethin I'm not sure yet.
I want commercial BGT game developers to feel comfortable porting their games over, which means that the manner in which the engine encrypts compiled bytecode and other resources has to be as well-protected as possible.
I could probably get behind the idea of an open source version with a few sensitive features excluded, but if you wanted to use that to compile protected work you'd have to add most notably encryption back in yourself.

Official server host for vgstorm.com and developer of the Manamon 2 netplay server.
PSA: sending unsolicited PMs or emails to people you don't know asking them to buy you stuff is disrespectful. You'll just be ignored, so don't waste your time.

2017-03-20 17:02:01

many encryption algorithms are open source. how do you expect they are so well-tested tried and true methods. just because something is open source does not mean the encryption on a game or project is cracked.as long as someone uses a key with enough bits of entropy. to calculate bits of entropy take log2(dictionarylength)*keylength. where your keylength is the number of digits in the key and dictionarylength is the number of items in the dictionary you pulled digits of the key from. i say digits, but we all know just a numeric, 0-9 key, wont have enough entropy. take for example a standard 10digit telephone number:
log2(10)*10 = 33.219280948873624 bits of entropy.
this isn't sufficient when we really need at least 128 bits of entropy or prefferably 256 bits. for a key with only numeric values we would need to get a key at least 78 digits long:
256/log2(10) = 77.06367888997919
for a key generated from a dictionary of 9 the 95 typable characters: A-Z, a-z, 0-9, and 33 assorted other symbols, we would only need a key of 39 characters:
256/log2(95) = 38.965848758590305
this is all assuming you generated your key perfectly from a true source of chaos/entropy. any non-deterministic qualities of the key makes it exponentially easier to crack.
this long looping tangent to say it either should all be open source or not at all. just because you don't open source the encryption side of your language doesn't mean its more secure. if anything its less. there needs to be eyes on it. You should also never, ever, ever write your own crypto. its already been done. please don't reinvent the wheel.

I don’t believe in fighting unnecessarily.  But if something is worth fighting for, then its always a fight worth winning.
check me out on Twitter and on GitHub