2018-05-16 12:02:07

Hi all,
After a few posts I've seen on here, and after talking to Paul Iyobo, I find myself wondering what features, and what APIs people would want out of a game engine.

Now I think of it, Ideal is a pretty cool name for such an engine, so I reckon go with that.

So drop your thoughts here. Please keep them realistic - we probably can't integrate with Google's AI so we can have mobs that talk back - but please don't limit them either.

I'll start with my thoughts in a separate post.

3... 2... 1... go!

-----
I have code on GitHub

2018-05-16 12:14:01

Firstly let's assume it's written in Python. As I imagine I'll be doing at least some of the work, unless someone else wants to take lead, Python it is.

The engine should work like Flask or Klein, being as system-agnostic as possible while maintaining a high level of functionality. The engine should not care how data is stored - although I guess it should provide some sort of optional persistant storage framework. Maybe use sqlite :memory: database as a back end? That way complex questions can be asked of the world and tables can be modified and discarded as needed.

It should definitely be event-driven... Those events should be pluggable:

# Hypothetical code:

from ideal import event

WEAPON_USE = event.register('weapon_use')


@event.listen(WEAPON_USE)
def weapon_used(player, weapon):
    """<player> fired <weapon>."""
    # Make a bang and stuff.


event.fire(WEAPON_USE, player, player.weapon)

How do people feel about this API? I think it could be pretty powerful, with some predefined events already available in the system:

GAME_STARTED

,

WORLD_LOADED

that kind of thing.

I don't feel like it should provide anything any other module provides: It shouldn't provide UI, since we have pyglet and pygame. It also shouldn't provide networking because we have Twisted, although an external module (

ideal.ext.networking

?) could be written to support multiplayer stuff with Twisted as a backend.

That's what I got so far. It's possible there's already stuff out there that does what I'm banging on about, so please feel free to tell me I'm being an idiot. Other than that, hit this thread with ideas and suggestions, and if there's enough interest we could compile it into a roadmap for a new game engine.

Cheers,

Chris

-----
I have code on GitHub

2018-05-16 12:49:40

I agree with the ideal.ext.networking
Also I think people may choose which storage they want
maybe like:
from ideal.storage.sqlalchemy import dump, load
or:
from ideal.storage.yaml import dump, load
There should be a map implementation where you can choose if you want to build the map from a tool, or editing the map file it's self.
for the moment it's all I can think about.

Paul

2018-05-16 13:01:40

Yes, maps definitely need to happen... How do you imagein the tool to build them working? Pyglet? Command Line? WX?

Also I feel like the maps should be pluggable too, so you can subclass them to provide your own implementation... Maybe something like:

from ideal import maps


class MyGameMap(maps.GameMap):
    def f(self):
        print('This is my game map!')


maps.map_class = MyGameClass


village = maps.create_map(...)

So the create_map function would use the map_class module attribute to figure out how to build the map so you could add whatever you wanted to maps.

I guess every game has some kind of map... Side scrollers and 3d... What other stuff do we all feel games all need? I know there's the obvious ones of sounds and visuals, but I feel like pyglet provides gboth, and anything I / we do will just be duplication.

-----
I have code on GitHub

2018-05-16 13:21:33

Well not really sure about the tool. I believe the more options we have, the better is.

Paul

2018-05-16 20:05:04

Nice, nice. where do you guys "talk". Would love to talk more about this.

If you like what I do, Feel free to check me out on GitHub, or follow me on Twitter

2018-05-16 20:41:50

@1, What I think we need is something like your game-server-base project -- a core that provides the base implementations for what you need most i.e. audio, graphics, networking, ...) but which doesn't do any hand-holding like BGT does (i.e. providing you every single function you could ever need). I think C++ might be better for something like this; I'd be happy to release to you an engine I was going to use in-house for a game I wanted to develop but said development got stalled because of college and other things getting in the way. It provides you menus, audio, TTS and a very basic graphics/event handling API (as well as, of course, input handling through keyboard/joystick), but it definitely doesn't hold your hand at all. I know, people will complain about how C++ is so hard to use and whatnot, but as I and many others have said, C++ isn't a language that's overly difficult to learn once you get rid of the fobia that you have to manage all the memory allocations and all that. In fact, here's one of my map serialization functions that you could use to serialize a map into JSON and XML either to a file or to memory:

// First, define the map struct (for an RTs-like game)
// Includes (required)
#include <iostream>
#include <string>
#include <fstream>
#include <cereal/types/vector.hpp>
#include <cereal/types/unordered_map.hpp>
#include <cereal/types/utility.hpp>
#include <cereal/types/array.hpp>
#include <cereal/archives/json.hpp>
#include <cereal/archives/xml.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/archives/portable_binary.hpp>
#include <cereal/access.hpp>
// Define the struct now...
struct Map {
public:
unsigned long long Rows, Columns, Plains, Overpasses;
std::vector<std::string> NorthSouthPaths, EastWestPaths;
std::unordered_map<std::string, std::vector<std::string>> Scripts;
std::string Name, Objective;
std::array<unsigned long long, 4> Resources;
private:
friend class cereal::access;
template<class Archive>
void serialize(Archive &ar) {
ar(Name, Objective, Rows, Columns, Plains, Overpasses, NorthSouthPaths, EastWestPaths, Scripts, Resources);
}
};
// And now, let's look at the map serialization function that translates your map into XML:
bool SerializeMapToXML (std::string file, struct Map map) {
try {
std::ofstream stream (file, std::ios::trunc|std::ios::out);
cereal::XMLOutputArchive archive(stream);
archive(map);
return true;
} catch (...) {
return false;
}
}

While this may or may not demonstrate good or bad coding practices, it does demonstrate how you might go about serializing a map:

  1. Define a standard C++ struct like you would any other struct.

  2. In the struct, befriend cereal::access. This makes it possible to keep the archive() method private so that it cannot be called by anything or anyone else but Cereal.

  3. In your serialize() function, call your archive parameter (in my case 'ar') with all the items you want to serialize.

  4. Then, serialize it to a file like my function does. (The catch(...) exception handler indicates that we don't really care about the error; if one occurs, then just return false. I could've handled this better, but when I did write it I was a bit frustrated since I couldn't figure out a good map loading/saving library other than serial, so was just trying to throw together an implementation.)

That handles serialization to a file, but what about memory? Same concept with struct definition (you only need to define it once), different function:

std::stringstream SerializeMapToXMLInMemory (struct Map map) {
try {
std::stringstream ss;
cereal::XMLOutputArchive archive(ss);
archive(map);
return ss;
} catch (...) {
std::stringstream ss("error");
return ss;
}
}

And, what about deserialization both from a file and from memory? Again, same concept, different function:
From a file:

template<typename T> std::pair<T, bool> DeserializeMapFromXML (std::string file) {
try {
std::ifstream stream (file);
cereal::XMLInputArchive archive(stream);
Map map;
archive(map);
return std::make_pair(map, true);
} catch (std::exception &ex) {
return std::make_pair(ex.what(), false);
} catch (...) {
return std::make_pair("Unknown error", false);
}
}

And from memory:

template<typename T> std::pair<T, bool> DeserializeMapFromXMLInMemory (std::stringstream map) {
try {
cereal::XMLInputArchive archive(map);
Map map;
archive(map);
return std::make_pair(map, true);
} catch (std::exception &ex) {
return std::make_pair(ex.what(), false);
} catch (...) {
return std::make_pair("Unknown error", false);
}
}

Now, some explanations:

  • The templates for both of these functions isn't required, but allows you to check the .second value of the pair and determine its truthfulness; if its true, then you know that what's in .first is a stringstream containing map data. But if its false, you know that you've got an error.

  • The difference between these two functions is that when you serialize something to a format like XML or JSON in memory you're returned the raw, serialized data that you can do with what you like (i.e. send it over the network). The file serialization/deserialization is designed to be saved in something like cold storage, then loaded sometime later in the future. The file at that point may or may not be on the same machine it was saved from. Either way, such a thing doesn't matter.

And, of course, the serialized data shouldn't be modified or read by humans unless you know what value is what. smile

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2018-05-16 21:09:35 (edited by pauliyobo 2018-05-16 21:10:09)

@7:
I don't get what do you mean by it shouldn't hold your hands,
The engine should serve as a framework, if the developer can't code his self, he won't gain much from the engine.
The engine just serves as a quicker way to handle things. It doesn't create a game.

Paul

2018-05-16 21:10:43

@6: you can find me on twitter as @paul_iyobo

Paul

2018-05-16 21:11:20

Hi,
@NicklasMCHD: We just talk on Twitter honestly... Or Mindspace.

@Ethin: thanks for the suggestions and the code. I appreciate the offer of the engine, but I'm afraid I don't know C++, and have no interest in learning it, although what you wrote does give me some interesting stuff to think about.

You say about handling graphics and sounds and stuff... But we already have libs for them? I mean there's Pyglet, and Pygame, and Libaudioverse (although I'll personally be sticking away from it because of its non-maintained status)... OpenAl... That's all assuming we're going to use it for games to be played locally.

Honestly I personally hate single-player games, so for myself I want it as system-agnostic as possible so I can use it for writing servers. At the very least the client bits should be separated away from the server-relevant bits.

If you want to check out what I've knocked together so far (a fancy events framework), you can find it at its Github repository.

-----
I have code on GitHub

2018-05-16 23:12:15 (edited by Ethin 2018-05-16 23:39:02)

@10, like I said, its your choice what the engine uses. smile #8, a hand-holding engine is one that tries to implement as much functionality as possible and makes it ultra-easy to do practically anything. It gives new programmers a false sense of success because they think they can implement anything. Then they go and try and they discover its not as easy as they think it is.
@10, Yes, we have libraries for graphics an audio. But a game engine is a core framework that provides functions that do the heavy lifting for you; for example, to draw a rectangle in SFML, you'd us the sf::RectangleShape/sf::Rect classes, rather than using the low-level graphics interfaces provided by DirectX, OpenGL, Vulkan, etc. Allow me to place this into context: If you visit https://vulkan-tutorial.com, you'll see a lot of code related to drawing. A game engine is designed to abstract that away so that you don't have to write all that code every time you want to draw something. smile

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2018-05-16 23:20:22

I've never heard of SFML. Just goggled it and it looks very interesting.

Cheers,

Chris

-----
I have code on GitHub

2018-05-16 23:39:46

@12, indeed, SFML is very interesting. My game engine uses it for quite a bit of things. smile

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2018-05-17 01:07:31

@8 agreed!

@2: I like your idea of handling weapons as events; it's how I've always done it in my later games. Because then you can easily run the weapons on a different thread from the main game frame thread.

Also, your idea of Python is right on. People should focus on the games they want to make, not the low-level memory management that you must deal with in C++. It will allow people to really put their game-coding skills to work. And, above all else, Python is a more natural language than C-style languages (BGT and "simple" being used in one sentence has always baffled me.)

If I was to start over today, I'd write a game in Python.

As for maps, I think WX is a good choice. It builds accessible GUIs (at least on Windows,) something that, unfortunately, you don't get with QT Framework despite QT's maturity. At least, when I built a GUI in Python using PyQT, it wasn't accessible.

So, I like where this is going. Best of luck to you!

2018-05-17 01:23:15 (edited by Ethin 2018-05-17 01:29:37)

@14, I've always been curious why people think you have to do extreme memory management with C++. You don't. Not any more. You do if you want to write manual memory allocators, but you don't if you don't want to. You have std::smart_pointer, std::shared_pointer, .... I could go on. The game engine I've written that runs the game I started but couldn't finish doesn't use any memory management at all. Its all automatically done. In fact, this Stack Overflow question asks the very question of how much time you might spend on memory management. The very first answer -- the accepted one -- indicates that you don't need to spend any memory management "until you need to organize your memory by hand, mostly for optimization purpose, or if the context forces you to do it (think big-constraints hardware)."

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2018-05-17 06:26:37

@14: the idea is to use more options as possible.
So basically there will be a tool for the wx map building and the cmd tool for it.

Paul

2018-05-17 10:54:40

I don't see how BGT's available tools are significantly different from Pygame minus graphics, plus accessible_output2? What is hand-holdy about BGT but not Pygame or Pyglet? Timers and dynamic_menus (I agree that those are dangerous to rely on)? Is it that you get sound without having to spend 5 lines initializing engines and writing your own wrapper so that you get a sound object that's half as convenient? That can't be it, since pygame.mixer only takes one init, channel count adjustment, and writing something for panning. And it lacks pitch-bending/sample rate modification, unless that hid from me last time I checked. Is it the keyboard functions? If devs are being stopped from making anything decent because they can't cope with translating between event and polling based input, then maybe you have a point.
TBH BGT's sound and easy compilation to exe are all I really care about. I keep asking where I can find an audio engine comparably simple and "powerful", and keep getting pointed to things which are completely and utterly unintuitive, even compared to the manual buffer loading you have to do with Javax.sound.Clip. Like, I look at Pyo and Libaudioverse and the simplest examples I find are unreadable gobbledegook compared to the Javasound equivalent. I can wrap Javasound. All I get when I read the python examples is "something something init init server out out something buffer node resource init connect". Is there an audio engine that just fricken works? Pygame.mixer seems weaker than BGT's sound engine, but it's the closest I've found for python.

看過來!
"If you want utopia but reality gives you Lovecraft, you don't give up, you carve your utopia out of the corpses of dead gods."
MaxAngor wrote:
    George... Don't do that.

2018-05-17 13:10:44

@Ethin. I would love to see your game engine, if you'll let me smile
- NicklasMCHD

If you like what I do, Feel free to check me out on GitHub, or follow me on Twitter

2018-05-17 15:43:31

@18, gladly. I hope I'll remember to upload it today -- I can be forgetful like that. smile @17, There are much better sound engines other than BGT's one (FMOD comes to mind). If you download it and check out its documentation, its very simple to load a sound:

// What I do is hold a master FMOD::System object and an unordered map of all loaded sounds by filename:
FMOD::System     *system;
std::unordered_map<std::string, FMOD::Sound*> sounds;
// This function is required, though you could probably throw exceptions instead if your willing to create an exception for each error:
template<typename funcname, typename linenumber> void errcheck(funcname function, linenumber ln, FMOD_RESULT result) {
if(result!=FMOD_OK) {
// In my engine, I have a logging framework set up, so we log the error, and print the error to a UI so that the user knows:
LOGF << "FMOD Error " << result << " from function " << function << ", line " << ln << ": " << FMOD_ErrorString(result);
#if (BOOST_OS_WINDOWS == 1)
MessageBox (NULL, FMOD_ErrorString(result), "Error", MB_OK);
#else
Gtk::Main app(true);
Gtk::MessageDialog msg(FMOD_ErrorString(result), false, MESSAGE_ERROR, BUTTONS_OK, false);
msg.run();
#endif
}
}
// OK, let's load a simple sound, no special effects.
// We first initialize the audio system:
unsigned int version = 0;
errcheck(__func__, __LINE__, FMOD::System_Create (&system));
errcheck(__func__, __LINE__, system->getVersion(&version));
if (version<FMOD_VERSION) {
LOGF << "Version (" << version << ") is less than latest version (" << FMOD_VERSION << ")";
#if (BOOST_OS_WINDOWS == 1)
MessageBox (NULL, "The sound system version is less than the required version.", "Error", MB_OK);
#else
Gtk::Main app(true);
Gtk::MessageDialog msg("The sound system version is less than the required version.", false, MESSAGE_ERROR, BUTTONS_OK, false);
msg.run();
#endif
exit(1);
}
errcheck(__func__, __LINE__, system->init(4093, FMOD_INIT_NORMAL, (void*)0));
/*
What this code just did:
* It created the system object and retrieved the version of the FMOD API.
* We ensure that the version is equal to or greater than the version we need (FMOD_VERSION), since if our version is less than the version we need, we may be using deprecated or broken functions.
* We then initialize the system with 4,093 sound channels, with normal initialization, and no extra driver data.
*/

And now, for the fun part -- loading a sound!

// Abstract this in a function, makes things easier
void LoadSound(std::string file) {
// Is the sound already loaded?
if (sounds.find(file)==sounds.end()) {
// Nope, go ahead and load it.
LOGV << "Loading sound " << file;
FMOD::Sound *sound;
// Now: create the sound with the given filename, no extra loading flags/modes, and no extra info.
errcheck(__func__, __LINE__, system->createSound(file.c_str(), FMOD_DEFAULT, 0, &sound));
// Did it work? Try and insert/reassign the sound. (Insert if it doesn't exist, reassign if it does.)
if (sounds.insert_or_assign(file, sound).second) {
LOGV << "Sound " << file << " loaded.";
} else {
LOGV << "Sound " << file << " reassigned.";
}
}
}
// And now, we play the sound.
// You could abstract this away in a function too, but the channel is needed to stop the sound.
FMOD::Channel *channel=0;
// We should... uh... probably lower the volume. Sounds loaded at normal volume can be... ah... loud.
errcheck(__func__, __LINE__, channel->setVolume(0.40));
// Now we play it!
errcheck(__func__, __LINE__, system->playSound(sounds[filename], 0, false, &channel));

The above code assumes you've already loaded the sound, of course. Most of this can be abstracted away in a function to make it a lot easier to work with. After you've begun sound playback, you can do quite a few things (which happen so fast the user doesn't notice):

  • Add 3D reverb (with System->CreateReverb3D()).

  • Create DSPs, geometries, channel groups, sound groups and streams (system->createDSP()/system->createDSPByPlugin()/system->createDSPByType(), system->createChannelGroup(), system->createGeometry(), system->createSoundGroup(), and system->createStream())

  • Add and delete sync points (Sound::addSyncPoint(), Sound::deleteSyncPoint())

  • And a looooot more. smile

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2018-05-17 16:33:18

CAE, your post @17 reminded me of an old programming joke.

The first man programs in an older language, and basically builds whatever he needs.  Second man ridicules the first man, claiming that he isn't a "real" programmer because of what he uses.  The second man moves to a language that lets him go online and grab code written by other people.  The second man assembles his projects out of 95% stuff he didn't write, and only has enough knowledge to tie it together.  The first man writes 100% of his code himself.  Who is really the "real" programmer?

I guess joke is the wrong word, but it was passed off as a joke, or saying, or whatever when I was in school.  I've always loved it.  big_smile

- Aprone
Please try out my games and programs:
Aprone's software

2018-05-17 18:36:04

@20, I'd say though that the definitions of a "real" programmer, someone who "writes all their code by hand" and someone who doesn't are very subjective. For example, I consider a "real" programmer someone who doesn't use scripting languages to accomplish absolutely everything involving a particular area like gaming unless your doing something like sysadmin work, where scripts are probably a better option (like BGT -- yes, if you look at the type description of a .bgt file, it says "script", not something like "BGT source", and its method of compilation is more like embedding the BGT binary into the executable and compiling your BGT code into bytecode... no wonder AV programs throw fits about it). I say that the definitions of someone who writes all their code by hand and someone who doesn't subjective because one person might define someone who writes all their code by hand as someone who writes everything from scratch (including the game engine and all that, along with the game that runs on top of the engine) and someone who doesn't as someone who uses already available game engines that may or may not be inferior or superior to one they wrote themselves to accomplish the task. In particular, one thing that nearly every audio game is lacking are physics and a true fPS-like environment. In audio games, particular in FPSs, its center the person and shoot. There's nothing more about it. Oh, sure, you have to worry about ammunition ricocheting off of shields and walls and stuff, but you don't have true physics. In a mainstream game like Halo, on the other hand, centering isn't the only thing you do. You don't just center someone from far away and shoot to kill them. You instead get as close to that person as possible, center them, and shoot them, all the while dodging shots and projectiles from that person (and even other people).

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2018-05-17 20:01:38

OK, at the risk of sounding like a bit of a dick here... This is about the game engine people would love to see, and there's lots of posts here, none of them about the actual topic.

@    def map_load(self, map=None):
        if os.path.isfile('{}.yaml'.format(map)):
            with open(map, 'r') as f:

                load(f)
: Please don't use threads, they're a terrible idea. I used to rail against this point, but it's true.

@17: Do you want "good" or "simple"? If you want "simple" use Pygame. If you want "good" use libaudioverse.

-----
I have code on GitHub

2018-05-17 20:30:38

@22, I'm confused why you don't want people to use threads, and what type of threads your talking about. Do you mean Python's implementation of threads or threads in general? I can see why some (like myself) might have problems with threads but other than coroutines, which really aren't like threads, you don't really have any other way of performing parallel computing. Yes, you have microthreads, like goroutines, but those are just another type of thread.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2018-05-17 23:05:11

At 22: I think I'm either too dumb or too ADHD-addled to use Libaudioverse (at least Ethin's 100 line example for loading a sound was comprehensible), so Pygame it is.

If we're talking about maps, what type(s) would we be dealing with? I can think of four I've used just off the top of my head, and I'm sure I'm forgetting others. I suppose that if it is an Ideal engine, it could support all of the most common types, but that seems like an unrealistically high expectation.

看過來!
"If you want utopia but reality gives you Lovecraft, you don't give up, you carve your utopia out of the corpses of dead gods."
MaxAngor wrote:
    George... Don't do that.

2018-05-18 00:03:31

@24, 100-line example? Rofl... I'd need to check but I think it was under 30. big_smile Ah well, I'm glad you managed to understand it. I myself have never wrapped my head around libaudioverse either, especially since the documentation is so lacking in so many details its not even funny.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github