2014-08-22 03:10:34

I am currently messing around with multithreading and that is fun! Does anyone have any good techniques for dealing with threads?
I don't like the threadname.join() function because it freezes the program till that thread stops. I think daemon threads are what I should use, and just grab an instance each time the player fights a mob. I will avoid locks at all costs though, as locking threads would fill memory very fast. There is no garbage collection for dead threads and I don't know quite when they exit. I am going to write a script messing around more with this. But I hope to write another module that will do this junk for me.
Has anyone already done such a thing?

2014-08-22 04:14:33 (edited by stewie 2014-08-22 04:15:59)

Why are you using threads? for a single player small game you don't even really need to. Also using print will obviously slow the program down by a large degree.

Deep in the human unconscious is a pervasive need for a logical universe that makes sense. But the real universe is always one step beyond logic.

2014-08-22 09:33:16 (edited by frastlin 2014-08-22 09:43:11)

If you wish to make real-time combat like Swamp has or even for an ai in a side-scroller, in python, I don't think there is any way to do it other than by using threading. Many, many, many python applications use eather multiprocessing or multithreading, and both have their advantages and disadvantages, but IMO for games, you really need multithreading.


So I am still not 100% on everything that has to do with threading, but here it goes:
syntax:


import threading


def myfunkname(arg):
    print arg


my_thread = threading.Thread(target=myfunkname, args=("myfunk's arguments! With a comma",))


The major functions in threading.Thread (The class you deal with for each thread but the main one).

__init__

the thing you call with
my_thread = threading.Thread(target=myfunkname, args=(my_arguments,))
you need to put that extra ',' comma there for some reason...
You can change it, you just need to put
Thread.__init__()
right under your __init__(self, argu, men, ts):
But don't, it's too much work! smile


run():

like __init__, don't bother, although if you do mess with __init__, you will probably wish to modify it so you can run it properly.


start()

you call this for every start of a thread. After this function is called, your thread is on a roll and you can only stop it by the next functions!


daemon

I didn't forget the (), it is a poor little variable sad... But it is not any little variable, it is a daemon!
my_thread.daemon = True
means that if you exit the main thread, my_thread will die without any warning. So my_thread could be talking to his girlfriend on the phone and if you press escape to close the game, my_thread will die! (So sad, it was getting a little racy sad)


join()

Don't use this like start! It is really not good practice, it is just what everyone does. It nullifies daemon and makes the main thread wait for it to finish. (well not the main thread, but the calling thread, but nvm about that).
so if I said:
my_thread.start()
my_thread.join()
I have to wait for my_thread to get off the phone before killing him, then what's the point? He's already hung up, so you don't get to talk to the girl sad!...
So, if you wish to treat your threads as royalty, use
join()
but imo, daemons are a lot better!



Lock (acquire() and release())

This is another class like Thread, so it has its own functions! in yourfunk you will wish to create an object like:
lock = threading.Lock()
then use the acquire() and release() functions, sandwiching the access to the variable.
locks are used for changing variables that many threads are changing.
so if I have my variable:
dinner = 100



and my main thread (me), calls everyone to dinner, I would have:
mommy_thread
daddy_thread
my_thread
girlfriend_thread #mmm...
brother_thread


Now if I just let them all eat at the same time, they would all be bumping into one another and grabbing food at the same time. In the real world this is generally what people do when the pizza arrives, but in python this can be a bad thing. When all those threads grab at the variable at the same time, the pieces don't end up getting subtracted from dinner, so you get a loaves and fishes effect.
dinner = 83
mommy_thread
daddy_thread
both grab one piece when dinner is at 83
dinner is now 82 and both mommy and daddy threads have a piece!
Great if you are poor like me and need to feed all those people, but not-so-good when everyone is on a diet and can only have 1/5th of the dinner (oops, daddy_thread got 21 pieces while still eating his 1/5th of the 100 pieces!)
This is also really bad when you wish to have everyone write a letter to grandma!
mommy would write "Dear grandma, I went to the pool today..."
under it daddy wrote "dear grandma, I hate swimming and mommy makes me go to the pool to swim with her..."
Although you see there is a problem in mommy and daddy's relationship in the first 2 lines, once brother my_thread and girlfriend start writing their stuff randomly in between everyone else's lines, life becomes crazy and grandma will end up banging a hole in her wall with her head, trying to figure out who is having problems with who!
So this is why there is lock!
if we wish to have daddy write, he needs to put a lock on the letter so no one else can write to it while he is writing his bit.
The code looks like:

import threading


letter = []


def writing_function(name, text):
    global letter
    lock = threading.Lock()
    lock.acquire()
    letter.append("Dear Grandma, this is %s and %s" % (name, text))
    lock.release()

daddy_thread = threading.Thread(target=writing_function, args=("daddy", "I hate going to the pool. When I stick my leg into the water, I just can't stop thinking about all the little kids who have peed in the water"))
mommy_thread = threading.Thread(target=writing_function, args=("mommy", "I went to the pool today. It was so refreshing after a long day of work just to swim through the water.\nHope to see you soon\nMommy"))
brother_thread = threading.Thread(target=writing_function, args=("brother", "I just got a longbow in swamp and I've been running through sub 2, killing all the zombies TTYS!"))

daddy_thread.start()
mommy_thread.start()
brother_thread.start()


for line in letter:
    print line



RLock() ...

This works great if the thread needs to write everything at once, but what if daddy needs to go to the bathroom in the middle of his section?
This is where relock comes into play!
Daddy can take a break, go to the bathroom, make a call to his .... erm friend!... walk the dog and do what ever he wishes and mommy and brother need to wait till he is done!
Here is what our new function looks like. Everything else is the same.


def writing_function(name, text):
    global letter
    relock = threading.RLock()
    relock.acquire()
    letter.append("Dear Grandma, this is %s and " % name)
    for line in text.splitlines():
        relock.acquire()
        letter.append(line)
    for line in text.splitlines():
        relock.release()
    relock.release()



threading.Event() (set(), clear(), wait())

Now girlfriend_thread does not want to sit at the counter looking at the letter, waiting for daddy to sign, she wants to be on her computer playing Paladin of the Sky! (That's smart!)
So she sets up a fancy notification to let her know when the last person has finished. She knows that brother_thread is always the last, so she puts a little flag (argument e) on brother that lets her know when brother finishes. This requires brother to trigger the Event().set() function. Then girlfriend puts a check in her function that says:
if the set function is given, write her little bit and then clear her notification so my_thread can't use it, otherwise, my_thread, dog_thread and any thread with the same notification she has (Event().wait() function) will hear that brother is done and can come down and write pretty little hearts and whatnot.
So here is the new code:



import threading


letter = []
event = threading.Event()
lock = threading.Lock()


def writing_function(name, text, e=None):
    global letter
    lock.acquire()
    letter.append("Dear Grandma, this is %s and %s" % (name, text))
    lock.release()
    if e:
        event.set()

def line_writing():
    global letter
    if event.wait():
        letter.append("\nxoxxo<3<3<3xxoxxoooo")
        event.clear()

daddy_thread = threading.Thread(target=writing_function, args=("daddy", "I hate going to the pool. When I stick my leg into the water, I just can't stop thinking about all the little kids who have peed in the water"))
mommy_thread = threading.Thread(target=writing_function, args=("mommy", "I went to the pool today. It was so refreshing after a long day of work just to swim through the water.\nHope to see you soon\nMommy"))
brother_thread = threading.Thread(target=writing_function, args=("brother", "I just got a longbow in swamp and I've been running through sub 2, killing all the zombies TTYS!", True))
girlfriend_thread = threading.Thread(target=line_writing)

#she needs to start before everyone so she can get the alarm when it goes off!
girlfriend_thread.start()
daddy_thread.start()
mommy_thread.start()
brother_thread.start()

for line in letter:
    print line



threading.Timer() (start(), cancel())

my_thread is not as smart as girlfriend_thread, but he also has set up a notification on his phone. His lets him know after a set time that he should write his bit. He uses a timer.
The threading timer is a very timid thread, it can't be called with a join() function and the main thread will exit while it is still running. Also, anyone can cancel it.
so my_thread is taking a chance that the rest of the threads won't be mean.
Here is what he did. He just changed the bottom half, under girlfriend_thread's line_writing function, but he also included the time.sleep function, because otherwise the letter would be sent without him!

#right under line_writing
def my_function(text):
    global letter
    letter.append("\n\nHey grandma, %s" % text)



my_thread = threading.Timer(2, my_function, ("I'm going crazy playing Castaways! Aprone has made the new map really really hard!",))
daddy_thread = threading.Thread(target=writing_function, args=("daddy", "I hate going to the pool. When I stick my leg into the water, I just can't stop thinking about all the little kids who have peed in the water"))
mommy_thread = threading.Thread(target=writing_function, args=("mommy", "I went to the pool today. It was so refreshing after a long day of work just to swim through the water.\nHope to see you soon\nMommy"))
brother_thread = threading.Thread(target=writing_function, args=("brother", "I just got a longbow in swamp and I've been running through sub 2, killing all the zombies TTYS!", True))
girlfriend_thread = threading.Thread(target=line_writing)

my_thread.start()
girlfriend_thread.start()
daddy_thread.start()
mommy_thread.start()
brother_thread.start()

#making everyone wait, that's so diva!
import time
time.sleep(3)
for line in letter:
    print line



This is pretty much most of python's threading module. The other stuff is a little more advanced (there is a lock function you can set a limit on, there are some checks to see how many threads are still active, there are some more things you can do with locks... but the above functions are the important ones. Google the function like "using threading.RLock() in python" to read tutorials and answers on stackoverflow!
Here is the python refference just fyi!

2014-08-22 22:25:11

So I was thinking how you could do events without using the threading module.
You would need to use the time.clock() function a lot as benchmarks.
I replicated the threading.Timer functionality. It is used a little differently because I kind of bastardized the timer and threading classes, but the while loop is all controlled by you. You can press buttons and have another function take care of them. Does anyone have a better way to do this?


class Thread(object):
    def __init__(self, name=None, function=None, args=()):
#name is not used here. function is the name of the function you wish to run with the run() function and args are the args for that function.
        self.name = name
        self.function = function
        self.args = args
        self.current_time = time.clock()
#current_time is a set point in time, time.clock() doesn't update like normal.



    def run(self):
        self.function(*self.args)

    def wait(self, length):
#I'm basically saying below that if the moving clock time minus the snapshot of the clock we got above is greater than the length argument, then do the run function
        if time.clock() - self.current_time >= length:
            self.current_time = time.clock()
            self.run()
            return True
        else:
            return False

#the function that will run with the run function
def hello(w):
    print "hello %s" % w

#for the while statement
play = True
#creating our object, we are saying hello is our function name and our arguments are "fred"
t = Thread(function=hello, args=["Fred"])

while play:
#this will run the wait function every loop and return False unless the clock minus the static time are greater than the wait number
    if t.wait(2):
        play = False
#Sleeping 5 milliseconds every loop makes our program take up something like 90% less CPU
    time.sleep(0.05)



This could be used to randomly wait for a time between enemy attacks.

2014-08-23 22:19:10

Threads are not necessary.  Threads actually hurt.  Threads will kill your productivity.  I'm not going to go into why because a quick Google search will bring up many, many articles about just why threads are bad.  The secret to using threads is to use queues, at least in the very few cases you need them, and only have one thread on which all the work happens.  The only reason that people really use threads these days is networking, because it allows placing blocking I/O in a place where it can do so in languages without good support for callback functions.  The other reason-gaining extra performance from multicore systems-does not apply in Python unless 50% or so of your app is calling out to C code.  Shared state between threads is horrible for reasons which become apparent as soon as you get past line 200 or so and start having bugs you cant' reliably reproduce all over the place.
Have a look at audiogame_engine.  It's not super mature but works well enough for I3d, and I have no idea how well it plays with Pygame's sound modules.  But it's my answer, and also the answer that many other people have come up with (actually, it's almost exactly a port of how Microsoft tells you to do it with XNA).  The reason I3d isn't open sourced is that Camlorn_audio sucks and is deprecated, but I'm planning something in the near future with Libaudioverse that will use audiogame_engine (it's my chosen platform for desktop games and game clients).  I'm not sure how clear it is, as a few places use some magic to make Sdl become friendly.  But it should at least give you ideas.
The general trick is to not wait.  If you want something to happen in 5 seconds, you tel the main loop to call a function for you in 5 seconds, and then go on about your day.  Typically, you achieve basic game logic by putting it in tick functions on the classes representing your objects, and register them with the main loop to be called once a heartbeat.  I'd strongly suggest reading some sighted game programming tutorials before continuing down this rabbit hole.
Things that use this approach of ticking include all muds.  As I recall, Diku does not even spawn a thread anywhere, and I'd be willing to bet pretty much anything that Alter Aeon does not split the game logic across more than one thread.  LPC doesn't have support for it, nor does unmodified Moo.  I3d didn't use them for anything: input, waiting logic, physics simulation, etc. all run through tick functions on one thread.  Some sighted games will split rendering the 10000 polygons across multiple threads, but even there it's discouraged for game logic, save in the few cases where every single enemy in a group of 10000 needs to run really complicated AI logic individually.  Twisted and Node.js exist so that network applications can run very efficiently on one core and with only one thread.  The list goes on.  Going into threading land is like going to Oz: all the rules you knew don't quite apply or apply in different ways.
I'd strongly suggest staying away from these until you understand why they're useful.  There are only two major reasons I can think of: the problem really can't run on one core or the problem is what is called embarrassingly parallelizable.  In the latter case, it's something like squaring 6 billion numbers: you can ask thread one to square the first billion, thread two to do the second, and so on.  In this case, the threads don't "know" about each other, and thus you avoid literally all the problems with them.  I suspect the examples you are seeing are for problems of this nature.

My Blog
Twitter: @ajhicks1992

2014-08-24 02:03:32

Really? What is the bad part of python multithreading? From what I've read, it is that it slows down things and accessing shared variables is something you need to be careful about (see the lock section above).
Also, killing a daemon thread, if it is running a critical process, can be harmful.
So, why is this bad for use in a combat situation? Running 45 threads on my computer took 5% of my CPU when running most any of the other audio games that are out take up something like 30%.
Is using threading for mobs something that requires speed or a shared variable?
I need to research more about these ticks, but after a few days with just researching them, I've not found anything that explicitly stats that that multithreading is "bad", not like global variables. In Java it is bad and in most other languages it is bad, but the threading module in python is really really easy to use and won't make your program "faster", but it is sure not that program intensive, especially not for a single object, running on an engine.
I think multithreading has just gotten a bad wrap from all the languages that don't handle it well and the abuse or misconception that it will increase performance. But frankly, a mob engine is kind of like what you are talking about about the billion numbers.
I am going to research ticks and look at them closer, but I think ticks came from needing to run graphics.
So, would you say to use time.clock() or use a class accessible timer that I create?
X would add every 0.05 seconds or something like that and the mob functions would run every tick. (although ticks are too predictable IMO.

2014-08-24 03:23:41 (edited by camlorn 2014-08-24 03:45:11)

Ticks are not "too predictable".  You can easily do 60 or 100 a second.  You can, if you are good at math and adventurous, have the time between them vary.  You can have the engine detect when it's slow and tick faster to catch up, or make the time between ticks longer without slowing the game.  But all you really need is the basics: tick 30-60 times a second, keeping the interval the same.  Your players literally cant tell-it's way too fast to be noticeable unless the engine gets under too much of a load to keep it up (in audiogame land, should this happen, it's a bug).  most games for the sighted do this, even multimillion dollar titles.  If you want to learn more, you can find many, many books on game architecture.
Actually doing this is a bit more difficult, and I strongly suggest reading over audiogame_engine.  Audiogame_engine has a working implementation of a main loop and a bunch of other stuff you will eventually need.  Ticking 60 times a second is not "tick, wait 1/60th of a second", it's "time the tick and wait however much time is left in this 1/60th of a second fragment".  In audiogame_engine's case, the delta--the time since the last tick--is passed.  But I never used it, and you can just shrug and ignore that part.  All you need is time.time(), time.sleep(),  and some basic arithmetic in a while loop.  You can probably adapt the code to work with your stuff, and all the low-level bits and pieces are demonstrated therein.
You seem to have misconceptions as regards what is "fast".  You will not need more than one thread for any single-player audiogame I can think of, and Swamp probably does-at most-one thread per map.  You may choose to use them at some point, but do avoid them for now.  Drawing polygons is actually a bad example in that it's usually not even done on the CPU, but drawing even one polygon takes a much longer time than updating one npc's state (if I had to guestimate, on the order of 100x-1000x, but the graphics card parallelizes automatically).
Threads are amazingly easy to start in just about any language these days.  But the problem here is that the entire game is shared state.  Every thread will want to modify or read the world and, because it's running with other threads, will not see it right.  Consider a program that updates coordinates on your objects, running in a thread.  A second thread can see some objects in new positions while others are still in old ones, read off coordinates with updated x values but not updated y values, or (what you want) read everything between updates.  The solution is to throw a lock into the mix, but as soon as you start doing that you start losing any benefit at all.  Fine-grained locking is notoriously hard to get right, and deadlocks are notoriously hard to debug.  Having a global lock that every thread holds before it does stuff means you shouldn't have bothered in the first place.
An embarrassingly parallelizable problem is a problem in which each thread may live in its own universe, sharing no state save a notification that it has completed its task.  This is easy, and problems like this are definitely a good introduction.  Every example I can think of for the threading and multiprocessing modules are in this category.  Games are not embarrassingly parallelizable: every thread will need to touch everything at some point and to some extent.  The documentation you are reading is not teaching you the difference: it's assuming you're a good enough programmer to know when threads are, and are not, the solution; its purpose is to show you how they work from the perspective of someone who needs them.  to that end, the examples are about teaching a programmer who knows threads how to use threads in Python and not showing you or even touching on all the problems you get to deal with in real-world cases.
threads deserve their bad reputation.  They are the only way to solve a problem which is difficult for humans to think about in the first place, and the guarantees we can't make and still have fast computers make it even harder.  In lower-level languages, it's possible to see half-updated variables (as in, some of the bytes in your integer have been changed while others have yet to happen).  Python doesn't let you quite have this problem because of the GIL, but the GIL keeps you from getting any performance boosts on tasks written purely in Python (excluding i/o) anyway.  Nothing you are doing warrants putting yourself through this particular hell-and yes, I'd really go that far in this case.
Let me put it this way.  Some of the largest web sites out there, handling thousands of concurrent users, run in one thread.  This is the purpose of Node.js, Twisted, and a bunch of other stuff.  Camlorn_audio has so many threading issues related to fine-grained locking that I couldn't fix them if I wanted, and I once spent 8 hours debugging a deadlock and wrote a 15-line comment explaining what I should never ever do if I wanted it to keep working.
If you go down this road, especially before you're at the point of truly needing it, you're going to waste months.  What used to be deterministic if-a-then-b programming becomes more difficult, in that running the program 5 times or even 100 times might trigger the bug you want to get rid of once because the threads didn't magically interact in the incorrect manner.  Most of the 8 hours above was me running the same program over and over and over, trying to get it to break so I could get a debugger running on it.  I do not know of anyone who goes and says "Yay! Threads!".  I have seen articles by the people behind C++ that discuss why they're bad and the problems, and there are many, many major companies researching alternatives to the standard lock models right now.  This isn't an unsolved problem, but it's kind of like trying to build a house with only a hammer-you can, if you're exceptionally clever, but it's going to be difficult.
Edit:
To find things on why threads don't solve all your problems, you need to google for that.  There is a difference between how and why, and you're looking at the how.  The long and the short of it is that the tutorials on implementing game ticks tend to be written for new game programmers, and the tutorials for new game programmers stay away from threads because explaining exactly why they're bad takes a discussion of low-level architecture and--depending on how far you go--assembly.  The explanations can spend pages and pages talking about 10-line examples and all the bugs they contain, so it's usually in a dedicated article elsewhere that you find good examples and whatnot.  The question here is not to do with performance, and the reason 45 threads is taking 5% of your CPU is (1) because you didn't do anything and they died and (2) because Python's GIL will prevent you from getting a gain.  In terms of CPU overhead, an idle thread costs almost nothing, so also (3) because you didn't do any work on them.

My Blog
Twitter: @ajhicks1992

2014-08-24 06:06:26

Well, what is a mob doing most of the time? waiting for its next step, waiting to respond, waiting for its next attack, it is a lot of waiting. I ran my 45 threads removing numbers from a a variable every random time between 1 and 5 seconds, and sending a print to the prompt. In reading what makes python slow, it says that doing a lot of if statements with non local variables, so basically anything with a dot in it, will significantly slow down productivity.
In thinking about how I can do this kind of system, I'm thinking that dicts are going to be everywhere and there will be a lot of if statements running from all the currently loaded creatures.
My brother was saying that he uses something called "delta time" in the game engine he uses, but I was totally lost. I know about ticks, but didn't connect the two at all.
So, is the little script above what you were saying? time.clock() - the_last_time_snapshot is an int that will eventually be like a 2 or something. So in a sense, my ticks are the built in clock. I can delay my game as much as I wish, so I think that using time.clock() won't be any more memory than implementing my own counter. What do you think? And, does each creature run off its own wait time? like one creature will attack every 2-5 seconds and another will be attacking every 1-3 seconds. In the tick games I've played, all creatures attacked at the same speed.
I'll look at audiogame engine now.

2014-08-24 14:01:08

OK, I took a look at audiogame_engine and I wish I knew about this before I got into pygame! LOL... It is basically a light-weight pygame.
Now, I just need a game example to see how they work together with one another.
I didn't understand everything, but I get that your main game loop pauses for something like 0.0166666 seconds each loop and then runs again. (I normally pause for 0.05 seconds and no one notices). But do you like keeping your own ticker vs keeping time.clock() or keeping your own ticker vs time.time() minus the last time.time()?
I guess delta time is "real-time" in cases where your game logic makes the game run a little slower than the 0.1666 second pause. But wouldn't just using time.clock fix that? Or is time.clock impossibly slow?
But really, if you made a couple little button smashers with libaudioverse and audiogame_engine, I would totally just use your audiogame_engine, it is much smaller than pygame. Or, you could create the same function calls as pygame and just substitute audiogame_engine as pygame instead. (Although the functionality of the calls looked a little different than pygame's calls).
But I would call this a wrapper around the SDL library rather than a full-blown game engine (engine to me is something like audiogame_engine.screen() to create the screen and I don't need to worry about it, or audiogame_engine.key1() for keydown events and key2 for key pressed events.
But you may have that, I just didn't notice (I don't know enough about how audiogame_engine works to translate all the code to what it does).
BTW, I keep refreshing the forum to see if libaudioverse is posted yet! smile

2014-08-24 17:51:00 (edited by camlorn 2014-08-24 17:52:22)

Libaudioverse got delayed because the first week of school is awful, I missed the deadline and decided to go ahead and port the media player example to Python, and the blog post about it won't come out the way I want.  I then decided that, since I was going to do it anyway, I'd go ahead and do the major thing that would break backward compatibility (rename device to simulation and physical device index to device index).  There is a reason I do not like setting deadlines.  And do note that I've said "preview": this is not alpha quality, isn't supported on Linux and Mac,  and will break horribly for at least some people.
Mobs wait, but this does not mean the engine does.  Mobs that wish to wait can do one of a few things, the simplest of which is count down an integer in their tick method.  This is when you explicitly don't use threads: threads are for things that need to continuously run, not things that need to spend most or all of their time waiting.  If you can provide a performance reason for wanting threads, we can talk about it on another topic; until then, my answer will always, always be don't.  If you want me to go into why threads are so hard, I'll go find an article-it takes a good deal of writing to set things up before you knock them down hard.
Audiogame_engine has a utility object that you can tick that handles scheduling things to happen in the future; the difference is that you specify "the future" in ticks, not seconds.  It does indeed have input handling: see keyboard_handler and mouse_handler; all screens give you one.  It's also got "layering": pausing the game is as simple as making a pause screen and pushing it on top of the stack of screens; the pause screen simply sets itself not to forward events to things below it, which actually stops time until you pop it off.  As I said, Audiogame_engine is basically exactly what Microsoft says to do in C# for XNA.  They're far from the only one to advocate such approaches, however; many articles exist on the internet that show you how to make your own.
The reason Audiogame_engine manages time is that a tick needs to happen in an instant, even if it doesn't.  You're simulating one infinitesimal bit of time, and so Audiogame_engine gives you the thing you're interested in: how long since the last one?  This is called the delta.  What your brother is referring to is this: if you specify speeds in velocities and know a bit about vectors, you can make it such that the game ticks as fast as the computer will let it and still have it come out right everywhere.  This is particularly important for games that might slow down because of CPU constraints, but I'd not worry about it just yet.  I did calculate the delta for completeness, but I've not yet had a project that needs it.
Be very concerned about time.sleep calls.  You should have exactly one in your entire program, and it should be exactly in the same place as Audiogame_engine puts it.  If you do so, you don't need threads anymore.
A few things to note: you're missing the half that handles input.   Look at ScreenStack and Screen for a not-half-bad approach to managing state in your game, i.e. opening and closing minigames, inventories, and pause menus.  I have yet to actually implement menu screens that just take menu item names, but when Libaudioverse is ready and I begin using the project again, those aren't far behind.  The reason I've not been pushing it everywhere is because it is, indeed, missing good examples and docs; everything should have docstrings, however.  Finally, tick speed is configurable when you make your MainLoop instance.
See keyboard.py, mouse.py, and event_responder.py to understand how that part of it works.  ScreenStack and Screen are just a concrete implementation of the interface defined in event_responder.py aimed at having good state management that doesn't involve nesting if statements all day long.  It's only 300 lines, which makes it about the size of any tic tac toe in the command prompt implementation I've ever done.  I'm not surprised that you're having trouble with it, especially since this community advocates the nested if statements and polling approaches heavily. You seem to be at the point where you can benefit from understanding it.  I do not believe that Pygame provides the higher-level primitives, but you could implement them on top of it instead if you wanted.
And as for the name, well, I needed one.  It was there.  I will be adding more stuff to it, but it's never going to be what most people think of as an engine.  Perhaps I will rename it someday, if I can come up with something better.

My Blog
Twitter: @ajhicks1992

2014-08-25 07:08:59

I've got 2 different ways of doing screens:
1. bring out another "screen" with another window being stacked on top of the main window. This is really fast and what I've got already. The downside is that it is a little annoying to have game-wide commands imported every time this happens. It is also a little slow. What I have been working out is using a "scene" attribute that changes to something like "game1" or "movement" and when ever the screen changes, append the function that checks for all those keys after the global check.
for example:

current_scene = "movement"
scenes_dict = {"movement": functionObjectWithSceneMovement@080939, "fight": functionWithFightingCommands@94847543}

then in the main loop:
while playing:
    k = global_keyconfig(current_key)
    if not k:Scene_class.scenes_dict[Scene_class.current_scene](current_key)

This is much faster and one really has more control over the screens.
As for delta time, so it compares my game speed to the time.clock speed and counts how many should have happened in a time compared to how many did happen and makes adjustments accordingly?



Really, threading is looking so much easier than this tick thing. Here is what I came up with:


import time, random

class Ticker(object):
    """ticks has not been implemented yet.
    framerate is the amount of time the main loop rests before running again.

    def __init__(self, tick=0.6, framerate=60):
        self.tick = tick
        self.playing = True
        self.functions = []
        self.framerate = 1.0/framerate

    def wait(self, wait_time=0, name="g"):
        try:
            self.dict1[name] += self.framerate
        except AttributeError:
            self.dict1 = {}
            self.dict1[name] = self.framerate
        finally:
            if self.dict1[name] >= wait_time:
                return True

    def main_loop(self):
        while self.playing:
            [x() for x in self.functions]
            time.sleep(self.framerate)


a = Ticker()


def hello():
    if a.wait(1, "fred"):
        print "hello world"
        a.functions.remove(hello)


a.functions.append(hello)


a.main_loop()

2014-08-25 15:10:32

You're overcomplicating things tremendously.  I think someone else needs to try to explain why threads are going to screw you in the end, but you're taking the molehill of ticks and turning it into a mountain.  You can feel free to disbelieve me about threads, but I'd bet literally anything that you'll be back in two months with a failed project and the exact same conversation we're having now.  Possibly word-for-word, or at least post-for-post.  I'm having trouble finding an article that is at your level.  I suspect this is because programmers at your level do not usually jump hard on the thread bandwagon and usually not just by going "hey, look at the threading module"-i.e. the tutorial or whatever has a chance to tell you about the problems.  I'll try to be explicit about it at the end of this post.
The delta is simply the time since the main loop of your game called tick last.  It *can* be used to make the game catch up.  Or you can flat-out ignore it, pretend that it is always 1/20 or 1/30 or 1/60 or whatever, and write the whole game that way.  As I said, nothing is wrong with the latter, especially in this case.
Audiogame_engine screens are exactly like opening another window.  Except that you don't re-import anything, you don't load a second Python interpreter, etc.  Audiogame__engine just sends all input to the topmost screen on the ScreenStack, and that's it.  There's basically no performance hit for doing so, and the code that actually handles it is simpler than what you're posting.  Larger, but simpler conceptually.  Just subclass screen, override tick, and make use of self.keyboard_handler and self.mouse_handler to set up functions to be called when you've got input ready.
A few things.  First, never ever use AttributeError for initialization.  __init__ is where you're supposed to set everything to default values.  Second, the delta is not the framerate: it is the time from the beginning of a tick call to the beginning of the next tick call.  If your tick is taking more time than the framerate, your delta becomes larger than the framerate.  Third, you do not wait framerate seconds, you wait framerate-last_tick_time seconds, Where you get last_tick_time as follows:

start_time = time.time()
tick()
end_time = time.time()
tick_time = end_time-start_time
wait_time = framerate-tick_time
if wait_time > 0:
    time.sleep(wait_time)

I do not think it really matters whether or not you use time.time() instead of time.clock().  In this case, it shouldn't matter much.  Do note that the windows clocks are notorious for being imprecise, i.e. you may not be getting more than 50 ms precision on older systems.  Again, this shouldn't matter much if at all for this case.
So, threads.
You're looking at this tick thing and going "but the thread API is so easy.  I can just time.sleep whenever."  And that statement is true, but only as far as that statement.
When a program without threads runs and receives some input, it will always do the exact same thing if that input is the same.  This is always true unless the processor melts, or something drastic like that.  We can even argue such drastic cases as hard drives exploding count as input, but that's just arguing over inanities.  You have nothing more than the basic programming problems.  If your program bugs on some input, you can make it bug every time.  You always know where the problem is, or at least how to cause the problem.  Debugging is a simple process: cause the bug, fix the exception or explore with a debugger.  In this case, you take two or three weeks at most to understand audiogame_engine or one of the numerous alternatives that do exactly the same thing.  You're then on top of the world, and nothing is left that will surprise you about your programs.
Now, I could tell you the threading story, but I'm going to just give an illuminating example, followed by a rewrite of the illuminating example to show the problem:

import threading
x = 0
def my_thread():
    global x
    for i in xrange(500000):
        x = x+1
t1 = threading.Thread(target = my_thread)
t2 = threading.Thread(target = my_thread)
t1.start()
t2.start()
t1.join()
t2.join()
print x

The answer is obviously 1000000, right?
No.
This will return a number between 0 and 1000000.  The number will be different every time you run it.  You will easily observe that addition operations are going "missing".  This is the simplest example that shows problems, and if you've looked around a lot, you'll throw a lock in because that's super easy to do in this case.  But see how nondeterministic it is?  You know there is a bug because it's not always printing 1000000.  I made the number high on purpose, however.  If you bring it down to 10, this program can and will run correctly sometimes.
The specific reason why is because x = x+1 is actually 3 operations, and can be rewritten as follows:

tmp = x
x = tmp+1

The third operation is tmp+1, but you can't put that on a line by itself.  These operations are run in the order they are written, but any other thread may "interrupt" and run one or more of its operations in the middle.  Because you're in Python, we don't have to talk about the wonderful difficulties that come in in C, where x = 5 can also be more than one operation.
The solution is locks.  Except that locks don't really solve anything.  If you do this, every other line of your program is going to be mylock.lock().  Most of the remaining ones are going to be mylock.unlock().  if you want any performance gain at all, you have to use more than one--typically one per object, and you have to then be sure you locked the 5 appropriate locks before doing the next line.  And you still can't just sleep whenever: if you do so while holding a lock, you're going to make any other thread that wants to get the lock sleep along with you-whether it's executing an attack or who knows.
X is called shared state because more than one thread may touch it.  Everything in a game is shared state.  Global statements are the obvious sign, but anything that's a reference type that you pass around can easily become shared state.  In the case of a game, it has to be so that the displaying thread can handle displaying it or the AI thread can know about it or etc.
Now consider something that takes 10 lines instead of 1, and how many places (including some in the middle of lines) another thread might decide to run.  The complexity isn't something that you "learn" how to get around.  It stays with you forever.  You're going to spend sunny afternoons running your program over and over and over, hoping the bug you found that one time you ran it yesterday will happen.  Your program is going to crash, half-crash (like crashing but worse because only some threads do it), and generally act insane.
So if you go with threads, good luck.  You'll need it, and you'll need it until the end of time.  The ticking approach is only hard at the beginning, and all the standard debugging techniques work fine.

My Blog
Twitter: @ajhicks1992

2014-08-25 19:16:23

Hello.
Here is my opinion.
A good program needs to only use threads in situations that are critical,
like reading large amounts of data from Internet without blocking the main
program.
But I say: Don't use threads.
Why?
Threads are difficult to debug and some types of threads can lock your
main program don't letting the user close it.
If you want to make side scrollers or pygame based games with AI, you
don't need threads.
You'll need a time counter, with time.time(), time.clock(),
pygame.time.get_ticks() or whatever other similar functions exist.

2014-08-25 23:26:05

OK, you've convinced me, no threads!!! LOL
Now I need to figure out how to either use the module in audiogame_maker or create my own.

2014-08-25 23:51:54

I *think* audiogame_engine will mix fine with Pygame's sound stuff.  There may be sdl issues; I am uncertain, as I obviously only used it with my own stack.
Writing your own is actually a good idea, if you want to learn.  It's a manageable project in terms of complexity and size.  I'd rate it at intermediate: you're at the point of understanding that functions are things like numbers too, that inheritance is a powerful thing, and that time.sleep causes so many problems and actually solves nothing.
Best of luck, whatever you do.

My Blog
Twitter: @ajhicks1992

2014-10-09 00:34:21

Since this topic apparently is the "Discuss Python Here!" topic and I can't resist, let me add a few of my own issues here.
Camlorn, or rather, all of you, would you mind helping me out iwth something? I would like to code a GNU Compiler for Python (GCP (correct me if I'm wrong, but to my knowledge, there is no GNU compiler that has the initials GCP)), and I need a little help.
Do you guys think you could provide me with a "good", Ogreat", or "excellent" implementation of the argparse for a compiler?

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    — Charles Babbage.
My Github

2014-10-09 01:18:27

You can't.  Not in a time period less than 3 or 4 years.  This is a project that people have earned doctorates with and it's still not done.  Grad students work on Pypy all the time, and the only real reason that Pypy works is because it's a JIT (otherwise, it would be much too slow to actually be worth the gains).  If I need to talk about this more, I will.
The project you're looking for is Nuitka.  This is actually a trick: it asks Python to implement the functionality, removes the Python main loop, and applies *some* optimizations.  Also, I ran it on a real app once.  8 hours later (i.e. the next morning), it still hadn't finished running.  I gave up at that point.  Nuitka has been undergoing development for 2 years, minimum, and my understanding is that the guy doing it is a compiler expert.
Going further than this is not really possible due to the nature of Python and languages like it: your code can self-modify at runtime, types aren't fixed, class members aren't fixed, functions are effectively garbage collected (really), functions can be assigned to other functions (you can replace sys.exit, if you're insane), and there's no actual language spec (problem: how do you be 100% compatible without a standard?).  Even getting this far requires masters level computer science or higher; compilers are half programming and half mathematical proof.  Some optimization A must be provably correct in something not dissimilar to the mathematical sense; you cannot test it on all inputs.  Optimization A is usually not simple, though there are a very few low-hanging fruit like constant folding.
I personally do not know of any programmer capable of even starting this project.  I cannot name anyone except the biggest names in computer science.  Doing this project is a topic of ongoing computer science research; being able to handle languages of this type efficiently would be a boon beyond belief.  But at best, you still have to include an interpreter with your compiler (handling eval is fun), and at worst--say, because you don't know how to implement optimizations and do this as a learning project--it's slower than the original language in the first place.
But as usual, I'm sure everyone is going to say I'm wrong and we'll be beginning another game of disbelieve the Camlorn.  To that end, know first that starting with the command line frontend is a mistake of epic proportions: once it compiles something, then worry about it.  Argparse is literally the simplest part of the project; if you can't underestand it, stop now and save yourself a huge headache.  Should you actually manage to produce a compiler for Python that can compile a significant subset of the language (say, significant enough that I can make an audiogame in it), I'll maintain your argparse interface for as long as you like.  But, if you can make it across the giant gulf of conceptual lava in your way, you'll look at argparse as the most trivial thing ever and won't need me.  I'll also write everyone I know who is involved with academia, because they will probably want to descend from the heavens and bestow on to you the wonders of a degree in computer science (or at least a great many scholarships).
I have been through this phase.  It is an interesting phase and you'll learn a lot from it.  Leave it behind as soon as possible.  This phase of learning to program yields no useful code.  Take the knowledge of basic computer architecture, what you understand of compilers, etc.  Use them for something, anything, else.  They're worth a lot, but they aren't worth a compiler.  If it's something you truly want to do one day, get into college for computer science as soon as possible, make sure it's a program with an emphasis on compiler and language design, and stick with it through at least a masters.

My Blog
Twitter: @ajhicks1992

2014-10-21 20:49:31 (edited by frastlin 2014-10-21 20:52:11)

Hello,
I have been dabbling with this queue problem and delta for the last month or so off and on. I finally figured it out in several ways.
One has 3 options when using a system like pygame or pyglet:
1. give everything its own addition each loop, won't be pretty if you have many objects. But basically, you can add to the object's elapsed time every iteration of the game loop. Just make sure you are sending the sleeping time value to the objects.


2. You can create a queue for all your object (either a set or a list, I don't think it matters), and run a loop each game loop to do: first, add the sleeping time to the time elapsed for each object. second, check if the object now should run and if so, run it. 3rd, if the object should run, it checks if the object should repeat (stay on the queue), or if it should remove the object from the queue. Other than that, it just goes on to the next iteration.

3. You have a built-in queue system you use for keyboard input and system events in pygame already. pygame.event... I have not quite figured out how using this for custom events works, but it is totally possible!


In other news, I found a fantastic tutorial on sighted game programming that I have been going through. I am commenting the code with comments about what each graphic and rendering thing does, I've been using my small amount of sight to see the drawing and text events that happen as well as reading the awesome descriptions in the tutorial of what is supposed to happen. So I hope that I can manage to make incorporating basic graphics (like text and whatnot) second nature. It is really not hard once you figure out all the things you need to do. There is just a ton of initializing, creating objects, drawing to objects then putting those objects into the internal screen, then updating the hardware's screen for each graphic.
I was amusing myself by drawing heads and lines and making stuff run around the screen.
What I still need to know is:
Can pygame render graphics in half a pixel or less? Because there is this huge complaint by sighted people when graphics are not "smooth", which means that they are jumping too far each frame.


good things I have found out:
When creating the screen for any of my stuff, the normal screen is this really really small box up on the top right corner of the screen that no sane person can see. (well they probably can, it is just like an inch by 2 inches though)
So I keep their values, but here is what I do to make the screen full screen:


displaySurface = pygame.display.set_mode((400, 300), pygame.FULLSCREEN)


This goes right under pygame.init() and replaces the other values they put in pygame.display.set_mode. There is often a 0, 32 after the size tuple and those two numbers are not important, they will default to what they need to be.
second, in the game loop they write something about if event.quit:pygame.quit(). But this does not work with a keyboard. All that does is quit the screen if you can click on the little x at the top of the screen. Instead, make your keyboard loop look like this:
    for event in pygame.event.get():
        if event.type == KEYDOWN and event.key == K_ESCAPE:
            pygame.quit()
            sys.exit()


Press escape to exit. Otherwise, hit alt-tab to get to your command prompt and hit ctrl C to kill the game's operation.

Third, when working with graphics, you need to call the update to your screen each game loop. This line looks like: pygame.display.update(). The tutorial puts in sys.exit() under pygame.quit() and says it is there because the ide they say for you to use crashes without sys.exit(). But the real reason why they need that sys.exit() is because there is this pesky video error when you call pygame.display.update() after pygame.quit(). To get around this there are two options:
1. do what they say to do in the tutorial and do:
pygame.quit()
sys.exit()
2. right proper code and make a variable above the while loop that then becomes False when you hit escape and call pygame.quit() at the bottom of the script.
3. just ignore the video error because it really is not going to change how your code runs!

2014-10-21 21:58:00

You cannot draw in half a pixel.  Nothing can draw in half a pixel.  Drawing in half a pixel is literally the same  as turning on half a light bulb.  This isn't even an analogy.  It's the literal truth.
The structures you are defining are much too complicated.  There is no reason why you need queues at this point, or really ever.  Your game will literally never have enough objects that need to tick that they can't handle the counter themselves.  Your main loop should literally be this:

sleep_for = 1/20.0 #1/20 is good enough, up this if you want
while game_is_running:
 start_time = time.time()
 for i in ticking_objects:
  i.tick()
 duration = time.time()-start_time
 sleep_needed = sleep_for - duration
 if sleep_needed > 0:
  time.sleep(sleep_needed)

Objects which need to act every 3 frames or every 5 frames or whatever can maintain internal counters.  In terms of efficiency, it's the same.  If you actually get to the point of needing the extra efficiency of variable ticks, keeping in mind that this actually comes with a higher long-term complexity in some ways, we can talk about how you write the data structure for it.  The way you're doing it, presuming I understand your explanation, is actually just as slow if not slower (floating point comparison vs. integer comparison).
And that's it.  Make tickers a list, put objects that need to tick there.  I'm pretty sure this is what SoundRTS does but could be wrong; SoundRTS is borderline on the needs advanced strategies front.
As for Pygame-I'm now kinda recommending Pyglet.  It makes sound easier and does all the stuff audiogame_engine does with event layering, etc.  It's got a full set of tutorials.  My recommendations are kinda a special case, though-I can churn out the parts of Pygame needed for audiogame-style input in a couple hours (and have) and personally have two different audio frameworks I wrote myself.  In this case, a couple people I know also said "wow, great find", so take it for what it's worth.
And graphics?  Not worth it.  You're not going to be able to approach anywhere near the quality that the sighted expect.  It will be like audiogames: only those with an association to our community are going to want to do it.  Against my better judgement, I'm going to go here because it is at least interesting to learn.  Just know that it is 99% likely not to matter long-term.
So:
Smooth graphics literally changes everything about how you write your game.  Everything.
To do it, you need to divorce the tick for graphics from the tick for gameplay.  I'd suggest looking at XNA tutorials, as they all do a good job of explaining this.  Basically, your graphics ends up ticking at least 60 times a second.  Your game doesn't, though.  The smoothness doesn't come from half a pixel, it comes from moving an object 2 pixels per frame for 4 frames instead of 8 pixels in one frame every 4 frames.  To use sequences, it's like how (2, 2, 2, 2) is smoother than (0, 0, 0, 8).
You do this by fundamentally changing how objects move about.  Instead of moving an object in your game logic, you specify what its velocity should be.  You can still move objects in the game logic, i.e. for teleportation.  But for stuff like walking, you no longer do so.  The graphics tick is responsible for updating position values.  You do this by specifying velocity as 2-dimensional vectors of the form (magnitude*cos(theta), magnitude*sin(theta)), such that theta is radians counterclockwise from east.  This will lead to some choppiness with collisions.
To deal with it, you do what you're suggesting about scheduled ticks, you integrate with a physics engine, you learn about how to specify things in forces instead of velocities (basic physics knowledge and trigonometry are prerequisite requirements), you move about 50% of your code into callbacks that the physics engine calls when things collide, and then you fight with the fact that you personally can't see the difference.  In truth, going to C/C++ at this point may be required; 2D games can most likely get away, but if you decide that now it's time for 3D, you're screwed in Python.  For smooth graphics, you probably also need a bunch of math I can be pretty sure you don't have for the simple reason that 90% to 99% of blind people never learn it.

My Blog
Twitter: @ajhicks1992

2014-10-21 22:54:33

A queue is basically a list or set what you had in your audiogame engine, in the scheduler.py file. (BTW, there is 2 lines of code missing in line 18, you need to have an else:task.elapsed=0 after the self.tasks.remove(task) or the repeat flag does not work).
The second item I posted above is your audio game engine structure.


Graphics are good for a lot of things:
1. if you ever wish to work with sighted developers, they are going to have graphics, so it will be something you will see often.
2. In order to read most of the pygame tutorials, I need to either skip over all the graphics, or run the examples in a way I can so I can learn all possible from them.


I was deliberating the other day about changing to pyglet or staying with pygame and I still may do stuff in pyglet, but I think staying with the mainstream and supported product is better. I also couldn't figure out pyglet's audio. I could make a sound play, but didn't get how to fiddle with all the values.
I can also read pygame's docs and there are tons of guides using pygame where as there is not much for pyglet. Pyglet is easier, but pygame has many other advantages.
Also, when I wish to do more than basic audio, libaudioverse will be there!

2014-10-21 23:32:09

No, I don't think it is.  Audiogame_engine ticks regardless. There's some stuff in there that lets you hook into timing, but the main loop doesn't.  I've abandoned Audiogame_engine to be honest; I'm now going to use Pyglet.  The thing with Pygame, for your use case: you're going to have to duplicate a lot of what Pyglet is doing for you.  I'm not sure that Pyame is more supported either, but it's quite possible that Pyglet is aimed at a more advanced develope.
Also, do be aware that Queue has a very, very specific definition.  In the way you're using it you need to just say list.  The long and the short of it is that queue implies a lot of things about what you're doing that you don't actually mean.

My Blog
Twitter: @ajhicks1992