2019-06-21 00:20:35 (edited by amerikranian 2019-06-21 00:21:15)

I have been looking for a suitable answer and to this day have found nothing that completely answers the questions I have, hence me asking here.
What is networking?
Don't worry, I will go into greater detail. Let's use Scrolling Battles as an example. What happens when I click on login? Does the server store my login information and compare it against what I type in as my password? Does it use files to do so?
What happens when I click on "create an account"? Do I cause a server to generate a file and then store my credentials??
How does chatting work? I mean, I understand the most basic concept, a for loop that sends a message to everyone connected (we'll exclude channels for simplicity sake), but how does the server know that it's a chat and not, I don't know, a shutdown request or a chat containing a special command of some sort.
What allows me to hear others as they move around? I'm using SBYW again just because it's on my mind. In the game I am able to hear others and if I choose to, I can also hear their beacon. How does that work? Do the clients constantly send out a pulse which is only displayed when I turn on beacons? I know that pulse is not an exact word that fits, but it does get the point across.
How easy it is to intercept data between the server and the client? This might be a difficult one to answer, lol. I have the most basic knowledge of hacking, and I am aware of something called "man in the middle" attack, in which a third party, a hacker, sits on the same network and gets a glimpse of data as it's being sent between the server and the user. How easy could be accomplished when it comes to something like sbyw? Should it even be a concern?
With the question of hacking comes the question of encryption. What, if any, of the data that is being transmitted between the server and the client needs to be encrypted? Again, should encryption be a concern?
Different protocols?? I am aware that a lot of this will probably go over my head, but what is TCP, UDP, etc? Why were they created? Are they used for certain tasks, or it doesn't really matter.
Different networking methods? Again, probably a bit too advanced, but I'll ask anyway.. I heard that in BGT, a server application typically uses a switch statement and does different things depending on the case. On the contrary, some examples I found for python networking just have a function named "connectionLost" or "connectionFailed" in the client and nothing resembling a switch and case statement in the server script. Why? What changes? What is happening under the hood that makes the scripts completely different?
Server workload, I'm actually using a different game as an example this time, yea! Couple of months ago, i had a great idea. I said, "What if I tried to speedhack Swamp?" It was out of curiosity, I didn't mean to keep it permanent,, I just wanted to see if it would work. It didn't. More over, it was like the program didn't register that I had enabled speedhack at all. In something like Survive the Wild or Redspot the speedhack would work and then the game would promptly close, but in Swamp, it utterly failed to register, which makes me suspect that the walking time was controlled by the server. I am aware that my theory could be wrong, but it brings me to my next question: How much should the server do? What should it store, what should it control,, etc.
Tired of me yet? Don't worry,  this should be the last question. What are some good libraries for networking in python? I have done some research, and a lot of people seem to like Twisted. Are there any good tutorials that you personally have found concerning the lib? Do you use something else?
I know that this was a lot, and I want to say thank you for reading. I am sorry if some of my facts in regards to hacking and or BGT networking might be slightly off, I know next to nothing about the former and very little about the ladder. That said, I hope that you can still help me with at least some of this mess.

2019-06-21 01:26:00

Chat me on Skype and we'll go on tt. I could answer most of these questions but don't really feel like writing all of it out.

Ivan M. Soto.
Feel free to check out my work and services.
http://ims-productions.com

2019-06-21 01:50:58 (edited by magurp244 2019-06-21 01:54:01)

Networking is dark voodoo witchcraft. Oh wait, a better answer, right, yes...

There's a number of ways to structure network code, some requiring more authentication than others, some using raw packet data, function calls, etc. There's a lot to cover, so for the sake of simplicity lets start with TCP and UDP.

The first protocol to be created as [TCP], which stands for Transmission Control Protocol. Part of the problem with network communications is getting data from computer A to computer B, it sounds simple but a lot can go wrong on the way. For example what if computer A takes to long to send the data? Or it gets hung up? What if data gets lost along the way? How does computer A know computer B recieved its data, so it can start sending different data? What if computer A can talk to computer B, but computer B has problems talking to computer A, or vice versa? What if computer A starts sending new data before computer B got the previous data and gets mixed up? So, what TCP does is give a reliable, ordered and error checked stream of data between two points, so it makes sure that data from computer A gets to computer B, but the added problem is that if there's a problem like dropped packets or latency, it can get stuck and keep sending packets until it gets through, blocking anything else.

Now, [UDP] or User Datagram Protocol, is a bit different. You don't have to connect to any specific client with a UDP protocol, it just beams it all over the place, in any order, and doesn't care about quality assuarance. This makes it a faster than TCP but somewhat less reliable. So what you can do is set it up so computer A broadcasts a packet on the network, and computer B just listens to the broadcasts for packets it wants, then computer B can broadcast it recieved the packet and computer A can switch over, etc. Think of it like two people holding a conversation by shouting across a canyon. The reason this was made was more to side step the problems of bottlenecks in communication, if you need to transmit something faster you may not necessarily want to care about how it gets there, and instead pack the error correction into the program itself rather than the protocol.

Now, getting into network structures, there's Peer to Peer, and Client Server setups, and maybe some combinations thereof. In Peer to Peer every computer is connected to every other computer, which is how torrent swarms work. This can produce a very robust system, as if one computer goes down, other computers in the swarm all have a copy of whats going on and keep going. In a strict Server Client model, if the server goes down the entire game would end, for example. The problem with Peer to Peer, at least in game applications, is that it requires everyone to keep everyone else in the loop, which means you have to transmit alot of information. Lets say there are 3 users, now if user A does something, he has to tell user B and C what he just did, then user B and C have to let user A know they got the message, and the same goes for both user B and C with user A. Now what happens if you have 6 users? Then user A would have to send a message to user B, C, D, E, and F that he did something, and then they all have to send a response, and then THEY have to tell user A they did something, and well, it gets very cumbersome very quickly.

Now, its situations like that where a Client Server model really shines. In this model one user stands as the server, and everyone else is a client. When a client makes a move, it sents its data to the server, and in turn the server sends all the collected new data off to all the connected Clients. So when user A does something, it just sends it to the server, then the server tells users B and C what A did, and when B or C does something, they tell the server, and the server tells everyone else. This means that you have to send a lot less data, and store less data on the clients when doing things, but the trade off is that the server has to do a lot more number crunching, and it if goes down, the entire conversation ends.

Now as to your question of speed hacking. One of the golden rules in Client Server models, or any model for that matter, is: Do Not Trust The Client. Never ever take a client at its word that the data its sending is accurate and always have the server verify it. A great example of this is Diablo 1 by Blizzard, where they made the hilariously foolish mistake of depending on clients to manage character data. Lets just say there ware a lot of people walking around with infinite health and mana, hacked super gear, etc. In the case of speed hacking, what a client would likely do is lie about its position or velocity to the server, "I'm not at X, and actually at Y!", the server takes their word for it and the player zips along. As a result, various server's can be programed to be a lot more discriminating, like asking "well, if your at X and you say your at Y, is it likely you get to Y in the time you said you did?", if the answers no, the server ignores your attempt at cheating and either auto corrects, after all it could be a latency or lag problem, or straight up kicks you. Now, what a server controls or checks for it up to you, but generally the server handles all the game logic and user input, then sends that input to clients to handle the UI, like playing sounds and representing the actions that take place.

I'm not sure how Scrolling Battles or SBYW may handle their network code, but likely when clicking login it establishes a connection with the server and likely does compare the login into you provide against data it has stored. Any files would likely be stored server side, and creating an account likely would create a listing in a database or its own separate file. As for telling if a packet is for chat or not, packets are basically strings of data, so adjusting the header part of the packet it can be used to determine what kind of packet it is, or use separate function calls when communication that are dedicated for chat. Hearing others is likely purely handled client side, but with the server sending data on where those objects are, and hacking really depends on how the client server setup is handled. Man in the middle attacks generally require a 3rd computer to be between computer A and computer B that their talking across. On the wider internet this is more common to happen than over, say, a local area network.

Now to top off this novella, you can check out a python twisted example I posted awhile ago in [this] thread, post 15.

-BrushTone v1.3.3: Accessible Paint Tool
-AudiMesh3D v1.0.0: Accessible 3D Model Viewer

2019-06-21 01:59:48

Alrighty. Deep breath...
Here goes!
Bare in mind these are only things I'd do. I have no idea how scrolling battles, and indeed, how any other game does things, this is merely what I would do, were I to think about coding such a system.
What happens when you click login.
the client sends the server a message that tells it a new user wishes to log in.
the server may ignore this, or it may send back a request to the client that pops up an input field for username, depending on coding style.
Most of the time, from what I know(this could be wrong), a password is encrypted by the server when it receives it, and then compared to an encrypted password which it has stored, like so:
client asks users for password,
client sends password to server
server encrypts password
if(encrypted_password==stored_password):
it matches, log in.
I might be getting my encryption and sending round the wrong way, but there you go, basicly.
Does it store your user info in a file? Depends on the coder. some use files, others use databases.
What happens when you click create an account?
Probably about the same as login, a generic user is created, and modified as you go along.
Is a new file created?
probably not.
How does the server know it is a chat message, not a shutdown message?
A little guess work, but again, here's how I'd do it:
The client formats its messages to the server in a specific way, consider the below:
chat|$a0|Hi everyone, how are you.
shutdown|$a0|secret authorisation code
spawn|$a0|enemies and kill everyone.
the server would receive these messages, and split them. The phrase "|$a0|" isn't one anyone is ever likely to type, and so it should be relatively safe.
The first part then tells the server what the message is, and the second part tells the server more info.
For example, the chat message tells the server its a chat message, and the message is x.
What allows me to hear others as they move around?
at a guess, I'd say when a user hits an arrow key, or what ever they use to move, their client sends to the server a message that says, "hi, I moved to here!"
The server then says ok. Whose nearby?
Aha, Jimmy is nearby, jimmy, bob just moved.
Then Jimmy's client goes, ok, play a sound! boink.
How easy it is to intercept data between the server and the client?
Don't know, don't care.
Hacking is a deep dark hole, and I make it a point to avoid it at all costs. It's not worth it, and its far better to avoid it. the only people who hack are either very low on the social scale, or very, very bored and need to invest in hunting, fishing, or a similar recreational activity.
At a guess, providing the server is secured with SSL, TLS and all that fancy shmancy stuff, I'd say very difficult. Otherwise, I'd say probably relatively easy to those who know what they're doing.
Easy when you know how, I guess.
What data should be encrypted?
The more encryption the better, i always say.
Ok so the difference between TCP and UDP.
They are both networking protocols.
In a basic term, a UDP connection sends some information, and that's the end of that.
Where as tcp has an open stream. A bit like... ok.
UDp is posting a letter to someone.
TCP is having a phone call.
With UDP, there's not really a way to instantly respond.
Where as TCP is a stream, and can be responded to almost instantly, it keeps the sockets open and is generally good fun all round.
Switch and Case are things I've never used, but I'd imagine its up to coders preference.
How much should the server control?
EVERYYTHING! Big brother is watching youuuu! ahem. This is largely up to the coder at the end of the day, but really, I'm all for total server control, and the client just being a floppy endpoint for the user to connect through.
Good networking for python?
Twisted, so I'm told. I stick with the good and trusty socket module, but then other coders stick their nose in the air and utterly refuse to even think about talking about that module, so perhaps it is not the best choice.
I hope this helps.

Nathan Smith
Managing Director of Nathan Tech
It's not disability
It's ability!

2019-06-21 03:50:46

What happens when I click on login? Does the server store my login information and compare it against what I type in as my password? Does it use files to do so?
This is a very difficult question to answer because it depends on the server your communicating with. As post 3 said, networking goes all the way back to TCP/IP. TCP and UDP are the two major core protocols that all other protocols are built upon. If you are familiar with the internet protocol suite of networking and its layers, TCP and UDP would fall in the transport layer, along with DCCP, SCTP, and RSVP (among others). Protocols like LDAP, HTTP, SMTP, SNMP, etc., fall into the application layer.
Back to your question, and its related ones: again, this depends on how the protocol is designed. At a guess, I'd say that when you click login, you are immediately asked for a username/password combination, which is hashed and sent to the server. The server receives this hash, and the username associated with that hash, finds the username, compares it to the existing hash on file for that user account, and if both match, the authentication is complete. If any fails, authentication fails.
Does it use files? Maybe. If its BGT, probably. The optimal solution would be a database, because datbases offer what files canot -- ultra-fast query times and indexes (among others).
What happens when I click on "create an account"? Do I cause a server to generate a file and then store my credentials??
When you click on create account, you are asked for your creds. The application most likely then establishes a connection to the server and sends specially encoded messages to the server to designate it as a "create account"/"registration" message. The server may then store the authentication credentials in open, unsecured files or memory; in secure files/secure storage or memory; in a database on the server system; or may forward those credentials onto another server for secure storage and archival purposes. (It could also do anything else it likes with those credentials.)
How does chatting work? I mean, I understand the most basic concept, a for loop that sends a message to everyone connected (we'll exclude channels for simplicity sake), but how does the server know that it's a chat and not, I don't know, a shutdown request or a chat containing a special command of some sort.
Chatting involves encoding your message (in, say, base64), prepending it with a unique identifier identifying it as a chat message, and sending it. (The ID can be anywhere else, but should be there.) This eliminates the possibility of a user sending arbitrary commands to the server to invoke behavior that is normally disallowed. A shutdown request, therefore, has a completely unque identifier of its own.
How easy it is to intercept data between the server and the client? This might be a difficult one to answer, lol. I have the most basic knowledge of hacking, and I am aware of something called "man in the middle" attack, in which a third party, a hacker, sits on the same network and gets a glimpse of data as it's being sent between the server and the user. How easy could be accomplished when it comes to something like sbyw? Should it even be a concern?
First, your description of an MITM is slightly incorrect, though iscorrect if the servr is only available on the LAN. It is, however, possible to execute an MITM over the internet; this, howeve,r usually requires exact and precise control over the connection (on both ends) and can usually only be done by ISPs or malicious software on either end of the connection. In any case, if the data is unencrypted and sent raw over TCP, it is trivial to intercept (and even modify) said data with a packet sniffer.
With the question of hacking comes the question of encryption. What, if any, of the data that is being transmitted between the server and the client needs to be encrypted? Again, should encryption be a concern?
If your not using TLS/SSL, then all of it should be. If your using TLS/SSL, you should e confident that yoru not using a weak certificate or cipher algorithm. Anything such as RSA 2048 or greator is perfect. (I prefer RSA 16384, but that's just me.) There are lots of ways of doing encryption, and securely sending and receiving data, and I cannot cover them all.
Different protocols?? I am aware that a lot of this will probably go over my head, but what is TCP, UDP, etc? Why were they created? Are they used for certain tasks, or it doesn't really matter.
TCP and UDP were created to ensure that data could be sent both reliably and unreliably. As post 3 explained, TCP was created to ensue that all data would always be delivered, no matter how shaky the connection was, though connections can be lost. UDP was created to do the exact opposite: send data to the IP and forget about it. Its what makes UDP so awesome yet unrelable at the same time. TCP and UDP are used for *everything* that is sent and received over the internet.
Different networking methods? Again, probably a bit too advanced, but I'll ask anyway.. I heard that in BGT, a server application typically uses a switch statement and does different things depending on the case. On the contrary, some examples I found for python networking just have a function named "connectionLost" or "connectionFailed" in the client and nothing resembling a switch and case statement in the server script. Why? What changes? What is happening under the hood that makes the scripts completely different?
Each programmer has their own way of writing networking systems. In this case your forgetting syntax. Python does not use a switch/case statement (in fact, it doesn't have one). With a few hacks you can simulate one, but ti wouldn't be easy to work with. .The equivalent thing to a switch statement would be a bunch of if/elif/elif/... statements, with the else statement being the default case.
Server workload, I'm actually using a different game as an example this time, yea! Couple of months ago, i had a great idea. I said, "What if I tried to speedhack Swamp?" It was out of curiosity, I didn't mean to keep it permanent,, I just wanted to see if it would work. It didn't. More over, it was like the program didn't register that I had enabled speedhack at all. In something like Survive the Wild or Redspot the speedhack would work and then the game would promptly close, but in Swamp, it utterly failed to register, which makes me suspect that the walking time was controlled by the server. I am aware that my theory could be wrong, but it brings me to my next question: How much should the server do? What should it store, what should it control,, etc.
CheatEngine and speed hacking work on a DLL level. It is possible to detect and prevent DLL injection attacks, which is exactly how speed hacking works. This is probably what prohibited your speed hack from functioning.
As for server workload, that depends on a *lot*, and how much data the server can send/receive also depends on a *lot* of factors: how many sockets the OS will allow, how much RAM the OS has, etc.

"On two occasions I have been asked [by members of Parliament!]: 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out ?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."    ā€” Charles Babbage.
My Github

2019-06-21 04:04:58 (edited by amerikranian 2019-06-21 04:08:43)

Post 3,  is there a way to see who sent the data?  I am asking because if we move and we want somebody else to hear us, the person moving does not need to hear them selves move twice.
Also, in your example for the server, you never create a while loop. Does the reactor take care of that on its own? If so, how can you terminate it  besides killing it from the task manager?
you mentioned packet loss with TCP because somebody takes too long to reply.  What would happen  if such a case  was to occur  and what can I do to fix and prevent it? I am asking because I want to be prepared if it happens to me.
For parsing data, Iā€™m assuming I would do so in the data received function of the server echo class, Correct?
Also, how much security does twisted provide, or do you have to do that on your own?

2019-06-21 05:02:06

@6
Typically yes  In the twisted examples, each connected user would have their own unique Echo() class handler, so if you put them in a list you could easily tell who sent what packet of data. The data recieved function is where you'd handle the inbound data.

Now, the examples posts were just that, examples. I do have more game oriented examples written with pyglet available, generally speaking you have to be careful when setting up a game loop because you have to let the reactor take control each cycle so it can check incoming and outgoing packets and connections, otherwise your network layer will stall. What I used to do this is twisted's coiterate() function to call an update() function with a while loop in it, then used yield to return control to the reactor every cycle. I can dig around for an example of this, if you like. There were also other guides and information in the thread Ilinked to with the examples that may be of use, like the 1500 archers article.

With TCP, packet loss is just something that happens when a packet of data doesn't reach the other side, its not necessarily because someone takes too long. If that happens, computer B would keep waiting for computer A to finish sending the packet. To resolve this issue net code typically uses a timeout, so if a packet takes to long to transmit, computer A and computer B agree to skip that packet and focus on the next, if possible, or the connection can be dropped. Depending on how the code is structured, you could have clients dynamically join a server game and "catch up" with the other players packet flow and game state that way, much like how they have FPS games where you can just jump into a public server at any time.

For security, Twisted does allow you to encrypt packets, though you can encrypt them yourself if you prefer. I haven't played around with that much however.

-BrushTone v1.3.3: Accessible Paint Tool
-AudiMesh3D v1.0.0: Accessible 3D Model Viewer

2019-06-21 06:37:53

Yes, I would like some more game oriented examples.

2019-06-21 08:21:26 (edited by magurp244 2019-06-21 08:24:03)

Right, understand that this is still a basic example, there's quite a bit more that could be added to it, like recionnection attempts, timeouts, etc. In this setup the script acts as both client and server, so you need to press "s" to start the server, then "c" to connect to it locally. You can then press space where it will send a packet across, you'll also notice that it prints things twice, this is not a bug. When you connect it will add two user echo() classes to the user list, one is for the client side, the other for the server side. In practice there would be only one of these in the list depending on whether your strictly the server or the client, but thats not really relevant at the moment. You can press "q" to quit or "d" to disconnect clients from the server side.

import pyglet
from pyglet.window import key
from pyglet import clock

from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.internet.protocol import Protocol
from twisted.internet.protocol import Factory
from twisted.internet.protocol import ClientFactory
from twisted.internet import reactor
from twisted.internet import task

import pickle
import zlib
import random

pyglet.options['debug_gl'] = False

#primary window/game class
class Prototype(pyglet.window.Window):
    def __init__(self):
        super(Prototype, self).__init__(640, 480, resizable=False, fullscreen=False, caption="Test")
        self.clear()

    #server side connected user variable, when someone connects the Echo Class will add itself into it.
    #then just call self.user.sendData to transmit to the connected user or check self.user.inbox to get data.
        self.user = []

        self.server = None
        self.client = None

    #since the Reactor is in control, to properly run the main game loop you have to manually call the clock.
    #this variable is to keep track of the last time the clock was ticked.
        self.old_dt = clock.tick()
        self.fps_display = pyglet.clock.ClockDisplay()


#shutdown program gracefully
    def shutdown(self, result):
        reactor.stop()

#system error shutdown
    def bailout(self, reason):
        reason.printTraceback()
        reactor.stop()

#main game loop
    def update(self):
        while not self.has_exit:
        #manually dispatch window events (since twisted reactor is in control)
            self.dispatch_events()

        #Make sure events are timed properly while coiterating with network layer. That and tick clock events like the fps counter and game pacing.
            dt = clock.tick()
            self.old_dt = self.old_dt + dt

        #if enough time has passed, update game state
            if self.old_dt >= 0.025:
                self.old_dt = 0

                self.clear()

                for a in self.user:
                    if len(a.inbox) > 0:
                        print(a.inbox)
                        a.inbox = []

                self.fps_display.draw()


        #draw here and manually flip frame buffers
            self.flip()

        #return control to twisted Reactor
            yield 1

                    
    def on_key_release(self,symbol, modifiers):
    #start server
        if symbol == key.S:
            self.server = TCP4ServerEndpoint(reactor, 8007)
            self.server.listen(QOTDFactory())
            print("server initializing")
    #connect to server
        if symbol == key.C:
            self.client = reactor.connectTCP('localhost', 8007, EchoClientFactory())
            print("connecting")
    #disconnect client from server side
        if symbol == key.D:
            for a in range(0,len(self.user),1):
                self.user[a].disconnect()

        if symbol == key.Q:
            if self.client != None:
            #disconnect from the client side
                print('stopping client')
                self.client.disconnect()
            self.close()

        if symbol == key.SPACE:
            for a in self.user:
                a.sendData("space pressed")



#this handles two way communication with connected clients
class Echo(Protocol):
#data reciever class
    def connectionMade(self):
    #add this class to the main program
        window.user.append(self)
    #stored incoming data
        self.inbox = []

    def dataReceived(self, data):
        data = zlib.decompress(data)
        data = pickle.loads(data)
        self.inbox.insert(0,data)

    def sendData(self,data):
        data = pickle.dumps(data)
        data = zlib.compress(data)
        self.transport.write(data)

    def disconnect(self):
        self.transport.loseConnection()



#this monitors the port and automatically calls Echo() when someone connects, passing their address to it.
class QOTDFactory(Factory):
    def buildProtocol(self, addr):
        print(addr, "connected.")
        return Echo()



#This class establishes a connection to a server.
class EchoClientFactory(ClientFactory):
    def startedConnecting(self, connector):
        print('Started to connect.')

    def buildProtocol(self, addr):
        print('Connected.')
        return Echo()

    def clientConnectionLost(self, connector, reason):
        print('Lost connection. Reason:', reason)

    def clientConnectionFailed(self, connector, reason):
        print('Connection failed. Reason:', reason)



if __name__ == '__main__':
#initialize main window
    window = Prototype()
#port to listen to
    task.coiterate(window.update()).addCallback(window.shutdown).addErrback(window.bailout)
#start reactor loop
    reactor.run()
-BrushTone v1.3.3: Accessible Paint Tool
-AudiMesh3D v1.0.0: Accessible 3D Model Viewer

2019-06-21 10:04:43

Hi, @post 1, you have no idea for how long i have wanted to ask these types of questions. even though i don't have my menu right, networking has been a question since i used BGT

best regards
never give up on what ever you are doing.