AI 2008 Network Subsystem.

Wrote new code? Fixed a bug? Want to discuss technical stuff? Feel free to post it here.

Moderator: Moderators

Message
Author
User avatar
kLabMouse
Administrator
Administrator
Posts: 1301
Joined: 24 Apr 2008, 12:02

AI 2008 Network Subsystem.

#1 Post by kLabMouse »

OK. It's time to design a new "Network" subsystem.

Package design:
Network
|-Server (Server functions, create socket, listen, close socket, call parser, and send).
|-Client (Client functions, establish socket, listen, close socket, call parser, and send).
|-MessageTokenizer (Split's stream to packets. No Networking functions here, just return a whole packet, or store at local buffer.)
|-Send (Send packets to Server)
|~|-ServerTypeX
|-Receive (Receive packets from Server)
|~|-ServerTypeX
|-XKore_Send (Send packets to client)
|~|ServerTypeX
|-XKore_Receive (Receive packets from client)

NetworkSub (Network Sub System types, and constructors)
|-DirectConnection
|-XKore
|-XKore2
|-XKoreProxy
|-e.t.c

Comunication:
1) Every "NetworkSub" must have at least two functions "new" and "SendRaw" and one hash blessed by "Network::Send" called "$self->{send_to_server}".
2) "Network::Receive" must push all messages to "$environment_change_Queue". Those will be interpreted by "AI::Environment::Queue". <-- All messages need to be documented out.



Everybody is welcome to suggest it own ideas, and implementations!

User avatar
kLabMouse
Administrator
Administrator
Posts: 1301
Joined: 24 Apr 2008, 12:02

Re: AI 2008 Network Subsystem.

#2 Post by kLabMouse »

Today, I made Network::Send package.
So, we are one step closer to better OpenKore ever made.

Technology
Super Moderators
Super Moderators
Posts: 801
Joined: 06 May 2008, 12:47
Noob?: No

Re: AI 2008 Network Subsystem.

#3 Post by Technology »

Everybody is welcome to suggest it own ideas, and implementations!
Ok, on behalf of this here are my implementation thoughts, visions and idea's...

XKore serverTypes
I think that serverTypes for XKore are not needed,
we can use the unpack strings from the receive part and use them to pack data into a packet that we send to the client.
However, we need perfect unpacking in order to be able to do perfect packing. (look at it this way, its easier to unpack a suitcase than to pack it)

XKore/Posseidon server
I know nothing about posseidon but it looks like xkore 2 to me in a way.
Couldn't we merge the 2 and make posseidon a special case of xkore 2?
In xkore2 we could add more features like read only. (kore will send to the RO client but will only do something with the keepalive packet)

Complex Packets
i suggest we use the Convert::Binary::C package until perl adds native support to do what we want.
We will need to change the structure of the packet_list attribute to support normal unpacking or complex (un)packing:
OLD:

Code: Select all

'packetSwitch' => ['handler function','unpack string',[qw(argument names)]]
NEW:
0) normal (un)packing

Code: Select all

'packetSwitch' => ['handler function', 0,'unpack string',[qw(argument names)]]
1) complex (un)packing

Code: Select all

'packetSwitch' => ['handler function', 1,'struct name', [tag options...]]
these tags options will help us to do special unpacking like:

Code: Select all

$c->tag('test1.auction[0].seller', Format => 'String');
$c->tag('test1.auction[0].buyer', Format => 'String');
An alternative vs Convert::Binary::C could be this:
1) complex (un)packing

Code: Select all

'packetSwitch' => ['handler function', 1,'complex perl unpack function', 'complex perl pack function']]
This 'complex perl (un)pack function' will do the same thing as the Convert::Binary::C:
for unpacking it will return an unpacked perl hash structure to the 'handler function' (or straight to $environment_change_Queue)
for packing it will take a perl hash structure as an argument and pack this accordingly, so we can send it.

Send Part
We can re-factor the send part so that it looks more like its receive counterpart.
To do this we will have to separate the pack-strings (that we now use in the subs) into some kind of object that holds a table of all pack-strings.

Maybe it is possible to merge all pack and unpack into 1 single table.
(reference: look at Receive/Sakexe_0.pm, it seems like there is no packet-switch (packet ID) being used on both send and receive part, or am i wrong about this?)

Other
btw, is stringToBytes etc. really needed when we use the perl pack function instead of just appending a string to a string? It would be nice if they were obsolete.
Look at this example please. By only using the pack function we don't only make the code shorter, it also adds the possibility to do what is explained in the paragraph: Send Part.
old:

Code: Select all

sub sendChatRoomBestow {
	my ($self, $name) = @_;

	my $binName = stringToBytes($name);
	$binName = substr($binName, 0, 24) if (length($binName) > 24);
	$binName .= chr(0) x (24 - length($binName));

	my $msg = pack("C*", 0xE0, 0x00, 0x00, 0x00, 0x00, 0x00) . $binName;
	$self->sendToServer($msg);
	debug "Sent Chat Room Bestow: $name\n", "sendPacket", 2;
}
new:

Code: Select all

sub sendChatRoomBestow {
	my ($self, $name) = @_;
	my $msg = pack('v x4 Z24', 0x00E0, stringToBytes($name));
	$self->sendToServer($msg);
	debug "Sent Chat Room Bestow: $name\n", "sendPacket", 2;
}
Can someone that uses such foreign languages test this?
One ST0 to rule them all? One PE viewer to find them!
One ST_kRO to bring them all and in the darkness bind them...

Mount Doom awaits us, fellowship of OpenKore!

kali
OpenKore Monk
OpenKore Monk
Posts: 457
Joined: 04 Apr 2008, 10:10

Re: AI 2008 Network Subsystem.

#4 Post by kali »

Poseidon can be used as a central server. For example, 10 bots can run off a single Poseidon server. Having it be special case of XKore 2 means that you either have to limit poseidon requests to one client per bot, or have XKore 2 implement a "viewer" model that is read-only.
Got your topic trashed by a mod?

Trashing topics is one click, and moving a topic to its proper forum is a lot harder. You expend the least effort in deciding where to post, mods expend the least effort by trashing.

Have a nice day.

Technology
Super Moderators
Super Moderators
Posts: 801
Joined: 06 May 2008, 12:47
Noob?: No

Re: AI 2008 Network Subsystem.

#5 Post by Technology »

Hmm i didn't know multiple bots could connect to a poseidon server. :o
Still, the RO client connects to the poseidon server right? This looks pretty much like xkore2/proxy to me.
But yea, after the client is connected, all poseidon does is keeping the client alive and sending the challenges to it.
Whereas with the xkore2/proxy server we pass on all packets (which we don't block or alter) to the RO client.
One ST0 to rule them all? One PE viewer to find them!
One ST_kRO to bring them all and in the darkness bind them...

Mount Doom awaits us, fellowship of OpenKore!

kali
OpenKore Monk
OpenKore Monk
Posts: 457
Joined: 04 Apr 2008, 10:10

Re: AI 2008 Network Subsystem.

#6 Post by kali »

It's a well known secret (I should now, my small proof of concept evolved to what is now known as Poseidon :D)
The advantage of the Poseidon is that you don't need an RO server to connect to. So for example, you have a P3 computer running Poseidon stand alone, while yur bot farm connects to that PC through the LAN. Or something that the bRO admin did - he had a central Poseidon server and everyone else botting in bRO connected to it (he ran it through donations).
Got your topic trashed by a mod?

Trashing topics is one click, and moving a topic to its proper forum is a lot harder. You expend the least effort in deciding where to post, mods expend the least effort by trashing.

Have a nice day.

Technology
Super Moderators
Super Moderators
Posts: 801
Joined: 06 May 2008, 12:47
Noob?: No

Re: AI 2008 Network Subsystem.

#7 Post by Technology »

Ok, nvm the Xkore / Posseidon for now.

Lets focus on the very basics and fundamentals of OpenKore first, the packets.

kali was talking about packets that are aware of how they are parsed and created.
Here's how we could realize this:
We should separate the pack strings from the subs (like the receive part) to make a lot of 'serverType hacks' and code that is being used atm obsolete.
Then we could merge the send and receive part which would make it very easy for kore to act as both server/client packet-wise.
Imagine how convenient it would be if it were possible to be able to both easily parse or even create any packet from data to send that to the client. (for xkore usage).
(note: Complex packets will have 2 (un)pack functions instead of 1 (un)pack string, until perl allows us to do what we need)

kLabMouse was talking about a base class for all serverTypes.
Here's how we could realize this:
We could create the base class out of the Sakexe_0.pm and ServerType0.pm (which are identical in a lot of places, if not totally) on both send and receive part.
There are 2 cases:
- If a old packet changes, we create a new child serverType that bases on the previous (for kRO/eA) or adjust the child serverType (iRO, mRO, ...).
- If a new packet gets added, we add it to the base serverType.
This way we can ensure that:
- we never break support for one serverType by supporting another.
- we support every eA version and don't end up with the amount of kRO ST's we have now. (new packets might as well be included in the base ST)
One ST0 to rule them all? One PE viewer to find them!
One ST_kRO to bring them all and in the darkness bind them...

Mount Doom awaits us, fellowship of OpenKore!

User avatar
kLabMouse
Administrator
Administrator
Posts: 1301
Joined: 24 Apr 2008, 12:02

Re: AI 2008 Network Subsystem.

#8 Post by kLabMouse »

Technology
I really like it.
But, one question: What will happen to packets that are seen only on 2-3 ST's, and all other use new one?

Technology
Super Moderators
Super Moderators
Posts: 801
Joined: 06 May 2008, 12:47
Noob?: No

Re: AI 2008 Network Subsystem.

#9 Post by Technology »

kLabMouse wrote:But, one question: What will happen to packets that are seen only on 2-3 ST's, and all other use new one?
You mean like packets that are new but replace the functionality of old packets on the send side right?
Look in: Network::Send::kRO::RagexeRE_2008_09_10a
ex. sendMapLogin -> 0x0436 -> in packet version 24
Here's what we are doing now and can continue on doing:
We update the other ST's with the new packet and leave the 2-3 ST's untouched.
One ST0 to rule them all? One PE viewer to find them!
One ST_kRO to bring them all and in the darkness bind them...

Mount Doom awaits us, fellowship of OpenKore!

EternalHarvest
Developers
Developers
Posts: 1798
Joined: 05 Dec 2008, 05:42
Noob?: Yes

Re: AI 2008 Network Subsystem.

#10 Post by EternalHarvest »

Technology wrote:we can use the unpack strings from the receive part and use them to pack data into a packet that we send to the client.
However, we need perfect unpacking in order to be able to do perfect packing. (look at it this way, its easier to unpack a suitcase than to pack it)
Network modules can do that for a long time. Why it wasn't used at all, even for packets which don't require complicated packing?

Post Reply