Page 1 of 2

AI 2008 Network Subsystem.

Posted: 20 Oct 2008, 09:03
by kLabMouse
OK. It's time to design a new "Network" subsystem.

Package design:
Network
|-Server (Server functions, create socket, listen, close socket, call parser, and send).
|-Client (Client functions, establish socket, listen, close socket, call parser, and send).
|-MessageTokenizer (Split's stream to packets. No Networking functions here, just return a whole packet, or store at local buffer.)
|-Send (Send packets to Server)
|~|-ServerTypeX
|-Receive (Receive packets from Server)
|~|-ServerTypeX
|-XKore_Send (Send packets to client)
|~|ServerTypeX
|-XKore_Receive (Receive packets from client)

NetworkSub (Network Sub System types, and constructors)
|-DirectConnection
|-XKore
|-XKore2
|-XKoreProxy
|-e.t.c

Comunication:
1) Every "NetworkSub" must have at least two functions "new" and "SendRaw" and one hash blessed by "Network::Send" called "$self->{send_to_server}".
2) "Network::Receive" must push all messages to "$environment_change_Queue". Those will be interpreted by "AI::Environment::Queue". <-- All messages need to be documented out.



Everybody is welcome to suggest it own ideas, and implementations!

Re: AI 2008 Network Subsystem.

Posted: 02 Feb 2009, 14:25
by kLabMouse
Today, I made Network::Send package.
So, we are one step closer to better OpenKore ever made.

Re: AI 2008 Network Subsystem.

Posted: 27 Aug 2009, 16:05
by Technology
Everybody is welcome to suggest it own ideas, and implementations!
Ok, on behalf of this here are my implementation thoughts, visions and idea's...

XKore serverTypes
I think that serverTypes for XKore are not needed,
we can use the unpack strings from the receive part and use them to pack data into a packet that we send to the client.
However, we need perfect unpacking in order to be able to do perfect packing. (look at it this way, its easier to unpack a suitcase than to pack it)

XKore/Posseidon server
I know nothing about posseidon but it looks like xkore 2 to me in a way.
Couldn't we merge the 2 and make posseidon a special case of xkore 2?
In xkore2 we could add more features like read only. (kore will send to the RO client but will only do something with the keepalive packet)

Complex Packets
i suggest we use the Convert::Binary::C package until perl adds native support to do what we want.
We will need to change the structure of the packet_list attribute to support normal unpacking or complex (un)packing:
OLD:

Code: Select all

'packetSwitch' => ['handler function','unpack string',[qw(argument names)]]
NEW:
0) normal (un)packing

Code: Select all

'packetSwitch' => ['handler function', 0,'unpack string',[qw(argument names)]]
1) complex (un)packing

Code: Select all

'packetSwitch' => ['handler function', 1,'struct name', [tag options...]]
these tags options will help us to do special unpacking like:

Code: Select all

$c->tag('test1.auction[0].seller', Format => 'String');
$c->tag('test1.auction[0].buyer', Format => 'String');
An alternative vs Convert::Binary::C could be this:
1) complex (un)packing

Code: Select all

'packetSwitch' => ['handler function', 1,'complex perl unpack function', 'complex perl pack function']]
This 'complex perl (un)pack function' will do the same thing as the Convert::Binary::C:
for unpacking it will return an unpacked perl hash structure to the 'handler function' (or straight to $environment_change_Queue)
for packing it will take a perl hash structure as an argument and pack this accordingly, so we can send it.

Send Part
We can re-factor the send part so that it looks more like its receive counterpart.
To do this we will have to separate the pack-strings (that we now use in the subs) into some kind of object that holds a table of all pack-strings.

Maybe it is possible to merge all pack and unpack into 1 single table.
(reference: look at Receive/Sakexe_0.pm, it seems like there is no packet-switch (packet ID) being used on both send and receive part, or am i wrong about this?)

Other
btw, is stringToBytes etc. really needed when we use the perl pack function instead of just appending a string to a string? It would be nice if they were obsolete.
Look at this example please. By only using the pack function we don't only make the code shorter, it also adds the possibility to do what is explained in the paragraph: Send Part.
old:

Code: Select all

sub sendChatRoomBestow {
	my ($self, $name) = @_;

	my $binName = stringToBytes($name);
	$binName = substr($binName, 0, 24) if (length($binName) > 24);
	$binName .= chr(0) x (24 - length($binName));

	my $msg = pack("C*", 0xE0, 0x00, 0x00, 0x00, 0x00, 0x00) . $binName;
	$self->sendToServer($msg);
	debug "Sent Chat Room Bestow: $name\n", "sendPacket", 2;
}
new:

Code: Select all

sub sendChatRoomBestow {
	my ($self, $name) = @_;
	my $msg = pack('v x4 Z24', 0x00E0, stringToBytes($name));
	$self->sendToServer($msg);
	debug "Sent Chat Room Bestow: $name\n", "sendPacket", 2;
}
Can someone that uses such foreign languages test this?

Re: AI 2008 Network Subsystem.

Posted: 28 Aug 2009, 03:08
by kali
Poseidon can be used as a central server. For example, 10 bots can run off a single Poseidon server. Having it be special case of XKore 2 means that you either have to limit poseidon requests to one client per bot, or have XKore 2 implement a "viewer" model that is read-only.

Re: AI 2008 Network Subsystem.

Posted: 28 Aug 2009, 03:33
by Technology
Hmm i didn't know multiple bots could connect to a poseidon server. :o
Still, the RO client connects to the poseidon server right? This looks pretty much like xkore2/proxy to me.
But yea, after the client is connected, all poseidon does is keeping the client alive and sending the challenges to it.
Whereas with the xkore2/proxy server we pass on all packets (which we don't block or alter) to the RO client.

Re: AI 2008 Network Subsystem.

Posted: 28 Aug 2009, 06:25
by kali
It's a well known secret (I should now, my small proof of concept evolved to what is now known as Poseidon :D)
The advantage of the Poseidon is that you don't need an RO server to connect to. So for example, you have a P3 computer running Poseidon stand alone, while yur bot farm connects to that PC through the LAN. Or something that the bRO admin did - he had a central Poseidon server and everyone else botting in bRO connected to it (he ran it through donations).

Re: AI 2008 Network Subsystem.

Posted: 09 Sep 2009, 17:54
by Technology
Ok, nvm the Xkore / Posseidon for now.

Lets focus on the very basics and fundamentals of OpenKore first, the packets.

kali was talking about packets that are aware of how they are parsed and created.
Here's how we could realize this:
We should separate the pack strings from the subs (like the receive part) to make a lot of 'serverType hacks' and code that is being used atm obsolete.
Then we could merge the send and receive part which would make it very easy for kore to act as both server/client packet-wise.
Imagine how convenient it would be if it were possible to be able to both easily parse or even create any packet from data to send that to the client. (for xkore usage).
(note: Complex packets will have 2 (un)pack functions instead of 1 (un)pack string, until perl allows us to do what we need)

kLabMouse was talking about a base class for all serverTypes.
Here's how we could realize this:
We could create the base class out of the Sakexe_0.pm and ServerType0.pm (which are identical in a lot of places, if not totally) on both send and receive part.
There are 2 cases:
- If a old packet changes, we create a new child serverType that bases on the previous (for kRO/eA) or adjust the child serverType (iRO, mRO, ...).
- If a new packet gets added, we add it to the base serverType.
This way we can ensure that:
- we never break support for one serverType by supporting another.
- we support every eA version and don't end up with the amount of kRO ST's we have now. (new packets might as well be included in the base ST)

Re: AI 2008 Network Subsystem.

Posted: 10 Sep 2009, 02:08
by kLabMouse
Technology
I really like it.
But, one question: What will happen to packets that are seen only on 2-3 ST's, and all other use new one?

Re: AI 2008 Network Subsystem.

Posted: 10 Sep 2009, 09:22
by Technology
kLabMouse wrote:But, one question: What will happen to packets that are seen only on 2-3 ST's, and all other use new one?
You mean like packets that are new but replace the functionality of old packets on the send side right?
Look in: Network::Send::kRO::RagexeRE_2008_09_10a
ex. sendMapLogin -> 0x0436 -> in packet version 24
Here's what we are doing now and can continue on doing:
We update the other ST's with the new packet and leave the 2-3 ST's untouched.

Re: AI 2008 Network Subsystem.

Posted: 30 May 2011, 15:14
by EternalHarvest
Technology wrote:we can use the unpack strings from the receive part and use them to pack data into a packet that we send to the client.
However, we need perfect unpacking in order to be able to do perfect packing. (look at it this way, its easier to unpack a suitcase than to pack it)
Network modules can do that for a long time. Why it wasn't used at all, even for packets which don't require complicated packing?