Openkore.com

OpenKore Forums
It is currently 18 Nov 2018, 02:24

All times are UTC - 5 hours [ DST ]





Post new topic Reply to topic  [ 24 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 26 Jun 2009, 01:36 
Offline
Administrator
Administrator
User avatar

Joined: 24 Apr 2008, 12:02
Posts: 1301
Nice!

_________________
Join our Team. Click here.
Image


Image


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 05 Jul 2009, 12:36 
Offline
Super Moderators
Super Moderators
User avatar

Joined: 06 May 2008, 12:47
Posts: 801
Some stuff that we can integrate in the parsers:
- missing received packets in kore can be logged
- missing sent packets in kore can be logged using xkore (kali)

The user can sent in that log so the devs can add support.

_________________
One ST0 to rule them all? One PE viewer to find them!
One ST_kRO to bring them all and in the darkness bind them...

Mount Doom awaits us, fellowship of OpenKore!


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 29 Jul 2009, 16:02 
Offline
Super Moderators
Super Moderators
User avatar

Joined: 06 May 2008, 12:47
Posts: 801
Here is an example of the complex packet parsing we need.
In this example we parse an auction packet and a hotkey packet.

Required modules:
- Convert::Binary::C
- Data::Dumper (just to show how the perl datatype will look)

Perl script code:
Code:
#!/usr/bin/env perl
package main;
use strict;
use Convert::Binary::C;
use Data::Dumper;

# Z <-> char
# C <-> unsigned char
# v <-> unsigned short
# V <-> unsigned long

#---------------------------------------------
# Create a new object and parse embedded code
#---------------------------------------------
my $c = Convert::Binary::C->new->parse(<<ENDC);

struct test1 {
   unsigned char test1, test2;
   unsigned short size;
   unsigned long pages, amount;
   struct auction {
      unsigned long ID;
      char seller[24];
      unsigned short item, type, unknown, amount;
      unsigned char identify, attribute, refine;
      unsigned short card[4];
      unsigned long price, buynow;
      char buyer[24];
      unsigned long timestamp;
   } auction[amount];
};

struct test2 {
   unsigned char test1, test2;
   struct hotkey {
      unsigned char type;
      unsigned long ID;
      unsigned short lvl;
   } hotkey[];
};

ENDC

# configuring the Convert::Binary::C object to unpack with little endian byteorder
$c->configure(ByteOrder => 'LittleEndian');

my $packed1 = pack('C*',
0x52, 0x02, 0x05, 0x01, 0x01, 0x00, 0x00, 0x00,
0x03, 0x00, 0x00, 0x00, 0x12, 0x00, 0x00, 0x00,
0x48, 0x79, 0x70, 0x65, 0x72, 0x69, 0x6f, 0x6e,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0xb1, 0x04, 0x04, 0x00, 0x00, 0x00, 0x01, 0x00,
0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x39, 0x05, 0x00, 0x00, 0x3a,
0x05, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0xa7, 0x8f, 0x6d, 0x4a, 0x13,
0x00, 0x00, 0x00, 0x48, 0x79, 0x70, 0x65, 0x72,
0x69, 0x6f, 0x6e, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0xfd, 0x08, 0x05, 0x00, 0x00,
0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xd2, 0x04,
0x00, 0x00, 0x2e, 0x16, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xc8, 0x8f,
0x6d, 0x4a, 0x14, 0x00, 0x00, 0x00, 0x48, 0x79,
0x70, 0x65, 0x72, 0x69, 0x6f, 0x6e, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xb1, 0x04,
0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0xd2, 0x04, 0x00, 0x00, 0xd7, 0x11, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0xe8, 0x8f, 0x6d, 0x4a);

my $packed2 = pack('H*', 'b902011c0000000a0000590200000000011a000000000001490000000a00014d0000000000014b000000000001220000000a00011d0000000900014a0000000000010f200000000001112000000000012a20000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000018e0000000000');

# configuring the Convert::Binary::C object to unpack char name[len] as string instead of array
$c->tag('test1.auction[0].seller', Format => 'String');
$c->tag('test1.auction[0].buyer', Format => 'String');

# unpacking from bytestream to perl hash using C struct
my $unpacked1 = $c->unpack('test1', $packed1);
print Dumper($unpacked1);

my $unpacked2 = $c->unpack('test2', $packed2);
print Dumper($unpacked2);

# re-packing from perl hash to bytestream using C struct
my $packed3 = $c->pack('test1', $unpacked1);
print unpack('H*', $packed3) . "\n\n";

my $packed4 = $c->pack('test2', $unpacked2);
print unpack('H*', $packed4) . "\n\n";


Console output:
Code:
$VAR1 = {
          'auction' => [
                         {
                           'seller' => 'Hyperion',
                           'ID' => 18,
                           'item' => 1201,
                           'buynow' => 1338,
                           'attribute' => 0,
                           'card' => [
                                       0,
                                       0,
                                       0,
                                       0
                                     ],
                           'amount' => 1,
                           'timestamp' => 1248694183,
                           'identify' => 1,
                           'unknown' => 0,
                           'refine' => 0,
                           'buyer' => '',
                           'price' => 1337,
                           'type' => 4
                         },
                         {
                           'seller' => 'Hyperion',
                           'ID' => 19,
                           'item' => 2301,
                           'buynow' => 5678,
                           'attribute' => 0,
                           'card' => [
                                       0,
                                       0,
                                       0,
                                       0
                                     ],
                           'amount' => 1,
                           'timestamp' => 1248694216,
                           'identify' => 1,
                           'unknown' => 0,
                           'refine' => 0,
                           'buyer' => '',
                           'price' => 1234,
                           'type' => 5
                         },
                         {
                           'seller' => 'Hyperion',
                           'ID' => 20,
                           'item' => 1201,
                           'buynow' => 4567,
                           'attribute' => 0,
                           'card' => [
                                       0,
                                       0,
                                       0,
                                       0
                                     ],
                           'amount' => 1,
                           'timestamp' => 1248694248,
                           'identify' => 1,
                           'unknown' => 0,
                           'refine' => 0,
                           'buyer' => '',
                           'price' => 1234,
                           'type' => 4
                         }
                       ],
          'amount' => 3,
          'test1' => 82,
          'pages' => 1,
          'size' => 261,
          'test2' => 2
        };
$VAR1 = {
          'hotkey' => [
                        {
                          'ID' => 28,
                          'lvl' => 10,
                          'type' => 1
                        },
                        {
                          'ID' => 601,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 26,
                          'lvl' => 0,
                          'type' => 1
                        },
                        {
                          'ID' => 73,
                          'lvl' => 10,
                          'type' => 1
                        },
                        {
                          'ID' => 77,
                          'lvl' => 0,
                          'type' => 1
                        },
                        {
                          'ID' => 75,
                          'lvl' => 0,
                          'type' => 1
                        },
                        {
                          'ID' => 34,
                          'lvl' => 10,
                          'type' => 1
                        },
                        {
                          'ID' => 29,
                          'lvl' => 9,
                          'type' => 1
                        },
                        {
                          'ID' => 74,
                          'lvl' => 0,
                          'type' => 1
                        },
                        {
                          'ID' => 8207,
                          'lvl' => 0,
                          'type' => 1
                        },
                        {
                          'ID' => 8209,
                          'lvl' => 0,
                          'type' => 1
                        },
                        {
                          'ID' => 8234,
                          'lvl' => 0,
                          'type' => 1
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 0,
                          'lvl' => 0,
                          'type' => 0
                        },
                        {
                          'ID' => 142,
                          'lvl' => 0,
                          'type' => 1
                        }
                      ],
          'test1' => 185,
          'test2' => 2
        };
520205010100000003000000120000004879706572696f6e00000000000000000000000000000000
b1040400000001000100000000000000000000390500003a05000000000000000000000000000000
0000000000000000000000a78f6d4a130000004879706572696f6e00000000000000000000000000
000000fd080500000001000100000000000000000000d20400002e16000000000000000000000000
0000000000000000000000000000c88f6d4a140000004879706572696f6e00000000000000000000
000000000000b1040400000001000100000000000000000000d2040000d711000000000000000000
0000000000000000000000000000000000e88f6d4a

b902011c0000000a0000590200000000011a000000000001490000000a00014d0000000000014b00
0000000001220000000a00011d0000000900014a0000000000010f20000000000111200000000001
2a200000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000018e0000000000

_________________
One ST0 to rule them all? One PE viewer to find them!
One ST_kRO to bring them all and in the darkness bind them...

Mount Doom awaits us, fellowship of OpenKore!


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 29 Jul 2009, 17:01 
Offline
Administrator
Administrator
User avatar

Joined: 24 Apr 2008, 12:02
Posts: 1301
Technology
Nice!!!
Need just one addition, the eval for array elements number.

_________________
Join our Team. Click here.
Image


Image


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 29 Jul 2009, 17:43 
Offline
Super Moderators
Super Moderators
User avatar

Joined: 06 May 2008, 12:47
Posts: 801
kLabMouse wrote:
Technology
Nice!!!
Need just one addition, the eval for array elements number.

Can you explain what you mean exactly or by the hand of an example?

_________________
One ST0 to rule them all? One PE viewer to find them!
One ST_kRO to bring them all and in the darkness bind them...

Mount Doom awaits us, fellowship of OpenKore!


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 30 Jul 2009, 03:44 
Offline
Administrator
Administrator
User avatar

Joined: 24 Apr 2008, 12:02
Posts: 1301
Some packet's can have a structure:
Code:
short param_1
short param_2
struct {
  something
} array[(packet_len-2-4-2)/struct_size]
short param_3


You see the "array" ??? Well, number of elements is calculated from packet len, and there is additional param_3 at the end of packet.
So it need to be parsed good.

_________________
Join our Team. Click here.
Image


Image


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 15 Dec 2009, 20:18 
Offline
Super Moderators
Super Moderators
User avatar

Joined: 06 May 2008, 12:47
Posts: 801
I've taken a look at the structs and all inner structs/datatypes that have a [...] seem to have it at the end of the packet.

example:
Code:
// packet 0x155
struct PACKET_CZ_REQ_CHANGE_MEMBERPOS {
  /* this+0x0 */ short PacketType
  /* this+0x2 */ short PacketLength
  struct MEMBER_POSITION_INFO memberInfo[...] {
    /* this+0x0 */ int AID;
    /* this+0x4 */ int GID;
    /* this+0x8 */ int positionID;
  }
}


or also:
Code:
// packet 0x155
struct PACKET_CZ_REQ_CHANGE_MEMBERPOS {
   short PacketType;
   short PacketLength;
   struct MEMBER_POSITION_INFO {
      int AID;
      int GID;
      int positionID;
   } memberInfo[];
};



btw i think its possible to use this if you need to calculate lengths of struct arrays:
Quote:
The final and most powerful way to define a Dimension tag is to pass it a subroutine reference. The referenced subroutine can execute whatever code is neccessary to determine the size of the tagged array:
Code:
sub get_size
  {
    my $m = shift;
    return $m->{hdr}{len}[0] / $m->{hdr}{len}[1];
  }
 
  $c->tag('more_complex.data', Dimension => \&get_size);
 
  $u = $c->unpack('more_complex', $data);

As you can guess from the above code, the subroutine is being passed a reference to hash that stores the already unpacked part of the compound embedding the tagged array. This is the result:
Code:
  $u = {
    'hdr' => {
      'len' => [
        42,
        7
      ]
    },
    'data' => [
      1,
      2,
      3,
      4,
      5,
      6
    ]
  };


to do a bytesToString, we can use something like this:
Code:
$c->tag('ProtoId', Hooks => { pack   => \&ProtoId_pack,
                                unpack => \&ProtoId_unpack });

_________________
One ST0 to rule them all? One PE viewer to find them!
One ST_kRO to bring them all and in the darkness bind them...

Mount Doom awaits us, fellowship of OpenKore!


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 19 Jan 2010, 07:33 
Offline
Administrator
Administrator
User avatar

Joined: 24 Apr 2008, 12:02
Posts: 1301
Nice. Should We implement it? or only in 3.0 ?

_________________
Join our Team. Click here.
Image


Image


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 19 Jan 2010, 09:24 
Offline
OpenKore Monk
OpenKore Monk

Joined: 04 Apr 2008, 10:10
Posts: 474
Yeah looks really nice.

I guess once we release the next stable version, we can start doing this? It doesn't have to be 3.0 could be 2.1.1 or 2.2 or something.

_________________
Got your topic trashed by a mod?

Trashing topics is one click, and moving a topic to its proper forum is a lot harder. You expend the least effort in deciding where to post, mods expend the least effort by trashing.

Have a nice day.


Top
 Profile  
 
 Post subject: Re: parsing of variable lenght packets with sub elements
PostPosted: 19 Jan 2010, 09:44 
Offline
Administrator
Administrator
User avatar

Joined: 24 Apr 2008, 12:02
Posts: 1301
kali wrote:
It doesn't have to be 3.0 could be 2.1.1 or 2.2 or something.

Yep. But it's a big Update. so better 2.2 or even 3.0

_________________
Join our Team. Click here.
Image


Image


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 24 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group