EQExtractor2

**Derision** · #1 05-22-2010, 01:49 PM

I've been tooling around with Visual C# 2010 the past couple of days and one thing led to another and I wrote an EQ Packet decoder and extended it to dump out SQL statements for Doors and NPC Spawns.

I collected data for the revamped highholdpass and freeport west zones and thought I'd post the SQL for anyone interested:

<Link removed. This data is now incorporated into the PEQ database>

Note that this is just doors and static spawns, no grids or anything, and the mobs will spawn wherever they happened to be when I collected the data. It's all basic stuff, but I thought I'd throw it out there.

**Akkadius** · #2 05-22-2010, 03:32 PM

Although this seemed to work fine, in West Freeport I could really notice that some NPC's were cut off beyond a certain radius. NPC's from out by commonlands and the temple were non-existent, as well as some other NPC's in the outer perimeter.

Are you in the middle of developing this packet collector further hence just the SQL files?

Thanks for this Derision.

**Derision** · #3 05-22-2010, 03:55 PM

Quote:

Originally Posted by Akkadius

Although this seemed to work fine, in West Freeport I could really notice that some NPC's were cut off beyond a certain radius. NPC's from out by commonlands
and the temple were non-existent, as well as some other NPC's in the outer perimeter.

It would help a lot if you could give me a #loc where you are standing, so I can go on live and compare the mobs I collected with what are on live and see what are missing.

It maybe that Live only sends mobs in a certain radius of the player, so I need to run around the zone to get them all.

Quote:

Are you in the middle of developing this packet collector further hence just the SQL files?

This is my first C# program of any size, and certainly isn't anywhere near release quality (hard coded file names, very fragile etc.). I just thought it was cool to see spawns in those
two revamped zones which is why I posted the .SQL for them.

**Akkadius** · #4 05-22-2010, 04:21 PM

Quote:

Originally Posted by Derision

It would help a lot if you could give me a #loc where you are standing, so I can go on live and compare the mobs I collected with what are on live and see what are missing.

It maybe that Live only sends mobs in a certain radius of the player, so I need to run around the zone to get them all.

This is my first C# program of any size, and certainly isn't anywhere near release quality (hard coded file names, very fragile etc.). I just thought it was cool to see spawns in those
two revamped zones which is why I posted the .SQL for them.

Aye I was just wondering where you were on this because I could have alot of fun with it.

And yes, I am pretty positive for lag reasons there is a certain radius that you cache info wise. But from what it looked like in freeport is that you packet collected from the center. Basically every mob outside the same radius is missing. I would just run your query to check it out, I pulled it up on a MQ2 map read to see all the mob positioning. And yes, I only use MQ for map purposes.

**joligario** · #5 05-22-2010, 07:38 PM

Yeah, on live they implemented an anti-MQ2Map feature a while back where they now only send data on NPCs within a certain radius.

**trevius** · #6 05-22-2010, 08:49 PM

Well, that is only halfway true. It does help cut down the bandwidth usage and the usefulness of ShowEQ and MQ2 Maps. But, they do still send zone-wide spawns, they just don't send position updates after that point, and maybe not even new spawns, I am not sure. But, if you zone into Blightfire for example, you will get spawns for the entire zone on ShowEQ, so I am pretty sure it still sends all of them on the initial zone in.

**Derision** · #7 05-23-2010, 12:16 PM

I had a look at freeportwest with MQ2Map and also moved around the zone and I didn't see any obvious missing spawns.

As Trevius says, Live seems to send all the spawns when you zone in, then for mobs within a certain radius, you get rapid position updates for them (with deltas),
and for mobs outside that radius, you get less frequent updates.

I've extended my collector to create grids. Here is an updated .SQL for freeportwest with grids:

<Link removed. This data is now incorporated into the PEQ database>

You will see that for mobs inside the city, near to where I was, e.g the patrolling knights and guards, there are lots of grid entries, because the position updates come every couple
of seconds, whereas for the mobs outside the gates (snakes, bats and the like), there are considerably fewer updates.

Automatic grids created from packet collects are never going to be much good for that reason, although it does give the world builder a starting point I suppose.

Anyway, I guess the next step is for me to get the tool to the point where it is ready for release as an executable at least, so other people can get to play with it.
Probably have time to do that next weekend.

I should add that while in theory it should be possible to have it record a live EQ stream as you are playing, the way I have it setup is to use capture files collected with Wireshark.
So you go to char select, start Wireshark capturing, zone in with a character, come back 20 minutes later stop the capture, save the captured packets and run the extractor
on the .pcap file.

robinreg · #8 05-24-2010, 01:29 AM

This version of highpasshold would be good to replace the old highpass for the SoF/SoD that currently has problems with it. I tried replacing it with the new version in PEQ database. Only problem I am having is the zone point to zoning out of HighPassHold. I can zone in from Kithicor, Keep and East Karana but I can not get the zone out to work.

**Akkadius** · #9 05-28-2010, 10:55 PM

Derision, would you please please please packet collect the doors/objects for Oldhighpass? Could that be doable? Or are you getting closer to a working version of this for others?

**Derision** · #10 05-29-2010, 03:22 AM

Quote:

Originally Posted by Akkadius

Derision, would you please please please packet collect the doors/objects for Oldhighpass? Could that be doable? Or are you getting closer to a working version of this for others?

Is oldhighpass accessable on Live ? It seems like it should be part of Seeds Of Destruction, but I don't see it listed here: http://everquest.allakhazam.com/db/z...of+Destruction

Anyway, assuming it is a SoD zone and it has pre-req tasks for entrance, I don't have any way of getting in there.

I'll try and put a version of my tool out some time this weekend.

**Akkadius** · #11 05-29-2010, 04:03 AM

Quote:

Originally Posted by Derision

Is oldhighpass accessable on Live ? It seems like it should be part of Seeds Of Destruction, but I don't see it listed here: http://everquest.allakhazam.com/db/z...of+Destruction

Anyway, assuming it is a SoD zone and it has pre-req tasks for entrance, I don't have any way of getting in there.

I'll try and put a version of my tool out some time this weekend.

I had a feeling it wasn't either, I was about to check myself 8 /

**Derision** · #12 05-29-2010, 02:25 PM

I've uploaded the executable here: http://code.google.com/p/projecteqem...2.rar&can=2&q=

Prereqs: .NET Framework 4, a Live EQ account and Wireshark to capture the .pcap files that this program processes.

When capturing with Wireshark, ALWAYS start the capture with your character at char select, then zone-in once the capture has started.

The program will only process data for a single zone, so if you move to another zone, only the data for the first zone you entered after starting the capture will be processed.

This is still very much a work-in-progress, but feel free to report any bugs or suggestions here.

**cavedude** · #13 05-29-2010, 08:03 PM

Tried it out on moors, works perfectly! You can see the end results in the next PEQ database

**trevius** · #14 05-29-2010, 09:07 PM

Quote:

Originally Posted by cavedude

Tried it out on moors, works perfectly! You can see the end results in the next PEQ database

Haha, I tried it out on the same zone and yeah, it worked pretty awesomely. I have a few pcaps saved from random collects here and there and figure I might as well use those through this tool first, then will probably start collecting more zones later. Excellent work as always, Derision

I can't wait to play with this a bit more. It will definitely save me a ton of time on doors and ground spawns at the very least.

Some other info that might be good to pull would be zone points and maybe even newzone for zone settings, since most of the new zones are just copy/paste of a single example in the zone table. Now that the zone table can have multiple entries for the same zone, I think we could just add an id field as the primary key.

**Akkadius** · #15 05-30-2010, 12:42 AM

Quote:

Originally Posted by Derision

I've uploaded the executable here: http://code.google.com/p/projecteqem...2.rar&can=2&q=

Prereqs: .NET Framework 4, a Live EQ account and Wireshark to capture the .pcap files that this program processes.

When capturing with Wireshark, ALWAYS start the capture with your character at char select, then zone-in once the capture has started.

The program will only process data for a single zone, so if you move to another zone, only the data for the first zone you entered after starting the capture will be processed.

This is still very much a work-in-progress, but feel free to report any bugs or suggestions here.

There are 655 npc's in Dragonscale Hills. Simply amazing Derision. Wow.