EQExtractor2

[Akkadius]

Originally Posted by robinreg

I got all the DoD and PoR zones. the one I got for devastation is the Bolvark spawns. I labeled it on the PEQ log site. Also got most of the revamps zones. I'm currently working on GoD instance zones.

I'm grabbing quite a few of the PoR instance zones.

[Andrew80k]

Anyone doing TBS?

[cavedude]

I did the first couple of zones, but have since started working on other zones to get some zone_points and npc appearances working.

[Akkadius]

Originally Posted by Andrew80k

Anyone doing TBS?

I have been doing TBS, but most of the zones are instanced, I have most of the statics though.

I've got almost PoR (is done I think), TBS and DoN statics completely done. I don't post them all until they are done otherwise if I upload them gradually I have no idea where I am with the way you have to upload to CD's repo.

[Derision]

1.5 is up with these changes:

Fixed bug which caused the first spawn to always be skipped.
Update generated for fog_density column in zone table.
Populated Drakkin specific fields in npc_types.
If zone config and/or zone points are selected, the generated SQL will no longer be commented out.

I implemented the update for npc_types, but I have disabled it for now in the UI as I found some issues with it ...

First I was updating the size, then I found multiple NPCs with the same name but different sizes, so I took the size out of the update. Then I found multiple NPCs with the same name,
but some were wielding weapons and some not, e.g.

mysql> select id, name, d_meele_texture1, d_meele_texture2 from npc_types where name like 'a_Nokk_elder';
+--------+--------------+------------------+------------------+
| id     | name         | d_meele_texture1 | d_meele_texture2 |
+--------+--------------+------------------+------------------+
| 394319 | a_Nokk_elder |                0 |                0 |
| 394320 | a_Nokk_elder |            10608 |                0 |
| 394363 | a_Nokk_elder |                0 |                0 |
+--------+--------------+------------------+------------------+
3 rows in set (0.25 sec)

(this is with current live data, not existing PEQ tables).

Now my guess is there are multiple mobs with the same name but different facial features, but I didn't check that, so I'm really not sure what to do and await Cavedude's words
of wisdom on the matter

[cavedude]

The melee textures, could that possibly be an item the NPC has on their loottable? If so, I don't think it would be a big deal to 0 out those columns after they have been sourced into the db. I'm mainly looking for textures on guards and player races as everything else is usually handled by loot. It's no big deal if I have to go back and manually fiddle with the melee columns.

About the size and face data... That's tricky. How about a way to check which columns you want to dump in the UPDATE? Maybe have the default the "safe" options, but also have choices for things like level or size which may not be as safe. Not sure how much work that would be, but I think that would make everybody happy.

[steve]

What version of The Void did you use it in? It worked fine for me in Void A, B, & G.

[blackdragonsdg]

Originally Posted by steve

What version of The Void did you use it in? It worked fine for me in Void A, B, & G.

It is the one when you zone in from PoTime...no idea which version it is.

[steve]

Originally Posted by blackdragonsdg

It is the one when you zone in from PoTime...no idea which version it is.

If you do a /who on yourself, it will say in the zonename. Depending on how far along you are with SoD progression determines what version of the zone you get sent to. Void A is the default version with no progression completed.

[steve]

Are you guys camping out to character select after getting to each zone? Just wanted to make sure I wasn't doing something unneeded. And will multiboxing confuse the capture?

[Derision]

Originally Posted by steve

Are you guys camping out to character select after getting to each zone? Just wanted to make sure I wasn't doing something unneeded. And will multiboxing confuse the capture?

Multiboxing can confuse the extractor and appears to be the cause of blackdragonsdg's crash.

What happens is the extractor reads each packet in the .pcap file looking for an SOE 'Session Request' packet (that the client send to the server).
When it sees a session request, it 'tentatively locks on' to that session and waits until it sees a particular packet (which was OP_NewZone and is
the packet in which the server sends various details about the zone) before deciding this is a valid EQ connection and permanently locking onto
this stream and ignoring all others.

What can happen, is if you zone in a second character before the server has sent the OP_NewZone to the first one, the extractor get's confused
by the crossed streams (oblig. Ghostbusters reference).

I've altered the extractor locally to 'permalock' to a stream when it sees an OP_ZoneEntry packet from the client to the server. This is the second
packet sent and so it is much less likely that multi-boxing will cause an issue, although of course, only the packets to/from the first client to
zone-in will be processed, but that shouldn't be an issue.

I'll push this version out later today when I've looked at another issue.

EDIT: Oh, and you don't need to camp out when you have finished. Just stop the capture and save it.

[steve]

Originally Posted by Derision

Oh, and you don't need to camp out when you have finished. Just stop the capture and save it.

Gotcha. But if I want to start a new one, simply zoning isn't enough, correct? I need to start from the character select each and every time for a new capture?

[Derision]

I've not tested it (I will later), but in theory, if you start a new capture while you are already in a zone, the next time you zone, the extractor should recognise the new zone and generate the SQL for that one.

[trevius]

Another option may be to start the capture before you zone, and as you zone, watch the capture and find a packet you are sure is the new zone communicating with your client and then right click that packet and select Conversation Filter > UDP. That will cause it to filter out everything accept for that zone's communication to you. You can normally tell fairly easily which packets are the zone communicating to you when you zone by watching for a ton of packets coming in at the same time from the same IP to you. Also, it is good to note that if you scroll up a bit while collecting, it will remain at that part of the collect, so you can select your packet to filter easier. You can scroll back to the bottom after if you wish to continue watching them come in real time again.

Then, when you stop the capture and save it, be SURE that you select the option for "Displayed" in the "Packet Range" section of the Save As window. That will cause it to save only the packets you filtered to see. Otherwise, if you don't save it with that option, it still saves the unfiltered collect.

By doing this, it should mean you can start a collect, then zone, then collect and save it. Then start another collect and zone again and so on.

I haven't really tested this yet, but I am pretty sure it would work fine.

[Derision]

Originally Posted by Derision

I've not tested it (I will later), but in theory, if you start a new capture while you are already in a zone, the next time you zone, the extractor should
recognise the new zone and generate the SQL for that one.

OK, now I've tested it and it doesn't work so well on the current version, so I don't recommend starting a capture unless you are at character select,
unless, as Trevius suggests, you are comfortable with using Wireshark filters to just save the packets relating to the zone you are interested in.

As it stands now, starting a capture while you are already in a zone and then zoning will either cause the extractor to miss data when entering the new zone
(including crucial packets which make the collect unusable), or if a mob happens to spawn in the zone you are in, after you start the capture, but before you
zone, then that mob in the previous zone will be included in the data for the zone you are going to.