Starting Work on SoF Opcodes/Structs - Page 4

**trevius** · #46 01-07-2009, 06:37 PM

@ greldor - Thanks. I wound up paying to get my old EQLive account active again. I am currently using ShowEQ to watch all opcodes from Client to Server on Live where I need to be looking. Right now, I am still working on just logging in. Once I can get past that point, I think it won't be too hard for me to break down each individual structure that has changed and get them updated as well as start filling in missing opcodes. I have switched directions for now and am going to see if I can get the emu working with EQLive. It should be much easier to get it working with Live, because I can see exactly what the client wants and is expecting from looking at the SEQ logs. Then, even if I don't know what a certain packet is, I should be able to just duplicate what I saw the Live server sending from the SEQ logs and trick the client to log all of the way in until I can fill in the missing information. There isn't really much other people can help with right now until I can get logged in all of the way. Once I do that, I will update the SVN so others can assist in filling in and updating opcodes and packet structures and making any other needed tweaks to finalize it. I am not planning to try to chase Live by making the emu work with it, I am only trying to get in on an EQLive version because it should be easier than SoF. Once I get in game with the EQLive version, I will try to work backwards and get SoF working with it. I think that will be much easier than trying to work from Titanium to SoF, since Live seems much closer to SoF than Titanium is.

@ unknownhost - No, no relation to the name. My avatar name is named after my nephew Trever, but with a fantasy twist to it :P I am definitely no machine, I am just persistent enough to keep going even when I am failing miserably. There are certainly a few people on this project who have much more experience than me and probably would have had this working by now already. Unfortunately, none of them have SoF, and more importantly, I don't think they have the time needed to do it. I am more than willing to do all of the grunt work in getting this stuff to work, but I admit it would be nice if one of the more experienced people were able to help figure out some of the more problematic issues (like currently being stuck at the first main loop). The big time involvement here is getting each of the needed opcodes and verifying they are accurate, and also getting each of the packet structures and verifying they are as accurate as possible. To the point I am currently failing at, I think I have all of the required opcodes and most of the packet structures should be good enough to be working for it. I haven't worked much past that point yet, because there isn't a point to until I get to the point where I need them. I didn't know anything about coding 8 months ago and have only learned by examples from reading the emulator source code and working on it. I have learned alot and learn new things every day, but I am sure the more experienced people would probably be able to help alot if they had the time. I will keep working away at it and will get it eventually, hopefully. All of the time I am working on this has really forced me to learn things I wouldn't have otherwise, so that is actually a good thing, but VERY time involved. It has officially been more than a month that I have been working on this so far, and I have worked on it for hours every single night. I made alot of progress in the first week or so, but since I got stuck on this main loop, I haven't been able to get past it. At this point, I still don't know what is causing it to hang up at that point. It could be anything from a wrong opcode, packet structure, something else missing, or whatever. Without logs directly from a packet collect (preferrably from SEQ) from when the SoF eqgame.exe was being used (patched Sep 7th 2007), it is hard to know exactly what might be missing. That is why I am hoping getting it working for EQLive will help me work back to getting SoF working.

@ Derision - Thanks for the info

So, any opcode that isn't added to the patch_ops.h will use the eq_packet_structs.h for their structure? I have tried adding a few new opcodes to the anniversary_ops.h and it requires me to put structs into the eq_packet_structs.h for it to compile. If it is supposed to be using the anniversary_structs.h, then why does it require to have a struct in eq_packet_structs.h as well?

The main thing I am trying to figure out is how to know which packets are encoded/decoded. Is it only the ones that are put in the patch_ops.h and set to be encoded/decoded, or are all packets automatically encoded/decoded by default? The only raw packets I see from live are the acks and some of the sessions and network status stuff, everything else is encoded and some is also combined.

I am also trying to understand why anniversary.cpp seems to be telling the server how to handle the opcode packets, but we also have client_packet.cpp, which has handling instructions for each opcode as well. It seems that anything I add to anniversary_ops.h and anniversary.cpp also has to be added to client_packet.cpp. I am not really too clear on what is going on with that just yet. It seems like there is redundancy in some of this stuff and it makes it more complex for me, heh.

I verified most of the opcodes I need to get logged into the emu with an EQLive client last night. I just have to fill in a couple more and then adjust a couple of structures slightly and I should be able to at least get to the same point I am at trying to get in with SoF. I was able to get almost all of the player profile for EQLive mapped out just by looking at the SEQ hex output. I still have a bit more to do to finish that struct 100%, but it should be pretty accurate when I am done with it, at least for Live. But, Live and SoF have very similar structs. They are much closer than Titanium and SoF are.

**Derision** · #47 01-07-2009, 07:09 PM

Quote:

Originally Posted by trevius

@ Derision - Thanks for the info

So, any opcode that isn't added to the patch_ops.h will use the eq_packet_structs.h for their structure?

First thing to say is that what ShowEQ shows as a 'Decoded' packet has nothing to do with the with the EQEmu ENCODE/DECODE terminology.

You are correct, anything that isn't in <patch>_ops.h just gets passed straight through using the common/eq_packet_structs.h structure for that Opcode.

Quote:

I have tried adding a few new opcodes to the anniversary_ops.h and it requires me to put structs into the eq_packet_structs.h for it to compile. If it is supposed to be using the anniversary_structs.h, then why does it require to have a struct in eq_packet_structs.h as well?

By putting an Opcode in anniversary_ops.h, you are saying that this Opcode requires the struct to be translated from the version in common/eq_packet_structs.h to the version in Anniversary_structs.h, hence why you need a copy of the struct in both files. If the Opcode is a new Anniversary-Only opcode, you don't need to put it in Anniversary_ops.h, and if there is a struct associated with it, you can just put it in common/eq_packet_structs.h.

The only time you need to put something in Anniversary_ops.h is if the same opcode exists in 6.2/Titanium and/or Anniversary, but the structs are different. In that case you need to put the SoF struct in Anniversary_structs.h and code in Anniversary.cpp to encode the struct from the version in common/eq_packet_structs.h to the version in patches/Anniversary_structs.h to account for a different order of fields in the structure, or new fields, etc.

Quote:

The main thing I am trying to figure out is how to know which packets are encoded/decoded. Is it only the ones that are put in the patch_ops.h and set to be encoded/decoded, or are all packets automatically encoded/decoded by default?

The only packets that are encoded/decoded for a particular client version are those listed in patches/<patch>_ops.h . All other packets are passed along using the struct in common/eq_packet_structs.h. As I said, encoding/decoding is only required if the packet struct for a client version (SoF in this case) is different than that specified in common/eq_packet_structs.h

Quote:

The only raw packets I see from live are the acks and some of the sessions and network status stuff, everything else is encoded and some is also combined.

As above, when SEQ says it has 'decoded' a packet, this just means it recoginised the EQ Application Opcode and doesn't correspond to the encode/decode terminology used in EQEmu.

Quote:

I am also trying to understand why anniversary.cpp seems to be telling the server how to handle the opcode packets, but we also have client_packet.cpp, which has handling instructions for each opcode as well.

Anniversary.cpp isn't telling the server how to handle the packet. It is telling the server how to translate the struct associated with that Opcode from the version that particular version of the client is sending into the structure that is in common/eq_packet_structs.h. Once the struct the client has sent has been translated into the 'emu struct', the Opcode is then passed through to client_packet.cpp to handle.

**trevius** · #48 01-07-2009, 07:20 PM

Thank you VERY much, Derision! Understanding that better should help me quite a bit in getting more work done on this. It sounds like I will just have to look at the current examples of how the conversions from struct to struct depending on the patch version are being done. That sounds like it might be easier to do than I originally thought it was. I will check the files now that I have your explanation and hopefully I can make more sense of what is going on so I can get things adjusted properly where needed. When I got into working on this, I was under the assumption that it would just be finding opcodes for it and redoing structs where needed, but it seems there is a bit more to doing that than I originally thought.

All I know is that when I finally get past the point I have been stuck at for the past few weeks, I am probably going to yell loud enough to wake up my wife and daughter :P

**trevius** · #49 01-07-2009, 07:38 PM

After just looking at it quickly to see what is happening, it gave me a couple more questions;

It looks like unknowns are commented out in the Titanium.cpp ENCODEs, so I am wondering if the unknowns are somehow handled/converted automatically, or if they aren't needed at all. Doesn't it still need to know where to send unknowns out at to fill in the gaps?

The other question is about (OUT) vs eq-> usage. Some of the ENCODEs have (OUT) and then the struct field name. But, some of them have something like this "eq->deity = emu->deity;" instead. My guess is that they are both doing the same thing. If I am right, then "eq" is saying to get the eq_packet_structs.h structure and "emu" is saying to put that item from the structure into this position for this patch version. If I understand correctly, that is the same thing that (OUT) is doing. So maybe the eq = emu way is just the old way?

I have seen some warnings about eq and emu not being used when I am compiling. I am guessing that is because I was making blank structs, which also means blank encodes/decodes. And since they are blank, the eq and emu variables aren't being used, so it warns about it. Now that I think I understand it better, I don't think I should need those blank structs now to begin with. If I want to send a 0 size packet, I guess I can just make the new op and then have it get sent when needed directly from the client_packet.cpp. I don't think I would have to add anything anywhere else for them. I will mess with that more later tonight.

**KLS** · #50 01-07-2009, 08:29 PM

OUT(x) is the same thing as: eq->x = emu->x;

When you do your var allocation at the top it allocates both emu and eq; emu being the source struct from the emulator, eq being the dest struct that's going out on the wire.

That's for outgoing packets, as you imagine it's reversed for incoming. When you're decoding an inc packet IN(x) is the same thing as: emu->x = eq->x; and eq is the source from the wire and emu is the dest to the emulator.

greldor · #51 01-08-2009, 10:38 AM

As a side note, If anyone needs SoF to aid Trevius:

Newegg.com has Secrets of Faydwer for $19.99
with free shipping!

Greldor
The Grand Creation

**trevius** · #52 01-09-2009, 06:32 AM

Well, it looks like EQLive may no longer be an option at all for the emulator. Unless I am doing something wrong, I can't even get EQLive to connect to the eqemulator.net login server. I tried messing around with my host file (windows host file), but that didn't make any difference. I also tried adjusting my ini files in the EQ directory, but that didn't help either. I put in my username and password and hit connect, and it just hangs there indefinitely saying "Logging into the server. Please wait..." It never actually connects to the eqemu login server, and it never seems to time out either, it just stops doing anything else.

My guess is that they have some new opcode or structure or maybe encryption when logging into the LS, and it doesn't match what our LS uses. And, since we don't have access to the LS, it means we are basically out of luck other than developing a new login server (which has been discussed a little).

It isn't the worst thing in the world that the emu won't work with live anymore, but it does probably slow my progress down. Really, as long as I can get in game with SoF, that should be most of what I need to start working out the rest of the details. I mostly just need to get past the point I have been stuck at. The fact that EQEmu may not be able to work with EQLive is another reason why SoF is probably our best bet for a final version upgrade.

**KLS** · #53 01-09-2009, 08:21 AM

I connected to EMU login just 1 minute ago with a fully updated client so it must be: I am doing something wrong =p

EvoZak · #54 01-09-2009, 02:37 PM

Wow, how are you getting anything done with a fully patched client? Tomorrow's structs could be different from today's. Craziness.

secash · #55 01-10-2009, 03:54 AM

He probably means he was able to hit the login server, not actually functionally connect.

**trevius** · #56 01-10-2009, 05:03 AM

No, I wasn't even able to hit the Login Server. Thanks for confirming that it works, KLS! Now to figure out what I am doing wrong lol. I should know how to do this by now, LMAO!

Evozak, as long as I don't patch past the point my client is at now, I don't have to worry about the structs changing. I am not trying to get the emu to work with live and have us chase live as we did in the past at one time. I am just trying to get in game with Live for now so I can hopefully use some of that info to figure out what is stopping me from getting in with SoF. Since I have the exact breakdown of the structures (or at least I can see the actual packets from live with SEQ), I should be able to get everything I need set correctly to connect. I am hoping that if I can get in with a Live client, that I can use that info to help me figure out what is stopping me with SoF. If I have to, I should be able to force the exact packet structures from Live just to trick the client to logging all of the way in. Then, hopefully I can figure out the details later.

**trevius** · #57 01-11-2009, 05:48 AM

Well, I can't figure out why my live client won't connect to the emu LS... If I put the wrong info in the host file, it times out and says to check my network connection like normal. If I put the correct eqemu stuff in the host file and log in, it just hangs trying to connect. It is like it connects, but doesn't know what to do next. The only thing I can think of is that it is something to do with one of my INI files or maybe because I only own up to SoF. I tried messing around with my INI files as much as I could think of, but it didn't make any difference at all. So, maybe since I am using SoF, it has to do some extra check before going to the login server. I would guess it has to do the check to see if I want to purchase SoD or something, and since it is connecting to the emu LS, it fails that check and just hangs.

KLS, does your EQLive client have all expansions up to SoD? If so, maybe that is why you go right through, and I get stopped when it is trying to do that final check. I can't really think of what else it might be. All of my other client versions work fine to get to the LS, including my fresh SoF install from the DVDs. It was only after I patched to live that it stopped letting me connect to the LS.

Secrets · #58 01-11-2009, 08:11 AM

Quote:

Originally Posted by trevius

Well, I can't figure out why my live client won't connect to the emu LS... If I put the wrong info in the host file, it times out and says to check my network connection like normal. If I put the correct eqemu stuff in the host file and log in, it just hangs trying to connect. It is like it connects, but doesn't know what to do next. The only thing I can think of is that it is something to do with one of my INI files or maybe because I only own up to SoF. I tried messing around with my INI files as much as I could think of, but it didn't make any difference at all. So, maybe since I am using SoF, it has to do some extra check before going to the login server. I would guess it has to do the check to see if I want to purchase SoD or something, and since it is connecting to the emu LS, it fails that check and just hangs.

KLS, does your EQLive client have all expansions up to SoD? If so, maybe that is why you go right through, and I get stopped when it is trying to do that final check. I can't really think of what else it might be. All of my other client versions work fine to get to the LS, including my fresh SoF install from the DVDs. It was only after I patched to live that it stopped letting me connect to the LS.

I just tried with a live client and could confirm the same symptoms as you. Bad info, times out and gives an error. EQLive, connects to server select. EQEmu? Hangs and never gives an error. Extremely odd.

**trevius** · #59 01-11-2009, 10:38 AM

Maybe KLS is running on a trial account client? I thought about using a trial account before I decided to pay to renew my old Live account. Unfortunately, the trial accounts are not the same as a fully updated live client. They have some limited files, and seem to only patch to a certain bare minimum point to be able to handle just the Tutorial zone. I never tested the trial version to see if it could connect to the emu LS, but I imagine it can. Even the UI on the trial version is a really old one (looks like Titanium UI). I tried using SEQ on the trial version and it didn't work at all. So, the trial definitely uses something other than EQLive. That is why I had to just pay to renew my old one instead.

Thanks for confirming that, Secrets. At least I know I am not crazy now lol. I have been wasting too many hours on trying to figure out why EQLive won't connect to the emu LS. Can anyone else confirm if it works for them or not? I would love to get it working to help me progress, but if it doesn't work, there is no use for me to waste anymore time on it!

klamath · #60 01-11-2009, 12:31 PM

If you just want live packets dumped out into your eqemu server its simple to connect. EQmain.dll is everquest loginserver. Swap the live one with your titanium file and your live patch will allow you to login to eqemu login server. Dunno how far you will get from there tho.