Starting Work on SoF Opcodes/Structs

[KLS]

Given that the patch files are pretty much self contained I don't think it really needs a new branch. At least not unless you need to modify core parts of the server to make it all work, in which case yes I'd agree with a SoF branch.

[trevius]

Any modifications of the core files that are needed shouldn't affect the other clients. So far, the only core file changes I have made are to add a new opcode. The only other change I currently have planned for core files is to add a second way to handle the sendexpzonein opcode. Since Titanium sends the same op and expects it right back, the current files work fine for Titanium. They also work for SoF, but the order in the EQ Debugs is a bit off, which might be a concern. So, I plan to copy the function in client_packet.cpp for sendexpzonein and make a different version for SoF. That won't effect how the other clients work, only how it handles SoF logins. Basically, instead of sending the sendexpzonein and getting it back from the client, SoF will send the send the sendexpzonein and send back the new worldobjectssent opcode and handle it by sending that same opcode back to the client again.

Sorry if that sounds a little confusing, but I have it tested on my test box and it works fine and doesn't change anything about how the other clients currently work. At this time, I don't think another branch is needed really. I will just leave it set so you have to uncomment a #define to enable using/testing SoF. So, by default, servers won't be able to use it unless they make that change. This is really only because of the possible crash issues with the SoF client that should probably be avoided for now on non-test servers.

As KLS said, since the patch files are handled in their own separate way, the SVN updates I make for SoF should be ok.

Right now, I am still working on and testing some structure changes to see if I can get more stuff working. I think figuring out the clientupdate structure is an important one to focus on for now. It is a pretty small struct (only 36 in size), but it uses some split signed/unsigned blocks that I don't really understand yet.

Here is the structure of it that I am currently working on:

struct PlayerPositionUpdateClient_Struct_WIP //from Client Packet Breakdown and testing
{
/*0000*/ uint16 spawn_id;			// Player's spawn id - Verified!
/*0002*/ uint16	sequence;			//increments one each packet - Verified!
/*0004*/ float delta_heading;		// Change in heading - Looks Accurate
/*0008*/ float camera_angle;		//unused camera incline angle
/*0012*/ float x_pos;				// x coord - Verified!
/*0016*/ float heading;				//Heading - Looks Accurate
/*0020*/ signed z_pos:10;			//Not Positive
		 signed y_pos:10;			//Not Positive
		 unsigned padding1:12;
/*0024*/ float delta_z;				// Change in z - Looks Accurate
/*0028*/ signed delta_y:20;			//Not Positive
		 unsigned padding2:12;		// ***Placeholder
/*0032*/ signed delta_x:20;			//Not Positive
		 unsigned animation:12;		//Not Positive
/*0036*/ 
};

The actual packet example from that looks like this:

3F 00 24 3B 00 00 00 00 - 00 40 9a 00 d7 15 a8 c0
23 01 00 00 9a e0 a2 c0 - 00 00 00 00 00 00 00 00
00 00 00 00

Really, once the spawn struct and clientupdate structs are worked out, I think that should have most of the struct work done. I am sure there will still be more little things to work out in the other structs, but after that, it will mostly be finding missing opcodes. I still haven't worked out a way to find all missing opcodes just yet, but I have been verifying more new ones almost every day.

If I only had a packet collect from 9/7/2007 or shortly after that, it would make this whole process ALOT easier lol.

[KLS]

That's not really what I was talking about, that all really falls under patch files. For example when we wanted to fix inter-zoning we had to actually modify how you zone as 6.2 sent a different series of packets overall than titanium and so required changes to both the patch files and the zone server. When I talk about "core changes" that's things that aren't contained to a certain version and could potentially affect other versions. =p

[Yeormom]

Just do us a flavor and make SoF commits contain only SoF related changes!

[trevius]

As if I had time to make any other changes while I am working on SoF. This thing is like a full time job that I don't get paid for lol.

I will probably be doing an SVN commit for SoF tonight. There is alot more progress since the initial release. Spawns now show up (missing names though), you can actually move around in the zone instead of being stuck in place, doors spawn and ground spawns spawn (but still need work), and a few new opcodes are verified.

Items are a big one that needs work, but those shouldn't be too bad once I know how to do the serialization like a structure so it sends it the way it needs to. I think it is almost to the point where other people might actually be able to start helping out on it soon. I am probably going to try to write a beginner's guide in the wiki for packet structures and maybe some opcode stuff.

[Yeormom]

Originally Posted by trevius

This thing is like a full time job that I don't get paid for lol.

You just described the entire EQemu project!

I am probably going to try to write a beginner's guide in the wiki for packet structures and maybe some opcode stuff.

I do so enjoy documentation~

[trevius]

Back to working on the item serialization again, I broke down items into as close to an exact structure as I can figure out. This is for EQLive, but no new fields have changed from SoF to Live as far as I know, so it should also apply to SoF. This structure may not be 100%, but I think it is at least 95% accurate. This is based off of what I could find from 13th floor and packet collects from EQLive.

struct ItemSerialization_Struct {

/*0000*/	uint32 stacksize;
/*0004*/	uint32 unknown004; // 00 00 00 00
/*0008*/	uint32 price;
/*0012*/	uint32 slot;
/*0016*/	uint32 merchcount;
/*0020*/	uint32 unknown020; // 00 00 00 00
/*0024*/	sint32 serialnumber;
/*0028*/	uint32 instnodrop;
/*0032*/	uint32 typepotion;
/*0036*/	uint32 unknown036; // 00 00 00 00
/*0040*/	uint32 unknown040; // 00 00 00 00
/*0044*/	uint32 unknown044; // 00 00 00 00
/*0048*/	uint32 unknown048; // 00 00 00 00
/*0052*/	uint32 unknown052; // 00 00 00 00
/*0056*/	sint32 unknown056; // 00 00 00 01
/*0060*/	Item_Struct item;
/*0000*/	uint8	unknown00x1;		// 00 Break for separating augments within items?
/*0000*/	Item_Struct augments;		// Bag Slots/Augments within an item
/*0000*/	uint8	unknown00x2;		// 00 Break for separating items?	

}


/*
** Child struct of Item_Struct:
**	Effect data: Click, Proc, Focus, Worn, Scroll
**
*/
struct ItemClickEffect_Struct {
/*0000*/	sint32	Effect;
/*0004*/	uint32	Type;
/*0008*/	uint8	Level2;
/*0009*/	uint8	Level;
/*0010*/	sint32	MaxCharges;		// Max Charges of this effect
			union {
/*0014*/		uint16  Fulfilment;	// Food fulfilment (How long it lasts)
/*0016*/		sint16  CastTime;	// Cast Time for clicky effects, in milliseconds
			};
/*0018*/	uint32	RecastDelay;		// Delay on Recast in seconds
/*0022*/	uint32	RecastType;		// Recast Type from -1 to 18
/*0026*/	uint32	unknown0005;
/*0030*/	char	effectname[1];		//click name - Default is NULL
/*0031*/	uint32	unknown0007;
/*0035*/
};

struct ItemProcEffect_Struct {
/*0000*/	sint32	Effect;
/*0004*/	uint32	Type;
/*0008*/	uint8	Level2;
/*0009*/	uint8	Level;
/*0010*/	uint32	unknown0001;
/*0014*/	uint32	unknown0002;
/*0018*/	uint32	unknown0003;
/*0022*/	uint32	unknown0004;
/*0026*/	sint32	ProcRate;		// Proc Rate - 100 is default
/*0030*/	char	effectname[1];		//proc name - Default is NULL
/*0031*/	uint32	unknown0007;
/*0035*/
};

struct ItemWornFocusScrollEffect_Struct {
/*0000*/	sint32	Effect;
/*0004*/	uint32	Type;
/*0008*/	uint8	Level2;
/*0009*/	uint8	Level;
/*0010*/	uint32	unknown0001;
/*0014*/	uint32	unknown0002;
/*0018*/	uint32	unknown0003;
/*0022*/	uint32	unknown0004;
/*0026*/	uint32	unknown0005;
/*0030*/	char	effectname[1];		//focus, worn, or scroll name - Default is NULL
/*0031*/	uint32	unknown0007;
/*0035*/
};

struct ItemAugmentSlot_Struct {
/*0000*/	uint8	AugSlotType;		// LDoN: Augment Slot 1-5 Type
/*0001*/	sint32	AugSlotVisible;		// LDoN: Augment Slot 1-5 Unknown
/*0005*/	uint8	AugSlotUnk2;		// LDoN: Augment Slot 1-5 Unknown
/*0006*/
};	

#define MAX_AUGMENT_SLOTS 5

struct Item_Struct {
	// Non packet based field
	// uint8	MinStatus; 
	// bool	LoreFlag;		// This will be true if LoreGroup is non-zero
	// bool	SummonedFlag;		// Unused in SoF

	// Packet based fields
	// uint8	ItemClass;		// Item Type: 0=common, 1=container, 2=book
	char	Name[1];		// Variable Length String for Name
	uint8	unknown0001;		// 00 Break between Strings
	char	Lore[1];		// Variable Length String for Item Lore Text
	uint8	unknown0002;		// 00 Break between Strings
	char	IDFile[1];		// Variable Length String for Visible Model Number - IT63
	uint8	unknown0003;		// 00 Break between Strings
	uint32	ID;			// Unique ID (also PK for DB)
	uint8	Weight;			// Item weight * 10
	uint8	NoRent;			// No Rent: 0=norent, 255=not norent
	uint8	NoDrop;			// No Drop: 0=nodrop, 255=not nodrop
	uint8	Size;			// Size: 0=tiny, 1=small, 2=medium, 3=large, 4=giant
	uint8	Slots;			// Bitfield for which slots this item can be used in
	char	Price[8];		// Item cost (?) maybe int64?
	uint32	Icon;			// Icon Number
	uint8	unknown0013;		// UNK013 - 01
	uint32	unknown0014;		// UNK014 - One of these are probably int32	
	bool	BenefitFlag;		// Does this have a benefit flag?
	bool	Tradeskills;		// Is this a tradeskill item?
	sint8	CR;			// Save vs Cold
	sint8	DR;			// Save vs Disease
	sint8	PR;			// Save vs Poison
	sint8	MR;			// Save vs Magic
	sint8	FR;			// Save vs Fire
	sint8	Corruption;		// New Save vs Corruption ****
	sint8	AStr;			// Strength
	sint8	ASta;			// Stamina
	sint8	AAgi;			// Agility
	sint8	ADex;			// Dexterity
	sint8	ACha;			// Charisma
	sint8	AInt;			// Intelligence
	sint8	AWis;			// Wisdom
	sint32	HP;			// HP
	sint32	Mana;			// Mana	
	uint32	Endur;			// Endurance
	sint32	AC;			// AC	
	uint32	unknown0015;		// 
	uint32	unknown0016;		// 
	uint32	unknown0017;		// 
	uint32	Classes;		// Bitfield of classes that can equip item (1 << class#)
	uint32	Races;			// Bitfield of races that can equip item (1 << race#)
	uint32	Deity;			// Bitmask of Deities that can equip this item
	sint32	SkillModValue;		// % Mod to skill specified in SkillModType
	uint32	unknown0038;		// UNK038 - Default is 0
	uint32	SkillModType;		// Type of skill for SkillModValue to apply to
	uint32	BaneDmgRace;		// Bane Damage Race
	uint32	BaneDmgBody;		// Bane Damage Body
	uint32	BaneDmgRaceAmt;		// Bane Damage Race Amount
	sint32	BaneDmgAmt;		// Bane Damage Body Amount
	bool	Magic;			// True=Magic Item, False=not
	sint32	CastTime_;		// Cast Time in Milliseconds
	uint8	ReqLevel;		// Required Level to use item
	uint8	RecLevel;		// Recommended level to use item
	uint8	RecSkill;		// Recommended skill to use item (refers to primary skill of item)
	uint32	BardType;		// Bard Skill Type
	sint32	BardValue;		// Bard Skill Amount
	sint8	Light;			// Light
	uint8	Delay;			// Delay * 10
	uint32	ElemDmgType;		// Elemental Damage Type (1=magic, 2=fire)
	uint32	ElemDmgAmt;		// Elemental Damage
	uint32	Range;			// Range of item
	uint32	Damage;			// Delay between item usage (in 0.1 sec increments)
	uint32	Color;			// RR GG BB 00 <-- as it appears in pc
	uint32	ItemType;		// Item Type/Skill (itemClass* from above)
	uint32	Material;		// Item material type
	uint32	unknown0060;		// UNK060 - Default is 0
	uint8	unknown0061;		// UNK061 - Default is 0
	float	SellRate;		// Sell rate
	sint32	CombatEffects;		// PoP: Combat Effects +
	sint32	Shielding;		// PoP: Shielding %
	sint32	StunResist;		// PoP: Stun Resist %
	sint32	StrikeThrough;		// PoP: Strike Through %
	uint32	ExtraDmgSkill;		// PoP: Extra Damage Skill
	uint32	ExtraDmgAmt;		// PoP: Extra Damage Amount
	sint32	SpellShield;		// PoP: Spell Shield %
	sint32	Avoidance;		// PoP: Avoidance +
	sint32	Accuracy;		// PoP: Accuracy +
	uint32	CharmFileID;		// ID of the Charm File
	sint32	FactionMod1;		// Faction Mod 1
	sint32	FactionAmt1;		// Faction Amt 1
	sint32	FactionMod2;		// Faction Mod 2
	sint32	FactionAmt2;		// Faction Amt 2
	sint32	FactionMod3;		// Faction Mod 3
	sint32	FactionAmt3;		// Faction Amt 3
	sint32	FactionMod4;		// Faction Mod 4
	sint32	FactionAmt4;		// Faction Amt 4
	char	CharmFile[1];		// Name of the Charm File
	uint8	unknown0062;		// 00 Break between Strings
	uint32	AugType;
	uint16	AugRestrict;
	uint16	AugDistiller;
	ItemAugmentSlot_Struct augslots[MAX_AUGMENT_SLOTS]; //Augment Slots
	uint32	PointType;
	uint32	LDoNTheme;
	uint32	LDoNPrice;
	uint32	unknown0098;		// UNK098
	uint32	LDoNSold;
	uint8	BagType;		// 0:Small Bag, 1:Large Bag, 2:Quiver, 3:Belt Pouch ... there are 50 types
	uint8	BagSlots;		// Number of slots: can only be 2, 4, 6, 8, or 10
	uint8	BagSize;		// 0:TINY, 1:SMALL, 2:MEDIUM, 3:LARGE, 4:GIANT 
	uint8	BagWR;			// 0->100
	uint8	Book;			// 0=Not bool, 1=Book
	uint16	BookType;
	uint32	Filename;		// Filename for book data
	uint8	LoreGroup;		// LoreGroup seems to have replaced LoreFlag
	bool	ArtifactFlag;		// Flag as an Artifact
	bool	PendingLoreFlag;	// ???  Maybe just unknown?
	uint32	Favor;			// Individual favor
	uint32	GuildFavor;		// Guild favor
	uint8   FVNoDrop;		// Firiona Vie nodrop flag
	uint8	DotShielding;		// May need to be moved to the PoP stuff ****
	uint32	Attack;
	uint32	Regen;
	uint32	ManaRegen;
	uint32	EnduranceRegen;
	uint8	Haste;
	uint8	DamageShield;
	uint32	unknown0120;		// UNK120 - Default is -1 	
	uint8	unknown0121;		// UNK121 - Default is 0 		
	bool	Attuneable;
	bool	NoPet;
	uint16	unknown0124;		// UNK124 - Default 0
	bool	PotionBelt;
	uint32	PotionBeltSlots;
	uint8	StackSize;
	bool	NoTransfer;		//Swapped Positions with StackSize ****
	// bool	Stackable;		//Not in 13th Floor ???
	bool	QuestItemFlag;
	uint8	unknown0131;		// UNK131 - Default is 0	
	uint8	unknown0132[19];		// UNK132 - Default is 0	
	ItemClickEffect_Struct Click;
	ItemProcEffect_Struct Proc;
	ItemWornFocusScrollEffect_Struct Worn, Focus, Scroll;
	uint32	unknown0193;//UNK193 - Default is 0
	uint8	purity;			//purity - Newly Added - Default is 0, but some go up to 75
	uint8	dsmitigation;		//dsmitigation - Newly Added - Default is 0, but some are up to 2
	uint8	herostr;		//heroic_str - Newly Added - Default is 0
	uint8	heroint;		//heroic_int - Newly Added - Default is 0
	uint8	herowis;		//heroic_wis - Newly Added - Default is 0
	uint8	heroagi;		//heroic_agi - Newly Added - Default is 0
	uint8	herodex;		//heroic_dex - Newly Added - Default is 0
	uint8	herosta;		//heroic_sta - Newly Added - Default is 0
	uint8	herocha;		//heroic_cha - Newly Added - Default is 0
	uint32	heropoison;		//HeroicSvPoison - Newly Added - Default is 0
	uint32	heromagic;		//HeroicSvMagic - Newly Added - Default is 0
	uint32	herofire;		//HeroicSvFire - Newly Added - Default is 0
	uint32	herodisease;		//HeroicSvDisease - Newly Added - Default is 0
	uint32	herocold;		//HeroicSvCold - Newly Added - Default is 0
	uint32	herocorruption;		//HeroicSvCorruption - Newly Added - Default is 0
	uint32	healamt;		//healamt - Newly Added - Default is 0, but some are up to 9
	uint32	spelldmg;		//spelldmg - Newly Added - Default is 0, but some are up to 9
	uint32	clairvoyance;		//clairvoyance - Newly Added - Default is 0, but some are up to 10
	uint32	backstabdmg;		//backstabdmg - Newly Added - Default is 0, but some are up to 65
	uint32	evolvinglevel;		//evolvinglevel - Newly Added - Default is 0, but some are up to 7
	uint32	MaxPower;		//MaxPower - Newly Added
	uint32	Power;			//Power - Newly Added

};

Since items are handled in a different way than any other struct, I am not exactly sure how to implement this properly. Currently, the emulator sends items as a long string. For SoF, they need to be sent like any other normal structure. Right now, I can't figure out how to add these structs in without having to mess with the item serialization for other clients. I will keep working on it, but it is a bit above my skill level right now. For starters, I need to figure out how to change the serialization from using MakeAnyLengthString and instead for it to send the actual data as it is written in the struct.