EQEmulator Forums

EQEmulator Forums (https://www.eqemulator.org/forums/index.php)
-   Development::Bug Reports (https://www.eqemulator.org/forums/forumdisplay.php?f=591)
-   -   Client-Server Item Desyncronization Bugs (https://www.eqemulator.org/forums/showthread.php?t=35531)

lerxst2112 07-28-2012 04:53 PM

Quote:

Obviously, the first loop catches the cursor slot 30. How does slot 8000 not being occupied actually affect the operation
of the second loop?
They appear to be two separate unrelated loops. One moves items in slots 0-30, and the second moves items from the cursor queue to 8000-(8000 + N). I didn't look at how MoveItemToCorpse works, but I would guess since the slot numbers don't overlap it isn't an issue.

Quote:

And, what would happen if both slot 30 and slot 8000 were occupied? We should lose one of them, right?
Sounds like a good thing to test for.

Uleat 07-30-2012 01:39 AM

They are seperate, but they're not... I had to include both for reference.

The first one handles the 'visible' cursor slot 30. The second handles slot 8000, which supposedly is also the visible cursor slot.
(There are several references in the code that indicate this..mostly in remark statements.)


Here's a little test, with validation checks, to induce a bugged corpse:

- Ensure no items are on your cursor

- #summonitem 13201 (Blackbox A Series)
- #summonitem 13202 (Blackbox B Series)
- #summonitem 13203 (Blackbox C Series)
- #summonitem 13204 (Blackbox D Series)

- Open your database tool software and look at this character's inventory..it should read:
-- Slot 30 - 13201
-- Slot 8001 - 13202
-- Slot 8002 - 13203
-- Slot 8003 - 13204

- Now, #kill yourself..it should crash your client.

- Refresh you database table and reinspect the values:
-- Slot 30 - 13202
-- Slot 8001 - 13203
-- Slot 8002 - 13204

- At this point, we can assume that item 13201 made it to the corpse, or that it was deleted for some reason.

- Now, revive and you should have cursor items. (Best to do all of this next to your bind point...)
-- Three items total, Blackboxes B, C & D.

- Loot your corpse..in addition to your equipment, you will find Blackbox A at the end.

- You didn't actually lose anything this time, but now your corpse is bugged.


Something is crashing the client..probably while trying to process slot 8000.

I will have to work up an in-depth test to figure out what's going on exactly..or learn how to actually use the
debug feature...

I suspect the blob data of the corpse will contain either a bugged item 13201 or slot 8000 of some type.

IF tradeskilling is using slot 8000 for combines or returns, I suspect this same issue is at the root of that problem too.
(Not death, but the use of slot 8000...)


Since the problem could reach back into the kernel, I don't wan't to simply 'fix' it here. Some more research
is needed..but it is on the list for sure now...

Uleat 07-30-2012 04:00 AM

Ok, I managed to get EQ Editor 1.30 to work enough to be able to inspect a corpse's inventory.

What I found was that my bugged corpse had two items on it...

Item 1 - CHARM slot (0) -> ITEM ID 0
Item 2 - CHARM slot (0) -> ITEM ID 0


Obviously, something is not right with the way the server currently handles loot transfers to a corpse. Whether it is in the
cursor queue or elsewhere, the corpse DOES contain entries in the blob that keeps the server from properly decaying it.

(EQ Editor looks pretty thorough..Sorry George, I still haven't tried your tools yet... I was looking specifically for some code
to incorporate into a '#command' that can be used in-game and ran across this. AndMetal seems to have something on
the wiki that could be adapted to a 'live' command for checking blobs..I'd like to add that as well. I need to find out who
wrote EQ Editor and see if I can use thier code too since it would only have to be converted from php to c++ or perl.)


http://code.google.com/p/ultraeqeditor/ will take you to this program.

To get this working, in the interim, you will need to make at least three changes:

1 - edit login.php and remove the 'md5()' wrapper from the password check at the beginning
2 - edit functions.php and remove the 'md5()' wrapper from the password check in 'CheckAuth()'
3 - edit corpse.php and change 'parent_corpse_id' to 'id'

I had to remove the wrappers because I can't log into the world server when my http password is security hashed...
And account passwords seem to be encrypted with SHA and not MD5 anyways...


These changes will let you at least see the items on the bugged corpse that was created in the last post.

Of course, you will need the same setup installed and running that's required for AllaClone. I just moved my
AllaClone files out and copied these in. You will need to edit the configuration.php file as well.

Uleat 07-30-2012 05:20 AM

Apparently, this needs some more work..I can't seem to find newer corpses...

This corpse I was looking at was several weeks old, so I'll see what I can do with the search criteria.

Uleat 07-30-2012 06:30 AM

I was able to finally get to my recent corpses and see that they actually had three items of slot (0) -> itemid (0).

This matches the amount of queued items that I had on the cursor when I died (Blackboxes B, C & D.)


Ouside of the 'slot 8000' possibility, I need to follow the calls back and see if any of the queued items are actually
being allowed since these methods are processing them.

Uleat 07-30-2012 09:34 AM

Ok, sadly it appears that EQ Editor needs alot more work... I finally got to a corpse that had ALL of my equip and
it reported all slots and itemids as '0' as well...

Anyways... Back to trying to find out what is crashing the client.

Uleat 07-30-2012 01:26 PM

I gave up short term on the blob data..was taking too much time.

Instead, I just added a command to view the server data of a particular corpse.


What I found was (I left my cursor item prior to crash in place):

- item: 13201 -> equipslot: 30 -> lootslot: 27
- item: 13202 -> equipslot: 8000 -> lootslot: 65535
- item: 13203 -> equipslot: 8001 -> lootslot: 65535
- item: 13204 -> equipslot: 8002 -> lootslot: 65535


With my testing with my 'zopp' command, I saw that the client does not recognize the 8000-series of slots.
When a packet is sent in that range, the client just ignores it..meaning moving/deleting the cursor item
will not 'bump' the next item up if you assume that 8001 is the next client slot.

With that being obvious, 'MoveItemToCorpse' is calling 'DeleteItemInInventory.' This function, among other
things, sends a client delete item packet. I'm not sure if the equipslot or lootslot ID is causing the crash at this
point, but I'm still working on it.


If anyone has any idea on why the lootitem slots are 65535 and/or the possibility of the slot 8000 range
causing client crashes, you could save me a whole bunch of time. Every time I find something new, I have to
stop to find and learn why something is causing that behavior...

Uleat 07-31-2012 07:04 AM

I believe the bugged corpse issue can be resolved soon.

I found what needed to be changed to allow ALL items to be seen by the client and now all of my corpses are
COMPLETELY lootable.


However... There is a duplication problem that this fix brings to light.

Currently, when a player logs back in after the crash (dying with queued cursor items,) those items are back on the cursor.

Now, when a player loots their corpse, those items are there again as well. This is what was going on behind the scenes
with the current code, but was being blocked as well.


Now I need to pin-down the client crash (I have, but need to ensure no other calls are affected) and the duplication cause.

In addition, I need to resolve any issues with a queued cursor bag that has contents..I'll probably just push them to the
cursor so they are not lost (rememeber, queued cursor slots do not have queued bag slots.)

Caryatis 07-31-2012 10:30 AM

I'm sure eventually you will actually produce something useful but seriously man, 8 posts by you in a row like an hour apart. If you want to start a twitter feed to track your rambling around this problem, feel free but nobody needs to know that you havent used Georges tools and just got an out of date program to kinda work but then turns out it wasnt needed at all because that was a circuitous route around the actual issue.

You might take offense to this but you don't seem experienced enough to be tackling this all at once. I don't see why you don't break out each bit into its own diff and finalize those. You got some test code here, other code there and your scope keeps growing every post, at this rate you will never finish.

The other thing to take into account is that this thread doesn't really inspire confidence in your final product, so if you dump a 10k line diff at the end, that is going to have to be combed over like crazy before a commit since this is a very exploit prone area and nobody likes to go through a huge ass diff like that, shit it can take like a month to get a multi line diff added around here.

Uleat 07-31-2012 01:05 PM

I appreciate your candor Caryatis, and you are probably right on all counts. I am mature enough not to take criticism
personally, though.


I have devised a fairly decent system of keeping track of this mess. Any function that I modify is considered a 'Change.'
When changes must be deployed together, it becomes a 'Patch.'


As far as patches in this thread, it is not my intent that they be implemented. I will start a new one over in server code
submission specifically for a finalized product. Anything here is mainly for review and suggestions.

I will post a 'support' patch soon. The other issues, like bandolier swaps, arrow stacking issues, etc... will come later since
they are the most intrusive into the kernel.


I do tend to get a little 'manic' at times, so I'll try to do a better job of monitoring my 'tweet' rate!
</points to my signature block>

Uleat 08-01-2012 02:05 PM

Ok, I think I have got a working fix for the bugged corpses.

I fixed the 'invisible item' (bugged corpses) and client crash issues with queued cursor items.

I have even somewhat fixed the new item duplication bug that arose from that.


I am now looking for a way to clear the client-side cursor queue when your dead. I can remove the immediate cursor item,
but when a player is dead, the queue doesn't update and I end up with 'phantom' items when a player is revived.
Logging out and back in resets the client, so there is no problem there.

Any ideas on how to do this? I don't want to 'Kick()' them everytime they die with items on their cursor...

Uleat 08-06-2012 11:53 PM

Got a couple of questions, if anyone happens to know an answer or two...


With what I can gleen from from the code, it appears the server player profile is based on the Titanium model with translations
(Encode/Decode) and adaptations applied per client.

Are there any instances where this is not true?


I've also been playing around with the bugged corpse issue and have had some luck. There are two specific things I can't
deduce and wonder if different clients are handling things differently.


I've added code to basically allow any item that is not assigned to a bag slot to show up in the corpse list..but not necessarily
add them when the corpse is created.

Should the trader slots be a part of corpse creation? My testing shows that trade items are put back into the player's
inventory after the corpse is processed. (The server code does not currently process trader slots to a corpse.)


I'm also having trouble clearing the cursor of a dead player client-side. I can clear the server and database with no issues,
but the client won't release them until revived.

Do all clients have this behavior? SoF does, but the server IS currently coded to move cursor queue items to the corpse. (source of bugged corpses)
I can do a work-around, but I need to know what the desired behavior should be. (Leave queue items on cursor, or force
zone change event on rez/revive..like Evac does.)


Thanks!

sorvani 08-07-2012 01:33 AM

you are forgetting that the death hover window now exists. it is generally turned off because it was causing issues itself. but one would think that since all the clients are designed for the hover, that may be one reason some things do not clean up (client side) like you are thinking.

test trades. make a second toon, open a trade window then #kill it.

Uleat 08-08-2012 02:12 AM

Yeah, cancelled trades are definitely processed after the corpse is created. I see the item pop back into my inventory about
a half-a-second after I use #kill.

Setting 'Hover' mode to false definitely eliminates the issue since forcing a zone change event is what I was wanting to
use to correct the situation. The only problem is trying to figure out how to handle the CSD associated with it when
'Hover' is set to true. It's more cosmetic than anything since trying to place the 'phantom' items in your inventory results
in the server deleting them anyways.


I'll still leave that question out there in case someone has a decent idea of how to handle that.

I want to add a bit more code to let the player know if the corpse contains 'bugged' items and then tidy it up a
little bit, then I'll post an alpha patch here to let you guys see where this is going.

Uleat 08-08-2012 07:06 AM

Ok, here is an ALPHA patch for bugged corpses. It may still need a little work, but most issues should be ironed out:

[CSD Patch 4]
Code:

Index: inventory.cpp
===================================================================
--- inventory.cpp        (revision 2175)
+++ inventory.cpp        (working copy)
@@ -397,9 +405,11 @@
                LogFile->write(EQEMuLog::Debug, "DeleteItemInInventory(%i, %i, %s)", slot_id, quantity, (client_update) ? "true":"false");
        #endif
 
-        if(!m_inv[slot_id]) {
+        // Added 'IsSlotValid(slot_id)' check to both segments of client packet processing.
+        // Cursor queue slots were slipping through and crashing client
+        if(!m_inv[slot_id]) {
                // Make sure the client deletes anything in this slot to match the server.
-                if(client_update) {
+                if(client_update && IsValidSlot(slot_id)) {
                        EQApplicationPacket* outapp;
                        outapp = new EQApplicationPacket(OP_DeleteItem, sizeof(DeleteItem_Struct));
                        DeleteItem_Struct* delitem        = (DeleteItem_Struct*)outapp->pBuffer;
@@ -427,7 +437,7 @@
                    database.SaveInventory(character_id, inst, slot_id);
        }
 
-        if(client_update) {
+        if(client_update && IsValidSlot(slot_id)) {
                EQApplicationPacket* outapp;
                if(inst) {
                        if(!inst->IsStackable() && !isDeleted)
@@ -437,13 +447,13 @@
                                // Stackable, arrows, etc ? Delete one from the stack
                                outapp = new EQApplicationPacket(OP_DeleteItem, sizeof(MoveItem_Struct));
 
-                        DeleteItem_Struct* delitem        = (DeleteItem_Struct*)outapp->pBuffer;
-                        delitem->from_slot                        = slot_id;
-                        delitem->to_slot                        = 0xFFFFFFFF;
-                        delitem->number_in_stack        = 0xFFFFFFFF;
-                        for(int loop=0;loop<quantity;loop++)
-                                QueuePacket(outapp);
-                        safe_delete(outapp);
+                                DeleteItem_Struct* delitem        = (DeleteItem_Struct*)outapp->pBuffer;
+                                delitem->from_slot                        = slot_id;
+                                delitem->to_slot                        = 0xFFFFFFFF;
+                                delitem->number_in_stack        = 0xFFFFFFFF;
+                                for(int loop=0;loop<quantity;loop++)
+                                        QueuePacket(outapp);
+                                safe_delete(outapp);
                }
                else {
                        outapp = new EQApplicationPacket(OP_MoveItem, sizeof(MoveItem_Struct));
Index: PlayerCorpse.cpp
===================================================================
--- PlayerCorpse.cpp        (revision 2175)
+++ PlayerCorpse.cpp        (working copy)
@@ -352,13 +352,20 @@
                pp->silver = 0;
                pp->gold = 0;
                pp->platinum = 0;
+
+                // need to tell client that cash has changed..could be RespawnFromHover=true..need to check
+                // issue looks like a money duplication error, but server value is actually correct
       
                // get their tints
                memcpy(item_tint, &client->GetPP().item_tint, sizeof(item_tint));
       
                // solar: TODO soulbound items need not be added to corpse, but they need
                // to go into the regular slots on the player, out of bags
-       
+               
+                // personal and cursor bag slots (251-340) are moved to corpse..should be deleting db entries too.
+                // reworked code to return and merge a list for the query builder instead of adding MoveItemToCorpse
+                // code to each loop.
+
                // worn + inventory + cursor
        std::list<uint32> removed_list;
        bool cursor = false;
@@ -367,23 +374,27 @@
                        item = client->GetInv().GetItem(i);
                        if((item && (!client->IsBecomeNPC())) || (item && client->IsBecomeNPC() && !item->GetItem()->NoRent))
                        {
-                                MoveItemToCorpse(client, item, i);
-                removed_list.push_back(i);
+                                removed_list.merge(MoveItemToCorpse(client, item, i)); 
                        }
                }
 
                // cursor queue
+                // bumped starting assignment to 8001 because any in-memory 'slot 8000' item was moved above as 'slot 30'
+                // this was mainly for client profile state reflection..should match db player inventory entries now.
                iter_queue it;
-                for(it=client->GetInv().cursor_begin(),i=8000; it!=client->GetInv().cursor_end(); it++,i++) {
+                for(it=client->GetInv().cursor_begin(),i=8001; it!=client->GetInv().cursor_end(); it++,i++) {
                        item = *it;
                        if((item && (!client->IsBecomeNPC())) || (item && client->IsBecomeNPC() && !item->GetItem()->NoRent))
                        {
-                                MoveItemToCorpse(client, item, i);
+                                removed_list.merge(MoveItemToCorpse(client, item, i));
                cursor = true;
                        }
                }
               
-        if(removed_list.size() != 0) {
+                // client->DeleteItemInInventory is memory only..update_db is passed as false..not sure why unless it's a
+                // sync issue... client->Save() alone doesn't appear to handle inventory deletion with &inv[slot_id] being
+                // null. player inventory db entries did not reflect client.inv states after 'client->Save()' alone.
+                if(removed_list.size() != 0) {
            std::stringstream ss("");
            ss << "DELETE FROM inventory WHERE charid=" << client->CharacterID();
            ss << " AND (";
@@ -401,14 +412,40 @@
            ss << ")";
            database.RunQuery(ss.str().c_str(), ss.str().length());
        }
-       
+
        if(cursor) {
-            std::list<ItemInst*>::const_iterator start = client->GetInv().cursor_begin();
-            std::list<ItemInst*>::const_iterator finish = client->GetInv().cursor_end();
-            database.SaveCursor(client->CharacterID(),
-                start, finish);
+                        // cycle cursor to clear memory queue..will delete server queue, but not client queue with hover enabled...
+                        // if can figure out client issue, change argument 3 (client_update) to false if handled elsewhere.
+                        // might could solve by forcing zone entry event to reset client inventory or sending client clear and
+                        // resending bulkinventory. don't want to send client delete cursor on revive..might cause item loss...
+
+                        while(!client->GetInv().CursorEmpty()) {
+                                if (RuleB(Character, RespawnFromHover)) {
+                                        client->Message(0, "Attempting to delete %s from cursor.", client->GetInv().GetItem(SLOT_CURSOR)->GetItem()->Name);
+                                        client->DeleteItemInInventory(SLOT_CURSOR, 0, true, false);
+                                }
+                                else {
+                                        client->DeleteItemInInventory(SLOT_CURSOR, 0, false, false); // no need to update client without hover
+                                }
+                        }
+
+                        if (RuleB(Character, RespawnFromHover)) {
+                                client->Message(13, "Warning: Your cursor may contain duplicate items not found on the server!");
+                                client->Message(0, "Place these items into an empty inventory slot to resolve this problem.");
+                        }
+
+                        // this code didn't appear to accomplish anything db-wise. it looks like it just re-saved the cursor
+                        // queue data. with this code in-place, the cursor queue was re-saved and prevented lost items..which
+                        // were the items on the corpses causing the 'bugged' issue in the first place. this code can be
+                        // deleted after testing and verification of code submission.
+
+            //std::list<ItemInst*>::const_iterator start = client->GetInv().cursor_begin();
+            //std::list<ItemInst*>::const_iterator finish = client->GetInv().cursor_end();
+            //database.SaveCursor(client->CharacterID(),
+            //    start, finish);
        }
 
+                client->CalcBonuses(); // shouldn't matter to a corpse, but hey..at least Magelo will update correctly =)
                client->Save();
        } //end "not leaving naked corpses"
       
@@ -417,28 +454,35 @@
 }
 
 // solar: helper function for client corpse constructor
-void Corpse::MoveItemToCorpse(Client *client, ItemInst *item, sint16 equipslot)
+std::list<uint32> Corpse::MoveItemToCorpse(Client *client, ItemInst *item, sint16 equipslot)
 {
        int bagindex;
        sint16 interior_slot;
        ItemInst *interior_item;
+        std::list<uint32> returnlist;
 
        AddItem(item->GetItem()->ID, item->GetCharges(), equipslot, item->GetAugmentItemID(0), item->GetAugmentItemID(1), item->GetAugmentItemID(2), item->GetAugmentItemID(3), item->GetAugmentItemID(4));
-        if(item->IsType(ItemClassContainer))
+        returnlist.push_back(equipslot);
+
+        // Qualified bag slot iterations. processing bag slots that don't exist is not a good idea.
+        if(item->IsType(ItemClassContainer) && ((equipslot >= 22 && equipslot <=30))) // Limit the bag check to inventory and cursor slots.
        {
                for(bagindex = 0; bagindex <= 10; bagindex++)
                {
+                        // For empty bags in cursor queue, slot was being resolved as SLOT_INVALID (-1)
                        interior_slot = Inventory::CalcSlotId(equipslot, bagindex);
                        interior_item = client->GetInv().GetItem(interior_slot);
 
                        if(interior_item)
                        {
                                AddItem(interior_item->GetItem()->ID, interior_item->GetCharges(), interior_slot, interior_item->GetAugmentItemID(0), interior_item->GetAugmentItemID(1), interior_item->GetAugmentItemID(2), interior_item->GetAugmentItemID(3), interior_item->GetAugmentItemID(4));
+                                returnlist.push_back(Inventory::CalcSlotId(equipslot, bagindex));
                                client->DeleteItemInInventory(interior_slot, 0, true, false);
                        }
                }
-        }
+        } // */
        client->DeleteItemInInventory(equipslot, 0, true, false);
+        return returnlist;
 }
 
 // To be called from LoadFromDBData
@@ -981,19 +1025,43 @@
                ItemList::iterator cur,end;
                cur = itemlist.begin();
                end = itemlist.end();
+
+                // Observed some odd behavior concerning the last corpse slot as coded (SoF client in this case.)
+                // Clicking index 29 (zero-based) with an item in index 28 results in retrieving index 28 item.
+                // Clicking index 29 with no item in index 28 results in the closing of the corpse loot window.
+
+                int corpselootlimit = 30; // 30 is the original value
+                /* need actual corpse limit values per client (or client range)..if always 30, then these con checks are unneeded
+                // enumeration shouldn't be needed unless someone finds a use for this info elsewhere
+                if (client->GetClientVersion()>=EQClientVoA)
+                        corpselootlimit=30;
+                else if (client->GetClientVersion()>=EQClientHoT)
+                        corpselootlimit=30;
+                else if (client->GetClientVersion()>=EQClientUnderfoot)
+                        corpselootlimit=30;
+                else if (client->GetClientVersion()>=EQClientSoD)
+                        corpselootlimit=30;
+                else if (client->GetClientVersion()>=EQClientSoF) // SoF has 32 visible slots..change untested
+                        corpselootlimit=30;
+                else if (client->GetClientVersion()>=EQClientTitanium)
+                        corpselootlimit=30;
+                else if (client->GetClientVersion()>=EQClient62)
+                        corpselootlimit=30;
+                else
+                        corpselootlimit=30; // */
+
                for(; cur != end; cur++) {
                        ServerLootItem_Struct* item_data = *cur;
                        item_data->lootslot = 0xFFFF;
+                       
+                        // Dont display the item if it's in a bag
 
-                        // Dont display the item if it's in a bag
-                        if(!IsPlayerCorpse() || item_data->equipSlot <= 30 || tCanLoot>=3)
+                        // added cursor queue slots to corpse item visibility list. Nothing else should be making it to corpse.
+                        if(!IsPlayerCorpse() || item_data->equipSlot <= 30 || tCanLoot>=3 ||
+                                (item_data->equipSlot >= 8000 && item_data->equipSlot <= 8999))
                        {
-                                if (i >= 30)
+                                if (i < corpselootlimit) // < 30 (0 - 29)
                                {
-                                                Message(13, "Warning: Too many items to display. Loot some then re-loot the corpse to see the rest");
-                                }
-                                else
-                                {
                                        item = database.GetItem(item_data->item_id);
                                        if (client && item)
                                        {
@@ -1006,9 +1074,24 @@
                                                item_data->lootslot = i;
                                        }
                                }
+                                else if (i == corpselootlimit) // = 30
+                                {
+                                        client->Message(13, "Warning: This corpse contains more items than can be displayed!");
+                                        client->Message(0, "Remove items and re-loot corpse to access remaining inventory.");
+                                }
                                i++;
                        }
                }
+                if (i > corpselootlimit) // > 30 (remember 'i' is increased again after the last iteration, so no '=')
+                        client->Message(0, "(%s contains %i additional %s.)", GetName(), (i-corpselootlimit), (i-corpselootlimit)==1?"item":"items");
+
+                if (IsPlayerCorpse() && i == 0 && itemlist.size() > 0) { // somehow, corpse contains items, but client doesn't see them...
+                        client->Message(13, "Warning: This corpse contains items that you do not permission to access!");
+                        client->Message(13, "Contact a GM for assistance to determine if item replacement is necessary.");
+                        client->Message(0, "BUGGED CORPSE [DBID: %i, Name: %s, Item Count: %i]", GetDBID(), GetName(), itemlist.size());
+                        // if needed/wanted - create log dump->iterate corpse list..need pointer to log file
+                        // could add code to check for owning client and give list of bugged items on corpse
+                }
        }
       
        // Disgrace: Client seems to require that we send the packet back...
Index: PlayerCorpse.h
===================================================================
--- PlayerCorpse.h        (revision 2175)
+++ PlayerCorpse.h        (working copy)
@@ -111,7 +111,7 @@
        inline int GetRezzExp() { return rezzexp; }
 
 protected:
-        void MoveItemToCorpse(Client *client, ItemInst *item, sint16 equipslot);
+        std::list<uint32> MoveItemToCorpse(Client *client, ItemInst *item, sint16 equipslot);
 
 private:
        bool                p_PlayerCorpse;


Remember, this patch is NOT commit ready.

Uleat 08-13-2012 07:25 AM

I ran across a couple of client bugs that can be mentioned, but don't really need a new thread.


First, there's a odd issue with my client (SoF) where with a full corpse (30 items), clicking index 29 (zero-based) will
loot index 28. Clicking index 29 again, with 28 empty, will cause the loot window to close.

Second, with the submitted bugged corpse patch, and with the RespawnFromHover = false, dying outside of your
binding zone with certain items loaded on your cursor, will cause a glitch when you reload. The icon from one of the
cursor items will appear when you load into zone. If you die in your binding zone after this, but before leaving, this
issue doesn't appear.


[Random Items - no issue]
17005 (first item on cursor)
8005
13201
17003
1619

[Sequential Items, all have the same icon - issue arises]
13201 (first item on cursor)
13202
13203
13204
13205


I think it is a graphic glitch as putting this 'phantom' item in your inventory results in no server message. If you
use the #zopp command (slot 30, item id) and place it in your inventory, you get a server item deletion message.

Uleat 08-15-2012 04:47 AM

Moving On...
 
Now that I've confused, flustered and maddened everyone with my code submissions so far, let's move on to something more productive.


Here's what I'm seeing at first glance with the 'Bandolier' issue.


inventory.cpp::Client::SetBandolier(const EQApplicationPacket *app)
Code:

slot = m_inv.HasItem(m_pp.bandoliers[bss->number].items[BandolierSlot].item_id, 1, invWhereWorn|invWherePersonal|invWhereCursor);
Item.cpp::Inventory::HasItem(uint32 item_id, uint8 quantity, uint8 where)
Code:

if(where & invWhereCursor) {
        // Check cursor queue
        slot_id = _HasItem(m_cursor, item_id, quantity);
        if (slot_id != SLOT_INVALID)
                return slot_id;
}


The following internal function is overloaded allowing two argument sets. The checks in Inventory::HasItem all use the first definition with
the exception of the last check that uses the second one, which is m_cursor.


Item.cpp::Inventory::_HasItem(ItemInstQueue& iqueue, uint32 item_id, uint8 quantity)
Code:

for (it=iqueue.begin(); it!=iqueue.end(); it++) {
        ItemInst* inst = *it;
        if (inst)
        {
                if (inst->GetID() == item_id) {
                        quantity_found += (inst->GetCharges()<=0) ? 1 : inst->GetCharges();
                        if (quantity_found >= quantity)
                                        return SLOT_CURSOR;
                }


Notice how 'it' is iterated. If an item is found, it returns SLOT_CURSOR. MAJOR PROBLEM there unless it's found at slot 8000...

SetBandolier assigns the weapon slot based on 'HasItem.' Say, for instance, the item being sought is in the fourth queue position. The actual
item is in slot 8003, but the Bandolier function thinks it is in slot 30 (queue slot 8000) because of 'return SLOT_CURSOR;' This is creating a
CSD.

We can't change that to slot_id because the client doesn't allow cursor queue manipulation. We should consider only checking the queue
for items, but not allowing their removal unless all items up to that item are also removed, and possibly pushed back to the cursor.


I think this is one of the causes of the bandolier issue, and possibly the tradeskill cursor issue as well.


If you think my logic is flawed, your input is welcome. I need to understand this if I'm to stand any chance of fixing it.

sorvani 08-16-2012 02:04 AM

having done no research into this code at all, but reading your post and knowing some history of the issue, I wonder if things get broke because it is using the cursor slots?

What if someone hits forage then hits a bandolier swap right as the forage is placed on the cursor? or some other event that returns an item to the cursor?

Uleat 08-16-2012 03:10 AM

The testing I've done appears to be ok with only using slot 30. Anytime a bandolier item is found further back in the queue,
the first item is taken..explains why a 'Box of Abu-Kar' wound up as my primary weapon...

I imagine that any items in it would also go poof...


Also, the support function doesn't check the cursor for stacking before trying to move stackable items.

When moving an arrow out of worn inventory, the client adds it to the cursor stack, but the server puts it in an empty bag slot.
Further manual swaps show the CSD condition by making items disappear.


I'm going to try dropping the 'invWhereCursor' and add a single cursor check in addition to bumping up the stackable range
check to 30 from 29 and see where that takes us.


In regards to foraging, or any other timer-based action, we could just throw in a timer check in the beginning and exit out if
any of them are active..maybe throw a "You must finish your current action before using the bandolier" message to the client.


I also scanned through the tradeskill combine functions. They appear to only use the 'm_inv' list, but I'm trying to find where
the relationship is defined between list and slot to see if the cursor is also being checked. I found the index declarations.

Uleat 08-16-2012 08:04 AM

Both of the bandolier issue fixes appear to work so far. I'm at 100% success on items going where they're suppose to. (I'm not
checking the database on this yet since that part of the code should work. It was only where the items were going that was
the issue. I will check though before this gets posted down the road.)

This does not mean that issues won't still arise once the client becomes de-sync'd due to another reason. It just means that
the bandolier shouldn't be the cause of the de-sync.


As far as the timer possibility, I don't know how feasible that really is. There are no client packets sent in the SetBandolier
function, so all switching is handled client-side. Cancelling the process would create a major de-sync. We'll have to watch this.

Uleat 08-17-2012 10:22 PM

I looked through the ts and foraging (fish/forage) code and didn't see anything out of the ordinary. I'm still sorting through the 'inventory' calls.
There's several areas that I don't trust based on what I see, but I still need to find all of the ways to test it in-game.

(The two foraging actions call different methods, PushItemOnCursor vs. PutItemInInventory(SLOT_CURSOR), but the end result is the same..just a
little more overhead with one of them.)


As far as cursor items being out of order, unless I find something specific somewhere, I'm thinking the client may be receiving UDP packets out of order.

There's no unique identifier in a series of cursor items. If the server sends three packets (a, b & c) to the client with the summonitem argument, the
client won't know that b->c->a is the wrong order unless it has the ability to identify and re-order based on timestamps. (I just don't know...)

My server is setup with drop_packets->false and is on a 100Mbs LAN..single player as well. I probably won't be able to reproduce issues that arise with
public servers.

Uleat 08-21-2012 10:40 PM

I'm not sure what's going on with this code snippet, but I can't seem to get it to work properly when I add it to another function.

This code is copied directly from inventory.cpp::Client::DeleteItemInInventory. It handles client updates when deleting partial quantity from current stack charges.

Code:

// Stackable, arrows, etc ? Delete one from the stack
        outapp = new EQApplicationPacket(OP_DeleteItem, sizeof(MoveItem_Struct));

        DeleteItem_Struct* delitem        = (DeleteItem_Struct*)outapp->pBuffer;
        delitem->from_slot                = slot_id;
        delitem->to_slot                = 0xFFFFFFFF;
        delitem->number_in_stack        = 0xFFFFFFFF;
        for(int loop=0;loop<quantity;loop++)
                QueuePacket(outapp);
        safe_delete(outapp);

I've adapted it to my needs, but what is happening is the client deletes the entire stack from the cursor, then each iteration
after the first creates the 'Move Item Failed In Client Application' error message.

Changing 'number_in_stack' to 'quantity' and disabling the iteration produces no results. I've tried changing from delete_struct
to moveitem_struct, but can't seem to get the client to take charge change commands unless the client is the originator.


I think this is another area where CSDs could be creeping in..server tells client to delete 2 charges in a 5 charge stack,
but all are deleted..and the client generates a 'Move Item Failed' error.

I don't see where normal 'actions' are causing this, but it could creep up in scripted/quest calls.

Harakiri23 08-29-2012 09:58 AM

The client has 2 different opcodes for this - one is MSG_MoveCharges to move charges from a stackable item anywhere (or delete with 0xFFFF..) and one is MSG_MoveItem - which moves non-stackable items (or delete them), can also be used to "delete" a stack of items.

The opcode names for you may differ, but that is your issue to figure out the naming in eqemu.

image 08-29-2012 10:45 AM

have you tried OP_DeleteCharge instead?

Uleat 08-29-2012 01:21 PM

It's been a while since I messed with this portion of the code.

I can't remember what I did and did not try. But, when I get back to a point where I can, I will try both of those.

Thanks!

Uleat 09-01-2012 04:36 PM

I'm going to try and work on the CSD issues a little more. I think that I can manage keeping my home source code updated by manual file transfer.
Posting-to-reply times will be delayed by at least a day, however...

I've not heard any negative feedback on this CSD patch series, as yet. Just remember..if you are using the bandolier and become de-sync'd from another
unfixed bug, you can still lose items just as any 'normal' item swap would cause. So, keep an eye out for those de-sync messages and take corrective action
immediately. (I can't guarantee that you will receive a message on a de-sync that originates in code outside of this patch, so be careful out there with
bandolier use and any other action that creates/moves/deletes items without doing a zone change/re-log first.)


My next area to work on is code using inventory.cpp::Client::TryStacking(). I've noticed that the stack search order does not match
Client::MoveItemToInventory() or the client behavior that I noticed in the work with the Bandolier function. I need to find all of the ways that it is used in-game,
test them out, and verify client behavior while using them.

The two functions that call this procedure are Client::AutoPutItemInInventory() and Client::Handle_OP_ShopPlayerBuy(). ShopPlayerBuy() is a client event
driven call. The other function is called by Client::AdventureMerchantPurchase(), Client::AltCurrencyPurchase(), Corpse::LootItem() and Client::TryStacking()
(and any possible script calls.)

I will be looking through the forums in search of postings about these in-game actions. But, if you know definitively about de-sync's and item loss while using
these actions, please re-post them here. This can include mysterious item movements (especially involving the cursor) since players are likely to change
zones/log and restore syncronization before switching equipment around.


Thanks!

Uleat 09-03-2012 03:48 PM

Just a little note on item loss with overloaded inventory...


I do have a working theory on this one, but I HAVE to ensure that I can test every call to the affected code. (I'm assuming that I won't get any script calls
unless there is something defined to handle it. Otherwise, another function calls it.)

The problem likely lies in multiple areas, but the code in item.cpp::Inventory::FindFreeSlot() is most culprit. (Kinda the reverse issue that was in the Bandolier code.)

In the case of a corpse [Loot All]... When you have a full inventory (with an empty cursor), but still have occupied bags on your corpse:

- The next bag is pushed to SLOT_CURSOR and the contents are pushed to 331 - 340
- ALL subsequent bags are also pushed to SLOT_CURSOR, via the queue, and each bag's contents overwrite any previous bag's data in 331 - 340
- The bags themselves will end up in the cursor queue
- The contents of the first cursored bag will be:
-- The contents of the last corpse bag if it was full -or-
-- A mix of contents of both previous and last corpse bag items if it was not full
-- (I'll need to double-check, but I believe _putitem is used instead of _swapitem..which means existing items go bye-bye..changing it would create more problems)
- Any queued cursored bags will be empty

I'm not sure exactly what behavior would occur if a non-container item was first on the cursor since bag slots aren't moved in that case. A bag 'popping' into the
cursor slot 'may' avail any lost and un-deleted items in the 331 to 340 range. These items would now be assigned to that bag and any remaining bags would
still be empty.


This problem will probably arise with overloaded trades as well. I've seen posts regarding both loot all and trading with this issue.


Anyways... I can't seem to find an OP_Code that handles the 'Loot All' action. Does the client actually send a loot request op for each item in the corpse
window?

If it does, and the client DOES NOT move the items, we could cancel the transaction and leave the bags on the corpse. If the client DOES move the items
regardless, all we can do is minimize the damage..maybe by 'unpacking' the bags to the cursor queue, but we might have to bump up the active queue array
size. Dropping them to the ground would most likely lead to them being lost if the player can't clear their inventory to make room. (I guess we could unpack
the bags when the corpse is created as well... Wouldn't that be a mess...)

(What I've seen with the cursor queue size on the server and a SoF client: {client: 37; profile: 102; database: 1000})


(This conversation sounds familiar..sorry if I duped it from somewhere else...)

Uleat 09-08-2012 08:00 PM

Currently, I am working on a fix for item deletion during a de-sync condition.

It may, or may not, work depending on my success with manipulating the client...


Does anyone know of way to send a 'NULL' item instance to the client? I'm trying to avoid using 'OP_DeleteItem' because of the issues surrounding its use and
already empty slots. (I'd like to avoid ANY messages to the client with the exception of the one informing them of the 'Inventory Resync' itself.)

client->SendItemPacket(slot_id, item_inst, ItemPacketTrade);

If I can use this to 'empty' a slot, it would save a ton of overhead.

Uleat 10-12-2012 11:15 PM

I've bugged at least one of the dev's enough with pm's about this resync thing, so I'm taking it public.

I'm just getting back into this after a hiatus and wanted to get some feedback, if anyone has some good suggestions.


What I'm looking at doing is creating two structs for one opcode. First for incoming, and the second for outgoing.

MoveItem requires a 32-bit incoming because of the timestamp sent by the client. However, the client seems to crash when you send a 32-bit data value
(size-wise) to certain slots. Inventory appears to be ok, but bank and shared bank slots appear to be the cause of the crash.

BulkSendInventory uses a sint16 data value when sending initial equipment and I want to convert the outgoing MoveItem_Struct to sint16 as well... Obviously,
there is a lot that will require mod'ing, so I wanted to get some input from you guys before I devote that much time and effort into something that may not work.

Thanks!

lerxst2112 10-13-2012 12:46 AM

You can't just assume that because one opcode uses a particular sized variable that another would do the same.

You would need packet captures of all the cases you are trying to figure out and then you would need to reverse engineer the packets to determine the difference when moving to/from the problem slots.

Uleat 10-13-2012 07:01 PM

Yeah, I agree that would be the proper way to do it. I just don't have the means to actually do that and I don't want to involve anyone into my chaotic thought
processes beyond the Q & A at this point...

It could be that the client is not coded to handle server-originating move/delete OP's in those slots and is crashing instead of giving the normal 'failure' message.

Whether it's that or the struct size (or something else entirely,) I'd still like to pin-point the cause. The resync function works with the exception of those slots. If I
can figure this out, that would eliminate the de-sync'd item deletion completely.

The main problem with this concept is that I don't believe live servers or clients are coded to do what I'm trying to do..which is why I'm running into issues...
I believe that I was told there is a known issue with certain op code actions and slots, but they are rare in their occurrences.

I don't mind fruitless ventures as long as I can use it as a training exercise, so I'll see what becomes of it. If this does pan out, I'll see if I can get someone to
verify my results with a live packet collection.

Uleat 10-26-2012 11:51 PM

I'm not quite ready to play with adding a special out-going struct for move/delete item, so..two quick questions: (don't hit me Lerxst! :) )


1) The tooltip info for int32 says it's a typedef for an unsigned int32. However, MSDN .net (and/or silverlight) defines it as a signed int32...

Question: Does MSVC++ not define it the same way, is it re-defined somewhere in the code, or is this an issue between compilers (and languages?)


2) I had been playing around and discovered that a 16-bit value for 'to_slot' and/or 'number_in_stack' didn't crash the client (though it may not work...)

Question: Would a bad conversion in the client where an (unsigned) 32-bit (~0) is being forced to an (unsigned) 16-bit register cause the crash? (I'm thinking
that might be causing an overload, or at least data loss, if it's not that the client isn't coded to handle the request.)


Thanks!

lerxst2112 10-27-2012 01:02 AM

There is no such thing as an int32 in the C++ standard. In this project, as you've seen, it is a typedef, or alias, for unsigned int. It just happens to be 32 bits on the platforms the code is intended to compile on. A compiler could generate code using 64 bit integers and still be 100% standard compliant.

The packets sent between server and client are the result of packet capturing and reverse engineering. It is possible that some of the types used are not exactly the same as what the client expects, but the sizes of the packets are almost certainly correct which means changing values to a smaller type would result in an incorrectly sized packet and also offset anything that came after that value unless you add the appropriate amount of padding to match the size. If you believe a packet is being sent incorrectly the right thing to do is capture traces of those specific packets and check them against the existing code. Perhaps the packet you're looking at does expect 0xffff and not 0xffffffff in that position, but the only way to know for sure is to verify it against packets from live.

Uleat 10-28-2012 12:21 AM

Ok..duh on the int32. I didn't have my code def window up when I was looking or I would have found <types.h>

I am really leaning away from even testing a modified out-going structure at this point. My question about conversions was more about the possibility of
corrupted data when narrowing. I seem to get what I expect when using integers, so I'm guessing data loss is more prevelent with floats.


(Sorry for the re-iterations..I just want to clarify the actual issue.)

As far as capturing from live... I don't believe what I'm trying to do is ever done on live.


For instance:

When a player deletes a bank item, they first move it to their cursor. Then a client-originating moveitem deletes it (from->30: to->-1: num_in_stk->0)

When an item is on the cursor, we can issue a server-originating moveitem to delete, regardless of client action. (from->30: to->-1: num_in_stk->-1 seems to
work and is what is currently used in existing code.)

But there is no existing action (that I'm aware of) that will generate a server-originating direct bank slot deletion on live. All quest updates, tradeskill combines,
etc... come from the client's cursor or personal inventory (and bags), which I have no trouble deleting from.


That's why I don't think there's a way to actually capture this particular value for a direct bank slot delete action.

(I can overwrite the existing item with no problem. But, then we're left with a client that could have 'phantom' items in empty slots and we're back at starting
point for this whole resync business... I do have a method that will clear empty bank slots on the client, but the client will spam itself with 'deletion' messages
for each empty slot, which I'm trying to avoid.)

lerxst2112 10-28-2012 12:57 AM

If there's no way to legally generate a deletion from a bank slot then why are you worried about it? If it can't happen on live then it shouldn't happen on the emu. If it does, then that's the bug.

Uleat 10-28-2012 05:29 PM

This whole 'ResyncInv' thing is just a band-aid to keep issues at bay, and eliminate item deletions, until all of the code can be found and corrected that causes
desync's.

I'm pretty sure my work on the Bandolier patch alleviated it as being a source of desync's, but it can still faulter and delete items if the client was already in a
desync'd condition when the player uses it. Of course, any moveitem call would cause that under the same conditions.

Even less supportive: If a player were to switch items around in their bank while desync'd, they may travel half-way around the world and be in the middle of a
raid before they discover that the stack of poison-resist potions they thought they had is actually a stack of rabbit meat.

I'm just trying to ensure that this method actually does what it says, is all.

lerxst2112 10-28-2012 11:41 PM

Well, if the client does not handle it properly, then you're on a fool's errand.

Uleat 10-30-2012 03:16 PM

Lol! You have no idea Lerxst!

My past life incarnations include: village idiot, court jester, and dancing fool...

Uleat 11-02-2012 10:04 PM

The resyncinv method does appear to do what I want it to, so hopefully all that is left is tweaking it.


I'd like to move on to the remaining issues, but I still need some help with the 'Loot All' feature in clients > SoF.

Specifically:

- What OP_Code is used for this action?

- Does the client send a request for each item in the loot window, or just one?

- Does a failure on the server cancel the client action, or does it continue?

- When an character's inventory is 'over-loaded,' does the corpse still have items on it, and then a re-loot shows it empty?


Any help would be appreciated. With some basic behavior knowledge, I might be able to figure out what exactly is wrong with this process.

Thanks!


(sidenote: I know SOE doesn't condone/support eqemu..but I wonder if they could be convinced to allow Steam to sell the UF client again. It wouldn't hurt
their current online offering and it would bring in a little more revenue from an obselete piece of software...)

Uleat 11-05-2012 12:21 PM

Two quick questions...

First:
Is there something wrong with this? (trying to track down inventory issues)
Code:

uint8 ItemInst::FirstOpenSlot() const {

        int8 slots=m_item->BagSlots,i;

        for(i=0;i<slots;i++) {
                if (!GetItem(i))
                        break;
        }

        return (i<slots) ? i : 0xff;
}

I though that I had read that assignments were not allowed in multi-declaration statements.

Is that true? Or, was it true in the past?


Second:
Is there a problem with this delegate?
Code:

static std::list<type> Class::Function(type arg);
It compiles with no errors, but something I did is crashing zone after I select the character. I can trace the cause by process of elimination, but re-compiling takes
time. If there's a simple answer...


Thanks!


All times are GMT -4. The time now is 09:00 AM.

Powered by vBulletin®, Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.