World Crashes with latest source
Just curious if anyone is running the latest source and has noticed any world crashes from Heap corruptions. Below is crash dump.
[04-04-2015 :: 19:31:41] [Crash] EXCEPTION_BREAKPOINT [04-04-2015 :: 19:31:41] [Crash] SymInit: Symbol-SearchPath: '.;V:\EQ\EQEmuServer;V:\EQ\EQEmuServer;C:\Windows; C:\Windows\system32;SRV*C:\websymbols*http://msdl.microsoft.com/download/symbols;', symOptions: 530, UserName: 'Administrator' [04-04-2015 :: 19:31:41] [Crash] OS-Version: 6.2.9200 () 0x110-0x3 [04-04-2015 :: 19:31:41] [Crash] V:\EQ\EQEmuServer\world.exe:world.exe (00E60000), size: 7798784 (result: 0), SymType: 'PDB', PDB: 'V:\EQ\EQEmuServer\world.exe' [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\ntdll.dll:ntdll.dll (77460000), size: 1499136 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\ntdll.dll', fileVersion: 6.3.9600.17668 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\KERNEL32.DLL:KERNEL32.DLL (76C40000), size: 1310720 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\KERNEL32.DLL', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\KERNELBASE.dll:KERNELBASE.dll (751F0000), size: 880640 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\KERNELBASE.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Perl\bin\perl512.dll:perl512.dll (28000000), size: 1273856 (result: 0), SymType: '-exported-', PDB: 'C:\Perl\bin\perl512.dll', fileVersion: 5.12.3.1204 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\WS2_32.dll:WS2_32.dll (77120000), size: 327680 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\WS2_32.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\VERSION.dll:VERSION.dll (74600000), size: 32768 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\VERSION.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\USER32.dll:USER32.dll (74CC0000), size: 1388544 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\USER32.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\ADVAPI32.dll:ADVAPI32.dll (76BC0000), size: 507904 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\ADVAPI32.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.9600.17415_none_7c5 769eaaa0aa358\COMCTL32.dll:COMCTL32.dll (748D0000), size: 561152 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.9600.17415_none_7c5 769eaaa0aa358\COMCTL32.dll', fileVersion: 5.82.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\MSVCRT.dll:MSVCRT.dll (76F00000), size: 798720 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\MSVCRT.dll', fileVersion: 7.0.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\NSI.dll:NSI.dll (74E70000), size: 28672 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\NSI.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\RPCRT4.dll:RPCRT4.dll (76FD0000), size: 761856 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\RPCRT4.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\GDI32.dll:GDI32.dll (74FB0000), size: 1105920 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\GDI32.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\sechost.dll:sechost.dll (74E20000), size: 266240 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\sechost.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\SspiCli.dll:SspiCli.dll (74C60000), size: 122880 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\SspiCli.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\CRYPTBASE.dll:CRYPTBASE.dll (74C50000), size: 40960 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\CRYPTBASE.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\bcryptPrimitives.dll:bcryptPri mitives.dll (74BF0000), size: 344064 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\bcryptPrimitives.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\system32\IMM32.DLL:IMM32.DLL (75170000), size: 159744 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\system32\IMM32.DLL', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\MSCTF.dll:MSCTF.dll (74E90000), size: 1122304 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\MSCTF.dll', fileVersion: 6.3.9600.17664 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\system32\napinsp.dll:napinsp.dll (74BD0000), size: 69632 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\system32\napinsp.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\system32\NLAapi.dll:NLAapi.dll (74BB0000), size: 81920 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\system32\NLAapi.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\System32\mswsock.dll:mswsock.dll (74B40000), size: 307200 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\System32\mswsock.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\DNSAPI.dll:DNSAPI.dll (74AC0000), size: 516096 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\DNSAPI.dll', fileVersion: 6.3.9600.17481 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\System32\winrnr.dll:winrnr.dll (749A0000), size: 40960 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\System32\winrnr.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\System32\rasadhlp.dll:rasadhlp.dll (745F0000), size: 32768 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\System32\rasadhlp.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\IPHLPAPI.DLL:IPHLPAPI.DLL (74B90000), size: 131072 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\IPHLPAPI.DLL', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\WINNSI.DLL:WINNSI.DLL (74990000), size: 32768 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\WINNSI.DLL', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\System32\fwpuclnt.dll:fwpuclnt.dll (745A0000), size: 286720 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\System32\fwpuclnt.dll', fileVersion: 6.3.9600.17485 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\system32\uxtheme.dll:uxtheme.dll (73FB0000), size: 970752 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\system32\uxtheme.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\combase.dll:combase.dll (766B0000), size: 1560576 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\combase.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\system32\dwmapi.dll:dwmapi.dll (74610000), size: 106496 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\system32\dwmapi.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\system32\ole32.dll:ole32.dll (752D0000), size: 1212416 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\system32\ole32.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\kernel.appcore.dll:kernel.appc ore.dll (744C0000), size: 36864 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\kernel.appcore.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] C:\Windows\SYSTEM32\dbghelp.dll:dbghelp.dll (730A0000), size: 1314816 (result: 0), SymType: '-exported-', PDB: 'C:\Windows\SYSTEM32\dbghelp.dll', fileVersion: 6.3.9600.17415 [04-04-2015 :: 19:31:41] [Crash] f:\dd\vctools\crt\crtw32\misc\dbgheap.c (1376): _free_dbg_nolock [04-04-2015 :: 19:31:41] [Crash] f:\dd\vctools\crt\crtw32\misc\dbgheap.c (1265): _free_dbg [04-04-2015 :: 19:31:41] [Crash] f:\dd\vctools\crt\crtw32\misc\dbgdel.cpp (54): operator delete [04-04-2015 :: 19:31:41] [Crash] f:\dd\vctools\crt\crtw32\heap\delete2.cpp (21): operator delete[] [04-04-2015 :: 19:31:41] [Crash] v:\eq\source\common\servertalk.h (206): ServerPacket::~ServerPacket [04-04-2015 :: 19:31:41] [Crash] ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 00F5AC76) [04-04-2015 :: 19:31:41] [Crash] 00F5AC76 (world): (filename not available): ServerPacket::`scalar deleting destructor' [04-04-2015 :: 19:31:41] [Crash] v:\eq\source\world\zoneserver.cpp (1322): ZoneServer::Process [04-04-2015 :: 19:31:41] [Crash] v:\eq\source\world\zonelist.cpp (98): ZSList::Process [04-04-2015 :: 19:31:41] [Crash] v:\eq\source\world\net.cpp (464): main [04-04-2015 :: 19:31:41] [Crash] f:\dd\vctools\crt\crtw32\startup\crt0.c (255): __tmainCRTStartup [04-04-2015 :: 19:31:41] [Crash] f:\dd\vctools\crt\crtw32\startup\crt0.c (165): mainCRTStartup [04-04-2015 :: 19:31:41] [Crash] ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 76C57C04) [04-04-2015 :: 19:31:41] [Crash] 76C57C04 (KERNEL32): (filename not available): BaseThreadInitThunk [04-04-2015 :: 19:31:41] [Crash] ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 774BB54F) [04-04-2015 :: 19:31:41] [Crash] 774BB54F (ntdll): (filename not available): RtlInitializeExceptionChain [04-04-2015 :: 19:31:41] [Crash] ERROR: SymGetLineFromAddr64, GetLastError: 487 (Address: 774BB51A) [04-04-2015 :: 19:31:41] [Crash] 774BB51A (ntdll): (filename not available): RtlInitializeExceptionChain |
Haven't seen anything on my Linux box, so I updated and booted up my Windows server. Haven't seen this yet. Looks like it was trying to delete pack when it wasn't there (already deleted in the switch?). I'll leave that to the more experienced to look at. Can you look in your world log to see if there are any entries around that same timestamp?
EDIT: If you see it more, we may just check for existence of pack before deletion at the end of that process. |
I have some extra logging on, and when the next crash happens, I will have more data
|
Mine is crashing every 3-5 days with the same error from a February version, so that crash has been in since at least Feb 25th.
|
Try the latest push. If it was the cause of the crash, you will no longer see it. There will also be an entry in your world logs so we can catch it. Report here if you see the entry or if you still see related crashes.
|
Thanks joli. Will test this pull in the next 2 hours
|
Ok recompiled and its up and running, if I see a crash happen again, will post the details in this thread
|
Will make one more change this evening. Demonstar brought up a great point about delete.
|
Joli, seen some changes this morning from the pull. Will update the source during my downtime today. Had another crash today from yesterdays source.
Below is crash data: Code:
[04-08-2015 :: 10:47:39] [Crash] EXCEPTION_BREAKPOINT Code:
[04-08-2015 :: 10:43:52] [Zone Server] Time Broadcast Packet: EQTime [02:17 pm] |
Nothing in world log? Time of crash is 33s after zone shutdown.
|
nope, just normal text when people zone or log in. Turned on more logging and see if we get another one. Currently running from this mornings pull
|
Update Joli, its been 2.5 days since updating to the source on the 8th. No crashes as of this time
|
Nice! Good to hear.
|
crashes are back
Been encountering lots of crashes of the past few days.
Code:
[04-19-2015 :: 22:24:19] [Crash] EXCEPTION_BREAKPOINT |
Same heap error every time. I will screen shot it next time. Running Windows 2012r2
|
https://github.com/EQEmu/Server/blob...rver.cpp#L1409
The 'pack' destructor is called with this action: https://github.com/EQEmu/Server/blob...vertalk.h#L206 But, the pBuffer was referenced here: https://github.com/EQEmu/Server/blob...rver.cpp#L1384 And is being called through 's' after the pBuffer allocation is released (here: https://github.com/EQEmu/Server/blob...rver.cpp#L1412) |
Thanks Uleat. Will test today.
|
Pushed update to address this.
|
Thanks J!
My head is still wonky and I don't touch the code when it's like this..for obvious reasons... |
Server is updated with new source, been running about 6 hours so far, no errors yet
|
Another crash just happened.
Code:
[04-21-2015 :: 17:52:44] [Crash] EXCEPTION_BREAKPOINT http://i.imgur.com/uNuRO0c.jpg |
Completely different issue, btw.
EDIT: Do you know what client was drunk and what they were /saying? EDIT2: I guess it could be language training too. |
Any chance that you could grab the message that is being Garbled?
I'm willing to say that it could be part of the 'delimiter' code..but, I'd like to see if your trying to assert on a character that is out of range first (like a character value above 127.) |
Uleat, I was also curious if it could be but I couldn't break the delimiter code. You may be right about the non-standard ASCII values, though.
|
I found a similar issue regarding an 'isdigit' call:
https://connect.microsoft.com/Visual...debug-compiles In particular: "The assert ensures EOF (-1) and the values representable by an unsigned char (0 to 0xFF) are the only values allowed as input to this function. To ensure the implicit cast to int results in a valid value, be sure to cast char values to unsigned char." I'll push a fix for this..but, we may need to keep an eye on future crashes - if it is relevant. EDIT: Ok, pushed a fix for this particular case. If you run across any more, or if this causes issues, please post back. |
Will keep this thread updated for future crashes. Also, I did see a garbled message right before the crash this morning. Character was talking to me in /say and this text showed all boxes [] <----like this, but about 20 of them
|
Hey Uleat, noticed the changelog, and updated to that new source, no errors so far
|
So far so good. No more crashes.
|
Have you had any more occurrences (or reports of) text coming in with the undisplayable characters (boxes)?
(Thanks for the update, btw :D ) |
Yes, I witnessed it once yesterday, but it did not crash zone.exe like it had in the past.
|
Any info on the source of that text would be greatly appreciated :)
If it's quest or text link-related, we can look at server code..as well as any nullptr or unallocated references - you may see 'stripping' in that case (something like '00 CC 00 CC' or whatever variant the os uses for deallocation marking.) If it's being generated by the client itself (no packet injections,) I can test that with two clients and changing the default font..and maybe some other testing. |
it was in a tell. The character they were trying to type was this :)
|
Had another crash apparently this morning in Katta.
Code:
[04-27-2015 :: 09:14:25] [Crash] EXCEPTION_ACCESS_VIOLATION |
and the zone file.
Code:
[04-27-2015 :: 07:42:02] [Zone Server] Time Broadcast Packet: EQTime [01:40 am] |
This is a crash happening on zone shutdown.
Not really anything that is going to affect much. We've been fighting zone cleanup on shutdown for a while. Though, most of the frequent crashes have been fixed up it doesn't affect a zone process with any players |
yeah, it used to cause heap error and crash the server, now it just crashes that zone and the zone restarts
|
Quote:
|
another crash today
Seen a few crashes today.
Code:
[05-03-2015 :: 16:20:03] [Crash] EXCEPTION_BREAKPOINT |
All times are GMT -4. The time now is 09:02 AM. |
Powered by vBulletin®, Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.