Login Server Disconnects

[provocating]

Anyone know what would cause this? All four of my servers are setup very similar. I have covered my basis by checking resolv.conf, my interfaces, iptables and such. Iptables is very similar between Chronicles of Norrath and Legacy of FrostStone. The login server is on the Chronicles of Norrath VM and three servers plus a test server all share that one login server. Only one server has this disconnect issue, that is Legacy of FrostStone. It disconnects every few minutes and then reconnects. I would never post an issue here unless I had exhausted every single possibility. Since all the servers are near identical I am at a loss why just one would have these issues.

Here is a snippet.

[Network] [10.14.15 - 22:45:35] New world server connection from 192.168.0.20:48000
[Network] [10.14.15 - 22:45:35] World server already existed for 192.168.0.20, removing existing connection and updating current.
[Network] [10.14.15 - 22:45:36] World server disconnected from the server, removing server and freeing connection.
[Network] [10.14.15 - 22:45:40] New world server connection from 192.168.0.20:48003
[Network] [10.14.15 - 22:45:41] World server disconnected from the server, removing server and freeing connection.
[Network] [10.14.15 - 22:45:56] New world server connection from 192.168.0.20:48020
[World] [10.14.15 - 22:45:56] Server Legacy of FrostStone(Legacy of FrostStone) successfully logged in.
[Network Trace] [10.14.15 - 22:45:56] ServerOP_LSAccountUpdate sent to world
[Network] [10.14.15 - 22:46:01] New world server connection from 192.168.0.20:48023
[Network] [10.14.15 - 22:46:01] World server already existed for 192.168.0.20, removing existing connection and updating current.
[Network] [10.14.15 - 22:46:01] World server disconnected from the server, removing server and freeing connection.
[Network] [10.14.15 - 22:47:57] New world server connection from 192.168.0.20:48037
[World] [10.14.15 - 22:47:57] Server Legacy of FrostStone(Legacy of FrostStone) successfully logged in.
[Network Trace] [10.14.15 - 22:47:57] ServerOP_LSAccountUpdate sent to world
[Network] [10.14.15 - 22:48:29] New world server connection from 192.168.0.20:48040
[Network] [10.14.15 - 22:48:29] World server already existed for 192.168.0.20, removing existing connection and updating current.
[Network] [10.14.15 - 22:48:29] World server disconnected from the server, removing server and freeing connection.
[Network] [10.14.15 - 22:49:25] New world server connection from 192.168.0.20:48059
[Network] [10.14.15 - 22:49:26] World server disconnected from the server, removing server and freeing connection.
[Network] [10.14.15 - 22:49:29] New world server connection from 192.168.0.20:48062
[Network] [10.14.15 - 22:49:29] World server disconnected from the server, removing server and freeing connection.
[Network] [10.14.15 - 22:49:57] New world server connection from 192.168.0.20:48064
[World] [10.14.15 - 22:49:57] Server Legacy of FrostStone(Legacy of FrostStone) successfully logged in.
[Network Trace] [10.14.15 - 22:49:57] ServerOP_LSAccountUpdate sent to world
[Network] [10.14.15 - 22:52:09] New world server connection from 192.168.0.20:48091
[Network] [10.14.15 - 22:52:09] World server already existed for 192.168.0.20, removing existing connection and updating current.
[Network] [10.14.15 - 22:52:09] World server disconnected from the server, removing server and freeing connection.

EDIT: I just noticed one thing. When I setup Legacy of FrostStone it setup both nics as VMXNET 3 instead of e1000, which I normally use. I know the VMXNet3 nics are supposed to have better performance, but the e1000 driver is supposed to have better compatibility. I will change them when the players all log out. I am not optimistic that this is the issue though.

[Noport]

If using old bnc network cards replace the 50 ohm also check cat5 cables. I have an e1000 as well single high with slow data exchange. All else fails replace network card. When we setup class F network servers we had a fail safe server to take over if server fails. do you have access to a main internet router that routes the traffic if that was reset by you or your isp it might take 15-30min to send the packets out. i ran a whois on the servers no return went to another site ran whois gave me ns infomation. Check Power Options in advanced mode check power and sleep timers. just did a traceroute on server dsl ip.att.net request time out found you main route kc1 level 3 sent up north where our main internet routers was setup. If it's not on your end talk with your isp att trace route time out.

[provocating]

Originally Posted by Noport

If using old bnc network cards replace the 50 ohm also check cat5 cables. I have an e1000 as well single high with slow data exchange. All else fails replace network card. When we setup class F network servers we had a fail safe server to take over if server fails. do you have access to a main internet router that routes the traffic if that was reset by you or your isp it might take 15-30min to send the packets out. i ran a whois on the servers no return went to another site ran whois gave me ns infomation. Check Power Options in advanced mode check power and sleep timers. just did a traceroute on server dsl ip.att.net request time out found you main route kc1 level 3 sent up north where our main internet routers was setup. If it's not on your end talk with your isp att trace route time out.

These are all VM's. All of the servers are on the same host.

[provocating]

I have narrowed it down to not being the host based driver (VMXNET3). So my next step is probably going to be having the login server connection going out the wan side and not the lan side. It could at least help narrow the issue down a bit further.

[RussellMatt27]

Can you run a capture (Wireshark or similar) from the system having the issue? I'd wonder if its not getting some return data to maintain the connection within VMWare or whatever virtualization you use.

[provocating]

I have spent many hours on this and have things that I know it is not. I am just trying to narrow it down or I will keep spinning my wheels on this. I know that is is not related to that particular I.P. address for LoF, I changed the private address this morning. I know it is not the actual server binaries, the developmental server is running the same and it does not have issues. I know it is not the iptables firewall, I checked the other servers and they are identical.

I am nearly positive it is not the operating system, I fired up the login server on Legacy of FrostStone instead of it being on Chronicles of Norrath, the problem is still there. I went ahead and configured the other servers to use the Legacy of FrostStone login server instead and they stay attached, the LoF server constantly disconnects. Here is a debug log. You can see the server is attached and then reattaches.

[Network] [10.16.15 - 13:07:28] Application packet received from client (size 12)
   0: 04 00 00 00 00 00 00 00 - 00 00                    | ..........
[Network] [10.16.15 - 13:07:28] Server list request received from client.
[ ... ]
[Network Trace] [10.16.15 - 13:07:35] Application packet received from server: 0x1001, (size 12)
   0: 00 00 00 00 00 00 00 00 - 0A 00 00 00              | ............
[Network Trace] [10.16.15 - 13:07:35] World Server Status Recieved.
[Network Trace] [10.16.15 - 13:07:36] Application packet received from server: 0x1001, (size 12)
   0: 01 00 00 00 01 00 00 00 - 3D 00 00 00              | ........=...
[Network Trace] [10.16.15 - 13:07:36] World Server Status Recieved.
[Network Trace] [10.16.15 - 13:07:38] Application packet received from server: 0x1001, (size 12)
   0: 0B 00 00 00 0B 00 00 00 - 41 00 00 00              | ........A...
[Network Trace] [10.16.15 - 13:07:38] World Server Status Recieved.
[Network] [10.16.15 - 13:07:40] New world server connection from 192.168.0.20:54232
[Network] [10.16.15 - 13:07:40] World server already existed for 192.168.0.20, removing existing connection and updating current.
[Network] [10.16.15 - 13:07:40] World server disconnected from the server, removing server and freeing connection.
[Network Trace] [10.16.15 - 13:07:50] Application packet received from server: 0x1001, (size 12)
   0: 00 00 00 00 00 00 00 00 - 0A 00 00 00              | ............
[Network Trace] [10.16.15 - 13:07:50] World Server Status Recieved.
[Network Trace] [10.16.15 - 13:07:53] Application packet received from server: 0x1001, (size 12)
   0: 0B 00 00 00 0B 00 00 00 - 41 00 00 00              | ........A...
[Network Trace] [10.16.15 - 13:07:53] World Server Status Recieved.
[Network Trace] [10.16.15 - 13:08:05] Application packet received from server: 0x1001, (size 12)
   0: 00 00 00 00 00 00 00 00 - 0A 00 00 00              | ............
[Network Trace] [10.16.15 - 13:08:05] World Server Status Recieved.
[Network Trace] [10.16.15 - 13:08:08] Application packet received from server: 0x1001, (size 12)
   0: 0B 00 00 00 0B 00 00 00 - 41 00 00 00              | ........A...
[Network Trace] [10.16.15 - 13:08:08] World Server Status Recieved.

[provocating]

I also notice that you will see the server trying to log in, but it is already logged in. The login server will clear the connection, you will get a bunch of 0x3003 for a while and then this.

[Network Trace] [10.16.15 - 13:18:52] Application packet received from server: 0x1008, (size 653)

And that point the server connects again with it's credentials.

[rhyotte]

http://www.servethehome.com/cheap-10...nectx-2-cards/


Just a heads up....

[provocating]

This is all virtual, no way it is a nic.

[rhyotte]

Ah. It will be interesting to see what it ends up being. I have not rolled a VM Server yet, but the next one I tinker with might very well be.

[provocating]

I am taking good notes on what I have checked. I have narrowed down so much now, there is not much left. The strangest thing is that the firewall configs are exactly the same between the guest vm's, the operating systems are all the same version of debian. The are all VM's on the same host, same EqEmu binaries on two of the guest, a third is damn well close. But nonetheless, one server is constantly reconnecting to the login server.

[Noport]

Your welcome glad to help

0x1001 Too many applications
0x3003 B3 protocol not supported caused by computer overload at the remote end or an invalid configuration, locally or remotely.   
0x1008 OS Resource error (e.g. no memory)

OS Resource error Communication Error Codes 15 Code (Hex) Code (Dec)
Download pdf book error codes inside 
http://www.datainterchange.com/Downloads/Support/Manuals/EPIC/VM-0001-09%20Communications%20Error%20Codes.pdf

[provocating]

Originally Posted by Noport

Your welcome glad to help

0x3003 B3 protocol not supported
0x1008 OS Resource error (e.g. no memory)

OS Resource error Communication Error Codes 15 Code (Hex) Code (Dec)
Communication Error Codes VM -0001 -09    0x1008 OS Resource error (e.g. no memory)
Download pdf book error codes inside 
http://www.datainterchange.com/Downloads/Support/Manuals/EPIC/VM-0001-09%20Communications%20Error%20Codes.pdf

Awesome, let me get to reading.

[Uleat]

Any way that you could 're-arrange' the order/assignment of the servers?

That way if a different server goes down..but, on the same node as before..it might point to something more specific?


EDIT: I was heading towards a gc issue..but, Noport is probably spot-on.

[provocating]

Since I know it is not a source/binary issue nor anything related to vSphere could this be nothing but a Debian package issue? Although they are the same major revision, LoF is a more up to date. I think LoF is 7.8 but CoN is 7.2