Ok after further investigation I think I found the problem, but I don't know how to fix it.
Here's the outplog of my EQ Client - (well the end where it broke off the connection)
2008-06-19 08:48:07	Zone Connect -- 0 -- Received MSG_ZONE_ADDRESS
2008-06-19 08:48:08	Zone addr [##.##.107.142:23579] received...
2008-06-19 08:48:08	ZONING
2008-06-19 08:48:08	Networking: Connection Closed [0] with 0 pending bytes.
2008-06-19 08:48:08	Networking: using port [2283].
2008-06-19 08:48:08	Networking: Connection Established [1]
2008-06-19 08:49:08	Could not connect to ##.##.107.142:7004 [client

isconnectReasonConnectFail,server

isconne  ctReasonNone].  Negotiation count: 597.
2008-06-19 08:49:08	
2008-06-19 08:49:08	Networking: Connection Closed [0] with 0 pending bytes.
2008-06-19 08:49:08	Failed to connect to zoneserver (##.##.107.142, port 7004), result = 1, for MSG_ZONE_ADDRESS.
2008-06-19 08:49:08	
2008-06-19 08:49:08	THE SERVER IS NOT RESPONDING.
2008-06-19 08:49:08	
2008-06-19 08:49:08	end of DisplayScreen
2008-06-19 08:49:08	
2008-06-19 08:49:10	disconnected at C:\EQ\EverQuest\EverQuest.cpp:17153 (char. select) g_world == NULL
It looks like the client is able to connect to the server, but is not able to connect to the zoneserver. I have checked my config, but everything lines up and I have the correct ports forward. Would a antivirus program interfere with the connection perchance?
Im running Windows Vista (where my server is) with Symantec Endpoint, and I can connect to the server just fine when behind my router. Im forwarding Ports 7000-7100, 9000, 9080, 5998-5999 and a few others too I think just to be safe.
Any suggestions?