Long #repop and zone times after source upgrade
 

I am banging my head on this one. I noticed an error in the log about being unable to load plugins.pl not sure if that has anythng to do with it because my plugins all seems to work. #repop used to start working right away. After updating to newest source i started getting these problems.

I updated to the latest source this morning, and have not seen any errors so far on my test server. #repop is instantly on my box

Plugins.pl has nothing to do with your problem.

What version/Date of source were you previously running and which one are you using now? Are you seeing a lot of chat messages in game? If you have a lot of logging turned on, I have found that it tends to start queuing up and #commands aren't very responsive or won't work at all if you are getting spammed too much.

I haven't seen any issues with #repop or zoning other than when I had turned on perl debug logging and was using a massive loop that was sending non-stop quest::debug() messages to my chat window.

I tried recompiling to latest source just now. Still have repop issue.

I just setup a fresh server too and migrated my custom DB info to it and effectively did a playerdata content wipe.

In CB for example i have 200 mobs about, it takes ~20 seconds to repop and during that time players will disapear briefly, MQ2EQBC drop its connection, and / commands dont work.

I don't however get a lot or any abnormal amounts of debug info in the chat box, just occasional stuff.

@Trev Where would I go to check debug logging? As far as source goes, my last updated was about a week ago. I usually try and update it once a week.

Ah, that explains a lot! Thanks.

I tried renaming the logs, quests, and plugins folder to try and sniff out the problem. None of which had any significant effect on the problem.

14 Sec loading without logs, quests or plugin folder.
23 Sec Loading with log folder, quests, and plugin.

In my log folder under subdir called 'zone' i am getting big log files like 10mb to 30mb for a specific zone.

What is weird though is when the quest is commented away that loads the weapon plugin random weapons on spawn it still has a substantial lag in popping. (The script used to work with instant popping without issues)/

One last thing... I looked at my log.ini file in my server directory and everything there is set to = off

So no idea why logs are even trying to be generated.

On my client logs i get this as the only event that logs during a repop

If you read the changelog.txt you'd know that the logging system got an absolute overhaul:

http://wiki.eqemulator.org/p?Logging_System_Overhaul

Thanks for pointing that out.

So using CB as an example with ALL logging off repop is now about 12-14 sec. So i shaved off that extra bit i was getting with logging.

Another zone with signficantly less NPC ~30 repops in about 4 sec down from ~7.

This is definitely an improvement but nowhere as close to the instant repopping I enjoyed before. Any additional holes you can think of i can run down?

Make sure this is unchecked in your CMAKE:

http://i.imgur.com/fIwvVfd.png

I recompiled after making the change in CMAKE and that didn't really seem to make a measurable difference. Repop times aren't terrible but on a zone with only 30 mobs it seems to me I remember repopping being faster than 4 sec / 12 sec per zone respectively.

Well there are varying factors but my repops are instant.

Hmmm, the server machine should be well beyond the specs needed.
W8.1 x64 with 8gb of RAM and dual core 3.2ghz. I will have to setup a new test server and experiment more. If you think of anything please dont hesitate to drop by and leave a suggestion. I do miss instant repop.

Paste your log table settings

Right now everything is set to 0.

I noticed the quest logging was taking up a huge amount of time.

Throwing up errors for this code in global_NPC.pl

Keep in mind before this would add maybe 1 sec to repop times. Purpose is to give non named NPC random weapons. The quest logging under zone is making huge 30mb to 100mb files

It also spits out in blocks of PrimTYpe and Sec type not just alternating. With the quest code disabled that is where i get the 10 sec drop (writing to HD). Not sure how i can fix that error with weapons other than not logging (not ideal). But there still seems to be a missing chunk of time that scales based on # of NPCs in zone. 30 NPC = 4 sec 200 = 14sec.

I know you said everything is set to 0, but do you even have quests set to 0?

If you're getting that size of quest messages your file writing is going to slow you down big time

I literally have every field in the log control table set to 0 and restarted the server.

In North Karana I get a 12 sec repop time.

If you are compiling with VS2013 ultimate, you can profile your zone process while you are repopping and figure out what is chewing up CPU cycles by taking so long

VS is still installing. Keep in mind all logging is off but it seems like i am getting a hard disk write spike during repop. The spike goes to about 350kb/s not very high but it sustains through the repop. The moment a mob shows up the spike drops. CPU doesn't seem to change.

http://i.imgur.com/SPhLts0.png?1

OK, then what is being written when you open up resource monitor

mysqld.exe is the culprit it looks like. Still trying to write with logging disabled somehow?

If it is truly mysql, then while you repop do a:

#logs gmsay 36 1

And you can get an idea of what queries are trying to write.

You shouldn't be pushing MB/s of data on write

Not sure if this matters. But when i tried to use #logs i get the context for the command but if i do #logs gmsay 36 1 i dont get a response and the DB doesn't change. I had to manually change the DB. Any chance that might be related to the cause of problems. I will double check my perl to make sure its the one for plugins.

I meant:

#logs set gmsay 36 1

Potential problem with Tasks?

EDIT:: character_enabledtasks is empty

EDIT2:: Looking at a simple repop SQL seems to be doing about 60 records a second a second on repop

EDIT3::Not sure if it matters but i did double check perl and the #logs set gmsay 36 <1/0> gives feedback for enabling or disabling but doesn't actually change the DB.

Could this have something to do with xampp versus installing mysql standalone? I know on my old comp I never had this issue either. But now on the a fresh install of everything on my new pc, its taking 21-25 seconds to perform #repop

*highfive* brother we aren't alone!~ I did install XAMP but i skipped mysql and used mariadb10.0 instead

I will mess around with XAMPP and see if anything might be messing it up

EDIT:: Then again maybe not.
http://i.imgur.com/SgTYBPa.png

@Akka

I am setting up a fresh W8.1 Box on a XEON 4gb machine Let me know if there is anything outside of the normal windows server install setup guide you want me to do.

You've never really said anything about what your using for disks, since I got home and was able to check I could see my writes were happening like that because its DELETING and REPLACE INTO constantly which is kind of horrid.

I have a M.2 SSD so my read speads are up 1000MB + and latency is nuts, so my repop's are pretty quick.

But, since repops happen potentially hundreds of times during the developing process, I'm going to look at getting them sped up quite a bit by optimizing the database end. I think everyone would appreciate this as a quality of life enhancement.

I will also make note that if you are using windows and you have any logs to 'console' enabled for MYSQL queries, that will definitely slow things down. Same thing for Perl, outputting to windows console is really slow.

If you have a shit load of quest errors like you posted before, and they are going to windows console, that is your problem.

Everything that we have enabled by default going to console is usually used sparingly given the cost consideration of windows console being slow.

If you want to see quests errors, I would highly recommend using gmsay because that is going to be a much faster medium to see realtime than the console

Did you make any changes? I recompiled this morning with latest source 8 files changed 169 insertions 101 deletions and repops seem to be a lot faster already

Zone with 30 mobs is like 1-2 sec CB with 200 is like 2-4 so really great job (i assume this was you)

Yep, was me. Have more changes to make too.

Did some work to get these cut down dramatically to pretty much instant, will submit them shortly. I'm sure many will love this.

If only it were a smidgen faster. o.O :p

You sir are awesome. I know people call you the man and give you unlimited accolades of worship, but you are amazingly cool and helpful. Reading your old posts when you were a newb gives me hope I will get better and be able to contribute more in the future, thanks Akka.

No problem, everyone starts somewhere.