Live Data Collection (Need Community Help)

[Akkadius]

I'm going to make this short and sweet.

We have made huge ground parsing Magelo's site for large amounts of data to catch up PEQ all the way through the current expansion. I spent time to write a web crawler that has gathered all of the following data types from all expansions to now:

• Merchantlists
• Loot
• Tradeskills
• Spawnpoints

I will be working on getting these in one by one, my life has been super busy and it has been hard to make time for things but I'm still trying to get these things knocked out because they are rather large. I also am not rushing because Cavedude is in the middle of moving, I am probably going to end up moving here shortly myself.

Asking the Community for Help:
Anyways, I need help from the community to collect logs from live. These are the logs that are generated from /log in-game from any and all time frames that have ever been played.

What will this enable us to do?
We will be able to collect quite a wide variety of data that just simply can't be automated through web sources. This will allow us to collect:

• NPC Relative Health
• NPC Spell Casts
• NPC Faction hits
• NPC Damage Data
• NPC Aggro Emotes
• NPC Death Emotes
• Other things I'm not thinking of right now

If we can get log data from PEQ's more recent expansions (GoD, LDON) all the way through the current expansion the more we will be able to fill in the gaps of all of the data that gets manually processed through the development procedures and standards that PEQ has put in place today.

My effort is going to be to take logs from players anonymously through a form and have a script go through and tear through the data existing through those logs. These logs will be kept privately and will only be used for the benefit of this community and will only be seen by trusted developers.

The UPLOAD form can be found here: http://eoc.akkadius.com/eoc/EQEmuDat...gCollector.php

Simply upload a eqlog file of your EverQuest live characters and once the data parsing process begins, I will try to make a way to have people see what NPC's have been collected in what zones and what hasn't collected so we can take an accurate hitlist against what we need to collect as a community.

This will take a while as we get people to dump their historic logs from the past, I encourage everyone to reach out to anyone they might know to help this process move along so we can get more data.

How/What:

• Typing /log in game in eqemu will create a logfile under EverQuest/Logs/eqlog_charname_servername.txt
• We will need preferably HUGE logs that contain data spanning several zones, old and big logs are completely fine
• The more combat spam the better, I don't care about /gu or /t messages, I won't be caring about any of that

If anyone has any questions about this please feel free to post.

[namini]

Oh boy. Logs, if I can find the old hard drives that I used to play and guide on, shoot. We're talking a crazy amount. But, yeah.

Short and sweet answer, you need logs, from what expansion to which expansion?

[Noport]

what format would you like to receive them in ? what is the max size allowed ? are you talking about the log files an or are you asking for pcap files ? i have several live logs i started in 2003-present

[sorvani]

we are talking about standard log files created by the /log on command.

any and all logs are helpful, but mostly from GoD+ as everything else is already "completed" more or less. Akka will have to respond to file size restrictions if any.

[Akkadius]

The default format in which they are generated in is required:

eqlog_charname_server.txt

I don't have any size restrictions, just load em up!

And yes all expansions GoD to now log data should be uploaded. Can use any and all, this needs to be a big community effort.

[jdoran]

Would you like us to trim the logs to just those expansions, or can your parser sort it out. I have logs going back years.

[demonstar55]

Originally Posted by jdoran

Would you like us to trim the logs to just those expansions, or can your parser sort it out. I have logs going back years.

Well, they did mention wanting to parse NPC Aggro/Death emotes and I know we are missing a lot of them from pretty much every expansion ... But stuff that isn't on PEQ yet would probably still be more useful.

[Akkadius]

Originally Posted by jdoran

Would you like us to trim the logs to just those expansions, or can your parser sort it out. I have logs going back years.

Anything and everything, my parser will be able to differentiate the data.

[boweq]

where would i send them to akka?
, also anything on live that you need really bad?

still got active acounts ( 95sk/95ranger/95sham/95bard/90 rougue)

[Noport]

Originally Posted by sorvani

we are talking about standard log files created by the /log on command.

ok i'll send you a log of our guild chat i did type /log on in guild chat

[Akkadius]

Thanks for those who have uploaded logs, but so far this is looking pretty pathetic.

We really need to get a real density of logs before any parsing systems can really be developed around this.

If you know someone with a mother load of logs spanning lots of content please contact them and see if they are willing to contribute to this.

[Burningsoul]

Bowmaster's a pretty simple being... He's like an orc - point him in the direction you want him to smash, and let him have at it. With your team are you capable of HoT & VoA content Bow? That'd be the place to hit if you're capable. Hell I'd LOVE to see crystallos fully working, just to finally play through it rather than read about it.

[blackdragonsdg]

Hey Akkadius your website rejected the log I attempted to upload tonight. It said it was not a log file but it is a legit log file. I tried to upload the file a second time thinking maybe it was just a fluke but it rejected it again. Any ideas?

[Akkadius]

Originally Posted by blackdragonsdg

Hey Akkadius your website rejected the log I attempted to upload tonight. It said it was not a log file but it is a legit log file. I tried to upload the file a second time thinking maybe it was just a fluke but it rejected it again. Any ideas?

It's going to look for the eqlog in the name as well as .txt

[blackdragonsdg]

Originally Posted by Akkadius

It's going to look for the eqlog in the name as well as .txt

Well it wasn't the name that caused the problem. Just a guess but I think 100% packet loss around hop 22 may have had something to do with my upload not working.