Live Data Collection (Need Community Help) - Page 2

**Akkadius** · #16 09-30-2012, 11:24 AM

Quote:

Originally Posted by blackdragonsdg

Hey Akkadius your website rejected the log I attempted to upload tonight. It said it was not a log file but it is a legit log file. I tried to upload the file a second time thinking maybe it was just a fluke but it rejected it again. Any ideas?

It's going to look for the eqlog in the name as well as .txt

blackdragonsdg · #17 10-01-2012, 02:53 AM

Quote:

Originally Posted by Akkadius

It's going to look for the eqlog in the name as well as .txt

Well it wasn't the name that caused the problem. Just a guess but I think 100% packet loss around hop 22 may have had something to do with my upload not working.

**Akkadius** · #18 10-01-2012, 02:40 PM

Quote:

Originally Posted by blackdragonsdg

Well it wasn't the name that caused the problem. Just a guess but I think 100% packet loss around hop 22 may have had something to do with my upload not working.

Try it again, I changed some max upload size settings on the webserver end.

How large was the file that you were trying to upload?

blackdragonsdg · #19 10-01-2012, 02:57 PM

Quote:

Originally Posted by Akkadius

Try it again, I changed some max upload size settings on the webserver end.

How large was the file that you were trying to upload?

It uploaded successfully today and the file size 2.08MB.

**Akkadius** · #20 10-01-2012, 11:43 PM

Thanks for those who have uploaded logs, but so far this is looking pretty pathetic.

We really need to get a real density of logs before any parsing systems can really be developed around this.

If you know someone with a mother load of logs spanning lots of content please contact them and see if they are willing to contribute to this.

pfyon · #21 10-02-2012, 10:29 PM

Haha I was just thinking about this a couple of weeks ago. Glad someone got around to writing a parser before me

I found 80 from a variety of characters + another 26 that I renamed as part of archiving back from 2002.

**Akkadius** · #22 10-02-2012, 10:45 PM

Quote:

Originally Posted by pfyon

Haha I was just thinking about this a couple of weeks ago. Glad someone got around to writing a parser before me

I found 80 from a variety of characters + another 26 that I renamed as part of archiving back from 2002.

Oh wow, logs from 03/04. Now we're talking progress.

Once again if I can encourage anyone and everyone to try and get them to upload historic character log files this helps the project quite a bit.

If anyone has questions please let me know.

Thanks,
Akkadius

demonstar55 · #23 10-02-2012, 10:50 PM

Have you thought about posting this on like the EQ sub-reddit? I'm sure there would be people at least interested in submitting some and there is a larger chance they have logs.

pfyon · #24 10-02-2012, 11:35 PM

Any chance you can release some aggregated data once you're done with this project? I'd like to see how older mobs compare (hp/damage wise) to what we have in the peq DB.

**Akkadius** · #25 10-03-2012, 12:15 AM

Quote:

Originally Posted by pfyon

Any chance you can release some aggregated data once you're done with this project? I'd like to see how older mobs compare (hp/damage wise) to what we have in the peq DB.

Yeah I definitely will, I just need to collect a decent amount of logs before I start putting together the parser.

In the middle of working on some other rather large PEQ parsing projects to get loot, merchantlist, tradeskills from all expansions.

In the meantime I just need to start building the private repository so I have something to work with when the script is being created to parse through all of the text files.

After that I plan on having a web page that people can go to and see a hitlist of what has been collected and what hasn't

Maze_EQ · #26 10-03-2012, 12:18 PM

Question....Is test server data useful? I've got access to a multitude of accounts that can basically box everything up to VoA stuff.

pfyon · #27 10-03-2012, 09:37 PM

Finished my first 80 using the web interface, found another 46 (in addition to the archived ones).

After sitting in front of my computer for an hour doing those, I decided I'd script it. If you have a linux box and lots of logs, try the following script:

Code:

 for x in eqlog*.txt; do echo "Starting $x"; curl -F FileName=@$x -F EQEmuForumName=CHANGEME http://eoc.akkadius.com/eoc/EQEmuDataCollect/LogCollector.php &> /dev/null; done

Change or remove the '-F EQEmuForumName=CHANGEME' to your forum name if you want it to be included.

**Akkadius** · #28 10-03-2012, 10:08 PM

That's quite the collection you have there and that definitely will help.

I've updated the form to include how many logs we've collected so far and the size:

http://eoc.akkadius.com/eoc/EQEmuDat...gCollector.php

pfyon · #29 10-03-2012, 11:39 PM

How are you storing the logs? Are you using the original filename? I have multiple logs which share the same filename (I would copy them elsewhere and/or rename them to make parsing of more recent stuff easier) which, if you keep them as the same filename, could get overwritten.

**Akkadius** · #30 10-03-2012, 11:55 PM

Quote:

Originally Posted by pfyon

How are you storing the logs? Are you using the original filename? I have multiple logs which share the same filename (I would copy them elsewhere and/or rename them to make parsing of more recent stuff easier) which, if you keep them as the same filename, could get overwritten.

No, I thought that through beforehand. They never get saved as the original filename.