Sourcing Cave Dude's Database into PEQ

[Zengez]

Heres what i did...

I pulled up a list of NPC names in cavedude that wasn't in peq... pulled those out into a list

Then i ran a program to take every line of the insert statement from dumping cavedude's database where the name from the above list showed up, giving me several insert statements per name at times, all of which were not in the peq database...

Then i adjested the npcid's to make them larger than peq would ever use, so that there was no chance of a duplicate id entry forming when i sourced it over, nor would there be any chance of a future peq update overwriting the id that i assigned to these mobs... so it is (theoretically) 'update proof'

Thats how i got the npc_types table to work in terms of the npc's ids versus name...

I'm sure i missed some, it's not perfect, it's not pretty, but i gotta say i'm wandering around plane of valor right now on my peq database and looking at all the spawns that weren't there a minute ago

I looked around and found a free hosting site for a temporary host until someone else can host it for themselves, here's the link;


http://upload.ohshare.com/v/8413635/...o_PEQ.zip.html

Note, you do NOT need any of cavedude's database on your machine to use this, they are straight insert statements designed to be run by mysql to insert into peq database... there are linebreaks every 80 or so characters in order to allow them to be straight copy/pasted into the mysql command line if you wish to do so (note some of these files are pretty long, i've found trying to paste in more than about 25,000 lines of code at a time can cause the command prompt to crash, so paste in sections)

Enjoy everyone, hope it helps some of you out, if nothing else at least it's a starting point

[eq4me]

Thanks alot!

This adds 6648 spawns in 46 zones.

awk -F"'" '{print $2}' spawngroupnew.sql | awk -F"_|[0-9]" '{print $1}' | sort | uniq -c| sort -n
4 shadowrest
15 hohonorb
46 hohonora
49 mmcj
50 takd
51 pofire
53 tipt
54 poair
54 ruji
55 potorment
55 solrotower
60 qinimi
62 bothunder
64 natimbi
64 potranquility
74 povalor
82 potactics
89 veksar
104 codecay
112 postorms
117 harbingers
119 nedaria
125 hatesfury
128 powater
131 chardokb
132 riwwi
150 bloodfields
154 vxed
155 hateplaneb
156 qvic
187 barindu
189 poinnovation
200 potimea
209 podisease
218 draniksscar
229 pojustice
229 wallofslaughter
232 abysmal
233 ferubi
250 poeartha
255 poknowledge
258 provinggrounds
287 causeway
310 dranik
376 kodtaz
422 ponightmare


Just one question.

Why do formerly not populated zones do have spawngroup entrys like nedaria_63 and additions to PEQ zones entrys like podisease1640021? Is the _ missing or redundant?

[Zengez]

Alot of the spawnentries and grid info i did as educated guesswork... like i said it's not perfect, just a point to jump from... if anyone feels so ambitious i'd recommend going through them more meticulously to find which ones specifically are needed and which ones aren't for these tables and clean up those two files a bit more than what i did...

Not quite sure if that answers your question or not, if not, pm me with more specifics

[Angelox]

Originally Posted by Zengez

Heres what i did...

I pulled up a list of NPC names in cavedude that wasn't in peq... pulled those out into a list

Then i ran a program to take every line of the insert statement from dumping cavedude's database where the name from the above list showed up, giving me several insert statements per name at times, all of which were not in the peq database...

Then i adjested the npcid's to make them larger than peq would ever use, so that there was no chance of a duplicate id entry forming when i sourced it over, nor would there be any chance of a future peq update overwriting the id that i assigned to these mobs... so it is (theoretically) 'update proof'

Thats how i got the npc_types table to work in terms of the npc's ids versus name...

I'm sure i missed some, it's not perfect, it's not pretty, but i gotta say i'm wandering around plane of valor right now on my peq database and looking at all the spawns that weren't there a minute ago

I looked around and found a free hosting site for a temporary host until someone else can host it for themselves, here's the link;


http://upload.ohshare.com/v/8413635/...o_PEQ.zip.html

Note, you do NOT need any of cavedude's database on your machine to use this, they are straight insert statements designed to be run by mysql to insert into peq database... there are linebreaks every 80 or so characters in order to allow them to be straight copy/pasted into the mysql command line if you wish to do so (note some of these files are pretty long, i've found trying to paste in more than about 25,000 lines of code at a time can cause the command prompt to crash, so paste in sections)

Enjoy everyone, hope it helps some of you out, if nothing else at least it's a starting point

Hi, and thank you for your great work - this was very needed. If you need a space I can fix you up on my webserver, or I can post your stuff un my UGLY web page (ugly, but works). PM me with what you want.
I have a question; Are you using eight digit numbers? I'm asking, becuause I am using the eight digit numbers for my up dates to the zones, and don't want them to cross with yours.

[Zengez]

a bunch of them are 8 digit numbers, so you'd have to check those... most of them are 9 digit numbers however i believe, you can always use a 'where id > 100000000' clause to source in most of them then peel through the rest...

all mine end in 000 though since it was a straight *1000 so that should make looking through the id's easier for you.

[Angelox]

Originally Posted by Zengez

a bunch of them are 8 digit numbers, so you'd have to check those... most of them are 9 digit numbers however i believe, you can always use a 'where id > 100000000' clause to source in most of them then peel through the rest...

all mine end in 000 though since it was a straight *1000 so that should make looking through the id's easier for you.

You might want to check out nine digit numbers - I had problems to where my EQEMU wouldn't load the database when they were nine digit.

[eq4me]

How about generating the IDs with an fixed offset instead of an multiplier?
EDIT: On second though I dont think its that easy, you might have to discard the old IDs alltogether and give new IDs, counting upwards from the offset.

Be aware that I did not read through the whole documentation yet, so the below could be moot:
Maybe it would be nice to have some conventions what ID ranges to use for certain stuff. Eg. Half the usable range for PEQ itself. A well sized chunk for imports from Cavedudes and other databases(are there any?). And the rest for local server customizations.
I recon it would help server admins to make sure that their custom additions dont clash with further updates of the PEQ databases.

[Darg]

Where do I put the files I downloaded from the one post? And how do I get it to source in properly?

[Angelox]

Originally Posted by Darg

Where do I put the files I downloaded from the one post? And how do I get it to source in properly?

If I were you, I'd wait tell Zengez replies to my "nine digit" comment - You're database may not load after you do this.

[Zengez]

Originally Posted by Angelox

You might want to check out nine digit numbers - I had problems to where my EQEMU wouldn't load the database when they were nine digit.

Yea, i had that problem originally because the database only supports some of the upper end ranges because of the size limitation of the field, but it won't tell you you've reached the max until you've already hit it, then try to go higher a second time...

basically if you multiply by 1000, then do so again, half yer id's turn out to be the same number which won't let anything load, but it won't error until you try to multiply/divide it 'again' once they are all the same number... which is frustrating...

Anyway i ran into this problem by accident, figured out what the problem was, then resolved it... so (in my case anyway) everything loads just fine and i didn't break anything... I can't imagine anyone else having this problem if i didn't because i haven't actually customized my server very much... but I make no promisses as i don't know other people's code... my recommendation; dump your database files before you source these in, then if it kills something do a kill on the table and resource in your dumped original version...


As to how to source it, i just did a straight copy/paste from the text files into the mysql prompt in order to put it in, but remember not to do any more than 20-25k lines of code at a time or you may crash the prompt (won't destroy your database or anything, just crashed the prompt)

[BobThompson]

This worked fine for me. No problems loading the database and all the zones previously empty have caveman's spawns in them.

[John Adams]

Everyone has their own way of doing things, but starting out with the base of what the designers intended is always a good practice. Any work (or modifications to existing work) that I do will follow this basic format:

http://www.eqemulator.net/forums/sho...12&postcount=7

The only problem I really have with this is the merge confusion - which is not a problem to the PEQ folks, since that is not their focus - it's our, individual desire to have more content immediately that is driving this effort again. If you use REPLACE INTO, you run the risk of blowing up LOTS of PEQ data. If you use INSERT INTO, you'll bump into existing (unique) records and lose Cavedude's data. I see some tables with auto increment values, and others where there are none. The confusion for me lies in, say, Zone ID 202 having npc ID 202000, in spawn group <autoinc, so it could be anything> using loot table <auto, anything> with loot drops <auto, anything> etc... see where I'm going? Sure it cuts down redundancy to NOT tie npc loot, faction, or spawn id's to zone id's (* 1000) but at the same time, it becomes hell to manage.

As a learning exercise, this is where I am focusing my attention right now... learning the intertwinings of the database and seeing if I can make things make more sense to me. I doubt my effort will become an acepted norm, but at least I will understand what's going on when I am through.