It doesn't look like it can be disabled. :( Without the table, zones crash as soon as a spell is cast, with a blank table, the zones won't boot at all.

Been trying the export script for 2 hours now with no luck :-x


*waves the white flag* :x

Did you install the DBD::mysql module? It's got a new name and I don't remember what it is off the top of my head. You should be able to use ppm to find it though. I'm not sure what the mysqlPP is, but it typically means mysql Pure Perl. I'm not sure if that's the same thing or not. I don't run it on my windows box anymore so I don't know for sure.

Is it possible to retract an SVN submission? Would be pretty awesome to just be able to remove it and not have to either manually remove everything added in it, or commit an older version to revert it.

I was hoping to get the new food stats fix in that I got done last night, but couldn't because of the problems with this.

Sheesh, I go away for 17 hours and everyone freaks out...

Anyways, to address the questions:

cavedude pretty much got it. I would recommend using the -h (for [h]elp) switch, recommended in the changelog:
which gives the usage (which I also posted above):
By default, it will look for both eqemu_config.xml & spells_us.txt in the current directory (as indicated in the usage info for the switches, since they don't have any /'s before the names of the files), so you could easily just execute it from your server directory. For me, I would end up using something like this, cause even though symbolic links are awesome, I don't really like making them a lot:
Then, you can just add the export to your startup script so it exports at runtime if you wanna be one of the cool kids :-)

The idea is this will replace using the spells file. The only reason I didn't just straight take out the old stuff is so everyone can continue to use it if they want to (just like turning bots on, SPELL_EFFECT_SPAM, etc) by changing the #defines in zone/spdat.h:
One change I might make would be to create a default case if neither are defined:
However, then we're completely rid of the real old way of doing it (lots of hard coding). I honestly don't think anyone's using that old code, but who knows.

Minor typo. I was copying/pasting the top half of the scripts, and forgot to make that change.

I've actually been using this code, successfully, for just shy of a month (since October 27th to be specific). The only thing that took a little time was the scripts to import/export the spells (having a kid in the middle of all that did slow things down a bit). Not only that, but the main thing it's doing is loading spells into the spells[] "array" from the database instead of the spells_us.txt file. There were also a few unknown values in the SPDat_Spell_Struct struct that I came across, but I made sure it worked with the spells_us.txt file before I even tested with the database (which caused problems with sharedmem a few months ago when I first started working on it). I personally don't consider it to be a "large feature", rather just a bunch of copying, pasting, & tweaking of existing code.

I'm running Debian & have libdbi-perl & libdbd-mysql-perl. The scripts use DBI, so Perl probably just assumes MySQL when trying to connect to the server (automatically negotiated maybe?):
As far as I can tell, both of the DB packages are installed automatically when installing the MySQL 5.0 server, again at least on Debian.

I guess the question now is anyone actually having problems after getting the spells into the database? Because other than that, I'm not sure what all the ruckus is about...

I know it compiles, I know it runs, I know it works, and made sure to test it thoroughly before I got to this point. I even just checked my launcher status page & haven't had any zone crashes since my server was booted up at 6:28am today.

Got to talk to trev, and it appears there is a compiling issue when setting to NEW_LoadSPDat. Looks like I missed some +'s when I was copying from a diff (thanks again Google!):
zone/net.cpp
Took 'em out, recompiled with NEW_LoadSPDat, and works fine.

EDIT: This is in SVN as Rev 232.

You don't, but I do. It interacts with a core part of the server functionality so I would consider it big even if it doesn't seem all that complex. Also given that there are no real benefits yet (it's all a idea for future expansion) I'd rather retain backwards compatibility right now. People who want to use the new system can use it but until we start getting applications that take advantage of it I'm not gonna force people.

I'm glad you guys got it all sorted out before I got back, kinda hard to fix stuff with svn when on the road =/ Keep working with this it has potential even if we haven't found it yet.

Probably the biggest benefit is that web-based tools could be made for editing spells online. It might also be useful that we can have code that is able to check the table for certain things. I am sure there is more potential out there for it. If there was a way to convert the Ailia/Bleh spell editor to work with the new table, I think it would leave little need for the spell file. But, unless that table is added into the PEQ and AX databases, we would also need directions in the server setup wiki for new admins to set this up for their server.

Hmm, I think I might have come up with a good use for it already lol. Would it be possible to add 1 more column to the end that will let us enable/disable the spell from loading (enabled by default)? So, instead of removing a spell or setting it to be unusable by all classes, we could simply disable it. Maybe even having an extra "comments" field next to that one for notes about the spell, if it has been changed or removed for any reason. Or, if you wanted to replace the spell, you could put the old Spell ID number in the comments field and then change it to an unused Spell ID so that you can reuse the one it was previously using for another spell. Although, I imagine that would make the import export stuff hard to do and stay sorted properly with the comments and enabled/disabled field.

The reason I mention this idea is because all classes are capped at 400 spells max (by the client as far as I can tell). I tried changing the settings in the source, but it didn't add more than 50 pages to the player spell books. Since I have spells up to level 75 on my server, Druids exceed the 400 cap and lose access to many of the high level spells due to lack of a slot in the spell book. If I could easily disable spells in the table like mentioned, I could just go through and disable a bunch of unused ones like ports to new/locked zones etc.

It's already in PEQ CVS.

Yeah there's benefit but no one's tapped it yet + it's a new feature so I'm keeping it off by default.

I can agree to keeping it off by default. The main reason I turned it on by default was because I was getting the feeling from the feedback I was getting that, if I made the effort to make the spells relatively easy to import/export, that I should. However, I think the better way to do it (which is how I was leaning originally, but decided to take a little initiative) is what you recommend, which is to turn it off by default. If others start using it & it becomes more "mainstream", it's easy enough to change. At the very least, now everyone can use the same spells table for their tools :D

I run a custom spell file and when I ran the import script, I got this:
Do those messages mean it changed the name of the spells? Or are they anything to be concerned about?

The "Loading" info is more so you can see what the status is, although it goes so fast that you normally don't get to see pretty much any of it. I currently have it print the spell name after inserting the escape character (\) for the query to insert it into the database, mainly because it's the easiest, which is where the confusion about the name might be. Since it indicates it loaded all 13373 of the 13373 spells, it should be fine, although I'm not really sure why all of those ended up back-to-back on a single line instead of erasing each other. If there was an error loading 1 or more spells, it would output a line letting you know which one failed (1 line for each spell that failed to load).

Does that help to answer that question?

That happens with the item import script too (of which this is based off of I guess) I've never had a problem with the items it named, and as long as it claims to have loaded all the data it's fine. I would assume the same goes for the spell script.

It was :-)

After reading this whole thread, and being convinced this feature was "off by default", I updated to the latest SVN, did the SQL and import_spells.pl. Now my server is broken. Try to mem a spell and get a message "Unexpected error: spell id out of range" and now I cannot click on my spell book at all.

Without reverting to an earlier SVN, is there a way to disable this completely (for real) and use the spells_*.txt way?

Please don't remove the old way just because 3 people have tried it and think it works.

Ya know, the 30 second edit on this forum is bullshit, always hated it. :)

What I meant to say, "servers Spells are broken". The server itself appears ok.

I think you use Linux, so try a 'make clean' and then 'make'.

Yes I do use linux for this server. I keep forgetting about the make clean (LE probably wants to hit me with a rotten tuna as well over this :))

I'll give it a try.

Ugh... ~blush~ make clean once again makes a fool of me ;) Thanks Derision for the reminder. I need to nail a sticky to my forehead (though I still cannot understand the need to whack the old compile for new changes).

Spells do work with this new table, the import thingy, and "off by default" which I will eventually learn how to turn this on and give it a try. Just not today. :)

Thanks AndMetal, sorry if my initial post sounded all assy.

Np :)

It's probably something worth mentioning in the changelog, since the structure of the spells (for shared memory) changed since we discovered some of the unknown fields, I just haven't gotten off my lazy ass to update it, lol.

How has this been working since the initial stumbles? Flawlessly I hope?

If it's working well I actually have an idea I'd like to implement next time I get some free time dealing with this (sekret idea but it's really cool trust me).

Well, the new DB Spells might be working ok, but I found a major issue with the current SVN revision that basically breaks spells if you try to use the old spells_us.txt way. The spells all still cast properly, but for some reason, they take 4X as long to cast either from items or just from casting a spell. There were so many changes that went in with the new spell changes AndMetal made, that I don't have any idea where to start looking for the cause of this.

I will probably move to the new DB version soon, but as long as the old way still exists, it should at least work :P

I just did quite a bit of testing with SVN Rev 232 & 235 using the unmodified Titanium spell file, trev's slightly custom spell file, and an outdated version of Live's spell file and have yet to reproduce the casting delay either by casting from clickies or from the spell bar. For the clickies, I used a Brown Ornate Chain Bridle & Fabled Jaundiced Bone Bracer. I also used a combination of Ice Age & Greater Immobilize for casting spells.

The only thing I can think of is if there was a problem sourcing the spells into the database.

I've said it before, I'll say it again:

Oops, it looks like the issue must somehow be on my end. Sorry that I was too quick to assume that the spell changes you made were the cause of it. Apparently, it is an issue with the test server I am running and it happens in all builds I have tried so far and I don't know why. The test server is a Debian build running on VMWare on my Windows PC. I don't think that should cause a problem, but who knows? The only thing I suspect is that it could be due to me running 2 servers at the same time from the same internet connection. I know I can't forward the required ports to the server needed for players to log in, but it works fine for me to log in, since I am on the same LAN and don't get restricted by ports. Maybe I somehow missed something else important in the build, but I have never heard of this issue before.

I did a bit more testing on it and it seems that spells are taking exactly 7 times longer to cast than what their cast time says on the spell or clickie item. I have no idea why it is 7X, but that is what happens every time for all spells. I have tried both the old spells file way and the new DB spells way and both have the exact same issue. That is an issue for another post though, so I am not asking for answers in this one.

Sorry to derail the thread and my apologies to AndMetal for making an assumption without better testing.

To keep this somewhat on topic, I must admit that it is awesome to be able to sort the new spells table by any column to quickly check for certain things in spells. It made finding all of the mounts in the file a very simple and quick task. Once a tool comparable to the Ailia/Bleh spell editor is created for working with the new DB spells, I don't see any need to keep the old system around. I am sure we will find even more benefits from it soon.

This error is caused because the DBD-mysql perl module is NOT installed. I ran into this issue ever since the Nov update and could not get the import_spells.pl or export_spells.pl scripts to run. After getting around to conducting some research on it now that my time has freed up, the issue seems to be that the default PPM package availability list no longer has DBD-mysql listed as an option. The DBD-mysqlPP will NOT WORK for the sake of the spell scripts and you must install the normal DBD-mysql module. To install it, follow these steps:

1. open the Perl Package Manager
2. click the Edit menu - Preferences
3. click the Repositories tab
4. In the location box type "http://theoryx5.uwinnipeg.ca/ppms/"
5. Click Add
6. Click OK
7. Install the DBD-mysql package

OR

1. open a command line prompt
2. type "ppm install http://theoryx5.uwinnipeg.ca/ppms/DBD-mysql.ppd
"
3. Press Enter

At this point, the import/export scripts will run correctly.

Since I think this is now a proven alternative systems for spell management, I was wondering if we should consider making it on by default. Otherwise, maybe we could set a rule that will allow admins to chose which type of spell system they want to use (the file or the table) and only have to set it the one time in the rules table instead of changing the define each time before updating the code on their server. Any ideas or input?

will players still need to have their own spell file if spells will now be loaded into DB?

Yes, this changes nothing about how the client handles spells. It just lets the server load spells directly from the database instead of from the spells_us.txt file. Basically, it doesn't really make much of a difference yet which way they load them. There is alot of potential with loading from the DB though. For one, it means you could download the PEQ database and have spells included with it so you don't have to put move the spells file from your EQ directory over. That is fairly minor, but 1 less step in the setup process. A big benefit of having it in a table like this is that you can give your GMs access to the table and they can edit it very easily to fix broken spells or make custom ones. Kayen has made quite a few adjustments to the custom spells on Storm Haven already. Many changes he makes doesn't require an update to the spells file that the clients use. It all depends on what changes you are making and to what spells.

The main benefit I can think of for loading spells into the database with this system would be that we can add extra features for spells that just aren't an easy option for the spells_us.txt way. My first idea for this is to create 2 new columns in the spells table at the end of it that will allow you to set a name and a value that could maybe be tied into quest globals. Basically, the idea is that the default name and value might be something like 0 for those fields, and anyone will gain those spells when using the #scribespells command or a spell scriber quest. But, any spells that are set to another name or value won't be scribed unless that player has that particular global with the same value or greater. So, basically, you could create tiers of spells or define special spells that will only be able to be learned if the player has earned them via a quest or whatever. I think it would be a great system and shouldn't be all that hard to create using the new database spells system and quest globals. That would be pretty tough to do/manage if spells weren't loaded into the database like they can be now.

Unfortunately, I won't have time to even think about working on that new system for quite a while. But, it is definitely something I would like to get going at some point.

Another quick idea to make use of the new database spells would be to create an additional field that would let admins set certain spells to be automatically converted into another spell if it was cast between a Titanium and a SoF client. This would be useful to correct any discrepancies between spell files of the 2 clients and also would be useful for servers with custom spell files. I don't even know if this idea is possible, but it would be pretty cool if so. The field could be called something like SoF_SpellID and you would just put the spell id of the spell that you want the SoF client to use in place of what the Titanium client is casting or visa versa. There could probably also be a Titanium_SpellID column added as well that would convert an SoF spell into a Titanium spell ID. So, say that the buff Temperance was spell ID 100 in Titanium, but spell ID 200 in SoF, by setting 200 in the SoF_SpellID field, it would convert the spell ID to the correct one so that the correct effect happens on SoF. Though, this might require 2 spells tables, one for Titanium and one for SoF. They would have to both be loaded into memory though. I think with 2 spells tables, it might actually not be too hard to do a conversion system like this though. Just an idea anyway.

If no one has an objection, I am going to finally change the default to the spells_new table as apposed to loading spells from the spells_us.txt file tonight. I see no reason not to change it now, anyway.

Of course, anyone still using the spell file will either need to make the switch to using the table, or will need to change the define before compiling. I think Cavedude have been changing the default before compiling the binaries anyway, so no reason not to change it in the source. The old way will still be there and will probably remain there indefinitely or until it is decided that we never have need for it again.

Let me know if there are any objections before I change it tonight. If a reason does pop up later, it can always be changed back easily.

ah Trev, I may have a small objection :cool:
One thing about workign with spells, that txt file is far easier to swap around when something needs to be changed FAST. (like copy and paste for entire chunk of lines)

So instead of a define- could you make this option into a Rule instead? this won't even give an overhead since all spells load only ONCE when sever starts.

To change it to a rule would probably mean adding in duplicate of a ton of code in multiple files. I don't really think it is a good option or it probably would have been done already.

If you need to do some major changes like that, you can easily use the import and export scripts to export the spell file from the spells_new table, change it and them import it back into the table. That is what the scripts were made for :)

Another reason that the table is a really good way to go is because if people export it to create their server's spell file, the exported file will work perfectly with both Titanium and SoF, since it has the correct number of fields to function with both clients.

I think the binaries have been set to use the table loading for a while now. So, if you use those, you would have been stuck with the table loading anyway. If you don't want to use the table loading way, it is very simple to change the source to use the file loading way instead and then compile your own binaries. All you have to do is go to zone/spdat.h and change the following:

to this

Trev, isn't the both codes allready there for #define to be used?

binaries wise - they stil were using spells.txt as of R535 i last dowloaded - how else would i have been using my own spells =P (speaking of which its time to update the binaries on the oficial site)

unfortunately i don't have option to do my own compile =(

ALso what kind of loading script you refering to - could you post a sample?

Everyone has the option to compile their own source. It is only a matter of taking the time to figure out how to do it the first time :P

The import/export scripts aren't included with the Binaries yet as far as I can see, but when this change goes in, they should probably get added to the Binaries. Otherwise, you can get the import and export scripts by downloading the source code through the SVN, as it comes with it in the main directory there.

Here is some more information on the import/export scripts and how to use them. It is in this very thread we are in right now:

http://www.eqemulator.net/forums/sho...t=24193&page=3

See post #45 by AndMetal

well as you could see number of downloads of ready binaries is well over 200 =P so as you see people far from comfortable of compiling their own source =P

Once you learn how to compile your own source, it's pretty much just as easy as downloading a pre-build binary, but gives you the option of making changes to the code.

This page has all the info you should need for learning how to compile it on Windows:

http://www.eqemulator.net/wiki/wikka.php?wakka=VS2008

Or, you could setup a Linux server and make compiling as easy as typing "make" :P

IMO, if your main concern is having options, then you should take the time to figure out how to compile your own source. If you are not worried about options, then just use the Binaries and you will still have plenty of options but not as many as compiling yourself. Your main concern has been options for so long, you should have learned how to do this long ago! There is only so much we can do to give options with Binaries, but with compiling yourself, the options are practically limitless :P

well i will do my best to swith over to my own compiling, but just want to tell you to give other people advance warning that their custom spell file may stop working soon and then they will be all over the forum going "WTH?" =P

So i urge you to release CURENT last Rev in spell from txt format, and swith to Db spells AFTER that, and put a warning into changelog

There will definitely be a warning about it in the changelog. The change is simple enough that Cavedude could change it to the file loading before compiling the next Binaries if needed. But really, if you want to stay current with the EQEmu updates, there is always going to be 1 thing or another that needs to be adjusted whether that be database changes or whatever. We can't support all legacy options indefinitely or it would be very hard to make good progression with it. I think the change from the spell file to the database is a very good one. Sometime soon I would like to get some new features added that will really make use of the benefit of having spells in a table vs in the spell file.

Trev, my cusdom spell file is designed under Titanium.
the Spell table in DB is deisgned under SoF

are there mututaly exclusive properties between the two?
Can I source the T-spells into Db wihout problem, or does the spell structure layout different for SoF spells?