Max Number of Spells in spells_us.txt?

[trevius]

I am trying to find a way to allow my players to level from 70 to 75 in a non-traditional manor. Basically, in the current top tier zone I am making, there will be drops for a quest that will allow players to level up to 75 even though the server cap is set to 70.

I have decided to use a current spell file download from SOE or from Lucy (either should work fine). Almost all of the current 1-70 spells seem to work fine for the most part with only minor bugs seen so far.

The issue is that it doesn't seem to load the entire spell database as far as I can tell, cause many of the 70+ spells just don't show up after using the spell scriber. I recall reading somewhere (but I can't find it now) that there is a limit of 9k or maybe 10k spells set to be loaded by the emu. I am guessing that is the reason why many are missing, because the exceeded the cut-off. I know that items have a max number that is able to be created by default and I don't think the itemIDs can exceed something like 100k. This is another reason I think spells may work this same way.

I know that some of the spells just won't work because some of them have new features that aren't supported in the emu yet. That is fine with me. I can remove any that don't work properly, or any that I don't want players to have from server-side and they just won't be scribed or castable. I can edit that stuff later, so that isn't a big concern.

I would like to know if anyone knows how to up the max amount of spells that can be loaded by the emu, or if there even really is a max number. Maybe I am incorrect in thinking that. It could just be that some aren't showing up in-game because they have features in them that aren't supported, so they won't even load properly or something.

I have tried compiling the source in the past, but I was never able to get it to work properly. I am sure that if someone could find the code I need to change to correct the issue I am having, I could give it a try again and get it working with enough effort, lol. Of course, I wouldn't complain if there was a rule to adjust settings like max number of items to load or max number of spells. I am not concerned about max itemID number, I am just mentioning it, because it seems like it could be similar to that issue.

I really wanted to find a way to do this without requiring a file download for the clients, but I don't think that is possible. I tried just editing some current spells in the same spell line and upgrading them so there would be 1 spell per level from 71 to 75 for each class. Unfortunately, the client side spell file is responsible for more than I had imagined. I was hoping it would only show the wrong spell info, but it also determines cast times and even stat adjustments from buffs. An example was that I used the spell holy armor to make a new group buff for clerics that would be an upgrade to CoV. The spell worked as intended, but the HPs don't show up properly in the inventory stats screen even though they show up with #showstats properly. Also, it would appear that the spell only took 2 seconds to cast, but then would just do nothing until the 20 seconds of cast time passed that was set for the actual upgraded spell on the server. This is just too buggy and too hard for my players to keep track of.

Thanks in advance for any information on this. I will be making another post in relation to this non-traditional leveling method, since stats don't show up properly due to skill caps getting rolled over, but I will leave that for another post in another section.

[AndMetal]

I remember the post you're talking about, but have no idea where it wound up. I thought I replied to it, but I wasn't able to find one.

What I do remember is that, when the spells are loaded into shared memory (see EMuShareMem/Spells.cpp), it creates a dynamically sized array (max spell ID + 1 according to the source), so it shouldn't matter how many spells you have. The exception to this was an old way of loading the information, in which it was defined as exactly 3602 spell records (see zone/spdat.h).

[ChaosSlayer]

as of this year builds ( I am using 1090 rigth now) you CANNOT load more than 10k spells (and from what I know you never could - I tried)

I have been telling this to devs for a while now- please resize array to 20k

[Secrets]

Originally Posted by trevius

I am trying to find a way to allow my players to level from 70 to 75 in a non-traditional manor. Basically, in the current top tier zone I am making, there will be drops for a quest that will allow players to level up to 75 even though the server cap is set to 70.

I have decided to use a current spell file download from SOE or from Lucy (either should work fine). Almost all of the current 1-70 spells seem to work fine for the most part with only minor bugs seen so far.


...


I would like to know if anyone knows how to up the max amount of spells that can be loaded by the emu, or if there even really is a max number. Maybe I am incorrect in thinking that. It could just be that some aren't showing up in-game because they have features in them that aren't supported, so they won't even load properly or something.

It's around 9304 spells, and any spell after a certain number doesn't get loaded.

It's one of those things in the titanium client that can't be tricked, sadly.

Your best bet is to copy the spells you want (There's a number of them) and put a spells_us.txt available for download. Unfortunately the client doesn't have a download option for them, nor does it come bug-free if you don't download it. (Not to mention some of the fields are too long for the titanium client anyhow...)

But that's the only thing you can do. Limits, limits, oh how i'd love if they wern't there.

[trevius]

Well, the number of fields seems to be fine. They all work just great with the spell editor I am using which requires a standard 203 number of fields for every line in the file. It gives errors if any lines don't have that many fields and I get no errors on the downloaded version from Live.

Does anyone know how I would up the max number of spells in the source? Can I just adjust how many the array loads up? I went through with the spell editor and changed every field of every player usable spell that didn't have a value that the spell editor understands, which I assume means it is something that Titanium doesn't have support for. So, I think that should stop me from getting errors when casting those spells. I also removed all spells that are placeholders, test spells and even the multi version spell from level 71+ (I only left the original and removed RK II and RK III). Now I just need a way for it to allow me to load them all.

[trevius]

Originally Posted by Secrets

It's around 9304 spells, and any spell after a certain number doesn't get loaded.

It's one of those things in the titanium client that can't be tricked, sadly.

I did some testing last night and verified that the max spell ID that can be used right now is 9999. I tested this by importing an EQlive spell file (over 20k spells) into excel and copied the holy armor spell and pasted it over all spell columns accept for the spell ID. So essentially all spells were now Holy Armor. Copied that same file over to my eqemu and everquest directories and loaded the server and logged in. I then #cast spells and clicked off the buff each time after the cast. I found that 9999 would cast and put the buff up, but 10000 would cast and not show the buff.

So, if I am reading what Secrets wrote correctly, the 10000 limit is built into the client and NOT the emu? I tried searching through the emu code, but I don't understand code well enough to be sure that there isn't any issues with the server that might cause spells above 10000 to not show up properly when cast. They do actually cast, but no buff icons show up and the buff message doesn't appear. Oddly, the worn off message still does appear...

If I am using the live version of the spells_us.txt file on both client and server, I can do a scribe spell and it seems to load all up to 9999. Anything above that shows up as "unknown spell" in the spell book and you are unable to scribe it or see any benefits from it.

I am no coder, but I read through alot of the emu code trying to understand how it loads the spells file. First I see this:

files.h

#ifndef SPELLS_FILE
  #define SPELLS_FILE		BASEDIR "spells_us.txt"
#endif

Which I think means that spells_us.txt can now be referenced as SPELLS_FILE now.

Then I look for that:

net.cpp

#ifdef NEW_LoadSPDat
    // For NewLoadSPDat function
    const SPDat_Spell_Struct* spells;
    SPDat_Spell_Struct* spells_delete;
    sint32 GetMaxSpellID();


    void LoadSPDat();
    bool FileLoadSPDat(SPDat_Spell_Struct* sp, sint32 iMaxSpellID);
    sint32 SPDAT_RECORDS = -1;
#else
    #define SPDat_Location    "spdat.eff"
    SPDat_Spell_Struct spells[SPDAT_RECORDS];
    void LoadSPDat(SPDat_Spell_Struct** SpellsPointer = 0);

#ifdef NEW_LoadSPDat
sint32 GetMaxSpellID() {
    int tempid=0, oldid=-1;
    char spell_line_start[2048];
    char* spell_line = spell_line_start;
    char token[64]="";
    char seps[] = "^";
    const char *spells_file=ZoneConfig::get()->SpellsFile.c_str();

I don't really understand this part at all lol.

Another thing I checked out is:

spells.h

struct MMFSpells_Struct {
    uint32        SPDAT_RECORDS;    // maxspellid + 1, size of array
    SPDat_Spell_Struct spells[0];
};

I see the commet that says "maxspellid + 1, size of array", but I don't understand where the array is built or where maxspellid or even SPDAT_RECORDS are defined. If you look at the items.h, it is really clear how that array is built:

items.h

#define MMF_EQMAX_ITEMS        100000

struct MMFItems_Struct {
    uint32        MaxItemID;
    uint32        NextFreeIndex;
    uint32        ItemCount;
    uint32        ItemIndex[MMF_EQMAX_ITEMS+1];
    Item_Struct    Items[0];
};

The SPDAT code has me confused too. I see where it used to be defined by a number, but now I don't see any define for it. I am not sure why "#define SPELL_UNKNOWN 0xFFFF" is only 16 bit when SPELLBOOK_UNKNOWN are 32 bit. I also don't know why it mentions "#define SPDAT_RECORDS 3602" as an else below.

spdat.h

#define SPELL_UNKNOWN 0xFFFF
#define SPELLBOOK_UNKNOWN 0xFFFFFFFF        //player profile spells are 32 bit

//#define SPDAT_SIZE        1824000
/*
   solar: look at your spells_en.txt and find the id of the last spell.
   this number has to be 1 more than that.  if it's higher, your zone will
   NOT start up.  gonna autodetect this later..
*/

#define NEW_LoadSPDat

#define EFFECT_COUNT 12

enum SpellAffectIndex {
    SAI_Calm            = 12, // Lull and Alliance Spells
    SAI_Dispell_Sight    = 14, // Dispells and Spells like Bind Sight
    SAI_Memory_Blur        = 27,
    SAI_Calm_Song        = 43  // Lull and Alliance Songs
};

#ifdef NEW_LoadSPDat
    extern const SPDat_Spell_Struct* spells;
    extern sint32 SPDAT_RECORDS;
#else
    #define SPDAT_RECORDS    3602

Noting spells code from the emusharemem.h file here as well for reference.

emusharemem.h

////////////
// Spells //
////////////
typedef bool(*CALLBACK_FileLoadSPDat)(void*, sint32);

typedef bool(*DLLFUNC_DLLLoadSPDat)(const CALLBACK_FileLoadSPDat, const void**, sint32*, int32);
struct SpellsDLLFunc_Struct {
    DLLFUNC_DLLLoadSPDat DLLLoadSPDat;
};

I noticed that the spells is the only one that doesn't have a clear function in the emusharemem.cpp code below. I don't know if that is anything suspicious or if that is normal.

Emusharemem.cpp

    if (Loaded()) {
        Items.GetItem = (DLLFUNC_GetItem) GetSym("GetItem");
        Items.IterateItems = (DLLFUNC_IterateItems) GetSym("IterateItems");
        Items.cbAddItem = (DLLFUNC_AddItem) GetSym("AddItem");
        Items.DLLLoadItems = (DLLFUNC_DLLLoadItems) GetSym("DLLLoadItems");
        Doors.GetDoor = (DLLFUNC_GetDoor) GetSym("GetDoor");
        Doors.cbAddDoor = (DLLFUNC_AddDoor) GetSym("AddDoor");
        Doors.DLLLoadDoors = (DLLFUNC_DLLLoadDoors) GetSym("DLLLoadDoors");
        Spells.DLLLoadSPDat = (DLLFUNC_DLLLoadSPDat) GetSym("DLLLoadSPDat");
        NPCFactionList.DLLLoadNPCFactionLists = (DLLFUNC_DLLLoadNPCFactionLists) GetSym("DLLLoadNPCFactionLists");
        NPCFactionList.GetNPCFactionList = (DLLFUNC_GetNPCFactionList) GetSym("GetNPCFactionList");
        NPCFactionList.cbAddNPCFactionList = (DLLFUNC_AddNPCFactionList) GetSym("AddNPCFactionList");
        NPCFactionList.cbSetFaction = (DLLFUNC_SetFaction) GetSym("SetNPCFaction");
        Loot.DLLLoadLoot = (DLLFUNC_DLLLoadLoot) GetSym("DLLLoadLoot");
        Loot.cbAddLootTable = (DLLFUNC_AddLootTable) GetSym("AddLootTable");
        Loot.cbAddLootDrop = (DLLFUNC_AddLootDrop) GetSym("AddLootDrop");
        Loot.GetLootTable = (DLLFUNC_GetLootTable) GetSym("GetLootTable");
        Loot.GetLootDrop = (DLLFUNC_GetLootDrop) GetSym("GetLootDrop");
        Opcodes.GetEQOpcode = (DLLFUNC_GetEQOpcode) GetSym("GetEQOpcode");
        Opcodes.GetEmuOpcode = (DLLFUNC_GetEmuOpcode) GetSym("GetEmuOpcode");
        Opcodes.SetOpcodePair = (DLLFUNC_SetOpcodePair) GetSym("SetOpcodePair");
        Opcodes.DLLLoadOpcodes = (DLLFUNC_DLLLoadOpcodes) GetSym("DLLLoadOpcodes");
        Opcodes.ClearEQOpcodes = (DLLFUNC_ClearEQOpcodes) GetSym("ClearEQOpcodes");
        SkillCaps.LoadSkillCaps = (DLLFUNC_DLLLoadSkillCaps) GetSym("LoadSkillCaps");
        SkillCaps.GetSkillCap = (DLLFUNC_GetSkillCap) GetSym("GetSkillCap");
        SkillCaps.SetSkillCap = (DLLFUNC_SetSkillCap) GetSym("SetSkillCap");
        SkillCaps.ClearSkillCaps = (DLLFUNC_ClearSkillCaps) GetSym("ClearSkillCaps");
        SkillCaps.GetTrainLevel = (DLLFUNC_GetTrainLevel) GetSym("GetTrainLevel");


void LoadEMuShareMemDLL::ClearFunc() {
    Items.GetItem = 0;
    Items.IterateItems = 0;
    Items.cbAddItem = 0;
    Items.DLLLoadItems = 0;
    Doors.GetDoor = 0;
    Doors.cbAddDoor = 0;
    Doors.DLLLoadDoors = 0;
    NPCFactionList.DLLLoadNPCFactionLists = 0;
    NPCFactionList.GetNPCFactionList = 0;
    NPCFactionList.cbAddNPCFactionList = 0;
    NPCFactionList.cbSetFaction = 0;
    Loot.DLLLoadLoot = 0;
    Loot.cbAddLootTable = 0;
    Loot.cbAddLootDrop = 0;
    Loot.GetLootTable = 0;
    Loot.GetLootDrop = 0;
    Opcodes.GetEQOpcode = NULL;
    Opcodes.GetEmuOpcode = NULL;
    Opcodes.SetOpcodePair = NULL;
    Opcodes.DLLLoadOpcodes = NULL;
    Opcodes.ClearEQOpcodes = NULL;
    SkillCaps.LoadSkillCaps = NULL;
    SkillCaps.GetSkillCap = NULL;
    SkillCaps.SetSkillCap = NULL;
    SkillCaps.ClearSkillCaps = NULL;
    SkillCaps.GetTrainLevel = NULL;
    loaded = false;
}

I am curious about the SPELL_UNKNOWN and maxlevel referenced below. I would think current character level is the only thing that should factor into any spell files. I assume by maxlevel, it is referencing the rule set on the server? My server is set to level 70 max level in the rules, but I have a quest that lets players get to 75 and their spells disappear from their spell book when they zone. I don't see any reason to reference anything other than current level. Otherwise, is there a way I could change the max level references to refer to 75 instead of 70 which is what my rule is set for?

The SPELL_UNKNOWN stands out because the spells show up as "unknown spell" in the spell book and that is what is in the message when you #scribespells 75 for spell IDs over 9999.

client_packet.cpp

        //Validity check for memorized
        if(Admin() < minStatusToHaveInvalidSpells) {
            for (uint32 mem = 0; mem < MAX_PP_MEMSPELL; mem++)
            {
                if (m_pp.mem_spells[mem] < 1 || m_pp.mem_spells[mem] >= (unsigned int)SPDAT_RECORDS || spells[m_pp.mem_spells[mem]].classes[GetClass()-1] < 1 || spells[m_pp.mem_spells[mem]].classes[GetClass()-1] > GetLevel())
                    m_pp.mem_spells[mem] = SPELLBOOK_UNKNOWN;
            }
            int maxlvl = RuleI(Character, MaxLevel);
            for (uint32 bk = 0; bk < MAX_PP_SPELLBOOK; bk++)
            {
                if (m_pp.spell_book[bk] < 1
                    || m_pp.spell_book[bk] >= (unsigned int)SPDAT_RECORDS
                    || spells[m_pp.spell_book[bk]].classes[GetClass()-1] < 1
                    || spells[m_pp.spell_book[bk]].classes[GetClass()-1] > maxlvl)
                    m_pp.spell_book[bk] = SPELLBOOK_UNKNOWN;

        //I believe these effects are stripped off because if they
        //are not, they result in permanent effects on the player
        for (uint32 j1=0; j1 < BUFF_COUNT; j1++) {
            if (buffs[j1].spellid <= (int32)SPDAT_RECORDS) {
                for (uint32 x1=0; x1 < EFFECT_COUNT; x1++) {
                    switch (spells[buffs[j1].spellid].effectid[x1]) {
                        case SE_Charm:
                        //case SE_Rune:
                            buffs[j1].spellid = SPELL_UNKNOWN;
                            m_pp.buffs[j1].spellid = SPELLBOOK_UNKNOWN;
                            m_pp.buffs[j1].slotid = 0;
                            m_pp.buffs[j1].level = 0;
                            m_pp.buffs[j1].duration = 0;
                            m_pp.buffs[j1].effect = 0;
                            x1 = EFFECT_COUNT;
                            break;
                        case SE_Illusion:
                            if(m_pp.buffs[j1].persistant_buff != 1){ //anything other than 1=non persistant
                                buffs[j1].spellid = SPELL_UNKNOWN;
                                m_pp.buffs[j1].spellid = SPELLBOOK_UNKNOWN;
                                m_pp.buffs[j1].slotid = 0;
                                m_pp.buffs[j1].level = 0;
                                m_pp.buffs[j1].duration = 0;
                                m_pp.buffs[j1].effect = 0;
                                x1 = EFFECT_COUNT;

I am sure there is mostly irrelevant code in this post, but I just added anything that stood out to me. I am trying to confirm 100% wether the 9999 max spell id limit is determined by the Titanium client or the eqemu server. I think it would be a HUGE help to the community to find a way to allow more like 20k or 25k spell IDs.

If it is the client that limits this, is there any possible chance that the files could be changed or maybe update 1 or 2 from live that will up this? I highly doubt it, but worth asking lol.

[ChaosSlayer]

good work Trev!!
I need 20k spell limit like air - I am unable to make my new classes wihout them