Lua Scripting+Content Databasing system

[Drajor]

Originally Posted by c0ncrete

i'm getting the idea that nobody likes python but me around here.

nevertheless, my posts were more about wanting to see this lua thing work out than having someone embed python. from what i understand, lua will work just as well as perl, with a much smaller footprint than python, and easier integration than either of them.

I use about 30 tools written in python to create and manage the content on my server. <3 4 python.

[cavedude]

Originally Posted by Zaela_S

If the quest scripting option is the only thing that is wanted/justifiable, I'm fine just working on that and keeping the rest to myself. It would be a lot less work, and I don't really have anything to gain in working on it anyway, just figured I might as well offer. Integrating it into the Parser/QuestManager handler classes shouldn't be too hard, and I think that'll get it just where it's wanted.

Lua does seem to have a very minimalist, "if you want something done, write the module/library yourself" mindset behind it, which is fine for me but I can see where that would be problematic for a works-out-of-the-box project like EQEmu.

I'm actually more interested in the spawning stuff, to be honest. The quest scripting is more a it'd be nice to have it, but ultimately there is nothing wrong with Perl for that. But, don't kill yourself. If something can't be done easily, forget about it.

[Zaela_S]

Originally Posted by cavedude

I'm actually more interested in the spawning stuff, to be honest. The quest scripting is more a it'd be nice to have it, but ultimately there is nothing wrong with Perl for that. But, don't kill yourself. If something can't be done easily, forget about it.

Spawns would probably be the single easiest thing to implement without disturbing any of the existing systems -- just need to add a lua_spawn_id variable to NPCs, and then do all the LuaSpawn checks/processes after the Spawn2 versions. Wouldn't be any problem to have Lua-spawned NPCs use Perl quest scripts, of course.

Could probably have just the spawns and maybe path grids done in an evening or two. Then I'd just need to figure out where and how to commit stuff ;p

[sorvani]

Originally Posted by Zaela_S

Could probably have just the spawns and maybe path grids done in an evening or two. Then I'd just need to figure out where and how to commit stuff ;p

Just post a diff here for the server code changes

[lerxst2112]

It'll be more than a diff. Depending on the lua interpreter used it'll also be a static library + headers, or a bunch of c/cpp files and headers.

[sorvani]

If it is correctly optional a diff against the trunk should be viable.

[Harakiri23]

Just my 2cents,

converting the perl eqemu quest to lua isnt that big of a deal - just remember - perl only can parse perl =)

Example from convert for my own engine:

http://pastebin.com/fBQnYXv3

(havent implemented all methods so some are commented)

Writing the initial converter might have taken about 10h.

Originally Posted by cavedude

As you mentioned there isn't a whole lot Lua can do in the quest department that Perl can't.

Au contraire, it is a million times better than perl. Why would basically every game in the last 15+ years use lua as scripting backend?

Its small, no depencies, can be embedded easily, is 10times faster than perl.

Oh and the biggest aspects - coroutines!

npc:Say("Something...")
Wait(1second)
npc:Say("Something more..")
SpawnSomething()
Wait(10seconds)

you are basically writing a movie scripting without going forth and back with throwing timed events and if(timer==1) etc...

This is also not a real thread, just everything on the stack. No multithreading issues or objects going out of scope.

The most important thing, if you really want to redo the scripting engine - going back to the drawing board before coding anything really - this is the hardest part - designing a good API itself that is extend-able, takes more time then actual coding..

Good luck!

[cavedude]

Originally Posted by Harakiri23

The most important thing, if you really want to redo the scripting engine - going back to the drawing board before coding anything really - this is the hardest part - designing a good API itself that is extend-able, takes more time then actual coding..

That's the key right there, most of the problems we do have with Perl are mainly due to the way it was implemented into EQEmu, and not inherent to the language itself.

Au contraire, it is a million times better than perl. Why would basically every game in the last 15+ years use lua as scripting backend?

As far as games are concerned I fully agree with you. Lua is light years ahead of Perl. However, to be fair I think Perl has the edge otherwise. I use it to do so much of my bidding over at PEQ, at work, and I even have cron running Perl scripts on my personal boxes that do various things for me. Perl is clunky and slow no doubt, but it's robust and external libraries make it even more so.

What I meant is that there is no immediate need for Lua in regards to scripting, Perl handles our needs just fine. We haven't hit a wall yet forcing us to say well, Perl has to go. Though, if Lua is added to our code base, I suspect that will quickly change.

Its small, no depencies, can be embedded easily, is 10times faster than perl.

Oh and the biggest aspects - coroutines!

Preaching to the choir there, sir. Perl's implementing coroutines in version 6, but that's too little too late I think. Still, it'll come in handy for my utility usage of the language.

[Harakiri23]

Originally Posted by cavedude

However, to be fair I think Perl has the edge otherwise. I use it to do so much of my bidding over at PEQ, at work, and I even have cron running Perl scripts on my personal boxes that do various things for me. Perl is clunky and slow no doubt, but it's robust and external libraries make it even more so.

sorry, i was referring to lua as more powerful as in "for scripting engines in games" due small footprint, easy to read and coroutines, also a selection of binding wrappers

obviously perl takes the cake - it can do anything - but its a heavy weight

i considered python and lua actually, while i would have preferred python - its also bloated already

[Zaela_S]

I'm not very familiar with coroutines, would probably be up to someone else to figure out a good way to implement them into the event-based system if it gets that far ;p

Originally Posted by cavedude

That's the key right there, most of the problems we do have with Perl are mainly due to the way it was implemented into EQEmu, and not inherent to the language itself.

Again, I'm not very familiar with the Perl system, but I do see some bits and pieces of the Perl code that seem eerily similar to qst (which I'm more familiar with), so I think I might understand what you mean, at least to some extent.

I'm probably not the best person to try to write a fully fleshed-out quest parser; I don't have that much experience with Lua and mostly just make things up as they occur to me. Just for the sake of it though, a random grab bag of features in my Lua quest parser that seem significant from my point of view:

*Mobs, items, spells and timers are all treated as objects with their own methods and can be stored in variables etc. Adding new methods in either C++ or Lua is fairly straightforward.

*Two scripts are run as a zone is booting, before NPCs spawn: /quests/global/zone_init.lua and /quests/<zone shortname>/zone_init.lua; these give the opportunity to initialize variables or, more likely, to set user defined functions and/or hook extra code onto "core" functions from C++, either globally or for a particular zone (the zone-specific one runs second in case you want to redefine something from the global one). Can also use the opportunity to rename some functions if you don't like the default names. Some standard functions are defined in the global zone_init.lua by default, e.g. mob:say() is just a call to mob:text() with some pre-defined parameters, and GetDist(), IsBehind(), and InCone() are all mostly just math that Lua can handle itself, only making calls to C++ to find coords/heading.

*EVENT_SIGNAL's trigger does not go through C++ at all, letting us pass any kind of data in the signal, including tables, references to mobs/etc, and functions.

*Every EVENT call attempts to trigger two scripted functions: EVENT_<WHATEVER> and GLOBAL_<WHATEVER>. For clients the use is clear enough: GLOBAL versions of events should be defined in playerglobal.lua, while zone-specific EVENTs should be defined in a zone's player.lua. For NPCs, it's up to the user if/how to use the GLOBAL events -- they don't need to be truly global, though they easily can be. They could be used within a zone as a sort of inherited/shared behavior without clashing with normal npc-specific EVENTs, or similar. Lua is fast and can check whether an EVENT/GLOBAL exists without doing much processing, so the double check is not noticable, even if it's rarely used.

*General Lua features: every Mob is given its own environment to store its variables and functions, but these environments are ultimately just tables sitting in specific variables within the global table; we can easily peek into MobA's environment from within the environment/script of MobB. This can be questionable and messy, but there are some good uses, like defining the EVENTs for an add's "script" from within the script of the mob that spawns it (also making it easy to vary behaviors from add to add without needing to juggle multiple script files -- at the extreme end, you could redirect the adds' environments to tables within spawner's environment, completely encapsulating them). The global table can also be accessed easily, making it a convenient place to put data where multiple mobs/scripts can refer to it (and without being tied to the lifespan of any one mob). Mob environments also automatically inherit from the global table, offering another way to define default EVENTs (for example, if you wanted most of the NPCs in a city to share the same EVENT_SAY, you could just plop it straight into the global table; any NPCs given an EVENT_SAY in their own script will "overwrite" the one inherited from the global table) or take it a bit further and have some EVENT inheritance hierarchies (though that can get messy quickly).


...Some of that might not fit too well into the existing framework, though. Not sure how important keeping things familiar and hopefully easily to convert over from Perl is compared to letting it go its own way without having to bend to some possibly-questionable decisions made however many years ago.

[Zaela_S]

Originally Posted by Harakiri23

also a selection of binding wrappers

This thread has kind of died down and maybe there wasn't that much interest in the first place, but: I just re-read this and want to take a moment to make a case against using any auto-generated bindings/wrappers if a Lua quest scripting system does become a thing.

I realize that manually writing functions to expose C++ will take a lot of time and that in the process some bugs will inevitably creep in, requiring even more time to notice and fix them. But I want to argue that going that route will ultimately leave you with a more sensible, user-friendly system oriented towards its actual purpose -- i.e., writing quest and combat scripts for mobs -- rather than the as-close-to-the-C++-as-possible result a wrapper will likely give you (as far as I know, anyway). Main points:

Simplification

There are some things the script writer should just never have to worry about. CastToNPC/Client is probably the most obvious example. Handing a script function a Mob reference should always be enough -- any needed checks for NPC or Client should happen on the C++ side and split the function's behavior as necessary. (Maybe the Perl system has already eliminated the need for these, but I see them used in a few old scripts up on svn so I dunno.)

For another example, consider Mob::CastSpell(). Specifically, the fact that it takes an entity id for the target rather than a Mob reference. We know that it always expects an entity id. But we also know that, in our script system, whenever we want to cast a spell on something we will always have a Mob reference to the target before we have the target's entity id. Therefore, any function exposing CastSpell() to our script system should get the entity id implicitly. There's no reason every script-writer should need to write this

self:castspell(437,target:GetID())

every time when they could just write this

self:castspell(437,target)

instead.

Furthermore, CastSpell() does not have any default targetting. It seems reasonable to expect that if a script-writer writes this

self:castspell(437)

they should expect that the spell will self-target the given caster (or even better, target the caster's target if they have one and the spell is detrimental, and target the caster otherwise). We're given a fair amount of leeway to write pseudo-overloaded functions like this, so it seem sensible to use this to provide defaults that the C++ function in question does not.

Limiting Redundancy

Going back to automating NPC/Client casts and checks, consider the case of instantaneous intra-zone movement. When we want to teleport an NPC, we use GMMove(). When we want to teleport a client, we generally use MovePC while specifying the zone they're already in, since the movement is cleaner (unless we want to put them in mid-air, anyway).

But again, the script-writer should not need to know about this distinction. From the point of view of the user, all we care about is that the function instantly moves the given Mob. As such the script system should provide a single function that does this for both NPCs and Clients, with the GMMove/MovePC distinction being handled automatically on the C++ side. For Clients, clean movement is the most sensible default, but we can easily provide the mid-air variant with a single extra boolean parameter, like so:

target:move(x,y,z,heading) --clean
target:move(x,y,z,heading,true) --mid-air

As a simpler example, consider these two Client functions: IsInAGuild() and IsInGuild(id). It's a little thing, but we can save a function here too. If we see a call like this

if client:InGuild() then
...

with no argument, it's reasonable to assume that we're checking if the client is in any guild at all. On the other hand, if we see something like this

local guild_id = other:GetGuildID()
if client:InGuild(guild_id) then
...

it's reasonable to assume that we're checking if the client is in the guild of the given id. The less the user has to remember little distinctions between functions, and the less functions there are in total, the less they're going to be tripped up. There are lots of little examples like that -- anywhere the script system can combine similar functions without making things more obscure, it should.

Opportunistic Improvements

The other two are kind of about this as well: when you take the time to look at functions one by one and think about how they're going to be used, it gives you a good chance to customize how the function is handled, to make it more sensible without sacrificing any functionality. At the same time, it also gives the chance for you to say "you know, this function should really do x, in case anyone ever wants to do y."

As an example, on my server I made the Damage functions ultimately return the final damage value caused to the target, so that this value could be returned to Lua when used in scripts. This allows for easy scripted lifetaps at any percentage of the dealt damage, or simple tracking of dealt damage for other purposes. Or, if you feel like disabling the standard AI melee and making an NPC's melee damage cycle purely script-based, you can detect when the NPC's attacks are dodged, parried, riposted, etc. Just a general functionality that should really be available to anyone who writes custom raid-level encounters.


Maybe we could go back and do most these things after auto-generating functions with a wrapper, but I think much of the impetus would be lost if the functions are already there and, in the worst case, by the time someone gets to working on it they'll run into that dreaded situation where they have to choose between making an improvement or maintaining compatibility with the scripts that have already been made. Maybe that could be avoided by just using modules to provide simplified function calls on the Lua side of things, but modules would be optional and, in the end, we'd still have a needlessly bloated, needlessly unfriendly base system. No good!


Maybe this is all dumb and I just don't know enough about how wrappers work. In any case I think it's probably clear that opening up content development is kind of my main area of interest ;p

[Kayen]

At least my opinion on this as someone who does a lot of heavy duty event scripting with PERL for my servers is that I just can't contemplate the need to change the scripting language when I can literally do anything my heart desires with PERL and easily at that. It does take some time to learn the functions related to eqemu but honestly it is so powerful once you have it down. Is every function super optimal probably not, but most of the examples your citing, at least to me just elicit a visceral reaction of 'who cares that isn't a big deal' at least not enough to warrant changing languages. Please don't take that the wrong way, I am not trying to be negative towards your work I respect what your doing..

I think it would ultimately negatively impact the project to fork quest scripting languages. Just because something can be done doesn't necessarily mean it should.

Anyways just my opinion.

Kayen
GM Storm Haven

[c0ncrete]

there are at least a few of us that are VERY interested. some of us just spend loads of time reading and poking around with things while we're learning about them instead of commenting or asking questions.

as far as how the perl interfaces to c++ are implemented, i couldn't agree more. there are a number of functions that could be consolidated, expanded upon, moved, or just corrected.

for example... the functionality of Mob::EntityVariableExists, Mob::GetEntityVariable, and Mob::SetEntityVariable could be combined into Mob::EntityVariable(evName, [evValue]), and it would be assumed that we wanted to return the value (evValue) of the variable name (evName) if no value was passed. there are tons of other things like that and and the ones you covered that i've been puttering around with to see what i can come up with on my personal server.

i also realize that once things are implemented and used a bit, it's difficult to want to change them for everyone because of migration issues. do you leave the old code in there for a while until people get a chance to move it, or you just yank it out unceremoniously?

[ghanja]

Originally Posted by c0ncrete

there are at least a few of us that are VERY interested. some of us just spend loads of time reading and poking around with things while we're learning about them instead of commenting or asking questions.

Quoted for truth.

[addingice]

Do you have this lua variant on github? I would love to pull this from you.