Lua Scripting+Content Databasing system

[Harakiri23]

Originally Posted by cavedude

However, to be fair I think Perl has the edge otherwise. I use it to do so much of my bidding over at PEQ, at work, and I even have cron running Perl scripts on my personal boxes that do various things for me. Perl is clunky and slow no doubt, but it's robust and external libraries make it even more so.

sorry, i was referring to lua as more powerful as in "for scripting engines in games" due small footprint, easy to read and coroutines, also a selection of binding wrappers

obviously perl takes the cake - it can do anything - but its a heavy weight

i considered python and lua actually, while i would have preferred python - its also bloated already

[Zaela_S]

Originally Posted by Harakiri23

also a selection of binding wrappers

This thread has kind of died down and maybe there wasn't that much interest in the first place, but: I just re-read this and want to take a moment to make a case against using any auto-generated bindings/wrappers if a Lua quest scripting system does become a thing.

I realize that manually writing functions to expose C++ will take a lot of time and that in the process some bugs will inevitably creep in, requiring even more time to notice and fix them. But I want to argue that going that route will ultimately leave you with a more sensible, user-friendly system oriented towards its actual purpose -- i.e., writing quest and combat scripts for mobs -- rather than the as-close-to-the-C++-as-possible result a wrapper will likely give you (as far as I know, anyway). Main points:

Simplification

There are some things the script writer should just never have to worry about. CastToNPC/Client is probably the most obvious example. Handing a script function a Mob reference should always be enough -- any needed checks for NPC or Client should happen on the C++ side and split the function's behavior as necessary. (Maybe the Perl system has already eliminated the need for these, but I see them used in a few old scripts up on svn so I dunno.)

For another example, consider Mob::CastSpell(). Specifically, the fact that it takes an entity id for the target rather than a Mob reference. We know that it always expects an entity id. But we also know that, in our script system, whenever we want to cast a spell on something we will always have a Mob reference to the target before we have the target's entity id. Therefore, any function exposing CastSpell() to our script system should get the entity id implicitly. There's no reason every script-writer should need to write this

self:castspell(437,target:GetID())

every time when they could just write this

self:castspell(437,target)

instead.

Furthermore, CastSpell() does not have any default targetting. It seems reasonable to expect that if a script-writer writes this

self:castspell(437)

they should expect that the spell will self-target the given caster (or even better, target the caster's target if they have one and the spell is detrimental, and target the caster otherwise). We're given a fair amount of leeway to write pseudo-overloaded functions like this, so it seem sensible to use this to provide defaults that the C++ function in question does not.

Limiting Redundancy

Going back to automating NPC/Client casts and checks, consider the case of instantaneous intra-zone movement. When we want to teleport an NPC, we use GMMove(). When we want to teleport a client, we generally use MovePC while specifying the zone they're already in, since the movement is cleaner (unless we want to put them in mid-air, anyway).

But again, the script-writer should not need to know about this distinction. From the point of view of the user, all we care about is that the function instantly moves the given Mob. As such the script system should provide a single function that does this for both NPCs and Clients, with the GMMove/MovePC distinction being handled automatically on the C++ side. For Clients, clean movement is the most sensible default, but we can easily provide the mid-air variant with a single extra boolean parameter, like so:

target:move(x,y,z,heading) --clean
target:move(x,y,z,heading,true) --mid-air

As a simpler example, consider these two Client functions: IsInAGuild() and IsInGuild(id). It's a little thing, but we can save a function here too. If we see a call like this

if client:InGuild() then
...

with no argument, it's reasonable to assume that we're checking if the client is in any guild at all. On the other hand, if we see something like this

local guild_id = other:GetGuildID()
if client:InGuild(guild_id) then
...

it's reasonable to assume that we're checking if the client is in the guild of the given id. The less the user has to remember little distinctions between functions, and the less functions there are in total, the less they're going to be tripped up. There are lots of little examples like that -- anywhere the script system can combine similar functions without making things more obscure, it should.

Opportunistic Improvements

The other two are kind of about this as well: when you take the time to look at functions one by one and think about how they're going to be used, it gives you a good chance to customize how the function is handled, to make it more sensible without sacrificing any functionality. At the same time, it also gives the chance for you to say "you know, this function should really do x, in case anyone ever wants to do y."

As an example, on my server I made the Damage functions ultimately return the final damage value caused to the target, so that this value could be returned to Lua when used in scripts. This allows for easy scripted lifetaps at any percentage of the dealt damage, or simple tracking of dealt damage for other purposes. Or, if you feel like disabling the standard AI melee and making an NPC's melee damage cycle purely script-based, you can detect when the NPC's attacks are dodged, parried, riposted, etc. Just a general functionality that should really be available to anyone who writes custom raid-level encounters.


Maybe we could go back and do most these things after auto-generating functions with a wrapper, but I think much of the impetus would be lost if the functions are already there and, in the worst case, by the time someone gets to working on it they'll run into that dreaded situation where they have to choose between making an improvement or maintaining compatibility with the scripts that have already been made. Maybe that could be avoided by just using modules to provide simplified function calls on the Lua side of things, but modules would be optional and, in the end, we'd still have a needlessly bloated, needlessly unfriendly base system. No good!


Maybe this is all dumb and I just don't know enough about how wrappers work. In any case I think it's probably clear that opening up content development is kind of my main area of interest ;p

[Kayen]

At least my opinion on this as someone who does a lot of heavy duty event scripting with PERL for my servers is that I just can't contemplate the need to change the scripting language when I can literally do anything my heart desires with PERL and easily at that. It does take some time to learn the functions related to eqemu but honestly it is so powerful once you have it down. Is every function super optimal probably not, but most of the examples your citing, at least to me just elicit a visceral reaction of 'who cares that isn't a big deal' at least not enough to warrant changing languages. Please don't take that the wrong way, I am not trying to be negative towards your work I respect what your doing..

I think it would ultimately negatively impact the project to fork quest scripting languages. Just because something can be done doesn't necessarily mean it should.

Anyways just my opinion.

Kayen
GM Storm Haven

[c0ncrete]

there are at least a few of us that are VERY interested. some of us just spend loads of time reading and poking around with things while we're learning about them instead of commenting or asking questions.

as far as how the perl interfaces to c++ are implemented, i couldn't agree more. there are a number of functions that could be consolidated, expanded upon, moved, or just corrected.

for example... the functionality of Mob::EntityVariableExists, Mob::GetEntityVariable, and Mob::SetEntityVariable could be combined into Mob::EntityVariable(evName, [evValue]), and it would be assumed that we wanted to return the value (evValue) of the variable name (evName) if no value was passed. there are tons of other things like that and and the ones you covered that i've been puttering around with to see what i can come up with on my personal server.

i also realize that once things are implemented and used a bit, it's difficult to want to change them for everyone because of migration issues. do you leave the old code in there for a while until people get a chance to move it, or you just yank it out unceremoniously?

[ghanja]

Originally Posted by c0ncrete

there are at least a few of us that are VERY interested. some of us just spend loads of time reading and poking around with things while we're learning about them instead of commenting or asking questions.

Quoted for truth.

[addingice]

Do you have this lua variant on github? I would love to pull this from you.

[Zaela_S]

I'm in the process of re-writing everything to be cleaner, better organized, more compatible (the itemdb in particular I need to redo so that it'll be easier to convert directly to and from SQL) and have better error checking. I still need to learn the basics of Git too. But I do hope to get a test branch up somewhere while I work on it, hopefully not too long from now.

[addingice]

Pros of Lua vs Perl:

1. Smaller footprint, memory and cpu wise.
   
2. Can be distributed with the code base.
   
3. Removes an outside dependency.
   
4. Popular among game enthusiasts.
   
5. If it's used to replace Perl & MySQL then we also have the option of creating 'eq server in a box' linux distribution with a great deal less headache since it's only the server to deal with and not the server and sql issues.
   
6. Item look up side can become far more dynamic for the custom users. Easy set bonuses, for example.
   
7. Lua to c interface is far simpler and cleaner. Means that whole interfacing point could become much cleaner for scripter's. Examples where shown in the thread. Could lead to a standard library for EQ Lua scripts since it's far simpler to reuse.
   
8. Some tools might be easier to write since we can reuse much of the lua script code directly into the tools themselves. This is difficult to do with Perl as it stands.
   

Cons of Lua vs Perl:

1. Switching systems means we have people who need to learn Lua who all ready know Perl.
   
2. Perl has a huge number of packages all ready made (may not be relevant considering our current uses).
   
3. need to rewrite many of the current quests to use Lua.
   
4. Some will not switch since they have a large number of custom scripts.
   
5. Under some conditions the Lua GC can become finicky. (rare but can happen, usually just a set gc level change at the global scope)
   

Pro's of Lua vs MySQL:

1. Imperative as well as Declarative for those who want more dynamic content behavior or are used to imperative but not declarative systems.
2. A chance to clean up the item table and other components since it's getting a little....crazy? yeah. crazy.
3. Lua embedding is just as supported as SQL in almost all cases.
4. Easy starting out for modders / content creators. All that is required is a text editor.

Con's of Lua vs MySQL:

1. SQL is a very common language. Many developers know it. Lua? not as much.
   
2. Thousands of SQL tools out there.
   
3. very very very easy to do global modifications in SQL, may not (but doesn't have to be) easy to do this in Lua, depends on implementation.

What else am I missing? I'll edit as I people add.

[Harakiri23]

Originally Posted by Zaela_S

This thread has kind of died down and maybe there wasn't that much interest in the first place, but: I just re-read this and want to take a moment to make a case against using any auto-generated bindings/wrappers if a Lua quest scripting system does become a thing.

You want to simplify things but still dont want to use a wrapper ? I skimmed over your code - and the problem is - the same as with the perl system - only 1% will understand it and maintaining and extending will be done by copy&paste of existing code without really understanding it (e.g. people currently just paste code in the embperl classes for new functions - but thats not how its done, there should be a clean header file to perl exported functions and then the perlxs/convert is called which generates all the func bodies).

Whats worse - lua is stack based - thats a whole new level of understanding for most people. For perl, you were unable to expose complex objects (func args) to the quest system - now with direct lua calls you have the same issue - every complex object has to be manually coded - where in bindings its done in one line.

Look how WoW emus, EQ2, Ryzoom and other implemented their event system - and compare all the bad things and improve upon this - (BTW - they all use bindings). A quick example how a binding could look like for getters or setters

	  .def("GetName", &NPC::GetName)	  
	  .def("GetCleanName", &NPC::GetCleanName)	  
	  .def("GetID", &NPC::GetID)	  
	  .def("GetRace", &NPC::GetRace)	  
	  .def("GetClass", &NPC::GetClass)	  
          .def("Say", &NPC::Say)

Thats everything you need - number of args, arg types, etc all automatically handled - everybody understands this. No need for

static int Lua_GetName(lua_State* L) {
	int num_args = lua_gettop(L);
	if (!num_args || !lua_isuserdata(L,1))
		return 0;

	Mob* check = *(Mob**)lua_touserdata(L,1);
	if (check) {
		if (check->IsNPC()) {
			if (num_args > 1 && lua_isboolean(L,2) && lua_toboolean(L,2)) {
				lua_pushstring(L,check->GetName());
			}
			else {
				lua_pushstring(L,check->GetCleanName());
			}
		}
		else {
			lua_pushstring(L,check->GetName());
		}
		return 1;
	}
	return 0;
}

You still need at least one or two who understand lower level lua api - e.g. ever thought about garbage collection ? If you dont have the right approach here your embedded lua will be very slow and the memory will grow very quickly.

More things you need to take care of : script scoping - want to make sure that nobody can overwrite a var of another npc ? or do you want that ?

How can entities interact with each other directly?

[Zaela_S]

Originally Posted by Harakiri23

You want to simplify things but still dont want to use a wrapper ?

Simplify things for people who write scripts, not so much people who write the script system. There's a big difference. Using a wrapper is pretty much the opposite from what I've seen -- makes things easy for the system writer, but no real help to casual script writers.

Originally Posted by Harakiri23

You still need at least one or two who understand lower level lua api

I understand it, and I'm the one writing all the functions, so that's fine with me.

For the two or three who might care, currently have a work-in-progress repo here; trying to keep the repo wiki up-to-date with all the EVENTs and object functions and eventually some script examples, as well as how-to's on module writing and adding new object types etc.

[addingice]

Another positive to the Lua system, the more we move into it the more we can 'live update' while the system is running. not sure how much of an advantage it is, but hey there you go.

[Zaela_S]

Added scriptability for the SoF+ respawn window by way of an EVENT_RESPAWN_WINDOW. This allows respawn options to be customized: options can be added or removed based on the Client's level, the current zone, qglobals; options to respawn in guild hall, server hub zone, options to return to a questgiver if you were in the middle of a quest event, etc.

Full description and some example usage can be found at https://github.com/Zaela/LuaEQEmu/wi...respawn_window

Someone may want to make a Perl version of this if nothing else. I think it's a pretty neat thing.

Also, there is a bug with the respawn window and aggro; when the Client is hovering for respawn, they will continually aggro whatever killed them (aggro, lose aggro because the client is not a valid attack target, then aggro again, rinse and repeat) and mess up the Client's x-targets. This can spam trigger EVENT_AGGRO and possibly EVENT_COMBAT. To fix, replace this in aggro.cpp:

bool Mob::CheckWillAggro(Mob *mob) {
	if(!mob)
		return false;
	_ZP(Mob_CheckWillAggro);

	//sometimes if a client has some lag while zoning into a dangerous place while either invis or a GM
	//they will aggro mobs even though it's supposed to be impossible, to lets make sure we've finished connecting
	if(mob->IsClient() && !mob->CastToClient()->ClientFinishedLoading())
		return false;

with this:

bool Mob::CheckWillAggro(Mob *mob) {
	if(!mob)
		return false;
	_ZP(Mob_CheckWillAggro);

	//sometimes if a client has some lag while zoning into a dangerous place while either invis or a GM
	//they will aggro mobs even though it's supposed to be impossible, to lets make sure we've finished connecting
	if (mob->IsClient()) { 
		if (!mob->CastToClient()->ClientFinishedLoading())
			return false;
		//clients will continually re-aggro whatever killed them if respawn hover is enabled,
		//repeatedly triggering EVENT_COMBAT/AGGRO and messing up x-targets; this fixes that
		if (mob->CastToClient()->IsHoveringForRespawn())
			return false;
	}