Lua Scripting+Content Databasing system

[c0ncrete]

there's another post containing diffs (including one for cmake) to build what has been released to the public thus far.

http://www.eqemulator.org/forums/showthread.php?t=36401

[Zaela_S]

I'm in the process of re-writing everything to be cleaner, better organized, more compatible (the itemdb in particular I need to redo so that it'll be easier to convert directly to and from SQL) and have better error checking. I still need to learn the basics of Git too. But I do hope to get a test branch up somewhere while I work on it, hopefully not too long from now.

[addingice]

Pros of Lua vs Perl:

1. Smaller footprint, memory and cpu wise.
   
2. Can be distributed with the code base.
   
3. Removes an outside dependency.
   
4. Popular among game enthusiasts.
   
5. If it's used to replace Perl & MySQL then we also have the option of creating 'eq server in a box' linux distribution with a great deal less headache since it's only the server to deal with and not the server and sql issues.
   
6. Item look up side can become far more dynamic for the custom users. Easy set bonuses, for example.
   
7. Lua to c interface is far simpler and cleaner. Means that whole interfacing point could become much cleaner for scripter's. Examples where shown in the thread. Could lead to a standard library for EQ Lua scripts since it's far simpler to reuse.
   
8. Some tools might be easier to write since we can reuse much of the lua script code directly into the tools themselves. This is difficult to do with Perl as it stands.
   

Cons of Lua vs Perl:

1. Switching systems means we have people who need to learn Lua who all ready know Perl.
   
2. Perl has a huge number of packages all ready made (may not be relevant considering our current uses).
   
3. need to rewrite many of the current quests to use Lua.
   
4. Some will not switch since they have a large number of custom scripts.
   
5. Under some conditions the Lua GC can become finicky. (rare but can happen, usually just a set gc level change at the global scope)
   

Pro's of Lua vs MySQL:

1. Imperative as well as Declarative for those who want more dynamic content behavior or are used to imperative but not declarative systems.
2. A chance to clean up the item table and other components since it's getting a little....crazy? yeah. crazy.
3. Lua embedding is just as supported as SQL in almost all cases.
4. Easy starting out for modders / content creators. All that is required is a text editor.

Con's of Lua vs MySQL:

1. SQL is a very common language. Many developers know it. Lua? not as much.
   
2. Thousands of SQL tools out there.
   
3. very very very easy to do global modifications in SQL, may not (but doesn't have to be) easy to do this in Lua, depends on implementation.

What else am I missing? I'll edit as I people add.

[orionsun]

I'll edit as I people add.

Welcome to the forums, let us know how that works out for you.

[wolfwalkereci]

I know nothing at all about Lua but it sounds pretty cool.

[KLS]

Lua and MySQL aren't even in the same problem domain.

[addingice]

Lua and MySQL aren't even in the same problem domain.

I understand that, but it can be used in the same way, if we want. I'm just saying it's an option, not that we *should*

[KLS]

I think it's somewhat impractical is all.

Also a few of the developers do have plans to somewhere down the road change up the quest system and the database system but we're not close enough to be ready to talk about it yet.

[Zaela_S]

I like Lua for content databasing for a few reasons:

1) Content data is essentially static -- it only changes when new stuff is put in there, and doesn't really change at runtime. The only thing you really get out of having it be SQL is searchability and the ability to invalidate a whole table in a single brainfart. (Oh, and making sweeping changes, which I guess is nice for this project; I hope to make at least items and spells convertible back and forth from SQL, which would still allow that.)

2) For some things (like the spawnpoint and path grid stuff I put up) open-ended scriptability is a lot simpler than having to add new fields (sometimes new tables) and new code for every little new abstract functionality or feature you want to add.

3) Having everything organized into the folder of the zone it belongs to is nice. Not that e.g. the NPC database could not be changed to include zone and reduce the need for global NPCID numbers, but that's probably one of those "too late to consider changing" things.

Definitely would not want to force anyone to switch. But there are reasons both for and against.

[Harakiri23]

Originally Posted by Zaela_S

This thread has kind of died down and maybe there wasn't that much interest in the first place, but: I just re-read this and want to take a moment to make a case against using any auto-generated bindings/wrappers if a Lua quest scripting system does become a thing.

You want to simplify things but still dont want to use a wrapper ? I skimmed over your code - and the problem is - the same as with the perl system - only 1% will understand it and maintaining and extending will be done by copy&paste of existing code without really understanding it (e.g. people currently just paste code in the embperl classes for new functions - but thats not how its done, there should be a clean header file to perl exported functions and then the perlxs/convert is called which generates all the func bodies).

Whats worse - lua is stack based - thats a whole new level of understanding for most people. For perl, you were unable to expose complex objects (func args) to the quest system - now with direct lua calls you have the same issue - every complex object has to be manually coded - where in bindings its done in one line.

Look how WoW emus, EQ2, Ryzoom and other implemented their event system - and compare all the bad things and improve upon this - (BTW - they all use bindings). A quick example how a binding could look like for getters or setters

	  .def("GetName", &NPC::GetName)	  
	  .def("GetCleanName", &NPC::GetCleanName)	  
	  .def("GetID", &NPC::GetID)	  
	  .def("GetRace", &NPC::GetRace)	  
	  .def("GetClass", &NPC::GetClass)	  
          .def("Say", &NPC::Say)

Thats everything you need - number of args, arg types, etc all automatically handled - everybody understands this. No need for

static int Lua_GetName(lua_State* L) {
	int num_args = lua_gettop(L);
	if (!num_args || !lua_isuserdata(L,1))
		return 0;

	Mob* check = *(Mob**)lua_touserdata(L,1);
	if (check) {
		if (check->IsNPC()) {
			if (num_args > 1 && lua_isboolean(L,2) && lua_toboolean(L,2)) {
				lua_pushstring(L,check->GetName());
			}
			else {
				lua_pushstring(L,check->GetCleanName());
			}
		}
		else {
			lua_pushstring(L,check->GetName());
		}
		return 1;
	}
	return 0;
}

You still need at least one or two who understand lower level lua api - e.g. ever thought about garbage collection ? If you dont have the right approach here your embedded lua will be very slow and the memory will grow very quickly.

More things you need to take care of : script scoping - want to make sure that nobody can overwrite a var of another npc ? or do you want that ?

How can entities interact with each other directly?

[Zaela_S]

Originally Posted by Harakiri23

You want to simplify things but still dont want to use a wrapper ?

Simplify things for people who write scripts, not so much people who write the script system. There's a big difference. Using a wrapper is pretty much the opposite from what I've seen -- makes things easy for the system writer, but no real help to casual script writers.

Originally Posted by Harakiri23

You still need at least one or two who understand lower level lua api

I understand it, and I'm the one writing all the functions, so that's fine with me.

For the two or three who might care, currently have a work-in-progress repo here; trying to keep the repo wiki up-to-date with all the EVENTs and object functions and eventually some script examples, as well as how-to's on module writing and adding new object types etc.

[addingice]

Another positive to the Lua system, the more we move into it the more we can 'live update' while the system is running. not sure how much of an advantage it is, but hey there you go.

[Zaela_S]

Added scriptability for the SoF+ respawn window by way of an EVENT_RESPAWN_WINDOW. This allows respawn options to be customized: options can be added or removed based on the Client's level, the current zone, qglobals; options to respawn in guild hall, server hub zone, options to return to a questgiver if you were in the middle of a quest event, etc.

Full description and some example usage can be found at https://github.com/Zaela/LuaEQEmu/wi...respawn_window

Someone may want to make a Perl version of this if nothing else. I think it's a pretty neat thing.

Also, there is a bug with the respawn window and aggro; when the Client is hovering for respawn, they will continually aggro whatever killed them (aggro, lose aggro because the client is not a valid attack target, then aggro again, rinse and repeat) and mess up the Client's x-targets. This can spam trigger EVENT_AGGRO and possibly EVENT_COMBAT. To fix, replace this in aggro.cpp:

bool Mob::CheckWillAggro(Mob *mob) {
	if(!mob)
		return false;
	_ZP(Mob_CheckWillAggro);

	//sometimes if a client has some lag while zoning into a dangerous place while either invis or a GM
	//they will aggro mobs even though it's supposed to be impossible, to lets make sure we've finished connecting
	if(mob->IsClient() && !mob->CastToClient()->ClientFinishedLoading())
		return false;

with this:

bool Mob::CheckWillAggro(Mob *mob) {
	if(!mob)
		return false;
	_ZP(Mob_CheckWillAggro);

	//sometimes if a client has some lag while zoning into a dangerous place while either invis or a GM
	//they will aggro mobs even though it's supposed to be impossible, to lets make sure we've finished connecting
	if (mob->IsClient()) { 
		if (!mob->CastToClient()->ClientFinishedLoading())
			return false;
		//clients will continually re-aggro whatever killed them if respawn hover is enabled,
		//repeatedly triggering EVENT_COMBAT/AGGRO and messing up x-targets; this fixes that
		if (mob->CastToClient()->IsHoveringForRespawn())
			return false;
	}

[Astal]

Originally Posted by addingice

Pros of Lua vs Perl:

1. Smaller footprint, memory and cpu wise.
   
2. Can be distributed with the code base.
   
3. Removes an outside dependency.
   
4. Popular among game enthusiasts.
   
5. If it's used to replace Perl & MySQL then we also have the option of creating 'eq server in a box' linux distribution with a great deal less headache since it's only the server to deal with and not the server and sql issues.
   
6. Item look up side can become far more dynamic for the custom users. Easy set bonuses, for example.
   
7. Lua to c interface is far simpler and cleaner. Means that whole interfacing point could become much cleaner for scripter's. Examples where shown in the thread. Could lead to a standard library for EQ Lua scripts since it's far simpler to reuse.
   
8. Some tools might be easier to write since we can reuse much of the lua script code directly into the tools themselves. This is difficult to do with Perl as it stands.
   

Cons of Lua vs Perl:

1. Switching systems means we have people who need to learn Lua who all ready know Perl.
   
2. Perl has a huge number of packages all ready made (may not be relevant considering our current uses).
   
3. need to rewrite many of the current quests to use Lua.
   
4. Some will not switch since they have a large number of custom scripts.
   
5. Under some conditions the Lua GC can become finicky. (rare but can happen, usually just a set gc level change at the global scope)
   

Pro's of Lua vs MySQL:

1. Imperative as well as Declarative for those who want more dynamic content behavior or are used to imperative but not declarative systems.
2. A chance to clean up the item table and other components since it's getting a little....crazy? yeah. crazy.
3. Lua embedding is just as supported as SQL in almost all cases.
4. Easy starting out for modders / content creators. All that is required is a text editor.

Con's of Lua vs MySQL:

1. SQL is a very common language. Many developers know it. Lua? not as much.
   
2. Thousands of SQL tools out there.
   
3. very very very easy to do global modifications in SQL, may not (but doesn't have to be) easy to do this in Lua, depends on implementation.

What else am I missing? I'll edit as I people add.

Very nice, i was gonna ask this. Now I Know, my brain farted when he said switch from a DB to LUA i was like how will that work?

[Wolftousen]

Awesome, great to hear that Lua is of interest in the community and among the devs. Out of all the languages I know, I use lua for my personal projects 90% of the time and for work whenever I can get away with it.

However, I would highly recommend using LuaJIT over basic Lua. As noted in the pros/cons list, Lua GC does have some iffy scenarios that LuaJIT fixes. LuaJIT also makes the c/c++ binding far easier once you grasp the ffi library. It's also very hard for other scripting languages to beat it's performance.