Adding Placeable objects to azone generated map files

[cavedude]

Originally Posted by Flare83

very nice work Derision. Anyone working on a new map pack with models included?

I was meaning to start, but since each map needs to be manually created, it would take a while. Anybody care to help? We could divide and conquer.

[Angelox]

Here's Skyfire map

[Angelox]

Here's Frontier Mountains and GFaydark Maps.

Derision; Im having problems with eastwastes , it does not dump all the placed-object info. I saw where objects 1-15 were, there then jumped to 2077-2015. For example I can see CRYSCAVETEL202 as an object, but it's not in the LISTOBJ dump.

[Derision]

Originally Posted by Angelox

Im having problems with eastwastes , it does not dump all the placed-object info. I saw where objects 1-15 were, there then jumped to 2077-2015. For example I can see CRYSCAVETEL202 as an object, but it's not in the LISTOBJ dump.

I see what you mean. I'll take a look at it at some point during the week.

[KLS]

An unusual amount of -1 model entries for eastwastes. I've been going through them alphabetically... slowly... I dare someone to take a look at bloodfields, it made me cry and not with any amount of joy. =(

[Angelox]

There's another problem I noticed too, Skyfire has a lot of 'moonwalkers' (npc's get stuck and go a few steps back then forward a few, never get anywhere). I noticed at first in the entrance to Veeshan. Thought maybe it was the map , but the azone2 fix doesn't help. Then I started seeing this moonwalking bug all over the zone, with many npcs. If you can slow or freeze the frame rate , you'll see the mob gets stuck waist-deep in the ground where this is happening (the moonwalking is just an illusion). Skyfire Map I made is so gigantic because I detailed a lot of objects in hopes it would help some. But the most needed objects that are what cause the problem are not in the S3d files , they are on the Skyfire.wld files. I haven't checked any other zones, I Imagine the more irregular the terrain, the more prone to moonwalking. I saw some moonwalking in FM and really haven't check more zones - But Skyfire would be the zone to work this bug out.
In FM(Frontier Mts), most mobs walked fine, with the new fix (Dersion fix for 'BestZ Calculation for aggroed mobs), most the giants travel flawlessly over the big mountains, but once in a while you bump into a MJ fan (moonwalker). I think the 'BestZ Calculation' fixed a lot more than expected.

[Derision]

Originally Posted by Angelox

Derision; Im having problems with eastwastes , it does not dump all the placed-object info. I saw where objects 1-15 were, there then jumped to 2077-2015. For example I can see CRYSCAVETEL202 as an object, but it's not in the LISTOBJ dump.

I altered the code to print out the names of the objects it couldn't find the model for. In eastwastes, there are 2054:

$ ./listobj eastwastes | grep "not found" | wc -l
2054

Of those 2054 objects, they all use 1 of 3 models:

$ ./listobj eastwastes | grep "not found" | sort | uniq -c

      1 Model ELECTMONU200_DMSPRITEDEF not found
   1006 Model ICETREE200_DMSPRITEDEF not found
   1047 Model ICETREE203_DMSPRITEDEF not found

So all of them bar 1 are trees. I am guessing they use an animated model fragment that the .wld loader doesn't handle.

I checked all the other .s3ds, and the placeable objects with missing models generally seem to be trees, grass, flames, waterfalls, etc (all things that are likely animated), so this doesn't bother me too much, as I think they are generally the kind of objects that you wouldn't want to include in LOS anyway.

As for CRYSCAVETELE202, it is possible that the object mesh exists in the S3D, but was never actually placed in the zone by the designer, which is why LISTOBJ doesn't list it, as no objects use that model.

And KLS, I thought your comment about bloodfields meant there were loads of objects missing for that zone too, but I guess it is just the opposite

[Angelox]

Ok well, I guess what already was dumped has the houses and what not - did you have time to take a look at skyfire? I'm wondering why so many mobs there don't path properly

[Derision]

Originally Posted by Angelox

Ok well, I guess what already was dumped has the houses and what not - did you have time to take a look at skyfire? I'm wondering why so many mobs there don't path properly

I saw the two at the VP entrance, but I couldn't find any others. If you could give me a list of some other mobs affected (target, #wpinfo and give me the grid numbers), I'll look into is some more.

[Angelox]

Here's a couple more; Crushbone and Najena Maps.
I'm doing maps and posting them as I do other work, not really gonna go all out and do them all at once, but I'll get there eventually - and share the ones I do get done with you all.

[Flare83]

Here are kael maps.

kael.tar.gz

[Angelox]

Derision ;
I think I start to see a pattern in a problem -
BtW a good way to test things is with the bots, they basically make the same mistakes the Mobs do when theres a bugged area.
In Crushbone , inside the castle in the first big room where Darish spawns, there a big Table- that was a good place to camp mobs from, but any pull gets near the table and they fall through the floor and get waist deep. In Unrest building where Undead barkeep spawns, when you pull him, he tries run through the bar, and falls through the floor to the level below, then starts attacking you through the floor.
Seems when there's an object in the building like a bar, or a table, they get bugged and mobs fall though. I also noticed the bots getting caught up inside the walls in unrest building and stuck, I had to summon them out.
In the top floor, where the undead knight spawns, i set up camp and went to pull a mob from the next floor down: Big mistake! none came up the stairs, they came under the floor and attacked, eventually killed me - it was a train and the bots won't respond unless they have line of sight. Seems like there's still something missing in the maps, something that needs to be added to the azone and will solve all these problems at once. I have made new maps and still the problem remains. I made a new Unrest map with all the items incuded, still no dice.

[Angelox]

Tell me if you still want locs and I'll get them and post here.

[Leere]

I'm not sure if this is the right place for this, but since it seems to be applicable to the topic at hand...

I've been playing around with loading the various data files for the newer zones. (All the .EQG ones.) The following is a refinement of the information from both OpenEQ and also what Derision has kindly provided with his azone patches. The EQTZ format isn't fully worked out yet, but maybe someone here can finish those last pieces.

Please forgive the repeating of already known information.

(Note: When I use 'int' as an indicator for data below, I'm using it in the sense of a 32 bit entity. Using long or one of the *int_32 types, probably would be more accurate, but I've been spending too much time with java source code lately, so int has just crept up to being equated with the 32-bit incarnation. -- And please forgive the pascal esque pseudo-code, I hope it isn't too confusing.)


ZON

There are two flavors of this file. The first started to show up with the Omens of War expansion, when the EQGs were first introduced. The second I find first used for Prophecy of Ro outdoor zones. They are distinguished by the 4 byte magic string at the start.

EQGZ indicates the first format, while EQTZ indicates the second.

Format One:

The file header has already been pretty well covered.

4 bytes - magic string, indicating the format, always EQGZ
DWORD - Version (Observed ones, 1 and 2, see below for differences)
DWORD - hashsize (The size of the block of \0 terminated strings, that provide all names)
DWORD - meshcount
DWORD - objcount
DWORD - unk_count
DWORD - lightcount

Next we have the list of strings. A block of hashsize bytes.

Following that, a block of meshcount DWORDs. They are references into the namehash, each indicating a mesh that is part of the zone. Everything else that wants a mesh references this list.

The next part is what we really care about, the list of placed objects. There are objcount of these entries, and here is also the only difference I could find between versions 1 and 2.

Object format:
1 DWORD - Reference to the meshlist, for which mesh to use for the Object.
1 DWORD - Reference into the namehash, for a name for the Object.
3 DWORDs - Interpreted as floats, the location of the object. This is seen absolute.
3 DWORDs - Floats again, the rotation of the object. Entries are in radians.
1 DWORD - A float the scale of the object.

The following is only present is if the ZON is version 2.
DWORD - count
count*DWORDS

As a note, if the mesh referenced is actually a .ter, and not a .mod, then the location and rotation info has to be ignored. At least, it had to be for my tests to look right. If anyone can find a better way to handle that, I'd be more than happy.

The version 2 block seems to be some kind of coloring info, or something like that. Certainly no geometry we need to care about.

Next we have the unknown. Each of these are 40 bytes long. (Seemingly an int followed by 9 floats. Zone lines, teleporters, something like that at a guess.)

Finally, we have the light entries. Each of these is 36 bytes long. (1 int for a name reference, 8 floats for parameters. Not that it really matters for our needs.)


Format Two:
These are newline separated lines of XML-styled parameters only. The entire format is human readable, and has to be parsed as such.

The first line holds a 'P' and nothing else. The other parameters are MINLNG, MAXLNG, MINLAT, MAXLAT, QUADSPERTILE, UNITSPERVERT, ...

The only ones that we really need to generate a map file, are QUADSPERTILE and UNITSPERVERT.

The NAME property has always been identical to the name of the ZON file, which is NOT always equal to the name of the EQG. There is a .DAT in the EQG with the name indicated by it, and that holds the primary geometry info. (In particular, it's a height-map, split into tiles, made up out of quads.)


TER

MOD and TER files are largely identical. The only thing that a MOD file has, that the TER lacks, is the possibility to include a skeleton for model animation.

The format is already well enough handled by azone, and as such I'll just limit myself to some observations.

There is no BSP tree, or anything else really usefull for splitting up the zone in here. The intent seems to have been to provide big chunks of polygons sharing the same material, and just feeding that to the GPU. (Those chunks I've found so far seemed to be 200 or 400 polygons each. Maybe that can be translated into a quadtree or something else, but I haven't looked that closely yet.)

The Material Block
I've seen a reference to the data in here in another thread. If the information has already been found out since, I missed it.

For each of the parameters, of a material, there is a block of 3 DWORDs.

The first is a reference to the namehash of the file, the second a selector, and the third the data, interpreted as indicated by the second.

0 - As a float (values for parameters)
2 - Reference into the name hash (usually seen for texture names)
3 - As an int (usually color info)


The file ending. Version 1 and Version 3 end after any potential skeleton info. Version 2 ends on a DWORD, that is used as a flag. If it's 0, the file ends with that, for non-zero, there are vertex_count*8 bytes following. (Two floats per triangle, at a guess, a secondary texture coordinate pair, which version 3 pulls up into the vertex structure itself.)

My limited forrays into the meaning of the group parameter, for the polygon list, had some matches between materials using a water shader and -1 for the group (I think that is what azone calls it anyway, the non-material parameter anyway, that has a flag field vibe for me), but that's highly likely to be wrong.

So far, the existence of some 100k+ polygons for the TER entries has made my eyes want to bleed and refuse to even try to just try and match up material info to group info. Sadly, it's going to be the only place where info useable for water maps is going to be found, if it's found.


DAT

The format for the terrain files. I haven't really managed to display these yet, but they are at least parsing correctly.

From the ZON file, two parameters are needed to generate any kind of worthwhile terrain. QUADSPERTILE, to load the height values correctly, and UNITSPERVERT, to create an actual quad grid.

The format now. First the quasi header. I don't really have a clue what any of the first four entries means, but the fifth is the only thing needed to parse the file.

Header:
3 DWORDs - Three ints, that make no sense.
ASCIIZ String - One \0 terminated string. (/me mourns the nice namehash block)
DWORD - tile_count

Only tile_count is really helpful, since there are that many tile entries following now. The String in the header always had a .dds file, so perhaps it's about a kind of horizon, or something... I have no idea, anyway.

For the tiles below, some helpful variables.

quadcount = QUADSPERTILE ^ 2
vertcount = (QUADSPERTILE + 1) ^ 2

The quadcount is the number of actual quads within in a tile, and the vercount how many vertices we need to create those.

Tile format:
DWORD - LNG of the tile (longitude?) Stored with 100000 representing 0
DWORD - LAT (latitude?) See LNG for format
DWORD - unknown (some flags maybe? They showed no pattern.)

vercount * DWORDs - Height field of floats. The missing z values, for the quads
vercount * DWORDs - Color info for the verts? Something like that.
vercount * DWORDs - Something looking like color info again
quadcount * BYTE - At a guess, how the quad has to be split into triangles.
DWORD - float
DWORD - unk

if unk == 0x1 || unk == 0x2 then
  BYTE - flag

  if flag > 0 then
    4 DWORDs - Which seem to be floats
  end

  DWORD - float
else
  unk seems to be interpreted as a float in this case.
end

DWORD - layercount

layer # 0 (always present)
ASCIIZ string (name points to one of the ECO files in the EQG)

for layer = 1 to layercount-1 do
  ASCIIZ String - name of ECO for layer
  DWORD size
  size * size BYTEs (overlay map? coordinate system is confusing)
end

Parameter block (placed objects, split into 4 types)

DWORD - count#1
for i=1 to count#1 do
  ASCIIZ String - MOD name (or LOD file)
  ASCIIZ String - References an ECO file by name (just the name, no .eco)
  DWORD - LNG (identical format to the one at the start of the tile)
  DWORD - LAT
  3 DWORDs - (floats) Loc (relative to the tile)
  3 DWORDs - (floats) Rotation (in degrees)
  3 DWORDs - (floats) Scale
  BYTE - always was 0xFF
end

DWORD - count#2
for i=1 to count#2 do
  ASCIIZ String - File reference
  DWORD
  ASCIIZ String
  DWORD - LNG
  DWORD - LAT
  3 DWORDs - (floats) Loc (relative)
  3 DWORDs - (floats) rot (in degrees)
  3 DWORDs - (floats) scale
  3 DWORDs - (floats) Some kind of size?
end

DWORD - count#3
for i=1 to count#3 do
  ASCIIZ String
  ASCIIZ String
  BYTE
  DWORD - LNG
  DWORD - LAT
  3 DWORDs - (floats) Loc (relative)
  3 DWORDs - (floats) rot
  3 DWORDs - (floats) scale
  DWORD - (float) ?
end

DWORD - count#4
for i=1 to count#4 do
  ASCIIZ String
  DWORD - LNG
  DWORD - LAT
  3 DWORDs - (floats) loc (relative to tile)
  3 DWORDs - (floats) rot (in degrees)
  3 DWORDs - (floats) scale
  DWORD - (float) ?
end

The count#4 entries seemed to hold big zone geometry, like bridges, buildings and the likes. The count#2 entries looked to be about water.

The z coordinates of the objects all seemed to be relative to where on the tile the x,y pair places them. (Many 0.0s)


Since they are referenced in the DAT.

ECO - clear text, XML-esque style, defines the materials
LOD - Level of Detail, clear text, with range indications and MOD files to use then.
TOG - Some kind of model grouping, valdeholm uses these, but no idea how to find the references just yet.


I think that about sums up what I've found out about the EQTZ format. Hopefully it is of some help for the map generation project.

[Derision]

Originally Posted by Angelox

In Unrest building where Undead barkeep spawns, when you pull him, he tries run through the bar, and falls through the floor to the level below

I have code that may fix this. Essentially it changes the way FindBestZ works to a) detect if the mobs position is under the world/inside geometry and b) find all valid Z positions for the current X/Y (by valid, I mean surfaces that are upward facing and not 'under the world' and returns the closest Z.

It adds a bit more runtime overhead. For each possible Z position it casts a ray upwards. If the ray hits a face, it then checks if it is the back or front side of a polygon. E.g. if you were inside a building it would hit the front side of a downward facing poly that makes up the ceiling, so you are not under the world. If you were waist deep in the bar in unrest, the ray would hit the back side of the poly making up the top of the bar, and so you would be classed as under the world and the code would move you to the top of the bar (assuming that is 'nearer' than the floor below).

It absolutely depends on the server having Azone2 generated map files. The reason for this is that the stock azone builds EQG maps with the vertices ordered the opposite way round to S3D maps, basically meaning the 'under the world' test that works for S3D maps, returns the opposite result for EQG maps.

I'll do some more work on it one of these days.