EQ-Zip S3D/EQG Manager

[Shendare]

So, I made a thing.

I took a break for a couple of weeks from other projects to recharge and try something fresh. I had some fun replacing textures in the client's zone files with higher resolution ones with the same look and feel.

It was taking too many steps, though, to extract a texture, convert it to a format I and Windows Explorer could work with, find a replacement, make any edits I wanted to, convert the replacement back to .dds, and import the .dds over the existing file in the .s3d package. With the same extension, if the extension was different.

I looked up the latest S3DSpy source code to see if I could patch in some convenient shortcuts, but while I played with Pascal back in high school, I don't have a Delphi compiler or up-to-date knowledge on it. So that was a bust.

I decided to see if I could make my own to satisfy my needs. As I researched APIs and added features, and saw how well things we coming along, I felt a seductive motivation to see just how good I could make my application.

You're going to laugh at how much effort I ended up putting into it.

You can read about it and download a compiled executable or the source code from my website, which also has several screenshots:

http://www.shendare.com/EQ/Emu/EQZip/

Or you can download the source from the Github link:

https://github.com/Shendare/EQZip

Here's one screenshot for the forum thread, though:



Hope others find it useful, too!

[Akkadius]

Definitely cool

[Zaela_S]

Originally Posted by Shendare

convert the replacement back to .dds

Just want to chime in to say that this step is unnecessary. The client's image loader will accept PNG/JPEG/etc images just fine, as long as the filename is what it expects.

I also have an S3D/EQG extractor thing with an image viewer an export-as-PNG option I made about a year ago... probably not much use to you at this point, though. Plus it's written in Lua mostly.

[Shendare]

Thanks, Akka!

Zaela, I noticed that I could put non-dds images in as long as they were named the same, but the client doesn't appear to create mipmaps for them, so they get really nastily pixellated at angles. Floors were especially bad, because they're -always- seen at angles, never head-on.

I made it an option in this app to turn off auto-dds conversion with mipmaps, but I don't see why anyone would turn it off once they see it.

The larger the texture, the less noticeable it will be, but even at 1024x1024, it's still bad, especially when moving.

(Watch at 1080p, of course, to avoid mpeg artifacting.)

1024x1024 ground as a PNG (no mipmaps, flicker flicker flicker):

https://www.youtube.com/watch?v=b9E7zcLiGbU

1024x1024 ground as a DDS with mipmaps (less sharp, but not flickering):

https://www.youtube.com/watch?v=5lIzF2LooMQ

[Zaela_S]

Originally Posted by Shendare

Zaela, I noticed that I could put non-dds images in as long as they were named the same, but the client doesn't appear to create mipmaps for them, so they get really nastily pixellated at angles. Floors were especially bad, because they're -always- seen at angles, never head-on.

Ah! I didn't know that it didn't generate mipmaps. I've only really looked at PNG images on weapons and such, much harder to see at a distance I guess. Makes sense if they're always pre-generated, though.

On the other hand, makes me wonder how they handle images that really are BMP -- almost all of the images in the Titanium client .s3d files named ".bmp" really do have BMP headers, and there are still some from recent free-to-play downloads that are like that -- mostly old Mob/weapon textures, but some zone ones as well. I guess there could be some BMP extension to support mipmaps, or they could be some old transitional semi-dds bmp format, although google isn't helping me there. Or I guess the images could just be so low quality that running them through a filtering algorithm doesn't make a noticeable difference. Might be worth looking into, in any case.

[Shendare]

I wonder whether it's an optimization they did to speed up zoning times. Instead of building mipmaps at zone time for every zone texture, they decided sometime after Titanium to bake them into the zone files. Makes sense, in a way, for computers with slower CPU/GPUs.

[Shendare]

Oh my god, I took a look.

Not only are they actual Bitmaps with no mipmaps, but they're 64x64 8-bit palette indexed bitmaps. The ones I was looking at in RoF2 were 256x256 with 16-bit DXT compression, and even they looked blocky and low-def in-game.

I had no idea that when I was playing with the Titanium client, things were looking -that- bad. Walking around, the banding/flickering appeared to be present, meaning no mipmaps were in use, but the sheer low resolution of the textures alleviated the problem a bit, since the lack of definition made the textures so blurry.

Graphically speaking... Titanium looks like a last-resort type of client to use if you don't have anything newer.

[rhyotte]

Tagging in.

[Shendare]

Tonight I discovered Github's Releases feature, and that it can be used for uploading not only compiled binaries, but screenshots as well. I don't need to host on a separate website for those!

I've created the Release for EQ-Zip 1.0, and will be working to migrate my other projects to Github releases.

[knowom]

Originally Posted by Shendare

Oh my god, I took a look.

Not only are they actual Bitmaps with no mipmaps, but they're 64x64 8-bit palette indexed bitmaps. The ones I was looking at in RoF2 were 256x256 with 16-bit DXT compression, and even they looked blocky and low-def in-game.

I had no idea that when I was playing with the Titanium client, things were looking -that- bad. Walking around, the banding/flickering appeared to be present, meaning no mipmaps were in use, but the sheer low resolution of the textures alleviated the problem a bit, since the lack of definition made the textures so blurry.

Graphically speaking... Titanium looks like a last-resort type of client to use if you don't have anything newer.

Classic zones were revamped with Luclin expansion and can be found on those CD's with higher 256x256 resolution textures, but Titanium install didn't include those higher resolution s3d revamps because Sony was just too cheap to fit them onto another couple CD's.

LOD Bias also plays a significant part in flickering/shimmering or how sharp/blurry a texture is in game the sharper it is the more flicker/shimmering you get with it so it's a trade off really personally I like it sharp and can deal with the shimmer compromise myself. You need something like RivaTuner/Nvidia Inspector/ect to adjust LOD Bias though. The lower resolutions actually wasn't such a huge deal with CRT's, but they really bad today at 1080p or greater especially LCD's are much sharper as well.

[Shendare]

I was scratching my head over the higher res textures thing. I remembered them announcing it, and couldn't for the life of me picture it happening after DoD instead of long before. Cost cutting in the CDs totally explains it. Thanks for solving the mystery!

I've been toying with the idea of looking up sharpening algorithms to make the mipmaps less washed out after rescaling. I'm sure there's got to be someting out there I could implement.

[knowom]

Originally Posted by Shendare

I was scratching my head over the higher res textures thing. I remembered them announcing it, and couldn't for the life of me picture it happening after DoD instead of long before. Cost cutting in the CDs totally explains it. Thanks for solving the mystery!

I've been toying with the idea of looking up sharpening algorithms to make the mipmaps less washed out after rescaling. I'm sure there's got to be someting out there I could implement.

Unsharp Mask is a good image filter when used in moderation.

BIMP plugin for GIMP would be useful if you were trying to do a batch of unsharp mask filtering to group of textures.
http://registry.gimp.org/node/26259

DDS plugin is also useful for GIMP if you want to export to DDS/DXT file types which can also create mip maps as well for them.
http://registry.gimp.org/node/70

The Luclin upgrade textures weren't too bad and Psycher on p1999 did a nice character texture upgrade at one point too that was quite a decent improvement.

[Shendare]

7/28/2015 - Version 1.1

https://github.com/Shendare/EQZip/releases

Release Notes:

DXT1 Decompression - Improved Slightly
DXT2 Decompression - Enabled
DXT3 Decompression - Added
DXT4 Decompression - Enabled
DXT5 Decompression - Improved Significantly
V8U8 Decompression - Improved Significantly

Known Issues and Planned Updates:

• DDS conversion does not support compression. This is intended. DXT compression is very lossy ( see http://www.fsdeveloper.com/wiki/inde...sion_explained ), and graphics cards have plenty of video memory for uncompressed textures these days.
  
• Support for reading and writing 16-bit uncompressed DDS textures is planned, as it would cut image sizes in half with much lower loss of definition than DXT compression.
  
• A feature will be added to vertically flip a texture, as some in-game geometry expects a texture to be bottom-up, and it can be hard to tell until you see it in-game.

[knowom]

A 2048x2048 texture with compression artifacts you will only really see if you are looking for them is usually much nicer than a 1024x1024 downscaled and blurrier texture where the higher quality details have been lost completely. Both take the same amount of ram and memory bandwidth to read, but the perceived quality increase of having 4 times as many pixels is significant. The average user probably won't be able to tell if you are compressing their textures or not, but most will be able to see a halving of texel resolution quite easily. DXT3 and 5 are similar if you use them in the correct context (5 for smooth alpha gradients, 3 for sharp alpha transitions).
http://www.gamedev.net/topic/611111-...aled-textures/


For Grobb retexture project I used 2048x2048 textures with DXT3 for smooth alpha transitions though id probably use DXT5 instead now and also use that GIMP plugin BIMP with unsharp mask set to like 1 or 2 instead. DXT2-5 are 4:1 compression ratio so the size of the textures used offset most or all lossy behavior to be on par with Luclin or Luclin revamps to classic zone 256x256 and 512x512 texture sizes. Titanium came with embarrassingly poor worse classic zone textures some were 32x32 I think and others 64x64 compared to the revamped textures Luclin provided for older zones.

Just use higher resolution base textures to compensate for the 4:1 compression if your worried about the DXT 3/5 lossy aspect.

Something worth mentioning that is quite interesting is hq4x or xBRZ filters http://sourceforge.net/projects/xbrz/
https://code.google.com/p/hqx/

Character models and objects in particular would be handy do use it with in fact someone at p1999 did experiment with hqx3 awhile back I'm assuming hq4x wasn't released at that point in time. Even better perhaps would be to duplicate the base texture once or twice each for a overlay layer of XBRZ/HQx4 filter versions of original textures then merge it down to blend the two together. You could also use some transparency between the filter layers and a unsharp mask filter on the top layer prior to merge down or after it just as way to sharper the image a bit wouldn't hurt either.

I'd probably have a ZBRZ layer with unsharp mask 25-75% transparency overlayed on top of a HQx4 layer with 25-75% transparency overlayed onto the original texture merge it down and export it however you want DDS/DXT3/5.

The two filters work a bit differently and look a bit different, but combining the base texture with both the other two filters and overlay plus optional transparency should provide a good mixture that looks a bit more natural with a fairly gradual cell shaded look that's not so harsh in terms of aliasing, but also not over smooth and cartoon looking either.

[rhyotte]

Cool.

/10char