Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ZLIB compression support #4080

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

samr46
Copy link
Contributor

@samr46 samr46 commented Mar 7, 2025

This pull request adds zlib data compression and decompression support.

Functions (shared): encodeString / decodeString
New algorithm: zlib

Optional arguments:

  • compression: 0-9 (default = 9);
  • format: gzip, zlib, raw or window size number (default = gzip);
  • strategy: default, filtered, huffman, rle, fixed.

Returns:

Encoded / decoded string or zlib error code.

If no arguments are passed then encodeString will use gzip with highest compression level (9) and decodeString will try to automatically determine compressed format (wrapper).

For most use cases there is no reason to change compression (the size difference is negligible; 0 = no compression).

Strategy is a low-level option which based on documentation might tune compression algorithm to get better results in some cases. I tested it on different data types and the size difference is not significant or might be worse than default option.

Note

Please note that the header (wrapper) size for gzip is 18 bytes and only 6 bytes for zlib (so it's more compact).
raw (deflate) has no wrapping at all and usually used by programs to read / write .zip files.

Tip

Quick testing (runcode):

decodeString("zlib", encodeString("zlib", "Hello MTA:SA World"))

decodeString("zlib", encodeString("zlib", "Hello MTA:SA World", {format = "gzip", compression = 9, strategy = "default"}))

Test resource: zlib.zip

@Fernando-A-Rocha
Copy link
Contributor

Fernando-A-Rocha commented Mar 7, 2025

Congrats on this PR, I definitely think it's useful to have zlib in encode/decode string functions.

On the topic of compression, what would be your opinion on giving servers the option of serving their resources in 1 or more zips that get decompressed by the client? This would save bandwidth and reduce download times, it would likely be a more efficient method of caching server resources.

@samr46
Copy link
Contributor Author

samr46 commented Mar 7, 2025

On the topic of compression, what would be your opinion on giving servers the option of serving their resources in 1 or more zips that get decompressed by the client? This would save bandwidth and reduce download times, it would likely be a more efficient method of caching server resources.

External web server with gzip support is the most efficient solution in my opinion. Internal http server is old and inefficient.
I recommend using nginx server for this. By default it uses on-the-fly gzip compression (so CPU resources will be used to compress content before sending it to client). It also supports static gzip mode (in this mode it's possible to serve pre-compressed versions of files to save CPU resources).

If we want to implement compression of client side cache files on server level then I recommend creating pre-compressed versions (.gz) on resource check and serving it via internal http server (it already supports gzip / deflate decompression).
It's actually fairly easy with newly added functions (although for most efficiency I'd create another internal FileCopy function that would copy file from resources folder into cache and simultaneously create compressed version of this file if it doesn't exist or was modified).
It doesn't make sense to send all files in compressed form because for images (i.e. png which already uses zlib, etc.) and some other data types compression is nearly useless (or even worse in terms of bandwidth and required CPU resources to decompress). So creating .zip version of all resource files to serve isn't a good idea. And it's still easier and better to send all files separately despite the number of download requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants