|Title:||Feature Request: XZ, Zstd, and/or no compression options|
|Last Modified:||2020-01-16 18:29:19|
|Version Found In:||4824e7389653a46f|
anonymous added on 2020-01-16 18:29:19:
We have a potential use case (trying to consolidate 1000's of tiny files, which many parallel filesystems like GPFS have problems with) into sqlar archives, and there would likely be thousands of sqlar archives (it's storing science work on our HPC cluster).
It would be useful if sqlar supported xz (for space), Zstd (for speed), or no compression options, at least on a per-sqlar archive basis.
As a stretch goal, it would be useful if the compression was specifiable on a per-file basis, and the sqlar command would give options for changing the compression of a file or archive. That way, researchers could store their immediate results with no compression/Zstd for speed, and when spare cycles are available we could run a script to recompress the archive with xz to reduce long-term storage usage.