SQLite Archiver

Ticket Change Details
Login
Overview

Artifact ID: c4e365a36ae131b132844ba53e538f249e92f085
Ticket: bd983680e5810db5a6ef1d856248ebd2b1d3ab5c
Feature Request: XZ, Zstd, and/or no compression options
User & Date: anonymous 2020-01-16 18:29:19
Changes

  1. foundin changed to: "4824e7389653a46f"
  2. icomment:
    We have a potential use case (trying to consolidate 1000's of tiny files, which many parallel filesystems like GPFS have problems with) into sqlar archives, and there would likely be thousands of sqlar archives (it's storing science work on our HPC cluster).
    
    It would be useful if sqlar supported xz (for space), Zstd (for speed), or no compression options, at least on a per-sqlar archive basis.
    
    As a stretch goal, it would be useful if the compression was specifiable on a per-file basis, and the sqlar command would give options for changing the compression of a file or archive.   That way, researchers could store their immediate results with no compression/Zstd for speed, and when spare cycles are available we could run a script to recompress the archive with xz to reduce long-term storage usage.
    
  3. login: "anonymous"
  4. mimetype: "text/x-markdown"
  5. private_contact changed to: "4bc2781e7dbfc4608a75b13bd6057fede0171ed0"
  6. severity changed to: "Minor"
  7. status changed to: "Open"
  8. title changed to:
    Feature Request:   XZ, Zstd, and/or no compression options
    
  9. type changed to: "Feature_Request"