% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/compress.r
\name{h5_compression}
\alias{h5_compression}
\title{Define HDF5 Compression and Filter Settings}
\usage{
h5_compression(
  compress = "gzip",
  chunk_size = 1024 * 1024,
  checksum = FALSE,
  int_packing = FALSE,
  float_rounding = NULL,
  blosc2_delta = FALSE,
  blosc2_truncate = NULL
)
}
\arguments{
\item{compress}{A string specifying the compression algorithm and optional level
(e.g., \code{"none"}, \code{"gzip"}, \code{"zstd-7"}, \code{"lz4"}, \code{"blosc1-lz4-9"},
\code{"blosc2-gzip-3"}, \code{"blosc2-zstd"}). See the \strong{Valid Compression Strings}
section below for an exhaustive list of supported formats. Default is \code{"gzip"}.}

\item{chunk_size}{An integer specifying the target chunk size in bytes.
Default is \code{1048576} (1 MB).}

\item{checksum}{A logical value indicating whether to apply the Fletcher32
checksum filter at the end of the pipeline to detect data corruption. Default is \code{FALSE}.}

\item{int_packing}{Control the HDF5 Scale-Offset filter for integer datasets. \emph{(Note:
Incompatible with \code{szip}, \code{zfp}, \code{bshuf}, and Blosc2 pre-filters).}
\itemize{
\item \code{FALSE} (Default): Disabled.
\item \code{TRUE}: Automatically calculates and applies the mathematically optimal minimum
bit-width for each individual chunk.
\item Integer (e.g., \code{8}): Forces packing into exactly that many bits.
}}

\item{float_rounding}{Control the HDF5 Scale-Offset filter for floating-point
datasets.\emph{(Note: Incompatible with \code{szip}, \code{zfp}, \code{bshuf}, and Blosc2 pre-filters).}
\itemize{
\item \code{NULL} (Default): Disabled.
\item Integer (e.g., \code{3}): The number of base-10 decimal places of detail to preserve
before truncating and packing the values (e.g., \code{3.141}). Negative numbers
round to powers of 10.
}}

\item{blosc2_delta}{A logical value. If \code{TRUE} and a \code{blosc2} compressor is selected,
applies the Blosc2 Delta pre-filter before compression. Default is \code{FALSE}.}

\item{blosc2_truncate}{An integer. If provided and a \code{blosc2} compressor is selected,
applies the Blosc2 Truncate Precision pre-filter to floating-point data, preserving
exactly the specified number of uncompressed bits. Default is \code{NULL}.}
}
\value{
An S3 object of class \code{compress} containing the parsed pipeline parameters.
}
\description{
Constructs a comprehensive filter pipeline configuration to be passed
as the \code{compress} argument to \code{\link[=h5_write]{h5_write()}}. This function allows fine-grained
control over chunking, pre-filters, compression algorithms, and data scaling.
}
\section{Valid Compression Strings}{

The \code{compress} argument accepts a highly specific string syntax to define both
the codec and its operational level.
\subsection{Native / Core Codecs}{
\itemize{
\item \code{"none"}: No compression.
\item \code{"gzip-[level]"}: Levels \code{1} to \code{9}. Default is \code{5}. (e.g., \code{"gzip"} or \code{"gzip-9"}).
\item \code{"zstd-[level]"}: Levels \code{1} to \code{22}. Default is \code{3}. (e.g., \code{"zstd"} or \code{"zstd-7"}).
\item \code{"lz4-[level]"}: Levels \code{0} to \code{12}. Default is \code{0}. Level \code{0} is standard LZ4. Levels \verb{1+} trigger LZ4-HC.
}
}

\subsection{Bitshuffle Pre-filter}{

Forces the native Bitshuffle pre-filter before compression.
\itemize{
\item \code{"bshuf-lz4"}: Bitshuffle + LZ4.
\item \code{"bshuf-zstd-[level]"}: Bitshuffle + Zstd (Levels \code{1} to \code{22}).
}
}

\subsection{Blosc Meta-compressors}{

Blosc applies its own highly optimized bitshuffling and multi-threading.
\itemize{
\item \strong{Blosc2 (Recommended):} \code{"blosc2"} (blosclz), \code{"blosc2-lz4-[level]"}, \code{"blosc2-zstd-[level]"}, \code{"blosc2-gzip-[level]"}, \code{"blosc2-ndlz"}
\item \strong{Blosc1 (Legacy):}  \code{"blosc1"} (blosclz), \code{"blosc1-lz4-[level]"}, \code{"blosc1-zstd-[level]"}, \code{"blosc1-gzip-[level]"}, \code{"blosc1-snappy"}
}
}

\subsection{ZFP (Lossy Floating-Point Compression)}{

ZFP can be run standalone (for integers and floats) or inside Blosc2 (floats only). Unlike \verb{[level]}, \verb{[tolerance]} and \verb{[bits]} are required.
\itemize{
\item \strong{Accuracy Mode} (Absolute error tolerance): \code{"zfp-acc-[tolerance]"} or \code{"blosc2-zfp-acc-[tolerance]"} (e.g., \code{"zfp-acc-0.001"}).
\item \strong{Precision Mode} (Bits of precision): \code{"zfp-prec-[bits]"} or \code{"blosc2-zfp-prec-[bits]"} (e.g., \code{"zfp-prec-16"}).
\item \strong{Rate Mode} (Bits of storage per value): \code{"zfp-rate-[bits]"} or \code{"blosc2-zfp-rate-[bits]"} (e.g., \code{"zfp-rate-8"}).
\item \strong{Reversible Mode} (Standalone Lossless): \code{"zfp-rev"}.
}
}

\subsection{Legacy Codecs}{
\itemize{
\item \code{"szip-nn"}, \code{"szip-ec"}: SZIP Nearest Neighbor or Entropy Coding.
\item \code{"bzip2-[level]"}: Levels \code{1} to \code{9}. Default is \code{9}. (e.g., \code{"bzip2-4"}).
\item \code{"lzf"}, \code{"snappy"}: Fast, unconfigurable legacy compressors.
}
}
}

\section{Automatic Shuffling}{


To maximize compression ratios without requiring users to manually manage
complex pipeline interactions, \code{h5_compression} automatically configures the
optimal shuffling pre-filter based on the following strict hierarchy:

\strong{1. Blosc's Internal Bitshuffle (Preferred)}
If a Blosc meta-compressor is selected (e.g., \code{"blosc2-zstd"}), the pipeline
automatically enables Blosc's highly optimized, internal bitshuffle routine.
This achieves peak compression performance without requiring the standalone
Bitshuffle plugin to be installed.

\strong{2. Explicit Bitshuffle Plugin}
If a standard codec is explicitly prefixed with \verb{bshuf-} (e.g., \code{"bshuf-lz4"}),
the pipeline delegates to the standalone Bitshuffle plugin.

\strong{3. Native HDF5 Byte Shuffle (Fallback)}
If a standard compressor is selected (e.g., \code{"zstd-5"} or \code{"gzip"}),
the pipeline safely falls back to the core HDF5 library's native byte shuffle
filter. This guarantees improved compression while maintaining universal
compatibility.

\strong{4. Strict Mutual Exclusions (When Shuffling is Disabled)}
To prevent data corruption or wasted CPU cycles, all shuffling is
\strong{forcefully disabled} in the following scenarios:
\itemize{
\item \strong{Scale-Offset Active:} If \code{int_packing} or \code{float_rounding} is applied,
shuffling is disabled because scale-offset destroys the byte-alignment
that shuffling relies on.
\item \strong{ZFP & SZIP:} These algorithms perform mathematical compression directly
on numerical values and will corrupt if the bitstream is rearranged beforehand.
\item \strong{1-Byte Data:} Characters, booleans, and 8-bit integers cannot be meaningfully
shuffled, so the step is skipped.
}
}

\examples{
# 1. Simple fast compression (Zstd level 7)
h5_compression("zstd-7")

# 2. Optimal integer packing (Scale-Offset)
h5_compression("gzip-9", int_packing = TRUE)

# 3. Complex Blosc2 Pipeline (Delta + Zstd)
h5_compression("blosc2-zstd-5", blosc2_delta = TRUE)

# 4. Lossy ZFP compression (Tolerance of 0.05)
h5_compression("zfp-acc-0.05")

# Pass the compress object directly to h5_write
file <- tempfile(fileext = ".h5")
cmp  <- h5_compression("gzip-9", checksum = TRUE)
h5_write(combn(1:10, 3), file, "sets", compress = cmp)

print(cmp)

h5_inspect(file, "sets")

# Clean up
unlink(file)

}
\seealso{
\code{\link[=h5_write]{h5_write()}}, \code{vignette('compression')}
}
