% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/parquet-metadata.R
\name{read_parquet_metadata}
\alias{read_parquet_metadata}
\alias{parquet_metadata}
\title{Read the metadata of a Parquet file}
\usage{
read_parquet_metadata(file, options = parquet_options())

parquet_metadata(file)
}
\arguments{
\item{file}{Path to a Parquet file.}

\item{options}{Options that potentially alter the default Parquet to R
type mappings, see \code{\link[=parquet_options]{parquet_options()}}.}
}
\value{
A named list with entries:
\itemize{
\item \code{file_meta_data}: a data frame with file meta data:
\itemize{
\item \code{file_name}: file name.
\item \code{version}: Parquet version, an integer.
\item \code{num_rows}: total number of rows.
\item \code{key_value_metadata}: list column of a data frames with two
character columns called \code{key} and \code{value}. This is the key-value
metadata of the file. Arrow stores its schema here.
\item \code{created_by}: A string scalar, usually the name of the software
that created the file.
}
\item \code{schema}:
data frame, the schema of the file. It has one row for
each node (inner node or leaf node). For flat files this means one
root node (inner node), always the first one, and then one row for
each "real" column. For nested schemas, the rows are in depth-first
search order. Most important columns are:
\itemize{
\item \code{file_name}: file name.
\item \code{r_col}: integer, the column index in R, starting from one.
\code{NA} for the root node. For Parquet files without list columns
this is simply the column index in the file. A list column has
multiple schema rows, and they all have the same \code{r_col}.
\item \code{name}: column name.
\item \code{r_type}: the R type that corresponds to the Parquet type.
Might be \code{NA} if \code{\link[=read_parquet]{read_parquet()}} cannot read this column. See
\link{nanoparquet-types} for the type mapping rules.
\item \code{r_type}:
\item \code{type}: data type. One of the low level data types.
\item \code{type_length}: length for fixed length byte arrays.
\item \code{repettion_type}: character, one of \code{REQUIRED}, \code{OPTIONAL} or
\code{REPEATED}.
\item \code{logical_type}: a list column, the logical types of the columns.
An element has at least an entry called \code{type}, and potentially
additional entries, e.g. \code{bit_width}, \code{is_signed}, etc.
\item \code{num_children}: number of child nodes. Should be a non-negative
integer for the root node, and \code{NA} for a leaf node.
}
\item \verb{$row_groups}: a data frame, information about the row groups.
Some important columns:
\itemize{
\item \code{file_name}: file name.
\item \code{id}: row group id, integer from zero to number of row groups
minus one.
\item \code{total_byte_size}: total uncompressed size of all column data.
\item \code{num_rows}: number of rows.
\item \code{file_offset}: where the row group starts in the file. This is
optional, so it might be \code{NA}.
\item \code{total_compressed_size}: total byte size of all compressed
(and potentially encrypted) column data in this row group.
This is optional, so it might be \code{NA}.
\item \code{ordinal}: ordinal position of the row group in the file, starting
from zero. This is optional, so it might be \code{NA}. If \code{NA}, then
the order of the row groups is as they appear in the metadata.
}
\item \verb{$column_chunks}: a data frame, information about all column chunks,
across all row groups. Some important columns:
\itemize{
\item \code{file_name}: file name.
\item \code{row_group}: which row group this chunk belongs to.
\item \code{column}: which leaf column this chunks belongs to. The order is
the same as in \verb{$schema}, but only leaf columns (i.e. columns with
\code{NA} children) are counted.
\item \code{file_path}: which file the chunk is stored in. \code{NA} means the
same file.
\item \code{file_offset}: where the column chunk begins in the file.
\item \code{type}: low level parquet data type.
\item \code{encodings}: encodings used to store this chunk. It is a list
column of character vectors of encoding names. Current possible
encodings: "PLAIN", "GROUP_VAR_INT", "PLAIN_DICTIONARY", "RLE", "BIT_PACKED", "DELTA_BINARY_PACKED", "DELTA_LENGTH_BYTE_ARRAY", "DELTA_BYTE_ARRAY", "RLE_DICTIONARY", "BYTE_STREAM_SPLIT".
\item \code{path_in_scema}: list column of character vectors. It is simply
the path from the root node. It is simply the column name for
flat schemas.
\item \code{codec}: compression codec used for the column chunk. Possible
values are: "UNCOMPRESSED", "SNAPPY", "GZIP", "LZO", "BROTLI", "LZ4", "ZSTD".
\item \code{num_values}: number of values in this column chunk.
\item \code{total_uncompressed_size}: total uncompressed size in bytes.
\item \code{total_compressed_size}: total compressed size in bytes.
\item \code{data_page_offset}: absolute position of the first data page of
the column chunk in the file.
\item \code{index_page_offset}: absolute position of the first index page of
the column chunk in the file, or \code{NA} if there are no index pages.
\item \code{dictionary_page_offset}: absolute position of the first
dictionary page of the column chunk in the file, or \code{NA} if there
are no dictionary pages.
\item \code{null_count}: the number of missing values in the column chunk.
It may be \code{NA}.
\item \code{min_value}: list column of raw vectors, the minimum value of the
column, in binary. If \code{NULL}, then then it is not specified.
This column is experimental.
\item \code{max_value}: list column of raw vectors, the maximum value of the
column, in binary. If \code{NULL}, then then it is not specified.
This column is experimental.
\item \code{is_min_value_exact}: whether the minimum value is an actual
value of a column, or a bound. It may be \code{NA}.
\item \code{is_max_value_exact}: whether the maximum value is an actual
value of a column, or a bound. It may be \code{NA}.
}
}
}
\description{
This function should work on all files, even if \code{\link[=read_parquet]{read_parquet()}} is
unable to read them, because of an unsupported schema, encoding,
compression or other reason.
}
\examples{
file_name <- system.file("extdata/userdata1.parquet", package = "nanoparquet")
nanoparquet::read_parquet_metadata(file_name)
}
\seealso{
\code{\link[=read_parquet_info]{read_parquet_info()}} for a much shorter summary.
\code{\link[=read_parquet_schema]{read_parquet_schema()}} for column information.
\code{\link[=read_parquet]{read_parquet()}} to read, \code{\link[=write_parquet]{write_parquet()}} to write Parquet files,
\link{nanoparquet-types} for the R <-> Parquet type mappings.
}
