% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/rxp_populate.R
\name{rxp_populate}
\alias{rxp_populate}
\title{Generate Nix Pipeline Code}
\usage{
rxp_populate(derivs, project_path = ".", build = FALSE, py_imports = NULL, ...)
}
\arguments{
\item{derivs}{A list of derivation objects, where each object is a list of
five elements:
\itemize{
\item \emph{name}, name of the derivation,
\item \emph{snippet}, the nix code snippet to build this derivation,
\item \emph{type}, can be R, Python or Quarto,
\item \emph{additional_files}, character vector of paths to files to make available to build sandbox,
\item \emph{nix_env}, path to Nix environment to build this derivation.
A single deriv is the output of \code{rxp_r()}, \code{rxp_qmd()} or \code{rxp_py()}
function.
}}

\item{project_path}{Path to root of project, defaults to ".".}

\item{build}{Logical, defaults to FALSE. Should the pipeline get built right
after being generated? When FALSE, use \code{rxp_make()} to build the pipeline
at a later stage.}

\item{py_imports}{Named character vector of Python import rewrites. Names are
the base modules that rixpress auto-imports as "import name", and values
are the desired import lines. For example: c(numpy = "import numpy as np",
xgboost = "from xgboost import XGBClassifier"). Each entry is applied by
replacing "import name" with the provided string across generated
_rixpress Python library files.}

\item{...}{Further arguments passed down to methods. Use \code{max-jobs} and
\code{cores} to set parallelism during build. See the documentation of
\code{rxp_make()} for more details.}
}
\value{
Nothing, writes a file called \code{pipeline.nix} with the Nix code to
build the pipeline, as well as folder called _rixpress with required
internal files.
}
\description{
Generate Nix Pipeline Code
}
\details{
This function generates a \code{pipeline.nix} file based on a list of derivation
objects. Each derivation defines a build step, and \code{rxp_populate()} chains these
steps and handles the serialization and conversion of Python objects into R
objects (or vice-versa). Derivations are created with \code{rxp_r()}, \code{rxp_py()}
and so on. By default, the pipeline is also immediately built after being
generated, but the build process can be postponed by setting \code{build} to
FALSE. In this case, the pipeline can then be built using \code{rxp_make()} at
a later stage.
The generated \code{pipeline.nix} expression includes:
\itemize{
\item the required imports of environments, typically \code{default.nix} files generated by
the \code{rix} package;
\item correct handling of interdependencies of the different derivations;
\item serialization and deserialization of both R and Python objects, and conversion
between them when objects are passed from one language to another;
\item correct loading of R and Python packages, or extra functions needed to build
specific targets
}

The \verb{_rixpress} folder contains:
\itemize{
\item R, Python or Julia scripts to load the required packages that need to be
available to the pipeline.
\item a JSON file with the DAG of the pipeline, used for visualisation, and to
allow \code{rxp_populate()} to generate the right dependencies between derivations.
\item \code{.rds} files with build logs, required for \code{rxp_inspect()} and \code{rxp_gc()}.
See \code{vignette("debugging")} for more details.
}

Inline Python import adjustments
In some cases, due to the automatic handling of Python packages, users might
want to change import statements. By default if, say, \code{pandas} is needed to
build a derivation, it will be imported with \verb{import pandas}. However, Python
programmers typically use \verb{import pandas as pd}. You can either:
\itemize{
\item use \code{py_imports} to rewrite these automatically during population, or
\item use \code{adjust_import()} and \code{add_import()} for advanced/manual control.
See \code{vignette("polyglot")} for more details.
}
}
\examples{
\dontrun{
# Create derivation objects
d1 <- rxp_r(mtcars_am, filter(mtcars, am == 1))
d2 <- rxp_r(mtcars_head, head(mtcars_am))
list_derivs <- list(d1, d2)

# Generate and build in one go
rxp_populate(derivs = list_derivs, project_path = ".", build = TRUE)

# Or only populate, with inline Python import adjustments
rxp_populate(
  derivs = list_derivs,
  project_path = ".",
  build = FALSE,
  py_imports = c(pandas = "import pandas as pd")
)
# Then later:
rxp_make()
}
}
\seealso{
Other pipeline functions: 
\code{\link{rxp_make}()},
\code{\link{rxp_pipeline}()}
}
\concept{pipeline functions}
