% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/sdf_interface.R
\name{sdf_persist}
\alias{sdf_persist}
\title{Persist a Spark DataFrame}
\usage{
sdf_persist(x, storage.level = "MEMORY_AND_DISK", name = NULL)
}
\arguments{
\item{x}{A \code{spark_connection}, \code{ml_pipeline}, or a \code{tbl_spark}.}

\item{storage.level}{The storage level to be used. Please view the
\href{https://spark.apache.org/docs/latest/programming-guide.html#rdd-persistence}{Spark Documentation}
for information on what storage levels are accepted.}

\item{name}{A name to assign this table. Passed to [sdf_register()].}
}
\description{
Persist a Spark DataFrame, forcing any pending computations and (optionally)
serializing the results to disk.
}
\details{
Spark DataFrames invoke their operations lazily -- pending operations are
deferred until their results are actually needed. Persisting a Spark
DataFrame effectively 'forces' any pending computations, and then persists
the generated Spark DataFrame as requested (to memory, to disk, or
otherwise).

Users of Spark should be careful to persist the results of any computations
which are non-deterministic -- otherwise, one might see that the values
within a column seem to 'change' as new operations are performed on that
data set.
}
