% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/parallel_execute.R
\name{parallel_execute}
\alias{parallel_execute}
\title{Execute Parallel Text Processing with AI Models}
\usage{
parallel_execute(
  test_data,
  slicing_n,
  n_post = 15,
  cores = 8,
  seed = NULL,
  multi_core = FALSE,
  benchmarking = FALSE,
  verbose = TRUE
)
}
\arguments{
\item{test_data}{Data frame. Input data containing text to be processed.}

\item{slicing_n}{Integer. Number of rows to sample from the input data for processing.
The batch size will be automatically calculated by dividing this by the number of cores.}

\item{n_post}{Integer. Number of posts to process in each API call. Default is 15.
Must be between 1 and 1000. Larger values may be more efficient but use more API credits.}

\item{cores}{Integer. Number of CPU cores to use for parallel processing. Default is 8.
The function will use the provider specified in set_api_specs() to determine
whether to use OpenAI or GROQ services.}

\item{seed}{Integer or NULL. Random seed for reproducible sampling. If NULL (default),
sampling is random. If set to a number, ensures identical datasets across runs.}

\item{multi_core}{Logical. If TRUE, uses multicore backend for parallel processing
(faster but Unix/Mac only). If FALSE (default), uses multisession backend
(works on all platforms including Windows).}

\item{benchmarking}{Logical. If TRUE, returns processing time in seconds instead of
the result data frame. Default is FALSE.}

\item{verbose}{Logical. If TRUE (default), progress and status messages are
printed via \code{message()}. If FALSE, such messages are suppressed.}
}
\value{
Data frame. Processed results containing extracted variables,
with duplicates removed (when \code{multi_response=FALSE}) and sorted by ID.
}
\description{
This is the main function that orchestrates parallel processing of text data
using AI APIs for variable extraction. It optimizes for speed and cost by
distributing work across multiple cores and handling missing data efficiently.
}
\details{
The function implements a multi-stage parallel processing workflow:
\enumerate{
\item Validates parameters and tests API connectivity
\item Randomly samples data and splits into batches for parallel processing
\item Executes parallel processing using futures and tracks progress
\item Handles missing/failed extractions with a secondary processing stage
\item Combines and deduplicates results
}

The function requires a coding instruction to be set via \code{set_coding_instruction()}
before execution. This instruction tells the AI how to format its responses.
}
\examples{
\dontrun{
# Set up API specifications and coding instruction
set_api_specs(provider = "OpenAI", model = "gpt-4", temp = 0.7, api_key = "your_key")
set_coding_instruction("Extract sentiment: id,sentiment,confidence")

# Process data with random sampling (default)
results <- parallel_execute(
  test_data = my_data,
  slicing_n = 240,
  cores = 4
)

# Process data with reproducible sampling (for calculate_overlap)
results1 <- parallel_execute(
  test_data = my_data,
  slicing_n = 240,
  cores = 4,
  seed = 123
)

# Switch to GROQ for comparison
set_api_specs(provider = "Groq", model = "llama-3.3-70b-versatile",
  temp = 0.3, api_key = "your_groq_key")
results2 <- parallel_execute(
  test_data = my_data,
  slicing_n = 240,
  cores = 4,
  seed = 123
)

# Calculate overlap between identical datasets
overlap_scores <- calculate_overlap(list(results1, results2))

}

}
\seealso{
\code{\link[=set_api_specs]{set_api_specs()}}, \code{\link[=set_coding_instruction]{set_coding_instruction()}}, \code{\link[=set_parameters]{set_parameters()}}
}
