# Execution Plans and Streaming Workflows

Large exact jobs are not only about arithmetic. They are also about controlling how much work is done per block and where the results are stored. `bigKNN` addresses those two questions with:

- execution plans, which turn a memory budget into a reproducible block size
- streaming APIs, which write results into destination `big.matrix` objects instead of returning everything as dense R objects

This vignette uses a moderately sized toy example so the code stays readable, but the same patterns scale to much larger references and query batches.

# Why plans and streaming matter for large matrices

For a small exploratory search, the simplest thing is often best:

- call `knn_bigmatrix()`
- get a pair of dense `index` and `distance` matrices back
- keep working in ordinary R objects

That becomes less attractive as the job grows. The exact computation may still fit, but the output itself can become large:

- `k`-NN output scales like `n_query x k`
- radius output scales with the total number of matches, which may vary a lot from one query to the next
- exact search still needs a block size that matches the memory budget you want to spend

Execution plans and streaming let you control those two pressure points directly.

# Build a Repeatable Example

The reference matrix below has 160 rows and 4 columns. It is large enough to show real changes in derived block size, but still small enough that we can inspect outputs directly in the vignette.

    i <- seq_len(160)

    reference_matrix <- cbind(
      x1 = i,
      x2 = (i %% 7) + 1,
      x3 = (i %% 11) + 0.5,
      x4 = (i %% 13) + 2
    )

    reference <- as.big.matrix(reference_matrix)

    dense_query <- rbind(
      reference_matrix[5, ] + c(0.2, 0.0, 0.1, 0.0),
      reference_matrix[50, ] + c(-0.3, 0.2, 0.0, 0.1),
      reference_matrix[120, ] + c(0.4, -0.1, 0.2, 0.0),
      reference_matrix[151, ] + c(0.1, 0.2, -0.2, 0.3)
    )

    query_ids <- paste0("q", seq_len(nrow(dense_query)))

    dim(reference_matrix)
    #> [1] 160   4
    dense_query
    #>         x1  x2   x3   x4
    #> [1,]   5.2 6.0  5.6  7.0
    #> [2,]  49.7 2.2  6.5 13.1
    #> [3,] 120.4 1.9 10.7  5.0
    #> [4,] 151.1 5.2  8.3 10.3

# Building a plan with `knn_plan_bigmatrix()`

`knn_plan_bigmatrix()` converts a target memory budget into a search plan. The plan stores the metric, the requested thread count, whether progress reporting should be enabled, and most importantly the derived `block_size`.

    plan <- knn_plan_bigmatrix(
      reference,
      metric = "euclidean",
      memory_budget = "64KB",
      num_threads = 2L,
      progress = FALSE
    )

    plan
    #> <bigknn_plan>
    #>   metric: euclidean
    #>   memory_budget: 64KB
    #>   block_size: 72
    #>   num_threads: 2
    #>   shape: 160 x 4

The printed object is the key summary you usually need:

- `metric`: the exact distance used by plan-aware calls
- `memory_budget`: the requested budget in a normalized form
- `block_size`: the number of rows processed per block
- `num_threads`: the thread request forwarded to common BLAS/OpenMP variables
- `shape`: the reference dimensions the plan was built for

# How memory budget maps to block size

Block size is derived rather than guessed. With the same reference matrix, a larger budget yields a larger working block.

    plan_small <- knn_plan_bigmatrix(
      reference,
      metric = "euclidean",
      memory_budget = "4KB",
      num_threads = 2L,
      progress = FALSE
    )

    plan_large <- knn_plan_bigmatrix(
      reference,
      metric = "euclidean",
      memory_budget = "1MB",
      num_threads = 2L,
      progress = FALSE
    )

    data.frame(
      memory_budget = c(plan_small$memory_budget, plan$memory_budget, plan_large$memory_budget),
      block_size = c(plan_small$block_size, plan$block_size, plan_large$block_size),
      row.names = NULL
    )
    #>   memory_budget block_size
    #> 1           4KB         13
    #> 2          64KB         72
    #> 3           1MB        160

This is useful even though the search remains exact. The plan does not change which neighbours are returned. It changes how much data is processed at once, which makes resource use more predictable across runs and machines.

In practice:

- use a smaller budget when memory is tight or when multiple jobs share a host
- use a larger budget when you want fewer, larger blocks
- keep the plan object if you want the same resource policy to be reused across multiple calls

# Running planned in-memory search

Plans plug into the same search API you would use without them.

    planned_knn <- knn_bigmatrix(
      reference,
      query = dense_query,
      k = 3,
      plan = plan,
      exclude_self = FALSE
    )

    planned_knn
    #> <bigknn_knn_result>
    #>   metric: euclidean
    #>   k: 3
    #>   queries: 4
    #>   references: 160
    #>   backend: bruteforce
    knn_table(planned_knn, query_ids = query_ids)
    #>    query rank neighbor distance
    #> 1     q1    1        5  0.22361
    #> 2     q1    2        6  1.85740
    #> 3     q1    3        4  2.15640
    #> 4     q2    1       50  0.37417
    #> 5     q2    2       49  2.03470
    #> 6     q2    3       51  2.03470
    #> 7     q3    1      120  0.45826
    #> 8     q3    2      119  2.28250
    #> 9     q3    3      118  6.37260
    #> 10    q4    1      151  0.42426
    #> 11    q4    2      152  1.83850
    #> 12    q4    3      150  2.23160

The result contract is unchanged. You still get dense `index` and `distance` matrices. The difference is that `bigKNN` uses the plan’s block size and thread policy while computing them.

# Streaming `k`-NN output with `knn_stream_bigmatrix()`

When `n_query x k` is large enough that you do not want the full result held in ordinary R matrices, switch to `knn_stream_bigmatrix()`.

The destination requirements are straightforward:

- `xpIndex` must be a writable `big.matrix` with `n_query` rows and `k` columns
- `xpDistance`, if supplied, must have the same dimensions
- `xpIndex` should use integer or double storage
- `xpDistance` should use double storage

    index_store <- big.matrix(nrow(dense_query), 3, type = "integer")
    distance_store <- big.matrix(nrow(dense_query), 3, type = "double")

    streamed_knn <- knn_stream_bigmatrix(
      reference,
      query = dense_query,
      xpIndex = index_store,
      xpDistance = distance_store,
      k = 3,
      plan = plan,
      exclude_self = FALSE
    )

    bigmemory::as.matrix(streamed_knn$index)
    #>      [,1] [,2] [,3]
    #> [1,]    5    6    4
    #> [2,]   50   49   51
    #> [3,]  120  119  118
    #> [4,]  151  152  150
    round(bigmemory::as.matrix(streamed_knn$distance), 4)
    #>        [,1]   [,2]   [,3]
    #> [1,] 0.2236 1.8574 2.1564
    #> [2,] 0.3742 2.0347 2.0347
    #> [3,] 0.4583 2.2825 6.3726
    #> [4,] 0.4243 1.8385 2.2316

Those streamed outputs match the in-memory result exactly:

    identical(bigmemory::as.matrix(streamed_knn$index), planned_knn$index)
    #> [1] TRUE
    all.equal(bigmemory::as.matrix(streamed_knn$distance), planned_knn$distance)
    #> [1] TRUE

# Streaming radius output with `radius_stream_bigmatrix()`

Radius search needs one extra step before streaming: you have to know how much space to allocate for the flattened matches. The simplest pattern is:

1. call `count_within_radius_bigmatrix()` to get per-query counts
2. allocate `sum(counts)` rows for the flattened `index` and `distance`
3. allocate `length(counts) + 1` rows for the offset vector
4. run `radius_stream_bigmatrix()`

    radius_counts <- count_within_radius_bigmatrix(
      reference,
      query = dense_query,
      radius = 2.2,
      plan = plan,
      exclude_self = FALSE
    )

    radius_counts
    #> [1] 3 3 1 2
    total_matches <- sum(radius_counts)
    total_matches
    #> [1] 9

    radius_index_store <- big.matrix(total_matches, 1, type = "integer")
    radius_distance_store <- big.matrix(total_matches, 1, type = "double")
    radius_offset_store <- big.matrix(length(radius_counts) + 1L, 1, type = "double")

    streamed_radius <- radius_stream_bigmatrix(
      reference,
      query = dense_query,
      xpIndex = radius_index_store,
      xpDistance = radius_distance_store,
      xpOffset = radius_offset_store,
      radius = 2.2,
      plan = plan,
      exclude_self = FALSE
    )

    streamed_radius
    #> <bigknn_radius_result>
    #>   metric: euclidean
    #>   radius: 2.2
    #>   queries: 4
    #>   references: 160
    #>   matches: 9
    streamed_radius$n_match
    #> [1] 3 3 1 2

The offset vector is what makes flattened radius output usable. It tells you where each query’s matches begin and end.

    radius_offset <- as.vector(bigmemory::as.matrix(streamed_radius$offset))
    radius_index <- as.vector(bigmemory::as.matrix(streamed_radius$index))
    radius_distance <- as.vector(bigmemory::as.matrix(streamed_radius$distance))

    radius_offset
    #> [1]  1  4  7  8 10
    radius_slice_table(radius_index, radius_distance, radius_offset, query_ids, 1)
    #>   query neighbor distance
    #> 1    q1        5  0.22361
    #> 2    q1        6  1.85740
    #> 3    q1        4  2.15640
    radius_slice_table(radius_index, radius_distance, radius_offset, query_ids, 2)
    #>   query neighbor distance
    #> 1    q2       50  0.37417
    #> 2    q2       49  2.03470
    #> 3    q2       51  2.03470

If you prefer, you can skip the streamed route and use `radius_bigmatrix()` to have `bigKNN` allocate the flattened vectors for you. Streaming becomes useful when those flattened vectors are large or when you want explicit control over their storage.

# Dense versus sparse query inputs

The public API also accepts sparse query matrices. At the moment they are densified on the R side before exact computation, so sparse input is primarily an interface convenience rather than a sparse-native backend.

    sparse_query <- Matrix::Matrix(dense_query, sparse = TRUE)

    sparse_knn <- knn_bigmatrix(
      reference,
      query = sparse_query,
      k = 3,
      plan = plan,
      exclude_self = FALSE
    )

    identical(sparse_knn$index, planned_knn$index)
    #> [1] TRUE
    all.equal(sparse_knn$distance, planned_knn$distance)
    #> [1] TRUE

That makes sparse queries especially handy when your upstream workflow already produces `Matrix` objects, and you want to use the same exact search call without manually converting first.

# Practical guidance for choosing output modes

A simple rule of thumb works well:

- use `knn_bigmatrix()` when `n_query x k` is small enough that ordinary R matrices are convenient
- use `knn_stream_bigmatrix()` when `n_query x k` is large and you want control over where those matrices live
- use `radius_bigmatrix()` when flattened radius output is still manageable in memory
- use `radius_stream_bigmatrix()` when the total number of matches is large or highly variable across query rows
- use `knn_plan_bigmatrix()` whenever you want repeatable memory and threading policy instead of ad hoc defaults

Plans and streaming do not make the search approximate. They make the exact workflow more operationally predictable, which is often the difference between an algorithm that works in principle and one that is pleasant to run at scale.