# `ExFastembed`
[🔗](https://github.com/elchemista/ex_fastembed/blob/v0.1.2/lib/ex_fastembed.ex#L1)

Loads FastEmbed text embedding and reranker models through `fastembed-rs`.

Use `load/1` followed by `embed_text/1` to generate vectors, or
`load_reranker/1` followed by `rerank/3` to score documents for a query.

Each BEAM VM shares one embedding model and one independent reranker. Loading
a replacement changes the model used by all processes; failed initialization
preserves the previous model. Loading, inference, and unloading within each
family run serially: inference and state inspection wait while a load is in progress,
including downloads. Applications own their request queues and must stop
submitting work before unloading or deleting a model.

Model files are downloaded on first use and cached in `.fastembed_cache`.
Set `FASTEMBED_CACHE_DIR` before loading a model to use another directory.
If `HF_HOME` is exported when the VM starts, FastEmbed uses that directory
instead. Partial downloads reuse the cached revision for consistent model files.
Loading and inference block the caller while the native work runs on a dirty
scheduler, allowing other BEAM processes to continue running.

All public functions return error tuples for invalid input. Strings must be
valid UTF-8. Model names are matched case-insensitively against the bundled
FastEmbed catalog, including the legacy aliases.

Recoverable Rust panics return error tuples. Initialization panics preserve the
previous session; inference panics unload the affected session because its
internal state may be inconsistent. Reload that model before retrying.

See the [model lifecycle guide](model_lifecycle.html) for memory, disk usage,
cache configuration, and deletion examples.

# `cache_directory`

```elixir
@spec cache_directory() :: String.t()
```

Returns the absolute effective cache root without creating it.

Respects `HF_HOME` from the native process environment at VM startup, otherwise
`FASTEMBED_CACHE_DIR`, with `.fastembed_cache` as the default. Existing roots
are canonicalized, resolving symlinks.

# `embed_models`

```elixir
@spec embed_models() :: [String.t()]
```

Returns text embedding model names accepted by `load/1`.

Includes repository names, explicit FastEmbed variant names, and legacy aliases.
Names are matched case-insensitively.

The list is sorted and contains both full-precision and quantized variants.
A repository shared by several variants selects its non-quantized model when
available; use an explicit name such as `EmbeddingGemma300MQ4` for a specific variant.

## Examples

    iex> "BGESmallENV15" in ExFastembed.embed_models()
    true

# `loaded_models`

```elixir
@spec loaded_models() :: {:ok, [model_info()]} | error()
```

Lists the models currently held by the native runtime, with their original cache paths.

Returns `{:ok, []}` when nothing is loaded. Unlike `models/0`, this also finds
models loaded from a different `FASTEMBED_CACHE_DIR` before it was changed.
Metadata is a snapshot and can change immediately under concurrent calls.

# `model_info`

```elixir
@spec model_info(String.t(), model_kind()) :: {:ok, model_info()} | error()
```

Resolves an accepted model name to its variant, metadata, and local cache status.

`kind` must be `:embedding` or `:reranker`. Resolution follows the same aliases
and case-insensitive matching as `load/1` and `load_reranker/1`. See `models/0`
for the meaning of `cached`.

## Examples

    iex> {:ok, info} = ExFastembed.model_info("BAAI/bge-small-en-v1.5", :embedding)
    iex> {info.name, info.dimension}
    {"BGESmallENV15", 384}

# `models`

```elixir
@spec models() :: [model_info()]
```

Lists distinct model variants with their description, repository, dimension, and cache status.

Checks the cache locally without network access or loading a model. `cached`
means that the selected ONNX weights, additional data, and four tokenizer/config
files are present and non-empty. It does not validate their contents or mean that
the model is loaded in the VM. Call `load/1` or `load_reranker/1` before inference.

Uses `FASTEMBED_CACHE_DIR`, defaulting to `.fastembed_cache`. Embedding variants
appear first, then rerankers, sorted by name within each family. Rerankers have
a `nil` dimension.

Includes `loaded`, the absolute `cache_dir` and repository `path`, cached
`revision`, relative required `files`, and per-file `file_details` with snapshot
paths and byte sizes. `variant_bytes` is the sum of all required non-empty files
(`nil` if incomplete). `disk_bytes` counts regular file bytes across the entire
repository, including other variants, revisions and partial downloads, without
following symlinks or counting snapshot links twice (`nil` on read errors).
These are local file sizes, not RAM usage or remote download estimates. Shared
repositories repeat `disk_bytes`; deduplicate by `path` before summing.

`loaded` matches the variant and current cache root. Use `loaded_models/0` to
find sessions loaded from other cache roots. Discovery may wait for active
operations while reading runtime state.

## Examples

    iex> Enum.any?(ExFastembed.models(), &(&1.name == "BGESmallENV15"))
    true

# `reranker_models`

```elixir
@spec reranker_models() :: [String.t()]
```

Returns reranker model names accepted by `load_reranker/1`.

Includes repository names, explicit FastEmbed variant names, and legacy aliases.
Names are matched case-insensitively.

## Examples

    iex> "BAAI/bge-reranker-base" in ExFastembed.reranker_models()
    true

# `delete_model`

```elixir
@spec delete_model(String.t(), model_kind()) :: {:ok, true} | error()
```

Deletes the downloaded model files from disk, unloading matching native sessions first.

The cache directory is the model's on-disk storage: deletion physically removes
its repository directory, including ONNX weights, tokenizer/config files,
blobs, and partial downloads. To release only RAM and keep these files, use
`unload/0` or `unload_reranker/0`.

Accepts the same names and kinds as `model_info/2`. **All cached variants and
revisions in the selected repository are removed**, including shared tokenizer
files and blobs. Both model families are checked before removal. Unrelated
repositories and models loaded from a different cache root remain untouched.

Returns `{:ok, true}` if removed or already absent. Repository symlinks and
non-directory entries are rejected. Internal symlinks are removed without
following their targets. If file removal fails after unloading, deletion can
be partial and the model stays unloaded; correct the error and retry.

Stop submitting work and drain application queues first, as for `unload/0`.
Deletion is serialized with library loads in this VM. Other VMs or external
processes sharing the cache must be coordinated by the application.

# `download`

```elixir
@spec download(String.t(), model_kind()) :: {:ok, true} | error()
```

Downloads an embedding or reranker model's required files without initializing it.

Accepts the names and kinds from `model_info/2`, using the current cache root.
Returns `{:ok, true}` when all files are present and non-empty, including an
already cached model. Reuses existing files and repairs incomplete snapshots.
File contents are not validated as an ONNX model until `load/1` or
`load_reranker/1` is called. Neither loaded session is replaced or unloaded.
Serializes with loads and cache deletion in this VM; existing inference can continue.

Useful for warming a cache in a Docker build or before starting inference.

# `unload`

```elixir
@spec unload() :: {:ok, true} | error()
```

Unloads the shared embedding model while keeping downloaded files.

Waits for an active load or inference, then drops the ONNX session, weights,
and its owned native buffers. Returns `{:ok, true}` even if already unloaded.
The reranker is independent and remains available.

Stop submitting model work and drain application queues before calling this
function. The library does not manage or cancel jobs; concurrent calls are
serialized without an ordering guarantee. A subsequent load can recreate the
session. Allocators may retain freed memory, so process RSS need not immediately
decrease by the size of the model. Already returned embeddings remain owned by
the calling BEAM processes.

# `unload_reranker`

```elixir
@spec unload_reranker() :: {:ok, true} | error()
```

Unloads the shared reranker while keeping downloaded files and the embedding model.

Returns `{:ok, true}` even if already unloaded. The synchronization, application
queue ownership, and memory reclamation behavior described in `unload/0` also apply.

# `embed_documents`

```elixir
@spec embed_documents([String.t()]) :: {:ok, [embedding()]} | error()
```

Embeds retrieval documents, returning one vector per text in input order.

Adds `passage: ` for multilingual E5, `search_document: ` for Nomic and
ModernBERT, and `title: none | text: ` for EmbeddingGemma. Other variants pass
text through unchanged. Pass unprefixed texts. For an EmbeddingGemma document
with a title, supply the complete formatted prompt through `embed_text/1`.
Use the same model for `embed_query/1` and the documents it will search.
An empty list returns `{:ok, []}` without requiring a loaded session.

# `embed_documents`

```elixir
@spec embed_documents([String.t()], keyword()) :: {:ok, [embedding()]} | error()
```

Embeds retrieval documents with the `:batch_size` option from `embed_text/2`.

# `embed_query`

```elixir
@spec embed_query(String.t()) :: {:ok, embedding()} | error()
```

Embeds one retrieval query, returning a single vector.

Adds the retrieval query prefix for multilingual E5, BGE v1.5, Nomic,
Snowflake Arctic Embed, Mxbai Embed Large v1, ModernBERT and EmbeddingGemma,
including their quantized variants. Other catalog variants pass text through
unchanged, including BGE-M3. Pass unprefixed text: prefixes are always added.
Use `embed_text/1` for manual instructions or tasks other than retrieval.
The prefix is chosen under the same lock as inference, from the active session.
See the [retrieval prefix table](model_lifecycle.html#retrieval-prefixes).

# `embed_query`

```elixir
@spec embed_query(String.t(), keyword()) :: {:ok, embedding()} | error()
```

Embeds one retrieval query with the `:batch_size` option from `embed_text/2`.

# `embed_text`

```elixir
@spec embed_text([String.t()]) :: {:ok, [embedding()]} | error()
```

Embeds a list of strings with the loaded text embedding model.

Call `load/1` before calling this function. An empty list returns `{:ok, []}`
without running inference.

Results preserve the input order, with one vector per text. A non-empty input
returns an error if no embedding model is loaded. Tokenization and truncation
follow the selected model's FastEmbed defaults.

## Examples

    iex> ExFastembed.embed_text([])
    {:ok, []}

    iex> ExFastembed.embed_text(["document", 123])
    {:error, "Invalid input: texts must be a list of strings"}

# `embed_text`

```elixir
@spec embed_text([String.t()], keyword()) :: {:ok, [embedding()]} | error()
```

Embeds texts with an optional `:batch_size` (positive integer or `nil`).

`nil` uses FastEmbed's default. Smaller batches can reduce peak inference memory;
all returned vectors still reside in memory. Dynamic quantized models reject a
batch size smaller than the number of texts because their vectors would not be
comparable across batches. Batch-size errors preserve the loaded session.

Text is passed unchanged, with no retrieval prefixes. See `embed_text/1` for
input and output semantics. Unknown options and invalid batch sizes return an
error even for an empty input.

# `load`

```elixir
@spec load(String.t()) :: {:ok, pos_integer()} | error()
```

Loads a text embedding model and returns its embedding dimension.

A successful call replaces the embedding model shared by all processes.
A failed download or initialization leaves the previous model available.
Calls using that model, including state inspection, wait until loading finishes.
An initialization panic also preserves the previous session. If cleanup of a
replaced session panics, the error states that the replacement is already loaded.
See `embed_models/0` for accepted names. The returned dimension is the length
of each vector generated by `embed_text/1`.

## Examples

```elixir
{:ok, 384} = ExFastembed.load("BAAI/bge-small-en-v1.5")
```

    iex> ExFastembed.load("invalid-model")
    {:error, "Model not recognized or not implemented: invalid-model"}

# `load_reranker`

```elixir
@spec load_reranker(String.t()) :: {:ok, true} | error()
```

Loads a reranker model.

A successful call replaces the reranker shared by all processes, independently
of the embedding model. Failed initialization preserves the previous reranker.
Reranker inference and state inspection wait while loading is in progress.
Panic recovery follows the behavior described in `load/1`.
See `reranker_models/0` for accepted names.

## Examples

```elixir
{:ok, true} = ExFastembed.load_reranker("BAAI/bge-reranker-base")
```

    iex> ExFastembed.load_reranker("invalid-reranker")
    {:error, "Reranker model not recognized: invalid-reranker"}

# `rerank`

```elixir
@spec rerank(String.t(), [String.t()], boolean()) ::
  {:ok, [rerank_result()]} | error()
```

Reranks documents for a query using the loaded reranker model.

Call `load_reranker/1` before calling this function. An empty document list
returns `{:ok, []}` without running inference.

Results are sorted by descending relevance score. Each result contains the
document's zero-based index in the original list, its score, and its text when
`return_docs` is `true` (`nil` otherwise). Scores are model-specific and are
not necessarily probabilities. A non-empty input requires a loaded reranker.

## Examples

    iex> ExFastembed.rerank("query", [], false)
    {:ok, []}

# `rerank`

```elixir
@spec rerank(String.t(), [String.t()], boolean(), keyword()) ::
  {:ok, [rerank_result()]} | error()
```

Reranks documents with an optional positive `:batch_size`; `nil` uses FastEmbed's default. See `rerank/3`.

# `embedding`

```elixir
@type embedding() :: [float()]
```

A dense vector with the dimension reported by `load/1`.

# `error`

```elixir
@type error() :: {:error, String.t()}
```

An input, model-loading, or inference failure with a human-readable reason.

# `model_file`

```elixir
@type model_file() :: %{
  name: String.t(),
  path: String.t() | nil,
  size_bytes: pos_integer() | nil
}
```

A required file, its snapshot path when present, and its non-empty size in bytes.

# `model_info`

```elixir
@type model_info() :: %{
  name: String.t(),
  kind: model_kind(),
  repository: String.t(),
  description: String.t(),
  dimension: pos_integer() | nil,
  cached: boolean(),
  loaded: boolean(),
  cache_dir: String.t(),
  path: String.t(),
  revision: String.t() | nil,
  files: [String.t()],
  file_details: [model_file()],
  variant_bytes: non_neg_integer() | nil,
  disk_bytes: non_neg_integer() | nil
}
```

A model variant with runtime state, local file paths, and disk byte sizes.

# `model_kind`

```elixir
@type model_kind() :: :embedding | :reranker
```

The two supported model families.

# `rerank_result`

```elixir
@type rerank_result() :: {non_neg_integer(), float(), String.t() | nil}
```

A zero-based document index, relevance score, and optional document text.

---

*Consult [api-reference.md](api-reference.md) for complete listing*
