> ## Documentation Index
> Fetch the complete documentation index at: https://r2r-patch-decruft-openapi-json.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Local LLMs

> Learn how to run a Retrieval-Augmented Generation system locally using R2R

## Introduction

To run R2R with default local LLM settings, execute `r2r serve --docker --config-name=local_llm`.

R2R supports RAG with local LLMs through the Ollama library. You may follow the instructions on their [official website](https://ollama.com/) to install Ollama outside of the R2R Docker.

## Preparing Local LLMs

Next, make sure that you have all the necessary LLMs installed:

```bash
# in a separate terminal
ollama pull llama3.1
ollama pull mxbai-embed-large
ollama serve
```

These commands will need to be replaced with models specific to your configuration when deploying R2R with a customized configuration.

## Configuration

R2R uses a TOML configuration file for managing settings, which you can [read about here](/documentation/configuration/introduction). For local setup, we'll use the default `local_llm` configuration. This can be customized to your needs by setting up a standalone project.

<AccordionGroup>
  <Accordion icon="gear" title="Local Configuration Details">
    The `local_llm` configuration file (`core/configs/local_llm.toml`) includes:

    ```toml
    [completion]
    provider = "litellm"
    concurrent_request_limit = 1

      [completion.generation_config]
      model = "ollama/llama3.1"
      temperature = 0.1
      top_p = 1
      max_tokens_to_sample = 1_024
      stream = false
      add_generation_kwargs = { }

    [database]
    provider = "postgres"

    [embedding]
    provider = "ollama"
    base_model = "mxbai-embed-large"
    base_dimension = 1_024
    batch_size = 32
    add_title_as_prefix = true
    concurrent_request_limit = 32

    [ingestion]
    excluded_parsers = [ "mp4" ]
    ```

    This configuration uses `ollama` and the model `mxbai-embed-large` to run embeddings. We have excluded media file parsers as they are not yet supported locally.

    <Note>
      We are still working on adding local multimodal RAG features. Your feedback would be appreciated.
    </Note>
  </Accordion>
</AccordionGroup>

For more information on how to configure R2R, [visit here](/documentation/configuration/introduction).

## Summary

The above steps are all you need to get RAG up and running with local LLMs in R2R. For detailed setup and basic functionality, refer back to the \[R2R Quickstart]\((/documentation/quickstart/introduction). For more advanced usage and customization options, refer to the \[basic configuration]\((/documentation/configuration/introduction) or join the [R2R Discord community](https://discord.gg/p6KqD2kjtB).