Prompt Management in R2R

R2R provides a flexible system for managing prompts, allowing you to create, update, retrieve, and delete prompts dynamically. This system is crucial for customizing the behavior of language models and ensuring consistent interactions across your application.

Default Prompts

R2R comes with a set of default prompts that are loaded from YAML files located in the py/core/providers/prompts/defaults directory. These default prompts provide a starting point for various tasks within the R2R system.

For example, the default RAG (Retrieval-Augmented Generation) prompt is defined as follows:

default_rag:
  template: >
    ## Task:

    Answer the query given immediately below given the context which follows later. Use line item references to like [1], [2], ... refer to specifically numbered items in the provided context. Pay close attention to the title of each given source to ensure it is consistent with the query.


    ### Query:

    {query}


    ### Context:

    {context}


    ### Query:

    {query}


    REMINDER - Use line item references to like [1], [2], ... refer to specifically numbered items in the provided context.

    ## Response:
  input_types:
    query: str
    context: str

Default Prompt Usage

Here’s a table showing the purpose of some key default prompts:

Certainly! I’ll create an expanded table that explains all the prompts you’ve listed, with links to their respective GitHub files. Here’s the updated table:

Prompt FilePurpose
default_rag.yamlDefault prompt for Retrieval-Augmented Generation (RAG) tasks. It instructs the model to answer queries based on provided context, using line item references.
few_shot_ner_kg_extraction.yamlUsed for few-shot Named Entity Recognition (NER) and Knowledge Graph (KG) extraction. It provides examples to guide the model in identifying entities and relationships.
few_shot_ner_kg_extraction_with_spec.yamlSimilar to the above, but includes a specific schema or specification for the extraction process.
graphrag_community_reports.yamlUsed in GraphRAG to generate reports about communities or clusters in the knowledge graph.
graphrag_map_system_prompt.yamlSystem prompt for the “map” phase in GraphRAG, used to process individual nodes or edges.
graphrag_reduce_system_prompt.yamlSystem prompt for the “reduce” phase in GraphRAG, used to combine or summarize information from multiple sources.
graphrag_triples_extraction_few_shot.yamlFew-shot prompt for extracting subject-predicate-object triplets in GraphRAG, with examples.
graphrag_triples_extraction_few_shot.yamlZero-shot prompt for extracting subject-predicate-object triplets in GraphRAG, without examples.
hyde.yamlRelated to Hypothetical Document Embeddings (HyDE) for improving retrieval performance.
kg_search.yamlUsed for searching the knowledge graph, possibly to formulate queries or interpret results.
kg_search_with_spec.yamlSimilar to kg_search.yaml, but with a specific schema or specification for the search process.
rag_agent.yamlDefines the behavior and instructions for the RAG agent, which coordinates the retrieval and generation process.
rag_context.yamlUsed to process or format the context retrieved for RAG tasks.
rag_fusion.yamlUsed in RAG fusion techniques, possibly for combining information from multiple retrieved passages.
system.yamlContains general system-level prompts or instructions for the R2R system.
zero_shot_ner_kg_extraction.yamlUsed for zero-shot Named Entity Recognition and Knowledge Graph extraction, without providing examples.
zero_shot_ner_kg_extraction_with_spec.yamlSimilar to the above, but includes a specific schema or specification for the extraction process.

You can find the full list of default prompts and their contents in the defaults directory.

Prompt Provider

R2R uses a R2RPromptProvider class to manage prompts. This provider handles the storage, retrieval, and manipulation of prompts, leveraging both a PostgreSQL database and YAML files for flexibility and persistence.

Key features of the R2RPromptProvider:

  1. Database Storage: Prompts are stored in a PostgreSQL table, allowing for efficient querying and updates.
  2. YAML File Support: Prompts can be loaded from YAML files, providing an easy way to version control and distribute default prompts.
  3. In-Memory Cache: Prompts are kept in memory for fast access during runtime.

Prompt Structure

Each prompt in R2R consists of:

  • Name: A unique identifier for the prompt.
  • Template: The actual text of the prompt, which may include placeholders for dynamic content.
  • Input Types: A dictionary specifying the expected types for any dynamic inputs to the prompt.

Managing Prompts

R2R provides several endpoints and SDK methods for managing prompts:

Adding a Prompt

To add a new prompt:

from r2r import R2RClient

client = R2RClient()

response = client.add_prompt(
    name="my_new_prompt",
    template="Hello, {name}! Welcome to {service}.",
    input_types={"name": "str", "service": "str"}
)

Updating a Prompt

To update an existing prompt:

response = client.update_prompt(
    name="my_existing_prompt",
    template="Updated template: {variable}",
    input_types={"variable": "str"}
)

Retrieving a Prompt

To get a specific prompt:

response = client.get_prompt(
    prompt_name="my_prompt",
    inputs={"variable": "example"},
    prompt_override="Optional override text"
)

Listing All Prompts

To retrieve all prompts:

response = client.get_all_prompts()

Deleting a Prompt

To delete a prompt:

response = client.delete_prompt("prompt_to_delete")

Security Considerations

Access to prompt management functions is restricted to superusers to prevent unauthorized modifications to system prompts. Ensure that only trusted administrators have superuser access to your R2R deployment.

Best Practices

  1. Version Control: Store your prompts in version-controlled YAML files for easy tracking of changes and rollbacks.
  2. Consistent Naming: Use a consistent naming convention for your prompts to make them easy to identify and manage.
  3. Input Validation: Always specify input types for your prompts to ensure that they receive the correct data types.
  4. Regular Audits: Periodically review and update your prompts to ensure they remain relevant and effective.
  5. Testing: Test prompts thoroughly before deploying them to production, especially if they involve complex logic or multiple input variables.

Advanced Usage

Dynamic Prompt Loading

R2R’s prompt system allows for dynamic loading of prompts from both the database and YAML files. This enables you to:

  1. Deploy default prompts with your application.
  2. Override or extend these prompts at runtime.
  3. Easily update prompts without redeploying your entire application.

Prompt Templating

The prompt template system in R2R supports complex string formatting. You can include conditional logic, loops, and other Python expressions within your prompts using a templating engine.

Example of a more complex prompt template:

complex_template = """
Given the following information:
{% for item in data %}
- {{ item.name }}: {{ item.value }}
{% endfor %}

Please provide a summary that {% if include_analysis %}includes an analysis of the data{% else %}lists the key points{% endif %}.
"""

client.add_prompt(
    name="complex_summary",
    template=complex_template,
    input_types={"data": "list", "include_analysis": "bool"}
)

This flexibility allows you to create highly dynamic and context-aware prompts that can adapt to various scenarios in your application.

Conclusion

R2R’s prompt management system provides a powerful and flexible way to control the behavior of language models in your application. By leveraging this system effectively, you can create more dynamic, context-aware, and maintainable AI-powered features.

For more detailed information on other aspects of R2R configuration, please refer to the following pages: