Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: how-to on default configuration for Ragbits #359

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ludwiktrammer
Copy link
Collaborator

No description provided.

@ludwiktrammer ludwiktrammer linked an issue Feb 12, 2025 that may be closed by this pull request
Copy link
Contributor

Trivy scanning results.

Copy link
Contributor

github-actions bot commented Feb 12, 2025

badge

Code Coverage Summary

Filename                                                                                                        Stmts    Miss  Cover    Missing
------------------------------------------------------------------------------------------------------------  -------  ------  -------  ------------------------------------------------------
packages/__init__.py                                                                                                0       0  100.00%
packages/ragbits-cli/src/ragbits/cli/__init__.py                                                                   31       4  87.10%   73-74, 81-82
packages/ragbits-cli/src/ragbits/cli/_utils.py                                                                     23       4  82.61%   45, 62-64
packages/ragbits-cli/src/ragbits/cli/state.py                                                                      79       3  96.20%   50-51, 61
packages/ragbits-cli/tests/unit/test_state.py                                                                      72       2  97.22%   103-104
packages/ragbits-conversations/src/ragbits/conversations/__init__.py                                                0       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/__init__.py                                        0       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/__init__.py                            3       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/base.py                               10       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/llm.py                                29       1  96.55%   79
packages/ragbits-conversations/tests/unit/history/test_llm_compressor.py                                           64       0  100.00%
packages/ragbits-core/src/ragbits/core/__init__.py                                                                  6       2  66.67%   8-9
packages/ragbits-core/src/ragbits/core/cli.py                                                                       6       0  100.00%
packages/ragbits-core/src/ragbits/core/config.py                                                                   17       0  100.00%
packages/ragbits-core/src/ragbits/core/options.py                                                                  17       0  100.00%
packages/ragbits-core/src/ragbits/core/types.py                                                                     9       0  100.00%
packages/ragbits-core/src/ragbits/core/audit/__init__.py                                                           74       5  93.24%   42-45, 52-54
packages/ragbits-core/src/ragbits/core/audit/base.py                                                               44       1  97.73%   119
packages/ragbits-core/src/ragbits/core/audit/cli.py                                                                69       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                                       4       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/base.py                                                          20       2  90.00%   56, 69
packages/ragbits-core/src/ragbits/core/embeddings/exceptions.py                                                    17       7  58.82%   7-8, 17, 26-27, 36, 45
packages/ragbits-core/src/ragbits/core/embeddings/litellm.py                                                       38      18  52.63%   78-112
packages/ragbits-core/src/ragbits/core/embeddings/noop.py                                                           8       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/__init__.py                                                             4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/base.py                                                                66       8  87.88%   61, 73, 174, 193-201, 205
packages/ragbits-core/src/ragbits/core/llms/exceptions.py                                                          17       7  58.82%   7-8, 17, 26-27, 36, 45
packages/ragbits-core/src/ragbits/core/llms/factory.py                                                             12       3  75.00%   30, 41, 51
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                             95      29  69.47%   94, 100-121, 165, 175, 204-229, 251-256, 267
packages/ragbits-core/src/ragbits/core/llms/local.py                                                               56      27  51.79%   9-12, 66-74, 86-87, 108-119, 140-156
packages/ragbits-core/src/ragbits/core/llms/mock.py                                                                29       9  68.97%   63-74
packages/ragbits-core/src/ragbits/core/metadata_stores/__init__.py                                                  3       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/base.py                                                     11       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/exceptions.py                                                4       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/in_memory.py                                                17       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/__init__.py                                                           2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/_cli.py                                                              44      21  52.27%   25-33, 47-49, 63-65, 73-75, 89-97
packages/ragbits-core/src/ragbits/core/prompt/base.py                                                              20       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                           35       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                           126       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/__init__.py                                                 2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/prompt_discovery.py                                        33       2  93.94%   54-55
packages/ragbits-core/src/ragbits/core/utils/__init__.py                                                            0       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/_pyproject.py                                                         38       1  97.37%   113
packages/ragbits-core/src/ragbits/core/utils/config_handling.py                                                    72       8  88.89%   16, 54-55, 62-63, 152-154
packages/ragbits-core/src/ragbits/core/utils/decorators.py                                                         29       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/dict_transformations.py                                               72       3  95.83%   24, 27, 108
packages/ragbits-core/src/ragbits/core/vector_stores/__init__.py                                                    3       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/_cli.py                                                       51       4  92.16%   63, 85, 91, 121
packages/ragbits-core/src/ragbits/core/vector_stores/base.py                                                       40       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/chroma.py                                                     59       1  98.31%   77
packages/ragbits-core/src/ragbits/core/vector_stores/in_memory.py                                                  38       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/pgvector.py                                                  124      12  90.32%   173-181, 215-217, 289
packages/ragbits-core/src/ragbits/core/vector_stores/qdrant.py                                                     62       1  98.39%   80
packages/ragbits-core/tests/cli/__init__.py                                                                         0       0  100.00%
packages/ragbits-core/tests/cli/test_cli_trace_handler.py                                                          48       3  93.75%   30, 43, 56
packages/ragbits-core/tests/cli/test_vector_store.py                                                              105       0  100.00%
packages/ragbits-core/tests/integration/vector_stores/test_vector_store.py                                         31       0  100.00%
packages/ragbits-core/tests/unit/__init__.py                                                                        0       0  100.00%
packages/ragbits-core/tests/unit/test_options.py                                                                   21       0  100.00%
packages/ragbits-core/tests/unit/audit/__init__.py                                                                  0       0  100.00%
packages/ragbits-core/tests/unit/audit/test_cli.py                                                                102       0  100.00%
packages/ragbits-core/tests/unit/audit/test_trace.py                                                               93       3  96.77%   14, 17, 20
packages/ragbits-core/tests/unit/embeddings/test_from_config.py                                                    14       0  100.00%
packages/ragbits-core/tests/unit/llms/__init__.py                                                                   0       0  100.00%
packages/ragbits-core/tests/unit/llms/test_from_config.py                                                          16       0  100.00%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                              70       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/__init__.py                                                           0       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_get_default_llm.py                                              12       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/__init__.py                                                        0       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/test_from_config.py                                               11       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/test_in_memory.py                                                 22       0  100.00%
packages/ragbits-core/tests/unit/prompts/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_parsers.py                                                           65       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_prompt.py                                                           165       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/__init__.py                                                      0       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/prompt_classes_for_tests.py                                     30       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/test_prompt_discovery.py                                        18       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/__init__.py                       2       1  50.00%   3
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/__init__.py               3       2  33.33%   2-4
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt1.py          14       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt2.py          14       0  100.00%
packages/ragbits-core/tests/unit/utils/__init__.py                                                                  0       0  100.00%
packages/ragbits-core/tests/unit/utils/test_config_handling.py                                                     64       2  96.88%   27-28
packages/ragbits-core/tests/unit/utils/test_decorators.py                                                          26       2  92.31%   17, 39
packages/ragbits-core/tests/unit/utils/test_dict_transformations.py                                                69       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_find.py                                                      13       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_config.py                                                 9       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_instace.py                                               37       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/__init__.py                                                          0       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chroma.py                                                      61       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_from_config.py                                                 42       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_in_memory.py                                                   84       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_pgvector.py                                                   173       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_qdrant.py                                                      40       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/__init__.py                                            2       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                             119       4  96.64%   266-273, 276
packages/ragbits-document-search/src/ragbits/document_search/cli.py                                                39       2  94.87%   85, 104
packages/ragbits-document-search/src/ragbits/document_search/documents/__init__.py                                  0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/document.py                                 61       2  96.72%   99, 146
packages/ragbits-document-search/src/ragbits/document_search/documents/element.py                                  78      12  84.62%   87, 167-174, 183-185
packages/ragbits-document-search/src/ragbits/document_search/documents/exceptions.py                               11       5  54.55%   7-8, 17, 26-27
packages/ragbits-document-search/src/ragbits/document_search/documents/source_resolver.py                          20       5  75.00%   5, 51-52, 55-56
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py                                 180      15  91.67%   166, 204, 246, 383-386, 390-391, 421, 428-429, 443-445
packages/ragbits-document-search/src/ragbits/document_search/ingestion/__init__.py                                  0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py                       33       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/__init__.py             5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/base.py                25       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/batched.py             18       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/distributed.py         30       8  73.33%   8-9, 36, 64-71
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/sequential.py          13       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py                        3       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/base.py                           19       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/dummy.py                          20       7  65.00%   33, 54-60
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/__init__.py           4       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/default.py           46       4  91.30%   98, 103-104, 137
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/images.py            50      19  62.00%   73-80, 87-99, 111, 124
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/pdf.py               19       6  68.42%   23, 35-43
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/utils.py             38      11  71.05%   71, 82-83, 98-101, 110, 121-123
packages/ragbits-document-search/src/ragbits/document_search/retrieval/__init__.py                                  0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py                       6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/base.py                           9       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/llm.py                           25       4  84.00%   47-50
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/multi.py                         27       4  85.19%   51-54
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/noop.py                           6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/prompts.py                       26       2  92.31%   65, 87
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                        3       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/base.py                           17       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/litellm.py                        18       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                           11       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/reciprocal_ranked_fusion.py       22       2  90.91%   50, 60
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/rerankers_answerdotai.py          21       0  100.00%
packages/ragbits-document-search/tests/__init__.py                                                                  0       0  100.00%
packages/ragbits-document-search/tests/helpers.py                                                                   3       0  100.00%
packages/ragbits-document-search/tests/cli/test_ingest.py                                                          21       0  100.00%
packages/ragbits-document-search/tests/cli/test_search.py                                                          71       0  100.00%
packages/ragbits-document-search/tests/integration/__init__.py                                                      0       0  100.00%
packages/ragbits-document-search/tests/integration/test_rerankers.py                                               24       6  75.00%   20-42
packages/ragbits-document-search/tests/integration/test_sources.py                                                 23      10  56.52%   22-32, 40-45
packages/ragbits-document-search/tests/integration/test_unstructured.py                                            48      10  79.17%   52-58, 71-77
packages/ragbits-document-search/tests/unit/test_config.py                                                         57       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_processor.py                                             17       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                                               233       1  99.57%   466
packages/ragbits-document-search/tests/unit/test_documents.py                                                      13       0  100.00%
packages/ragbits-document-search/tests/unit/test_elements.py                                                       19       0  100.00%
packages/ragbits-document-search/tests/unit/test_local_file_source.py                                              13       0  100.00%
packages/ragbits-document-search/tests/unit/test_processing_strategies.py                                          25       0  100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                                      41       0  100.00%
packages/ragbits-document-search/tests/unit/test_rephrasers.py                                                     40       0  100.00%
packages/ragbits-document-search/tests/unit/test_rerankers.py                                                      81       1  98.77%   25
packages/ragbits-document-search/tests/unit/test_source_discriminator.py                                           35       0  100.00%
packages/ragbits-document-search/tests/unit/test_sources.py                                                        52       8  84.62%   20-21, 56-61
packages/ragbits-document-search/tests/unit/testprojects/project_with_instance_factory/__init__.py                  0       0  100.00%
packages/ragbits-document-search/tests/unit/testprojects/project_with_instance_factory/factories.py                20       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/__init__.py                                                      0       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/base.py                                                         15       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/openai_moderation.py                                            19       5  73.68%   29-33
packages/ragbits-guardrails/tests/unit/test_openai_moderation.py                                                   35       0  100.00%
TOTAL                                                                                                            5207     351  93.26%

Diff against main

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 7fd16de

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

Copy link
Collaborator

@kzamlynska kzamlynska left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way you use 'default' (component/configuration) is not clear for me. I haven't comment all the places I have problem with understanding it. But as I am not very good in reading instructions treat my comments as suggestions :)

docs/cli/main.md Outdated
@@ -2,6 +2,8 @@

Ragbits comes with a command line interface (CLI) that provides a number of commands for working with the Ragbits platform. It can be accessed by running the `ragbits` command in your terminal.

Functions that operate on Ragbits components (like [`ragbits vector-store`](#ragbits-vector-store)) try to use default values for the components if they are not provided explicitly. To learn how to set these defaults in your project, see the [How to Set Default Configuration for Components](../how-to/core/configuration.md) guide.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First sentence is not clear for me, maybe:
Whenever possible, functions that operate on Ragbits components (such as the Ragbits vector store) should be used with their default values
or:
With functions/In the functions ....

```

#### DocumentSearch Specific Behavior
While you can provide `DocumentSearch` with a default configuration in the same way as other components (by setting the `document_search` key in the YAML configuration file), there is also a shortcut. If you don't provide a default configuration for `DocumentSearch` explicitly, it will look for the default configuration of all the components it uses (like `vector_store`, `provider`, `rephraser`, `reranker`, etc.) and use them as its own default configuration. This way, you don't have to configure those components twice (once for `DocumentSearch` and once for the component itself).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't provide a default configuration for DocumentSearch explicitly, it will look...
It is not clear what is "it". Maybe: program, system. Or maybe:
The default configuration of all the components will be used.

# How to Set Default Configuration for Components

## Introduction
When you use Ragbits in your project, you can set default configurations for different types of components (like embedders, vector stores, LLMs, etc.) in the project configuration. Typically, there are many different implementations for each type of component, and each implementation has its own configurations. Ragbits allows you to choose the implementation you want to use by default for each type of component and set configurations to be used along with it.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ragbits allows you to choose the implementation you want to use by default ...

I think it will be more clear without 'by default'. For me by default means that it is set in the system and I don't have to do anything to set it.

When you use Ragbits in your project, you can set default configurations for different types of components (like embedders, vector stores, LLMs, etc.) in the project configuration. Typically, there are many different implementations for each type of component, and each implementation has its own configurations. Ragbits allows you to choose the implementation you want to use by default for each type of component and set configurations to be used along with it.

## Using the Default Components
Your default configuration is used automatically by the [Ragbits CLI](../../cli/main.md). The `ragbits` commands that work on components (like [`ragbits vector-store`](../../cli/main.md#ragbits-vector-store), [`ragbits document-search`](../../cli/main.md#ragbits-document-search), etc.) will use the default component configuration unless instructed otherwise.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here also I would skip deafult in 'Your default configuration'
Your configuration ....


You can also use the default configuration in your code by instantiating the component using the `subclass_from_defaults()` factory method of [the base class of the given component type](#list-of-component-types). This method will create an instance of the default implementation of the component using the default configuration.

For example, this will create an instance of the default vector store implementation with the default configuration (as long as you have set the default vector store in the project configuration):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about an example how to set default vector store in project configuration

Note that `VectorStore` itself is an abstract class, so the instance created by `subclass_from_defaults()` will be an instance of one of the concrete subclasses of `VectorStore` that you have set as the default in the project configuration.

## Setting the Default Configuration
You can specify the default configuration for components in two different ways: either by providing a factory function that creates the default instance of the component or by providing a YAML configuration file that contains the default configuration.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"You can specify the default configuration for components"
Maybe: "You can specify the your own configuration of the components"
default configuration I undesratnd as the one which doesn;t need to be set. For example for Vectore Store we have defined VectorStoreOptions k=5 and max_distance=None and it is default configuration. I can use this one or set my own parameters

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

docs: how-to on default configuration for Ragbits
2 participants