Refactor the docs and the examples folders.

Author: drazvan

CHANGELOG.md

@@ -17,7 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 
 - Support for [custom configuration data](./docs/user_guide/configuration-guide.md#custom-data).
-- Example for using [custom LLM and multiple KBs](./examples/multi_kb/README.md)
+- Example for using [custom LLM and multiple KBs](examples/configs/multi_kb/README.md)
 - Support for [`PROMPTS_DIR`](./docs/user_guide/advanced/prompt-customization.md#prompt-configuration).
 - [#101](https://github.com/NVIDIA/NeMo-Guardrails/pull/101) Support for [using OpenAI embeddings](./docs/user_guide/configuration-guide.md#the-embeddings-model) models in addition to SentenceTransformers.
 - First set of end-to-end QA tests for the example configurations.
@@ -82,8 +82,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - Support for chat models i.e. prompting with a sequence of messages.
 - Experimental support for allowing the LLM to generate [multi-step flows](docs/user_guide/configuration-guide.md#multi-step-generation).
 - Example of using Llama Index from a guardrails configuration (#40).
-- [Example](examples/llm/hf_endpoint) for using HuggingFace Endpoint LLMs with a guardrails configuration.
-- [Example](examples/llm/hf_pipeline_dolly) for using HuggingFace Pipeline LLMs with a guardrails configuration.
+- [Example](examples/configs/llm/hf_endpoint) for using HuggingFace Endpoint LLMs with a guardrails configuration.
+- [Example](examples/configs/llm/hf_pipeline_dolly) for using HuggingFace Pipeline LLMs with a guardrails configuration.
 - Support to alter LLM parameters passed as `model_kwargs` in LangChain.
 - CLI tool for running evaluations on the different steps (e.g., canonical form generation, next steps, bot message) and on existing rails implementation (e.g., moderation, jailbreak, fact-checking, and hallucination).
 - [Initial evaluation](nemoguardrails/eval/README.md) results for `text-davinci-003` and `gpt-3.5-turbo`.

docs/README.md

@@ -21,11 +21,11 @@ The getting started section covers two topics:
 
 Five reference examples are provided as a general demonstration for building different types of rails:
 
-* [Topical Rail](../examples/topical_rail/README.md): Making the bot stick to a specific topic of conversation.
-* [Moderation Rail](../examples/moderation_rail/README.md): Moderating a bot's response.
-* [Fact Checking and Hallucination Rail](../examples/grounding_rail/README.md): Ensuring factual answers.
-* [Secure Execution Rail](../examples/execution_rails/README.md): Executing a third-party service with LLMs.
-* [Jailbreaking Rail](../examples/jailbreak_check/README.md): Ensuring safe answers despite malicious intent from the user.
+* [Topical Rail](../examples/_deprecated/topical_rail/README.md): Making the bot stick to a specific topic of conversation.
+* [Moderation Rail](../examples/_deprecated/moderation_rail/README.md): Moderating a bot's response.
+* [Fact Checking and Hallucination Rail](../examples/_deprecated/grounding_rail/README.md): Ensuring factual answers.
+* [Secure Execution Rail](../examples/_deprecated/execution_rails/README.md): Executing a third-party service with LLMs.
+* [Jailbreaking Rail](../examples/_deprecated/jailbreak_check/README.md): Ensuring safe answers despite malicious intent from the user.
 
 > **Note:** These examples are meant to showcase the process of building rails, not as out-of-the-box safety features. Customization and strengthening of the rails is highly recommended.
 

docs/architecture/README.md

@@ -170,7 +170,7 @@ The stream of events processed by the guardrails runtime (a simplified view with
 
 As shown in the examples [here](../README.md#examples), the event-driven design allows us to hook into the process and add additional guardrails.
 
-For example, in the [grounding rail](../../examples/grounding_rail/README.md) example, we can add an additional fact-checking guardrail (through the `check_facts` action) after a question about the report.
+For example, in the [grounding rail](../../examples/bots/info/README.md) example, we can add an additional fact-checking guardrail (through the `check_facts` action) after a question about the report.
 
 ```colang
 define flow answer report question

docs/user_guide/advanced/red-teaming.md

@@ -12,7 +12,7 @@ To run a red teaming process, there are three steps involved:
 2. Create a set of challenges (`challenges.json`) and add them to the `config` folder.
 3. Start the server `nemoguardrails server` and use the Chat UI to interact with various configurations.
 
-For a server configuration template with two guardrails configuration and a set of challenges, check out [this example](../../../examples/red-teaming).
+For a server configuration template with two guardrails configuration and a set of challenges, check out [this example](../../../examples/configs/red-teaming).
 
 ## Challenges
 
@@ -54,4 +54,4 @@ The UI enables the user to rate the attack's success (No Success, Some Success,
 
 ## Recording the results
 
-The sample configuration [here](../../../examples/red-teaming) includes an example of how to use a "custom logger" to save the ratings, including the complete history of the conversation, in a CSV file.
+The sample configuration [here](../../../examples/configs/red-teaming) includes an example of how to use a "custom logger" to save the ratings, including the complete history of the conversation, in a CSV file.

docs/user_guide/configuration-guide.md

@@ -122,7 +122,7 @@ You can specify additional parameters when using NeMo LLM models using the `para
 
 The `api_host`, `api_key`, and `organization_id` are fetched automatically from the environment variables `NGC_API_HOST`, `NGC_API_KEY`, and `NGC_ORGANIZATION_ID`, respectively.
 
-For more details, please refer to the NeMo LLM Service documentation and check out the [NeMo LLM example configuration](../../examples/llm/nemollm).
+For more details, please refer to the NeMo LLM Service documentation and check out the [NeMo LLM example configuration](../../examples/configs/llm/nemollm).
 
 #### Custom LLM Models
 

docs/user_guide/integration-with-langchain.md

@@ -42,7 +42,7 @@ Existing LangChain chains can be used as actions when defining guardrails in Col
       bot remove last message
       bot $updated_msg
   ```
-Checkout [this sample script](../../examples/demo_chain_as_action.py) for the complete example.
+Checkout [this sample script](../../examples/scripts/demo_chain_as_action.py) for the complete example.
 
 ## Add guardrails to existing chains
 
@@ -73,7 +73,7 @@ define flow
   bot $answer
 ```
 
-Checkout [this sample script](../../examples/demo_chain_with_guardrails.py) for a complete example.
+Checkout [this sample script](../../examples/scripts/demo_chain_with_guardrails.py) for a complete example.
 
 ## Use chains in your custom actions
 

docs/user_guide/python-api.md

@@ -138,7 +138,7 @@ You can register a Langchain chain as an action using the [LLMRails.register_act
 app.register_action(some_chain, name="some_chain")
 ```
 
-When a chain is invoked as an action, the parameters of the action correspond to the input keys of the chain. For the return value, if the output of the chain has a single key, the value will be returned. If the chain has multiple output keys, the dictionary of output keys and their values is returned. See [demo_chain_with_guardrails](../../examples/demo_chain_with_guardrails.py) and [demo_chain_as_action](../../examples/demo_chain_as_action.py) as examples.
+When a chain is invoked as an action, the parameters of the action correspond to the input keys of the chain. For the return value, if the output of the chain has a single key, the value will be returned. If the chain has multiple output keys, the dictionary of output keys and their values is returned. See [demo_chain_with_guardrails](../../examples/scripts/demo_chain_with_guardrails.py) and [demo_chain_as_action](../../examples/scripts/demo_chain_as_action.py) as examples.
 
 ### Custom Actions
 

examples/README.md

@@ -20,26 +20,26 @@ doesn't deviate from a specified topic of conversation. This example covers:
     - Writing basic flows and messages
     - Covers querying and using a Knowledge Base
     - Labels: `Topical`; `good first example`
-    - [Link to example](./topical_rail/README.md)
+    - [Link to example](_deprecated/topical_rail/README.md)
 - **Factual Question Answering:** The example largely focuses on two key aspects - ensuring that
 the bot's response is accurate and mitigates hallucination issues. This example:
     - Covers querying and using a Knowledge Base
     - Ensures answers are factual
     - Reduces hallucination risks
     - Labels: `Topical`
-    - [Link to example](./grounding_rail/README.md)
+    - [Link to example](bots/info/README.md)
 - **Moderating Bots:** Moderation is a complex, multi-pronged approach task. In
 this example, we cover:
     - Ensuring Ethical Responses
     - Blocking restricted words
     - "2 Strikes" ~ shutting down a conversation with a hostile user.
     - Labels: `Safety`; `Security`;
-    - [Link to example](./moderation_rail/README.md)
+    - [Link to example](_deprecated/moderation_rail/README.md)
 - **Detect Jailbreaking Attempts:** Malicious actors will attempt to overwrite a
 bot's safety features. This example:
     - Adds jailbreak check on user's input
     - Labels: `Security`
-    - [Link to example](./jailbreak_check/README.md)
+    - [Link to example](_deprecated/jailbreak_check/README.md)
 - **Safe Execution:** LLMs are versatile but some problems are better solved by
 using pre-existing solutions. For instance, if Wolfram|Alpha is great at
 solving a mathematical problem, it is better to use it to solve mathematical
@@ -49,4 +49,4 @@ This example:
     - Walks through some security guideline
     - Showcases execution for a third-party service
     - Labels: `Security`
-    - [Link to example](./execution_rails/README.md)
+    - [Link to example](_deprecated/execution_rails/README.md)

examples/custom_prompt_context/README.md renamed to examples/_deprecated/custom_prompt_context/README.md

@@ -4,7 +4,7 @@ This example shows how you can include custom data in your custom prompt.
 
 A common use case is to let the LLM know the current date and time so that it can respond appropriately to user queries involving a time component, e.g., "What day is today?". To achieve this, you need to:
 
-1. Register an additional [prompt context variable](../../docs/user_guide/advanced/prompt-customization.md#prompt-variables) (i.e., an additional variable that will be available when the prompts are rendered). This can be achieved using the `register_prompt_context(name, value_or_fn)`.
+1. Register an additional [prompt context variable](../../../docs/user_guide/advanced/prompt-customization.md#prompt-variables) (i.e., an additional variable that will be available when the prompts are rendered). This can be achieved using the `register_prompt_context(name, value_or_fn)`.
 
 2. Change the desired prompts to include the new variable(s).
 

examples/custom_prompt_context/config.co renamed to examples/_deprecated/custom_prompt_context/config.co

@@ -168,12 +168,12 @@ Below is a small snippet of the conversation we can provide the bot:
 ### Managing conversation
 
 For building this bot, developers need to work with `messages`, `flows`, and
-`actions`. We write `messages` & `flows` in their [canonical forms](../../docs/getting_started/hello-world.md#hello-world-example). Let's
+`actions`. We write `messages` & `flows` in their [canonical forms](../../../docs/getting_started/hello-world.md#hello-world-example). Let's
 examine them individually.
 
 **Note:** Think of messages as generic intents and flows as pseudo-code
 for the flow of the conversation. For a more formal explanation, refer to this
-[document](../../docs/architecture/README.md#the-guardrails-process).
+[document](../../../docs/architecture/README.md#the-guardrails-process).
 
 #### User and Bot Messages
 
@@ -205,7 +205,7 @@ for the application. For situations where a flow or a message isn't defines,
 the underlying large language model comes up with the next step for the bot or with
 an appropriate canonical form. It may or may not leverage the existing rails
 to do so, but the mechanism of flows and messages ensures that the LLM can come
-up with appropriate responses. Refer to the [colang runtime description guide](../../docs/architecture/README.md#canonical-user-messages) for more information on the same.
+up with appropriate responses. Refer to the [colang runtime description guide](../../../docs/architecture/README.md#canonical-user-messages) for more information on the same.
 
 #### Using Flows
 
@@ -234,7 +234,7 @@ define flow
   execute wolfram alpha request
   bot respond to math question
 ```
-The Wolfram|Alpha action comes with the library ([found here](../../nemoguardrails/actions/math.py)). Let's take a look at the key elements in the implementation of the action.
+The Wolfram|Alpha action comes with the library ([found here](../../../nemoguardrails/actions/math.py)). Let's take a look at the key elements in the implementation of the action.
 ```
 async def wolfram_alpha_request(
     query: Optional[str] = None, context: Optional[dict] = None
@@ -334,7 +334,7 @@ new_message = rails.generate(messages=[{
 }])
 print(new_message)
 ```
-Refer to [Python API Documentation](../../docs/user_guide/interface-guide.md#python-api) for more information.
+Refer to [Python API Documentation](../../../docs/user_guide/interface-guide.md#python-api) for more information.
 ### UI
 Guardrails allows users to interact with the server with a stock UI. To launch the
 server and access the UI to interact with this example, the following steps are
@@ -345,17 +345,17 @@ the UI
 * Click "New Chat" on the top left corner of the screen and then proceed to
 pick `execution_rail` from the drop-down menu.
 
-Refer to [Guardrails Server Documentation](../../docs/user_guide/interface-guide.md#guardrails-server) for more information.
+Refer to [Guardrails Server Documentation](../../../docs/user_guide/interface-guide.md#guardrails-server) for more information.
 ### Command Line Chat
 
 To chat with the bot with a command line interface simply use the following
 command while you are in this folder.
 ```
 nemoguardrails chat --config=sample_rails
 ```
-Refer to [Guardrails CLI Documentation](../../docs/user_guide/interface-guide.md#guardrails-cli) for more information.
+Refer to [Guardrails CLI Documentation](../../../docs/user_guide/interface-guide.md#guardrails-cli) for more information.
 
 Wondering what to talk to your bot about?
 * Write your own action!
 * Try connecting your existing Machine Learning Pipeline via an action!
-* [Explore more examples](../README.md#examples) to help steer your bot!
+* [Explore more examples](../../README.md#examples) to help steer your bot!

examples/custom_prompt_context/config.py renamed to examples/_deprecated/custom_prompt_context/config.py

@@ -130,7 +130,7 @@ That certainly sounds reasonable, but there's a problem! If you look over the re
 
 ## Fact Checking Rail
 
-The fact checking rail enables you to check the validity of the bot response based on the knowledge base. It takes as inputs the bot response and the relevant chunk from the knowledge base, and makes a call to the LLM asking if the response is true based on the retrieved chunk. The actual format of the LLM call can be seen in [`actions/fact_checking.py`](../../nemoguardrails/library/factchecking/actions.py).
+The fact checking rail enables you to check the validity of the bot response based on the knowledge base. It takes as inputs the bot response and the relevant chunk from the knowledge base, and makes a call to the LLM asking if the response is true based on the retrieved chunk. The actual format of the LLM call can be seen in [`actions/fact_checking.py`](../../../nemoguardrails/library/factchecking/actions.py).
 
 Let's modify our flow from before to add the fact checking rail. Now, when the bot provides its answer, we'll execute the `check_facts` action, and store the response in the `accurate` variable. If the fact checking action deems the response to be false, we'll remove that message from the response and let the user know that the bot doesn't know the answer.
 
@@ -201,11 +201,11 @@ Everything up until `Finished chain.` is part of the bot's "internal reasoning",
 
 ## Hallucination Rail
 
-While the fact checking action works well when we have a relevant knowledge base to check against, we'd also like to guard against hallucination when we don't have a pre-configured knowledge base. For this use case, we can use the [`check_hallucination`](../../nemoguardrails/library/hallucination/actions.py) action.
+While the fact checking action works well when we have a relevant knowledge base to check against, we'd also like to guard against hallucination when we don't have a pre-configured knowledge base. For this use case, we can use the [`check_hallucination`](../../../nemoguardrails/library/hallucination/actions.py) action.
 
 The hallucination rail uses a self-checking mechanism inspired by the [SelfCheckGPT](https://arxiv.org/abs/2303.08896) technique. Similar to the fact-checking rail, we ask the LLM itself to determine whether the most recent output is consistent with a piece of context. However, since we don't have a knowledge base to pull the context from, we use the LLM to generate multiple additional completions to serve as the context. The assumption is that if the LLM produces multiple completions that are inconsistent with each other, the original completion is likely to be a hallucination.
 
-You can view [`actions/hallucination/hallucination.py`](../../nemoguardrails/library/hallucination/actions.py) to see the format of the the extra generations and the hallucination check call.
+You can view [`actions/hallucination/hallucination.py`](../../../nemoguardrails/library/hallucination/actions.py) to see the format of the the extra generations and the hallucination check call.
 
 The current implementation only supports OpenAI LLM Engines.
 
@@ -272,7 +272,7 @@ Again taking a look at the detailed output log, we see that the LLM produced sev
 
 With a basic understanding of building the rails, the next step is to try out the bot and customize it! You can continue to interact with the bot via the API, or use the `nemoguardrails` CLI to launch an interactive command line or web chat interface. Customize your bot by adding in new flows or documents to the knowledge base, and test out the effects of adding and removing the rails explored in this notebook and others.
 
-Refer [Python API Documentation](../../docs/user_guide/interface-guide.md#python-api) for more information.
+Refer [Python API Documentation](../../../docs/user_guide/interface-guide.md#python-api) for more information.
 
 ### UI
 
@@ -286,7 +286,7 @@ the UI
 * Click "New Chat" on the top left corner of the screen and then proceed to
 pick `grounding_rail` from the drop-down menu.
 
-Refer [Guardrails Server Documentation](../../docs/user_guide/interface-guide.md#guardrails-server) for more information.
+Refer [Guardrails Server Documentation](../../../docs/user_guide/interface-guide.md#guardrails-server) for more information.
 
 ### Command Line Chat
 
@@ -296,6 +296,6 @@ command while you are in this folder:
 ```bash
 nemoguardrails chat --config=.
 ```
-Refer [Guardrails CLI Documentation](../../docs/user_guide/interface-guide.md#guardrails-cli) for more information.
+Refer [Guardrails CLI Documentation](../../../docs/user_guide/interface-guide.md#guardrails-cli) for more information.
 
-* [Explore more examples](../README.md#examples) to help steer your bot!
+* [Explore more examples](../../README.md#examples) to help steer your bot!