md/02.Application/06.MoE/Phi3/azure-ai-search-nvidia-rag.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Azure AI Search with NVIDIA NIM and LlamaIndex Integration\n",
    "\n",
    "In this notebook, we'll demonstrate how to leverage NVIDIA's AI models and LlamaIndex to create a powerful Retrieval-Augmented Generation (RAG) pipeline. We'll use NVIDIA's LLMs and embeddings, integrate them with Azure AI Search as the vector store, and perform RAG to enhance search quality and efficiency.\n",
    "\n",
    "## Benefits\n",
    "- **Scalability**: Leverage NVIDIA's large language models and Azure AI Search for scalable and efficient retrieval.\n",
    "- **Cost Efficiency**: Optimize search and retrieval with efficient vector storage and hybrid search techniques.\n",
    "- **High Performance**: Combine powerful LLMs with vectorized search for faster and more accurate responses.\n",
    "- **Quality**: Maintain high search quality by grounding LLM responses with relevant retrieved documents.\n",
    "\n",
    "## Prerequisites\n",
    "- 🐍 Python 3.9 or higher\n",
    "- 🔗 [Azure AI Search Service](https://learn.microsoft.com/azure/search/)\n",
    "- 🔗 NVIDIA API Key for access to NVIDIA's LLMs and Embeddings via the NVIDIA NIM microservices\n",
    "\n",
    "## Features Covered\n",
    "- ✅ NVIDIA LLM Integration (we'll use [Phi-3.5-MOE](https://build.nvidia.com/microsoft/phi-3_5-moe))\n",
    "- ✅ NVIDIA Embeddings (we'll use [nv-embedqa-e5-v5](https://build.nvidia.com/nvidia/nv-embedqa-e5-v5))\n",
    "- ✅ Azure AI Search Advanced Retreival Modes\n",
    "- ✅ Document Indexing with LlamaIndex\n",
    "- ✅ RAG using Azure AI Search and LlamaIndex with NVIDIA LLMs\n",
    "\n",
    "Let's get started!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install azure-search-documents==11.5.1\n",
    "!pip install --upgrade llama-index\n",
    "!pip install --upgrade llama-index-core\n",
    "!pip install --upgrade llama-index-readers-file\n",
    "!pip install --upgrade llama-index-llms-nvidia\n",
    "!pip install --upgrade llama-index-embeddings-nvidia\n",
    "!pip install --upgrade llama-index-postprocessor-nvidia-rerank\n",
    "!pip install --upgrade llama-index-vector-stores-azureaisearch\n",
    "!pip install python-dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation and Requirements\n",
    "Create a Python environment using Python version >3.10.\n",
    "\n",
    "## Getting Started!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get started, you need a `NVIDIA_API_KEY` to use NVIDIA AI Foundation models:\n",
    "1) Create a free account with [NVIDIA](https://build.nvidia.com/explore/discover).\n",
    "2) Click on your model of choice.\n",
    "3) Under Input, select the Python tab, and click **Get API Key** and then click **Generate Key**.\n",
    "4) Copy and save the generated key as NVIDIA_API_KEY. From there, you should have access to the endpoints.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "# Load environment variables\n",
    "load_dotenv()\n",
    "\n",
    "if not os.environ.get(\"NVIDIA_API_KEY\", \"\").startswith(\"nvapi-\"):\n",
    "    nvidia_api_key = getpass.getpass(\"Enter your NVIDIA API key: \")\n",
    "    assert nvidia_api_key.startswith(\"nvapi-\"), f\"{nvidia_api_key[:5]}... is not a valid key\"\n",
    "    os.environ[\"NVIDIA_API_KEY\"] = nvidia_api_key\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RAG Example using LLM and Embedding\n",
    "### 1) Initialize the LLM\n",
    "`llama-index-llms-nvidia`, also known as NVIDIA's LLM connector, allows you to connect to and generate from compatible models available on the NVIDIA API catalog. See here for a list of chat completion models: https://build.nvidia.com/search?term=Text-to-Text\n",
    "\n",
    "Here we will use **mixtral-8x7b-instruct-v0.1**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core import Settings\n",
    "from llama_index.llms.nvidia import NVIDIA\n",
    "\n",
    "# Here we are using mixtral-8x7b-instruct-v0.1 model from API Catalog\n",
    "Settings.llm = NVIDIA(model=\"microsoft/phi-3.5-moe-instruct\", api_key=os.getenv(\"NVIDIA_API_KEY\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2) Initialize the Embedding\n",
    "`llama-index-embeddings-nvidia`, also known as NVIDIA's Embeddings connector, allows you to connect to and generate from compatible models available on the NVIDIA API catalog. We selected `nvidia/nv-embedqa-e5-v5` as the embedding model. See here for a list of text embedding models: https://build.nvidia.com/nim?filters=usecase%3Ausecase_text_to_embedding%2Cusecase%3Ausecase_image_to_embedding\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.embeddings.nvidia import NVIDIAEmbedding\n",
    "\n",
    "Settings.embed_model = NVIDIAEmbedding(model=\"nvidia/nv-embedqa-e5-v5\", api_key=os.getenv(\"NVIDIA_API_KEY\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3) Create an Azure AI Search Vector Store"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import sys\n",
    "import os\n",
    "import getpass\n",
    "from azure.core.credentials import AzureKeyCredential\n",
    "from azure.search.documents import SearchClient\n",
    "from azure.search.documents.indexes import SearchIndexClient\n",
    "from IPython.display import Markdown, display\n",
    "from llama_index.vector_stores.azureaisearch import AzureAISearchVectorStore, IndexManagement\n",
    "\n",
    "\n",
    "search_service_api_key = os.getenv('AZURE_SEARCH_ADMIN_KEY') or getpass.getpass('Enter your Azure Search API key: ')\n",
    "search_service_endpoint = os.getenv('AZURE_SEARCH_SERVICE_ENDPOINT') or getpass.getpass('Enter your Azure Search service endpoint: ')\n",
    "search_service_api_version = \"2024-07-01\"\n",
    "credential = AzureKeyCredential(search_service_api_key)\n",
    "\n",
    "# Index name to use\n",
    "index_name = \"llamaindex-nvidia-azureaisearch-demo\"\n",
    "\n",
    "# Use index client to demonstrate creating an index\n",
    "index_client = SearchIndexClient(\n",
    "    endpoint=search_service_endpoint,\n",
    "    credential=credential,\n",
    ")\n",
    "\n",
    "# Use search client to demonstrate using existing index\n",
    "search_client = SearchClient(\n",
    "    endpoint=search_service_endpoint,\n",
    "    index_name=index_name,\n",
    "    credential=credential,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vector_store = AzureAISearchVectorStore(\n",
    "    search_or_index_client=index_client,\n",
    "    index_name=index_name,\n",
    "    index_management=IndexManagement.CREATE_IF_NOT_EXISTS,\n",
    "    id_field_key=\"id\",\n",
    "    chunk_field_key=\"chunk\",\n",
    "    embedding_field_key=\"embedding\",\n",
    "    embedding_dimensionality=1024, # dimensionality for nv-embedqa-e5-v5 model\n",
    "    metadata_string_field_key=\"metadata\",\n",
    "    doc_id_field_key=\"doc_id\",\n",
    "    language_analyzer=\"en.lucene\",\n",
    "    vector_algorithm_type=\"exhaustiveKnn\",\n",
    "    # compression_type=\"binary\" # Option to use \"scalar\" or \"binary\". NOTE: compression is only supported for HNSW\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4) Load, Chunk, and Upload Documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.core import SimpleDirectoryReader, StorageContext, VectorStoreIndex\n",
    "from llama_index.core.text_splitter import TokenTextSplitter\n",
    "\n",
    "# Configure text splitter (nv-embedqa-e5-v5 model has a limit of 512 tokens per input size)\n",
    "text_splitter = TokenTextSplitter(separator=\" \", chunk_size=500, chunk_overlap=10)\n",
    "\n",
    "# Load documents\n",
    "documents = SimpleDirectoryReader(\n",
    "    input_files=[\"data/txt/state_of_the_union.txt\"]\n",
    ").load_data()\n",
    "storage_context = StorageContext.from_defaults(vector_store=vector_store)\n",
    "\n",
    "# Create index with text splitter\n",
    "index = VectorStoreIndex.from_documents(\n",
    "    documents,\n",
    "    transformations=[text_splitter],\n",
    "    storage_context=storage_context,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5) Create a Query Engine to ask questions over your data\n",
    "\n",
    "Here is a query using pure vector search in Azure AI Search and grounding the response to our LLM (Phi-3.5-MOE)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       " The speaker mentioned the Ukrainian Ambassador to the United States, along with other members of Congress, the Cabinet, and various officials such as the Vice President, the First Lady, and the Second Gentleman, as being present in the chamber."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "query_engine = index.as_query_engine()\n",
    "response = query_engine.query(\"Who did the speaker mention as being present in the chamber?\")\n",
    "display(Markdown(f\"{response}\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a query using hybrid search in Azure AI Search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       " The Russian stock market experienced a significant drop, losing 40% of its value. Additionally, trading had to be suspended due to the ongoing situation."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Source Nodes:\n",
      "file_path: data\\txt\\state_of_the_union.txt\n",
      "\n",
      "building a coalition of other freedom-loving nations from Europe and the Americas to Asia and Africa to confront Putin. \n",
      "\n",
      "I spent countless hours unifying our European allies. We shared with the world in advance what we knew Putin was planning and precisely how he would try to falsely justify his aggression.  \n",
      "\n",
      "We countered Russia’s lies with truth.   \n",
      "\n",
      "And now that he has acted the free world is holding him accountable. \n",
      "\n",
      "Along with twenty-seven members of the European Union including France, Germany, Italy, as well as countries like the United Kingdom, Canada, Japan, Korea, Australia, New Zealand, and many others, even Switzerland. \n",
      "\n",
      "We are inflicting pain on Russia and supporting the people of Ukraine. Putin is now isolated from the world more than ever. \n",
      "\n",
      "Together with our allies –we are right now enforcing powerful economic sanctions. \n",
      "\n",
      "We are cutting off Russia’s largest banks from the international financial system.  \n",
      "\n",
      "Preventing Russia’s central bank from defending the Russian Ruble making Putin’s $630 Billion “war fund” worthless.   \n",
      "\n",
      "We are choking off Russia’s access to technology that will sap its economic strength and weaken its military for years to come.  \n",
      "\n",
      "Tonight I say to the Russian oligarchs and corrupt leaders who have bilked billions of dollars off this violent regime no more. \n",
      "\n",
      "The U.S. Department of Justice is assembling a dedicated task force to go after the crimes of Russian oligarchs.  \n",
      "\n",
      "We are joining with our European allies to find and seize your yachts your luxury apartments your private jets. We are coming for your ill-begotten gains. \n",
      "\n",
      "And tonight I am announcing that we will join our allies in closing off American air space to all Russian flights – further isolating Russia – and adding an additional squeeze –on their economy. The Ruble has lost 30% of its value. \n",
      "\n",
      "The Russian stock market has lost 40% of its value and trading remains suspended. Russia’s economy is reeling and Putin alone is to blame. \n",
      "\n",
      "Together with our allies we are providing support to the Ukrainians in their fight for freedom. Military assistance. Economic assistance. Humanitarian assistance. \n",
      "\n",
      "We are giving more than $1 Billion in direct assistance to Ukraine. \n",
      "\n",
      "And we will continue to aid the Ukrainian people as they defend their country and to help ease their suffering.  \n",
      "\n",
      "Let me be clear, our forces are not engaged and will not engage in\n",
      "file_path: data\\txt\\state_of_the_union.txt\n",
      "\n",
      "our forces are not engaged and will not engage in conflict with Russian forces in Ukraine.  \n",
      "\n",
      "Our forces are not going to Europe to fight in Ukraine, but to defend our NATO Allies – in the event that Putin decides to keep moving west.  \n",
      "\n",
      "For that purpose we’ve mobilized American ground forces, air squadrons, and ship deployments to protect NATO countries including Poland, Romania, Latvia, Lithuania, and Estonia. \n",
      "\n",
      "As I have made crystal clear the United States and our Allies will defend every inch of territory of NATO countries with the full force of our collective power.  \n",
      "\n",
      "And we remain clear-eyed. The Ukrainians are fighting back with pure courage. But the next few days weeks, months, will be hard on them.  \n",
      "\n",
      "Putin has unleashed violence and chaos.  But while he may make gains on the battlefield – he will pay a continuing high price over the long run. \n",
      "\n",
      "And a proud Ukrainian people, who have known 30 years  of independence, have repeatedly shown that they will not tolerate anyone who tries to take their country backwards.  \n",
      "\n",
      "To all Americans, I will be honest with you, as I’ve always promised. A Russian dictator, invading a foreign country, has costs around the world. \n",
      "\n",
      "And I’m taking robust action to make sure the pain of our sanctions  is targeted at Russia’s economy. And I will use every tool at our disposal to protect American businesses and consumers. \n",
      "\n",
      "Tonight, I can announce that the United States has worked with 30 other countries to release 60 Million barrels of oil from reserves around the world.  \n",
      "\n",
      "America will lead that effort, releasing 30 Million barrels from our own Strategic Petroleum Reserve. And we stand ready to do more if necessary, unified with our allies.  \n",
      "\n",
      "These steps will help blunt gas prices here at home. And I know the news about what’s happening can seem alarming. \n",
      "\n",
      "But I want you to know that we are going to be okay. \n",
      "\n",
      "When the history of this era is written Putin’s war on Ukraine will have left Russia weaker and the rest of the world stronger. \n",
      "\n",
      "While it shouldn’t have taken something so terrible for people around the world to see what’s at stake now everyone sees it clearly. \n",
      "\n",
      "We see the unity among leaders of nations and a more unified Europe a more unified West. And we see unity among the people who are gathering in cities in large crowds around the world\n"
     ]
    }
   ],
   "source": [
    "from llama_index.core.query_engine import RetrieverQueryEngine\n",
    "from llama_index.core.vector_stores.types import VectorStoreQueryMode\n",
    "from IPython.display import Markdown, display\n",
    "from llama_index.core.schema import MetadataMode\n",
    "\n",
    "# Initialize hybrid retriever and query engine\n",
    "hybrid_retriever = index.as_retriever(vector_store_query_mode=VectorStoreQueryMode.HYBRID)\n",
    "hybrid_query_engine = RetrieverQueryEngine(retriever=hybrid_retriever)\n",
    "\n",
    "# Query execution\n",
    "query = \"What were the exact economic consequences mentioned in relation to Russia's stock market?\"\n",
    "response = hybrid_query_engine.query(query)\n",
    "\n",
    "# Display the response\n",
    "display(Markdown(f\"{response}\"))\n",
    "print(\"\\n\")\n",
    "\n",
    "# Print the source nodes\n",
    "print(\"Source Nodes:\")\n",
    "for node in response.source_nodes:\n",
    "    print(node.get_content(metadata_mode=MetadataMode.LLM))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Vector Search Analysis\n",
    "The LLM response accurately captures the key economic consequences mentioned in the source text regarding Russia's stock market. Specifically, it states that the Russian stock market experienced a significant drop, losing 40% of its value, and that trading was suspended due to the ongoing situation. This response aligns well with the information provided in the source, indicating that the LLM correctly identified and summarized the relevant details regarding the stock market impact as a result of Russia's actions and the sanctions imposed.\n",
    "\n",
    "#### Source Nodes Commentary\n",
    "The source nodes provide a detailed account of the economic consequences that Russia faced due to international sanctions. The text highlights that the Russian stock market lost 40% of its value, and trading was suspended. Additionally, it mentions other economic repercussions, such as the devaluation of the Ruble and the broader isolation of Russia's economy. The LLM response effectively distilled the critical points from these nodes, focusing on the stock market impact as requested by the query.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's take a look at a query where Hybrid Search doesn't give a well-grounded answer:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       " The provided context does not specify the exact date of Russia's invasion of Ukraine. However, it does mention that the events discussed are happening in the current era and that the actions taken are in response to Putin's aggression. For the precise date, one would need to refer to external sources or historical records."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Source Nodes:\n",
      "file_path: data\\txt\\state_of_the_union.txt\n",
      "\n",
      "our forces are not engaged and will not engage in conflict with Russian forces in Ukraine.  \n",
      "\n",
      "Our forces are not going to Europe to fight in Ukraine, but to defend our NATO Allies – in the event that Putin decides to keep moving west.  \n",
      "\n",
      "For that purpose we’ve mobilized American ground forces, air squadrons, and ship deployments to protect NATO countries including Poland, Romania, Latvia, Lithuania, and Estonia. \n",
      "\n",
      "As I have made crystal clear the United States and our Allies will defend every inch of territory of NATO countries with the full force of our collective power.  \n",
      "\n",
      "And we remain clear-eyed. The Ukrainians are fighting back with pure courage. But the next few days weeks, months, will be hard on them.  \n",
      "\n",
      "Putin has unleashed violence and chaos.  But while he may make gains on the battlefield – he will pay a continuing high price over the long run. \n",
      "\n",
      "And a proud Ukrainian people, who have known 30 years  of independence, have repeatedly shown that they will not tolerate anyone who tries to take their country backwards.  \n",
      "\n",
      "To all Americans, I will be honest with you, as I’ve always promised. A Russian dictator, invading a foreign country, has costs around the world. \n",
      "\n",
      "And I’m taking robust action to make sure the pain of our sanctions  is targeted at Russia’s economy. And I will use every tool at our disposal to protect American businesses and consumers. \n",
      "\n",
      "Tonight, I can announce that the United States has worked with 30 other countries to release 60 Million barrels of oil from reserves around the world.  \n",
      "\n",
      "America will lead that effort, releasing 30 Million barrels from our own Strategic Petroleum Reserve. And we stand ready to do more if necessary, unified with our allies.  \n",
      "\n",
      "These steps will help blunt gas prices here at home. And I know the news about what’s happening can seem alarming. \n",
      "\n",
      "But I want you to know that we are going to be okay. \n",
      "\n",
      "When the history of this era is written Putin’s war on Ukraine will have left Russia weaker and the rest of the world stronger. \n",
      "\n",
      "While it shouldn’t have taken something so terrible for people around the world to see what’s at stake now everyone sees it clearly. \n",
      "\n",
      "We see the unity among leaders of nations and a more unified Europe a more unified West. And we see unity among the people who are gathering in cities in large crowds around the world\n",
      "file_path: data\\txt\\state_of_the_union.txt\n",
      "\n",
      "building a coalition of other freedom-loving nations from Europe and the Americas to Asia and Africa to confront Putin. \n",
      "\n",
      "I spent countless hours unifying our European allies. We shared with the world in advance what we knew Putin was planning and precisely how he would try to falsely justify his aggression.  \n",
      "\n",
      "We countered Russia’s lies with truth.   \n",
      "\n",
      "And now that he has acted the free world is holding him accountable. \n",
      "\n",
      "Along with twenty-seven members of the European Union including France, Germany, Italy, as well as countries like the United Kingdom, Canada, Japan, Korea, Australia, New Zealand, and many others, even Switzerland. \n",
      "\n",
      "We are inflicting pain on Russia and supporting the people of Ukraine. Putin is now isolated from the world more than ever. \n",
      "\n",
      "Together with our allies –we are right now enforcing powerful economic sanctions. \n",
      "\n",
      "We are cutting off Russia’s largest banks from the international financial system.  \n",
      "\n",
      "Preventing Russia’s central bank from defending the Russian Ruble making Putin’s $630 Billion “war fund” worthless.   \n",
      "\n",
      "We are choking off Russia’s access to technology that will sap its economic strength and weaken its military for years to come.  \n",
      "\n",
      "Tonight I say to the Russian oligarchs and corrupt leaders who have bilked billions of dollars off this violent regime no more. \n",
      "\n",
      "The U.S. Department of Justice is assembling a dedicated task force to go after the crimes of Russian oligarchs.  \n",
      "\n",
      "We are joining with our European allies to find and seize your yachts your luxury apartments your private jets. We are coming for your ill-begotten gains. \n",
      "\n",
      "And tonight I am announcing that we will join our allies in closing off American air space to all Russian flights – further isolating Russia – and adding an additional squeeze –on their economy. The Ruble has lost 30% of its value. \n",
      "\n",
      "The Russian stock market has lost 40% of its value and trading remains suspended. Russia’s economy is reeling and Putin alone is to blame. \n",
      "\n",
      "Together with our allies we are providing support to the Ukrainians in their fight for freedom. Military assistance. Economic assistance. Humanitarian assistance. \n",
      "\n",
      "We are giving more than $1 Billion in direct assistance to Ukraine. \n",
      "\n",
      "And we will continue to aid the Ukrainian people as they defend their country and to help ease their suffering.  \n",
      "\n",
      "Let me be clear, our forces are not engaged and will not engage in\n"
     ]
    }
   ],
   "source": [
    "# Query execution\n",
    "query = \"What was the precise date when Russia invaded Ukraine?\"\n",
    "response = hybrid_query_engine.query(query)\n",
    "\n",
    "# Display the response\n",
    "display(Markdown(f\"{response}\"))\n",
    "print(\"\\n\")\n",
    "\n",
    "# Print the source nodes\n",
    "print(\"Source Nodes:\")\n",
    "for node in response.source_nodes:\n",
    "    print(node.get_content(metadata_mode=MetadataMode.LLM))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hybrid Search: LLM Response Analysis\n",
    "The LLM response in the Hybrid Search example indicates that the provided context does not specify the exact date of Russia's invasion of Ukraine. This response suggests that the LLM is leveraging the information available in the source documents but acknowledges the absence of precise details in the text.\n",
    "\n",
    "The response is accurate in identifying that the context mentions events related to Russia's aggression but does not pinpoint the specific invasion date. This showcases the LLM's ability to comprehend the information provided while recognizing gaps in the content. The LLM effectively prompts the user to seek external sources or historical records for the exact date, displaying a level of cautiousness when information is incomplete.\n",
    "\n",
    "### Source Nodes Analysis\n",
    "The source nodes in the Hybrid Search example contain excerpts from a speech discussing the U.S. response to Russia's actions in Ukraine. These nodes emphasize the broader geopolitical impact and the steps taken by the U.S. and its allies in response to the invasion, but they do not mention the specific invasion date. This aligns with the LLM response, which correctly identifies that the context lacks the precise date information.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       " The provided context does not specify the exact date of Russia's invasion of Ukraine. However, it mentions that the event occurred six days before the speech was given. To determine the precise date, one would need to know the date of the speech."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Source Nodes:\n",
      "file_path: data\\txt\\state_of_the_union.txt\n",
      "\n",
      "Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.  \n",
      "\n",
      "Last year COVID-19 kept us apart. This year we are finally together again. \n",
      "\n",
      "Tonight, we meet as Democrats Republicans and Independents. But most importantly as Americans. \n",
      "\n",
      "With a duty to one another to the American people to the Constitution. \n",
      "\n",
      "And with an unwavering resolve that freedom will always triumph over tyranny. \n",
      "\n",
      "Six days ago, Russia’s Vladimir Putin sought to shake the foundations of the free world thinking he could make it bend to his menacing ways. But he badly miscalculated. \n",
      "\n",
      "He thought he could roll into Ukraine and the world would roll over. Instead he met a wall of strength he never imagined. \n",
      "\n",
      "He met the Ukrainian people. \n",
      "\n",
      "From President Zelenskyy to every Ukrainian, their fearlessness, their courage, their determination, inspires the world. \n",
      "\n",
      "Groups of citizens blocking tanks with their bodies. Everyone from students to retirees teachers turned soldiers defending their homeland. \n",
      "\n",
      "In this struggle as President Zelenskyy said in his speech to the European Parliament “Light will win over darkness.” The Ukrainian Ambassador to the United States is here tonight. \n",
      "\n",
      "Let each of us here tonight in this Chamber send an unmistakable signal to Ukraine and to the world. \n",
      "\n",
      "Please rise if you are able and show that, Yes, we the United States of America stand with the Ukrainian people. \n",
      "\n",
      "Throughout our history we’ve learned this lesson when dictators do not pay a price for their aggression they cause more chaos.   \n",
      "\n",
      "They keep moving.   \n",
      "\n",
      "And the costs and the threats to America and the world keep rising.   \n",
      "\n",
      "That’s why the NATO Alliance was created to secure peace and stability in Europe after World War 2. \n",
      "\n",
      "The United States is a member along with 29 other nations. \n",
      "\n",
      "It matters. American diplomacy matters. American resolve matters. \n",
      "\n",
      "Putin’s latest attack on Ukraine was premeditated and unprovoked. \n",
      "\n",
      "He rejected repeated efforts at diplomacy. \n",
      "\n",
      "He thought the West and NATO wouldn’t respond. And he thought he could divide us at home. Putin was wrong. We were ready.  Here is what we did.   \n",
      "\n",
      "We prepared extensively and carefully. \n",
      "\n",
      "We spent months building a coalition of other freedom-loving nations from Europe\n",
      "file_path: data\\txt\\state_of_the_union.txt\n",
      "\n",
      "our forces are not engaged and will not engage in conflict with Russian forces in Ukraine.  \n",
      "\n",
      "Our forces are not going to Europe to fight in Ukraine, but to defend our NATO Allies – in the event that Putin decides to keep moving west.  \n",
      "\n",
      "For that purpose we’ve mobilized American ground forces, air squadrons, and ship deployments to protect NATO countries including Poland, Romania, Latvia, Lithuania, and Estonia. \n",
      "\n",
      "As I have made crystal clear the United States and our Allies will defend every inch of territory of NATO countries with the full force of our collective power.  \n",
      "\n",
      "And we remain clear-eyed. The Ukrainians are fighting back with pure courage. But the next few days weeks, months, will be hard on them.  \n",
      "\n",
      "Putin has unleashed violence and chaos.  But while he may make gains on the battlefield – he will pay a continuing high price over the long run. \n",
      "\n",
      "And a proud Ukrainian people, who have known 30 years  of independence, have repeatedly shown that they will not tolerate anyone who tries to take their country backwards.  \n",
      "\n",
      "To all Americans, I will be honest with you, as I’ve always promised. A Russian dictator, invading a foreign country, has costs around the world. \n",
      "\n",
      "And I’m taking robust action to make sure the pain of our sanctions  is targeted at Russia’s economy. And I will use every tool at our disposal to protect American businesses and consumers. \n",
      "\n",
      "Tonight, I can announce that the United States has worked with 30 other countries to release 60 Million barrels of oil from reserves around the world.  \n",
      "\n",
      "America will lead that effort, releasing 30 Million barrels from our own Strategic Petroleum Reserve. And we stand ready to do more if necessary, unified with our allies.  \n",
      "\n",
      "These steps will help blunt gas prices here at home. And I know the news about what’s happening can seem alarming. \n",
      "\n",
      "But I want you to know that we are going to be okay. \n",
      "\n",
      "When the history of this era is written Putin’s war on Ukraine will have left Russia weaker and the rest of the world stronger. \n",
      "\n",
      "While it shouldn’t have taken something so terrible for people around the world to see what’s at stake now everyone sees it clearly. \n",
      "\n",
      "We see the unity among leaders of nations and a more unified Europe a more unified West. And we see unity among the people who are gathering in cities in large crowds around the world\n"
     ]
    }
   ],
   "source": [
    "# Initialize hybrid retriever and query engine\n",
    "semantic_reranker_retriever = index.as_retriever(vector_store_query_mode=VectorStoreQueryMode.SEMANTIC_HYBRID)\n",
    "semantic_reranker_query_engine = RetrieverQueryEngine(retriever=semantic_reranker_retriever)\n",
    "\n",
    "# Query execution\n",
    "query = \"What was the precise date when Russia invaded Ukraine?\"\n",
    "response = semantic_reranker_query_engine.query(query)\n",
    "\n",
    "# Display the response\n",
    "display(Markdown(f\"{response}\"))\n",
    "print(\"\\n\")\n",
    "\n",
    "# Print the source nodes\n",
    "print(\"Source Nodes:\")\n",
    "for node in response.source_nodes:\n",
    "    print(node.get_content(metadata_mode=MetadataMode.LLM))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hybrid w/Reranking: LLM Response Analysis\n",
    "In the Hybrid w/Reranking example, the LLM response provides additional context by noting that the event occurred six days before the speech was given. This indicates that the LLM is able to infer the invasion date based on the timing of the speech, even though it still requires knowing the exact date of the speech for precision.\n",
    "\n",
    "This response demonstrates an improved ability to use context clues to provide a more informative answer. It highlights the advantage of reranking, where the LLM can access and prioritize more relevant information to give a closer approximation of the desired detail (i.e., the invasion date).\n",
    "\n",
    "### Source Nodes Analysis\n",
    "The source nodes in this example include references to the timing of Russia's invasion, specifically mentioning that it occurred six days before the speech. While the exact date is still not explicitly stated, the nodes provide temporal context that allows the LLM to give a more nuanced response. The inclusion of this detail showcases how reranking can improve the LLM's ability to extract and infer information from the provided context, resulting in a more accurate and informative response.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:**\n",
    "In this notebook, we used NVIDIA NIM microservices from the NVIDIA API Catalog.\n",
    "The above APIs, `NVIDIA (llms)`, `NVIDIAEmbedding`, and [Azure AI Search Semantic Hybrid Retrieval (built-in reranking)](https://learn.microsoft.com/azure/search/semantic-search-overview). Note, that the above APIs can also support self-hosted microservices. \n",
    "\n",
    "**Example:**\n",
    "```python\n",
    "NVIDIA(model=\"meta/llama3-8b-instruct\", base_url=\"http://your-nim-host-address:8000/v1\")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}