Rerank.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "QS3U4Ic8nrk2",
        "ewm7DaKoaPHf",
        "tnEpdg6r-Her"
      ],
      "toc_visible": true,
      "authorship_tag": "ABX9TyOf0MYDrKGS4dmOsjLKqWoM",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/hiwei93/rag-practice/blob/main/Rerank.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Rerank\n",
        "\n",
        "Cohere Rerank 实现参考：\n",
        "- [LangChain document: Cohere Reranker](https://python.langchain.com/docs/integrations/retrievers/cohere-reranker)\n",
        "\n",
        "Cross Encoder 实现参考：\n",
        "- [Issue: how to use reranker model with langchain in retrievalQA case?](https://github.com/langchain-ai/langchain/issues/13076#issuecomment-1839814250)\n",
        "- [sentence-transformers example: Retrieve & Re-Rank Demo over Simple Wikipedia](https://github.com/UKPLab/sentence-transformers/blob/master/examples/applications/retrieve_rerank/retrieve_rerank_simple_wikipedia.ipynb)"
      ],
      "metadata": {
        "id": "lchPBVWpo65_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 依赖安装"
      ],
      "metadata": {
        "id": "vsKo7Uc854wr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install langchain cohere chromadb --quiet"
      ],
      "metadata": {
        "id": "6hr9Ua9FscGo"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 加载数据"
      ],
      "metadata": {
        "id": "QS3U4Ic8nrk2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tqdm import tqdm\n",
        "import requests\n",
        "\n",
        "\n",
        "def http_get(url, path) -> None:\n",
        "    \"\"\"\n",
        "    Downloads a URL to a given path on disc\n",
        "    \"\"\"\n",
        "    if os.path.dirname(path) != \"\":\n",
        "        os.makedirs(os.path.dirname(path), exist_ok=True)\n",
        "\n",
        "    req = requests.get(url, stream=True)\n",
        "    if req.status_code != 200:\n",
        "        print(\"Exception when trying to download {}. Response {}\".format(url, req.status_code), file=sys.stderr)\n",
        "        req.raise_for_status()\n",
        "        return\n",
        "\n",
        "    download_filepath = path + \"_part\"\n",
        "    with open(download_filepath, \"wb\") as file_binary:\n",
        "        content_length = req.headers.get(\"Content-Length\")\n",
        "        total = int(content_length) if content_length is not None else None\n",
        "        progress = tqdm(unit=\"B\", total=total, unit_scale=True)\n",
        "        for chunk in req.iter_content(chunk_size=1024):\n",
        "            if chunk:  # filter out keep-alive new chunks\n",
        "                progress.update(len(chunk))\n",
        "                file_binary.write(chunk)\n",
        "\n",
        "    os.rename(download_filepath, path)\n",
        "    progress.close()"
      ],
      "metadata": {
        "id": "vONDcjVns7Zm"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import json\n",
        "import gzip\n",
        "import os\n",
        "\n",
        "wikipedia_filepath = 'simplewiki-2020-11-01.jsonl.gz'\n",
        "\n",
        "if not os.path.exists(wikipedia_filepath):\n",
        "    http_get('http://sbert.net/datasets/simplewiki-2020-11-01.jsonl.gz', wikipedia_filepath)\n",
        "\n",
        "passages = []\n",
        "with gzip.open(wikipedia_filepath, 'rt', encoding='utf8') as fIn:\n",
        "    for line in fIn:\n",
        "        data = json.loads(line.strip())\n",
        "\n",
        "        #Add all paragraphs\n",
        "        #passages.extend(data['paragraphs'])\n",
        "\n",
        "        #Only add the first paragraph\n",
        "        passages.append(data['paragraphs'][0])\n",
        "\n",
        "print(\"Passages:\", len(passages))\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2KVXIbXeoN5j",
        "outputId": "f26e3e9c-6065-4e1c-bf10-653648e21948"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Passages: 169597\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "average = sum((len(p) for p in passages)) / len(passages)\n",
        "max_length = max((len(p) for p in passages))\n",
        "\n",
        "print(average, max_length)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "25FifEP2tadw",
        "outputId": "0cac675c-2850-4b64-90a7-973c43141308"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "235.19272746569808 3644\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "passages = [p for p in passages if len(p) <= 512]"
      ],
      "metadata": {
        "id": "e8eHgle-xdDD"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "len(passages)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PNdljNtMxk1o",
        "outputId": "3dadf52c-0e67-4300-c9fd-fe0912b1f42e"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "160443"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "passages[:100]"
      ],
      "metadata": {
        "id": "PLOaTY_w9EP5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 构建向量查询器"
      ],
      "metadata": {
        "id": "rTEkCmtKmxpw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_community.embeddings import HuggingFaceInferenceAPIEmbeddings\n",
        "from langchain_community.vectorstores import Chroma\n",
        "\n",
        "\n",
        "from google.colab import userdata\n",
        "\n",
        "# 获取 Huggingface token\n",
        "inference_api_key = userdata.get('hf_token')\n",
        "\n",
        "embedding = HuggingFaceInferenceAPIEmbeddings(\n",
        "    api_key=inference_api_key, model_name=\"sentence-transformers/all-MiniLM-L6-v2\"\n",
        ")"
      ],
      "metadata": {
        "id": "VpmU_yzqm2i7"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vectordb = Chroma.from_texts(\n",
        "    texts=passages[:500],\n",
        "    embedding=embedding\n",
        ")"
      ],
      "metadata": {
        "id": "jt3mBMsUuPP1"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vectordb._collection.count()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pTpXMA442S02",
        "outputId": "714ef3e9-2a41-43a4-cfc9-19bc4fef00fa"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "500"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "vectordb.similarity_search_with_score(\"What's Chinese New Year?\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TBRs63oF2rgU",
        "outputId": "385b3eb9-daac-4b9c-8f5c-048121db65b4"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(Document(page_content='Chinese New Year, known in China as the SpringFestival and in Singapore as the LunarNewYear, is a holiday on and around the new moon on the first day of the year in the traditional Chinese calendar. This calendar is based on the changes in the moon and is only sometimes changed to fit the seasons of the year based on how the Earth moves around the sun. Because of this, Chinese New Year is never on January1. It moves around between January21 and February20.'),\n",
              "  0.44440481066703796),\n",
              " (Document(page_content='In Hinduism and Buddhism, a dakini is a female being like a goddess. They are mostly found in Tibetan Buddhism.Chinese: 空行母, Pinyin: Kōngxíng Mǔ and 狐仙,Pinyin:Hú xian ;明妃,Pinyin:Míng fēi｝ The dakini inspires spiritual practice. A dakini is often depicted as beautiful and naked. The nakedness represents the freedom of the mind.'),\n",
              "  1.4540748596191406),\n",
              " (Document(page_content='Lu Sheng-Yen (盧勝彥, Lú Shèngyàn) (27 June 1945), is the founder and spiritual leader of the True Buddha School, which is a religious group with teachings taken from Taoism and Buddhism. He is called Master Lu by his followers. Within his sect, he is also known as \"Living Buddha Lian Sheng\" (蓮生活佛, \"Liansheng Huófó\"). He is worshipped by his followers as a \"Living Buddha\".'),\n",
              "  1.5080631971359253),\n",
              " (Document(page_content='Pompeii was a Roman city. Now it is a UNESCO World Heritage Site.'),\n",
              "  1.5488247871398926)]"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Cohere Rerank"
      ],
      "metadata": {
        "id": "oCV1lRdqgPsX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import userdata\n",
        "cohere_token = userdata.get('cohere')"
      ],
      "metadata": {
        "id": "F68p9BeT8rZm"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.retrievers import ContextualCompressionRetriever\n",
        "from langchain.retrievers.document_compressors import CohereRerank\n",
        "\n",
        "compressor = CohereRerank(cohere_api_key=cohere_token)\n",
        "compression_retriever = ContextualCompressionRetriever(\n",
        "    base_compressor=compressor, base_retriever=vectordb.as_retriever()\n",
        ")\n",
        "\n",
        "compression_retriever.get_relevant_documents(\n",
        "    \"What's Chinese New Year\"\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PLRIUNiA8c60",
        "outputId": "9510e770-7c0d-4321-f58d-c3790ba51efc"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Document(page_content='Chinese New Year, known in China as the SpringFestival and in Singapore as the LunarNewYear, is a holiday on and around the new moon on the first day of the year in the traditional Chinese calendar. This calendar is based on the changes in the moon and is only sometimes changed to fit the seasons of the year based on how the Earth moves around the sun. Because of this, Chinese New Year is never on January1. It moves around between January21 and February20.', metadata={'relevance_score': 0.9995955}),\n",
              " Document(page_content='Lu Sheng-Yen (盧勝彥, Lú Shèngyàn) (27 June 1945), is the founder and spiritual leader of the True Buddha School, which is a religious group with teachings taken from Taoism and Buddhism. He is called Master Lu by his followers. Within his sect, he is also known as \"Living Buddha Lian Sheng\" (蓮生活佛, \"Liansheng Huófó\"). He is worshipped by his followers as a \"Living Buddha\".', metadata={'relevance_score': 0.08299415}),\n",
              " Document(page_content='In Hinduism and Buddhism, a dakini is a female being like a goddess. They are mostly found in Tibetan Buddhism.Chinese: 空行母, Pinyin: Kōngxíng Mǔ and 狐仙,Pinyin:Hú xian ;明妃,Pinyin:Míng fēi｝ The dakini inspires spiritual practice. A dakini is often depicted as beautiful and naked. The nakedness represents the freedom of the mind.', metadata={'relevance_score': 0.022802636})]"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Cross Encoder Rerank\n",
        "\n",
        "模拟 CohereRerank 实现 CrossEncoderRerank"
      ],
      "metadata": {
        "id": "Fxn7n9Cwo5fM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Cross-encoder 模型访问封装\n",
        "\n",
        "使用 Huggingface inference api 来调用 Cross-encoder 模型"
      ],
      "metadata": {
        "id": "ewm7DaKoaPHf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import requests\n",
        "class CrossEncoder(object):\n",
        "    url = \"https://api-inference.huggingface.co/models\"\n",
        "\n",
        "    def __init__(self, model_name: str, token: str):\n",
        "        self.model_name = model_name\n",
        "        self.token = token\n",
        "\n",
        "    def get_similary_score(self, query, answer) -> float:\n",
        "        url = f\"{self.url}/{self.model_name}\"\n",
        "        body = {\"text\": query, \"text_pair\": answer}\n",
        "        headers = {\"token\": self.token}\n",
        "        resp = requests.post(url, json=body, headers=headers)\n",
        "        print(resp.text)\n",
        "        result = resp.json()\n",
        "        return result[0]['score']"
      ],
      "metadata": {
        "id": "Vb1XEMjicdFg"
      },
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import userdata\n",
        "\n",
        "model_name = \"cross-encoder/ms-marco-MiniLM-L-12-v2\"\n",
        "token = userdata.get('hf_token')"
      ],
      "metadata": {
        "id": "1SVS1ngNvlFl"
      },
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "encoder = CrossEncoder(model_name, token)\n",
        "print(encoder.get_similary_score(\"Who like apples?\", \"I told all my friends that I like apples\"))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eLOyrWM0ghW1",
        "outputId": "326641ac-a54f-4dd7-8584-a4e298f756b6"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[{\"label\":\"LABEL_0\",\"score\":0.9944524168968201}]\n",
            "0.9944524168968201\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### CrossEncoderRerank 实现"
      ],
      "metadata": {
        "id": "tnEpdg6r-Her"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from typing import Optional, Sequence, Dict\n",
        "\n",
        "from langchain.callbacks.manager import Callbacks\n",
        "from langchain.retrievers.document_compressors.base import BaseDocumentCompressor\n",
        "from langchain_core.documents import Document\n",
        "from langchain_core.pydantic_v1 import Extra, root_validator\n",
        "\n",
        "\n",
        "class CrossEncoderRerank(BaseDocumentCompressor):\n",
        "    \"\"\"Document compressor that uses `Cross Encoder to Rerank`.\"\"\"\n",
        "\n",
        "    model: str = \"cross-encoder/ms-marco-MiniLM-L-12-v2\"\n",
        "    \"\"\"Model to use for reranking.\"\"\"\n",
        "    encoder: CrossEncoder = None\n",
        "    token: str = None\n",
        "\n",
        "    class Config:\n",
        "        \"\"\"Configuration for this pydantic object.\"\"\"\n",
        "        extra = Extra.forbid\n",
        "        arbitrary_types_allowed = True\n",
        "\n",
        "    @root_validator(pre=True)\n",
        "    def validate(cls, values: Dict) -> Dict:\n",
        "        model_name = values.get(\"model\")\n",
        "        token = values.get('hf_token')\n",
        "        values['encoder'] = CrossEncoder(model_name, token)\n",
        "        return values\n",
        "\n",
        "    def _compute_score(self, query, content) -> float:\n",
        "        return self.encoder.get_similary_score(query, content)\n",
        "\n",
        "    def compress_documents(\n",
        "        self,\n",
        "        documents: Sequence[Document],\n",
        "        query: str,\n",
        "        callbacks: Optional[Callbacks] = None,\n",
        "    ) -> Sequence[Document]:\n",
        "        \"\"\"\n",
        "        Compress documents using Cross Encoder Inference API.\n",
        "\n",
        "        Args:\n",
        "            documents: A sequence of documents to compress.\n",
        "            query: The query to use for compressing the documents.\n",
        "            callbacks: Callbacks to run during the compression process.\n",
        "\n",
        "        Returns:\n",
        "            A sequence of compressed documents.\n",
        "        \"\"\"\n",
        "        if len(documents) == 0:  # to avoid empty api call\n",
        "            return []\n",
        "        doc_list = list(documents)\n",
        "        _docs = [d.page_content for d in doc_list]\n",
        "        for doc in doc_list:\n",
        "            score = self._compute_score(query, doc.page_content)\n",
        "            doc.metadata[\"relevance_score\"] = score\n",
        "        doc_list.sort(key=lambda x: x.metadata[\"relevance_score\"], reverse=True)\n",
        "        return doc_list\n"
      ],
      "metadata": {
        "id": "m99iY_wAo2PA"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "reranker = CrossEncoderRerank(model=model_name, token=token)\n",
        "reranker._compute_score(\"Who like apples?\", \"I told all my friends that I like apples\")"
      ],
      "metadata": {
        "id": "VkV_HE7hgkB4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9e8d9420-658c-4ec3-a0c3-72b41a86f907"
      },
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[{\"label\":\"LABEL_0\",\"score\":0.9944524168968201}]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9944524168968201"
            ]
          },
          "metadata": {},
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### CrossEncodRerank 使用"
      ],
      "metadata": {
        "id": "jKdPK3DT-VFq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.retrievers import ContextualCompressionRetriever\n",
        "from langchain.retrievers.document_compressors import CohereRerank\n",
        "\n",
        "compressor = CrossEncoderRerank(model=model_name, token=token)\n",
        "compression_retriever = ContextualCompressionRetriever(\n",
        "    base_compressor=compressor, base_retriever=vectordb.as_retriever()\n",
        ")\n",
        "\n",
        "compression_retriever.get_relevant_documents(\n",
        "    \"What's Chinese New Year?\"\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "83ff3c62-a09e-4550-97db-e437563c1a23",
        "id": "9qKb_N8i-fOV"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[{\"label\":\"LABEL_0\",\"score\":0.9999721050262451}]\n",
            "[{\"label\":\"LABEL_0\",\"score\":2.4164222850231454e-05}]\n",
            "[{\"label\":\"LABEL_0\",\"score\":4.29442516178824e-05}]\n",
            "[{\"label\":\"LABEL_0\",\"score\":1.2482765669119544e-05}]\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Document(page_content='Chinese New Year, known in China as the SpringFestival and in Singapore as the LunarNewYear, is a holiday on and around the new moon on the first day of the year in the traditional Chinese calendar. This calendar is based on the changes in the moon and is only sometimes changed to fit the seasons of the year based on how the Earth moves around the sun. Because of this, Chinese New Year is never on January1. It moves around between January21 and February20.', metadata={'relevance_score': 0.9999721050262451}),\n",
              " Document(page_content='Lu Sheng-Yen (盧勝彥, Lú Shèngyàn) (27 June 1945), is the founder and spiritual leader of the True Buddha School, which is a religious group with teachings taken from Taoism and Buddhism. He is called Master Lu by his followers. Within his sect, he is also known as \"Living Buddha Lian Sheng\" (蓮生活佛, \"Liansheng Huófó\"). He is worshipped by his followers as a \"Living Buddha\".', metadata={'relevance_score': 4.29442516178824e-05}),\n",
              " Document(page_content='In Hinduism and Buddhism, a dakini is a female being like a goddess. They are mostly found in Tibetan Buddhism.Chinese: 空行母, Pinyin: Kōngxíng Mǔ and 狐仙,Pinyin:Hú xian ;明妃,Pinyin:Míng fēi｝ The dakini inspires spiritual practice. A dakini is often depicted as beautiful and naked. The nakedness represents the freedom of the mind.', metadata={'relevance_score': 2.4164222850231454e-05}),\n",
              " Document(page_content='Pompeii was a Roman city. Now it is a UNESCO World Heritage Site.', metadata={'relevance_score': 1.2482765669119544e-05})]"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 对比 see sharp 检索词的效果\n",
        "\n",
        "数据集前 500 个例子中，有一个 see sharp 的例子无法通过搜索找到，定位问题中。\n",
        "\n",
        "> C# (pronounced \"see sharp\") is a computer programming language. It is developed by Microsoft. It was created to use all capacities of .NET platform. The first version was released in 2001. The most recent version is C# 8.0, which was released in September 2019. C# is a modern language. C#\\'s development team is led by Anders Hejlsberg, the creator of Delphi.',\n",
        " 'Wilmington is a city in New Hanover County, North Carolina, United States."
      ],
      "metadata": {
        "id": "GMeiI_vM-0xN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install rank_bm25 --quiet"
      ],
      "metadata": {
        "id": "tyH2W_df_uZY"
      },
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 测试关键词搜索\n",
        "\n",
        "from langchain.retrievers import BM25Retriever\n",
        "\n",
        "# initialize the bm25 retriever and faiss retriever\n",
        "bm25_retriever = BM25Retriever.from_texts(\n",
        "    passages[:500]\n",
        ")\n",
        "bm25_retriever.k = 2"
      ],
      "metadata": {
        "id": "_NTe5yk5A8LS"
      },
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vectordb.similarity_search_with_score(\"see sharp\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DwivQ_kNAJBs",
        "outputId": "ba75873b-cfe3-4b36-c052-578387bd9df6"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(Document(page_content='A cross section is what one gets if one cuts an object into slices.'),\n",
              "  1.4852806329727173),\n",
              " (Document(page_content='In geometrical optics, a focus (also called an image point) is the point where light rays that come from a point on the object converge (come together).'),\n",
              "  1.522754430770874),\n",
              " (Document(page_content='\"For the band, see U2 (band).\"'), 1.6397788524627686),\n",
              " (Document(page_content='A high five is a hand gesture done with two people who want to express joy over an common achievement. The five refers to the five fingers on each hand.'),\n",
              "  1.6554793119430542)]"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "bm25_retriever.get_relevant_documents(\"see sharp\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XoIfdGjyBTbO",
        "outputId": "9e04f7d6-035d-46d9-9d3a-a96e4d488ab1"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Document(page_content='\"For the band, see U2 (band).\"'),\n",
              " Document(page_content='Bifocals are eyeglasses with lenses that are split between two different strengths. Usually the lower half of each lens is made to help the wearer read, while the upper one is to help the wearer see at a distance.')]"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def keyword_match(keyword):\n",
        "    return [p for p in passages[:500] if keyword in p]"
      ],
      "metadata": {
        "id": "gXeqxenlFPuF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "keyword_match(\"see sharp\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FVQQzxsUFgFC",
        "outputId": "dcdd2cb3-acb6-4674-96d6-0862556dfc97"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['C# (pronounced \"see sharp\") is a computer programming language. It is developed by Microsoft. It was created to use all capacities of .NET platform. The first version was released in 2001. The most recent version is C# 8.0, which was released in September 2019. C# is a modern language. C#\\'s development team is led by Anders Hejlsberg, the creator of Delphi.']"
            ]
          },
          "metadata": {},
          "execution_count": 87
        }
      ]
    }
  ]
}