examples/operator-rbac/3-client-rbac-test-local.ipynb

{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "\n",
    "# Feast Client with RBAC  \n",
    "### Example Using outside of Kubernetes for local testing\n",
    "\n",
    "This notebook will  test Feast authentication outside of Kubernetes for local testing.\n",
    "\n",
    "When running outside of Kubernetes, you need to manually set the service account token in the `LOCAL_K8S_TOKEN` environment variable. The token can be retrieved from a running pod using:  \n",
    "\n",
    "```sh\n",
    "\n",
    "kubectl exec <pod-name> -- cat /var/run/secrets/kubernetes.io/serviceaccount/token\n",
    "\n",
    "```\n",
    "\n",
    "To authenticate Feast externally, set the retrieved token as an environment variable:  \n",
    "\n",
    "```sh\n",
    "\n",
    "export LOCAL_K8S_TOKEN=\"your-service-account-token\"\n",
    "\n",
    "```  \n"
   ],
   "id": "bb0145c9c1f6ebcc"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Test Cases\n",
    "| User Type       | ServiceAccount               | RoleBinding Assigned | Expected Behavior in output                                |\n",
    "|----------------|-----------------------------|----------------------|------------------------------------------------------------|\n",
    "| **Read-Only**  | `feast-user-sa`              | `feast-reader`       | Can **read** from the feature store, but **cannot write**. |\n",
    "| **Unauthorized** | `feast-unauthorized-user-sa` | _None_               | **Access should be denied** in `test.py`.                  |\n",
    "| **Admin**      | `feast-admin-sa`             | `feast-writer`       | Can **read and write** feature store data.                 |"
   ],
   "id": "160681ba4ab3c2c5"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "###  Feature Store settings",
   "id": "6590c081efb1fe3c"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:56:16.529917Z",
     "start_time": "2025-03-14T14:56:16.361968Z"
    }
   },
   "cell_type": "code",
   "source": "!cat client/feature_repo/feature_store.yaml",
   "id": "fac5f67ff391b5cf",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "project: feast_rbac\r\n",
      "provider: local\r\n",
      "offline_store:\r\n",
      "    host: localhost\r\n",
      "    type: remote\r\n",
      "    port: 8081\r\n",
      "online_store:\r\n",
      "    path: http://localhost:8082\r\n",
      "    type: remote\r\n",
      "registry:\r\n",
      "    path: localhost:8083\r\n",
      "    registry_type: remote\r\n",
      "auth:\r\n",
      "    type: kubernetes\r\n",
      "entity_key_serialization_version: 3\r\n"
     ]
    }
   ],
   "execution_count": 17
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "**The Operator create client feature store ConfigMap** containing the `feature_store.yaml `settings. We can retrieve it and port froward to local as we are testing locally.",
   "id": "84f73e09711bff9f"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:56:42.630556Z",
     "start_time": "2025-03-14T14:56:42.455312Z"
    }
   },
   "cell_type": "code",
   "source": "!kubectl get configmap feast-sample-kubernetes-auth-client -n feast -o jsonpath='{.data.feature_store\\.yaml}' ",
   "id": "456fb4df46f32380",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "project: feast_rbac\r\n",
      "provider: local\r\n",
      "offline_store:\r\n",
      "    host: feast-sample-kubernetes-auth-offline.feast.svc.cluster.local\r\n",
      "    type: remote\r\n",
      "    port: 80\r\n",
      "online_store:\r\n",
      "    path: http://feast-sample-kubernetes-auth-online.feast.svc.cluster.local:80\r\n",
      "    type: remote\r\n",
      "registry:\r\n",
      "    path: feast-sample-kubernetes-auth-registry.feast.svc.cluster.local:80\r\n",
      "    registry_type: remote\r\n",
      "auth:\r\n",
      "    type: kubernetes\r\n",
      "entity_key_serialization_version: 3\r\n"
     ]
    }
   ],
   "execution_count": 18
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### The function below is executed to support the preparation of client testing.",
   "id": "ae61f4dca31f3466"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Run Port Forwarding for All Services for local testing ",
   "id": "28636825ae8f676d"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:57:03.137990Z",
     "start_time": "2025-03-14T14:57:03.096768Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import subprocess\n",
    "\n",
    "# Define services and their local ports\n",
    "services = {\n",
    "    \"offline_store\": (\"feast-sample-kubernetes-auth-offline\", 8081),\n",
    "    \"online_store\": (\"feast-sample-kubernetes-auth-online\", 8082),\n",
    "    \"registry\": (\"feast-sample-kubernetes-auth-registry\", 8083),\n",
    "}\n",
    "\n",
    "# Start port-forwarding for each service\n",
    "port_forward_processes = {}\n",
    "for name, (service, local_port) in services.items():\n",
    "    cmd = f\"kubectl port-forward svc/{service} -n feast {local_port}:80\"\n",
    "    process = subprocess.Popen(cmd, shell=True)\n",
    "    port_forward_processes[name] = process\n",
    "    print(f\"Port forwarding {service} -> localhost:{local_port}\")"
   ],
   "id": "c014248190863e8a",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Port forwarding feast-sample-kubernetes-auth-offline -> localhost:8081\n",
      "Port forwarding feast-sample-kubernetes-auth-online -> localhost:8082\n",
      "Port forwarding feast-sample-kubernetes-auth-registry -> localhost:8083\n"
     ]
    }
   ],
   "execution_count": 19
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Function to retrieve a Kubernetes service account token and set it as an environment variable",
   "id": "c0eccef6379f442c"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:57:42.336072Z",
     "start_time": "2025-03-14T14:57:42.332818Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import subprocess\n",
    "\n",
    "def get_k8s_token(service_account):\n",
    "    namespace = \"feast\"\n",
    "\n",
    "    if not service_account:\n",
    "        raise ValueError(\"Service account name is required.\")\n",
    "\n",
    "    result = subprocess.run(\n",
    "        [\"kubectl\", \"create\", \"token\", service_account, \"-n\", namespace],\n",
    "        capture_output=True, text=True, check=True\n",
    "    )\n",
    "\n",
    "    token = result.stdout.strip()\n",
    "\n",
    "    if not token:\n",
    "        return None  # Silently return None if token retrieval fails\n",
    "\n",
    "    os.environ[\"LOCAL_K8S_TOKEN\"] = token\n",
    "    return \"Token Retrieved: ***** (hidden for security)\"\n"
   ],
   "id": "70bdbcd7b3fe44",
   "outputs": [],
   "execution_count": 20
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "**Generating training data**. The following test functions were copied from the `test_workflow.py` template but we added `try` blocks to print only \n",
    "the relevant error messages, since we expect to receive errors from the permission enforcement modules."
   ],
   "id": "8c9e27ec4ed8ca2c"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "",
   "id": "856dcffcb8f19705"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:58:09.731552Z",
     "start_time": "2025-03-14T14:58:06.599552Z"
    }
   },
   "cell_type": "code",
   "source": "!cat client/feature_repo/test.py",
   "id": "934963c5f6b18930",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "import os\r\n",
      "\r\n",
      "from feast import FeatureStore\r\n",
      "from feast.data_source import PushMode\r\n",
      "from datetime import datetime\r\n",
      "import pandas as pd\r\n",
      "\r\n",
      "# Initialize Feature Store\r\n",
      "repo_path = os.getenv(\"FEAST_REPO_PATH\", \".\")\r\n",
      "store = FeatureStore(repo_path=repo_path)\r\n",
      "\r\n",
      "def fetch_historical_features_entity_df(store: FeatureStore, for_batch_scoring: bool):\r\n",
      "    \"\"\"Fetch historical features for training or batch scoring.\"\"\"\r\n",
      "    try:\r\n",
      "        entity_df = pd.DataFrame.from_dict(\r\n",
      "            {\r\n",
      "                \"driver_id\": [1001, 1002, 1003],\r\n",
      "                \"event_timestamp\": [\r\n",
      "                    datetime(2021, 4, 12, 10, 59, 42),\r\n",
      "                    datetime(2021, 4, 12, 8, 12, 10),\r\n",
      "                    datetime(2021, 4, 12, 16, 40, 26),\r\n",
      "                ],\r\n",
      "                \"label_driver_reported_satisfaction\": [1, 5, 3],\r\n",
      "                \"val_to_add\": [1, 2, 3],\r\n",
      "                \"val_to_add_2\": [10, 20, 30],\r\n",
      "            }\r\n",
      "        )\r\n",
      "        if for_batch_scoring:\r\n",
      "            entity_df[\"event_timestamp\"] = pd.to_datetime(\"now\", utc=True)\r\n",
      "\r\n",
      "        training_df = store.get_historical_features(\r\n",
      "            entity_df=entity_df,\r\n",
      "            features=[\r\n",
      "                \"driver_hourly_stats:conv_rate\",\r\n",
      "                \"driver_hourly_stats:acc_rate\",\r\n",
      "                \"driver_hourly_stats:avg_daily_trips\",\r\n",
      "                \"transformed_conv_rate:conv_rate_plus_val1\",\r\n",
      "                \"transformed_conv_rate:conv_rate_plus_val2\",\r\n",
      "            ],\r\n",
      "        ).to_df()\r\n",
      "        print(f\"Successfully fetched {'batch scoring' if for_batch_scoring else 'training'} historical features:\\n\", training_df.head())\r\n",
      "\r\n",
      "    except PermissionError:\r\n",
      "        print(\"\\n*** PERMISSION DENIED *** Cannot fetch historical features.\")\r\n",
      "    except Exception as e:\r\n",
      "        print(f\"Unexpected error while fetching historical features: {e}\")\r\n",
      "\r\n",
      "def fetch_online_features(store: FeatureStore, source: str = \"\"):\r\n",
      "    \"\"\"Fetch online features from the feature store.\"\"\"\r\n",
      "    try:\r\n",
      "        entity_rows = [\r\n",
      "            {\r\n",
      "                \"driver_id\": 1001,\r\n",
      "                \"val_to_add\": 1000,\r\n",
      "                \"val_to_add_2\": 2000,\r\n",
      "            },\r\n",
      "            {\r\n",
      "                \"driver_id\": 1002,\r\n",
      "                \"val_to_add\": 1001,\r\n",
      "                \"val_to_add_2\": 2002,\r\n",
      "            },\r\n",
      "        ]\r\n",
      "        if source == \"feature_service\":\r\n",
      "            features_to_fetch = store.get_feature_service(\"driver_activity_v1\")\r\n",
      "        elif source == \"push\":\r\n",
      "            features_to_fetch = store.get_feature_service(\"driver_activity_v3\")\r\n",
      "        else:\r\n",
      "            features_to_fetch = [\r\n",
      "                \"driver_hourly_stats:acc_rate\",\r\n",
      "                \"transformed_conv_rate:conv_rate_plus_val1\",\r\n",
      "                \"transformed_conv_rate:conv_rate_plus_val2\",\r\n",
      "            ]\r\n",
      "\r\n",
      "        returned_features = store.get_online_features(\r\n",
      "            features=features_to_fetch,\r\n",
      "            entity_rows=entity_rows,\r\n",
      "        ).to_dict()\r\n",
      "\r\n",
      "        print(f\"Successfully fetched online features {'via feature service' if source else 'directly'}:\\n\")\r\n",
      "        for key, value in sorted(returned_features.items()):\r\n",
      "            print(f\"{key} : {value}\")\r\n",
      "\r\n",
      "    except PermissionError:\r\n",
      "        print(\"\\n*** PERMISSION DENIED *** Cannot fetch online features.\")\r\n",
      "    except Exception as e:\r\n",
      "        print(f\"Unexpected error while fetching online features: {e}\")\r\n",
      "\r\n",
      "def check_permissions():\r\n",
      "    \"\"\"Check user role, test various Feast operations.\"\"\"\r\n",
      "    feature_views = []\r\n",
      "\r\n",
      "    # Step 1: List feature views\r\n",
      "    print(\"\\n--- List feature views ---\")\r\n",
      "    try:\r\n",
      "        feature_views = store.list_feature_views()\r\n",
      "        if not feature_views:\r\n",
      "            print(\"No feature views found. You might not have access or they haven't been created.\")\r\n",
      "        else:\r\n",
      "            print(f\"Successfully listed {len(feature_views)} feature views:\")\r\n",
      "            for fv in feature_views:\r\n",
      "                print(f\"  - {fv.name}\")\r\n",
      "    except PermissionError:\r\n",
      "        print(\"\\n*** PERMISSION DENIED *** Cannot list feature views.\")\r\n",
      "    except Exception as e:\r\n",
      "        print(f\"Unexpected error listing feature views: {e}\")\r\n",
      "\r\n",
      "    # Step 2: Fetch Historical Features\r\n",
      "    print(\"\\n--- Fetching Historical Features for Training ---\")\r\n",
      "    fetch_historical_features_entity_df(store, for_batch_scoring=False)\r\n",
      "\r\n",
      "    print(\"\\n--- Fetching Historical Features for Batch Scoring ---\")\r\n",
      "    fetch_historical_features_entity_df(store, for_batch_scoring=True)\r\n",
      "\r\n",
      "    # Step 3: Apply Feature Store\r\n",
      "    print(\"\\n--- Write to Feature Store ---\")\r\n",
      "    try:\r\n",
      "        store.apply(feature_views)\r\n",
      "        print(\"User has write access to the feature store.\")\r\n",
      "    except PermissionError:\r\n",
      "        print(\"\\n*** PERMISSION DENIED *** User lacks permission to modify the feature store.\")\r\n",
      "    except Exception as e:\r\n",
      "        print(f\"Unexpected error testing write access: {e}\")\r\n",
      "\r\n",
      "    # Step 4: Fetch Online Features\r\n",
      "    print(\"\\n--- Fetching Online Features ---\")\r\n",
      "    fetch_online_features(store)\r\n",
      "\r\n",
      "    print(\"\\n--- Fetching Online Features via Feature Service ---\")\r\n",
      "    fetch_online_features(store, source=\"feature_service\")\r\n",
      "\r\n",
      "    print(\"\\n--- Fetching Online Features via Push Source ---\")\r\n",
      "    fetch_online_features(store, source=\"push\")\r\n",
      "\r\n",
      "    print(\"\\n--- Performing Push Source ---\")\r\n",
      "    # Step 5: Simulate Event Push (Streaming Ingestion)\r\n",
      "    try:\r\n",
      "        event_df = pd.DataFrame.from_dict(\r\n",
      "            {\r\n",
      "                \"driver_id\": [1001],\r\n",
      "                \"event_timestamp\": [datetime.now()],\r\n",
      "                \"created\": [datetime.now()],\r\n",
      "                \"conv_rate\": [1.0],\r\n",
      "                \"acc_rate\": [1.0],\r\n",
      "                \"avg_daily_trips\": [1000],\r\n",
      "            }\r\n",
      "        )\r\n",
      "        store.push(\"driver_stats_push_source\", event_df, to=PushMode.ONLINE_AND_OFFLINE)\r\n",
      "        print(\"Successfully pushed a test event.\")\r\n",
      "    except PermissionError:\r\n",
      "        print(\"\\n*** PERMISSION DENIED *** Cannot push event (no write access).\")\r\n",
      "    except Exception as e:\r\n",
      "        print(f\"Unexpected error while pushing event: {e}\")\r\n",
      "\r\n",
      "if __name__ == \"__main__\":\r\n",
      "    check_permissions()\r\n"
     ]
    }
   ],
   "execution_count": 21
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:58:15.658631Z",
     "start_time": "2025-03-14T14:58:15.653740Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import os\n",
    "\n",
    "# Set the FEAST_REPO_PATH before importing check_permissions\n",
    "os.environ[\"FEAST_REPO_PATH\"] = \"client/feature_repo\""
   ],
   "id": "7c20ea331dc5a09b",
   "outputs": [],
   "execution_count": 22
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Test Read-Only Feast User \n",
    "**Step 1: Set the Token**"
   ],
   "id": "84e3f83699b8d83"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:58:27.147260Z",
     "start_time": "2025-03-14T14:58:24.035744Z"
    }
   },
   "cell_type": "code",
   "source": "get_k8s_token(\"feast-user-sa\")",
   "id": "f1fe8baa02d27d38",
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Token Retrieved: ***** (hidden for security)'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 23
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "**Step 2: Test permission functions to validate permission on fetching online, offline or perform write operation**",
   "id": "140c909fa8bcc6ab"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:58:57.328137Z",
     "start_time": "2025-03-14T14:58:54.623074Z"
    }
   },
   "cell_type": "code",
   "source": [
    "from client.feature_repo.test import  check_permissions\n",
    "\n",
    "# Call the function\n",
    "#Run test.py script from pod to test RBAC for client-readonly-user.\n",
    "# verify the logs for write operation will show below message \n",
    "# --- Write to Feature Store ---\n",
    "#*** PERMISSION DENIED *** User lacks permission to modify the feature store.\n",
    "\n",
    "check_permissions()\n"
   ],
   "id": "14b7ad38368db767",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "--- List feature views ---\n",
      "Handling connection for 8083\n",
      "Successfully listed 2 feature views:\n",
      "  - driver_hourly_stats\n",
      "  - driver_hourly_stats_fresh\n",
      "\n",
      "--- Fetching Historical Features for Training ---\n",
      "Handling connection for 8081\n",
      "Successfully fetched training historical features:\n",
      "    driver_id           event_timestamp  label_driver_reported_satisfaction  \\\n",
      "0       1001 2021-04-12 10:59:42+00:00                                   1   \n",
      "1       1002 2021-04-12 08:12:10+00:00                                   5   \n",
      "2       1003 2021-04-12 16:40:26+00:00                                   3   \n",
      "\n",
      "   val_to_add  val_to_add_2  conv_rate  acc_rate  avg_daily_trips  \\\n",
      "0           1            10   0.229559  0.502340              613   \n",
      "1           2            20   0.697800  0.872357              891   \n",
      "2           3            30   0.933721  0.885730              781   \n",
      "\n",
      "   conv_rate_plus_val1  conv_rate_plus_val2  \n",
      "0             1.229559            10.229559  \n",
      "1             2.697800            20.697800  \n",
      "2             3.933721            30.933721  \n",
      "\n",
      "--- Fetching Historical Features for Batch Scoring ---\n",
      "Handling connection for 8081\n",
      "Successfully fetched batch scoring historical features:\n",
      "    driver_id                  event_timestamp  \\\n",
      "0       1001 2025-03-14 14:58:55.852884+00:00   \n",
      "1       1002 2025-03-14 14:58:55.852884+00:00   \n",
      "2       1003 2025-03-14 14:58:55.852884+00:00   \n",
      "\n",
      "   label_driver_reported_satisfaction  val_to_add  val_to_add_2  conv_rate  \\\n",
      "0                                   1           1            10   0.815464   \n",
      "1                                   5           2            20   0.445888   \n",
      "2                                   3           3            30   0.287972   \n",
      "\n",
      "   acc_rate  avg_daily_trips  conv_rate_plus_val1  conv_rate_plus_val2  \n",
      "0  0.251485              465             1.815464            10.815464  \n",
      "1  0.566106              646             2.445888            20.445888  \n",
      "2  0.948905              916             3.287972            30.287972  \n",
      "\n",
      "--- Write to Feature Store ---\n",
      "\n",
      "*** PERMISSION DENIED *** User lacks permission to modify the feature store.\n",
      "\n",
      "--- Fetching Online Features ---\n",
      "Handling connection for 8082\n",
      "Successfully fetched online features directly:\n",
      "\n",
      "acc_rate : [None, None]\n",
      "conv_rate_plus_val1 : [None, None]\n",
      "conv_rate_plus_val2 : [None, None]\n",
      "driver_id : [1001, 1002]\n",
      "\n",
      "--- Fetching Online Features via Feature Service ---\n",
      "Handling connection for 8082\n",
      "Successfully fetched online features via feature service:\n",
      "\n",
      "conv_rate : [None, None]\n",
      "conv_rate_plus_val1 : [None, None]\n",
      "conv_rate_plus_val2 : [None, None]\n",
      "driver_id : [1001, 1002]\n",
      "\n",
      "--- Fetching Online Features via Push Source ---\n",
      "Handling connection for 8082\n",
      "Successfully fetched online features via feature service:\n",
      "\n",
      "acc_rate : [None, None]\n",
      "avg_daily_trips : [None, None]\n",
      "conv_rate : [None, None]\n",
      "conv_rate_plus_val1 : [None, None]\n",
      "conv_rate_plus_val2 : [None, None]\n",
      "driver_id : [1001, 1002]\n",
      "\n",
      "--- Performing Push Source ---\n",
      "Unexpected error while pushing event: \n"
     ]
    }
   ],
   "execution_count": 24
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### Test Unauthorized Feast User ",
   "id": "e5e63a172da6d6d7"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:59:12.042904Z",
     "start_time": "2025-03-14T14:59:11.959795Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# Retrieve and store the token\n",
    "get_k8s_token(\"feast-unauthorized-user-sa\")"
   ],
   "id": "a7b3a6578fcf5c3c",
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Token Retrieved: ***** (hidden for security)'"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 25
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:59:14.629414Z",
     "start_time": "2025-03-14T14:59:14.375404Z"
    }
   },
   "cell_type": "code",
   "source": "check_permissions()",
   "id": "7aea5658325ab008",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "--- List feature views ---\n",
      "No feature views found. You might not have access or they haven't been created.\n",
      "\n",
      "--- Fetching Historical Features for Training ---\n",
      "\n",
      "*** PERMISSION DENIED *** Cannot fetch historical features.\n",
      "\n",
      "--- Fetching Historical Features for Batch Scoring ---\n",
      "\n",
      "*** PERMISSION DENIED *** Cannot fetch historical features.\n",
      "\n",
      "--- Write to Feature Store ---\n",
      "\n",
      "*** PERMISSION DENIED *** User lacks permission to modify the feature store.\n",
      "\n",
      "--- Fetching Online Features ---\n",
      "\n",
      "*** PERMISSION DENIED *** Cannot fetch online features.\n",
      "\n",
      "--- Fetching Online Features via Feature Service ---\n",
      "\n",
      "*** PERMISSION DENIED *** Cannot fetch online features.\n",
      "\n",
      "--- Fetching Online Features via Push Source ---\n",
      "\n",
      "*** PERMISSION DENIED *** Cannot fetch online features.\n",
      "\n",
      "--- Performing Push Source ---\n",
      "Unexpected error while pushing event: Unable to find push source 'driver_stats_push_source'.\n"
     ]
    }
   ],
   "execution_count": 26
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Test Admin Feast User",
   "id": "cb78ced7c37ceb4c"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:59:23.745351Z",
     "start_time": "2025-03-14T14:59:23.647728Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# Retrieve and store the token\n",
    "get_k8s_token(\"feast-admin-sa\")"
   ],
   "id": "4f10aae116825619",
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Token Retrieved: ***** (hidden for security)'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 27
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:59:28.583546Z",
     "start_time": "2025-03-14T14:59:26.745747Z"
    }
   },
   "cell_type": "code",
   "source": "check_permissions()",
   "id": "7a6133f052b9cfe1",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "--- List feature views ---\n",
      "Successfully listed 2 feature views:\n",
      "  - driver_hourly_stats\n",
      "  - driver_hourly_stats_fresh\n",
      "\n",
      "--- Fetching Historical Features for Training ---\n",
      "Handling connection for 8081\n",
      "Successfully fetched training historical features:\n",
      "    driver_id           event_timestamp  label_driver_reported_satisfaction  \\\n",
      "0       1001 2021-04-12 10:59:42+00:00                                   1   \n",
      "1       1002 2021-04-12 08:12:10+00:00                                   5   \n",
      "2       1003 2021-04-12 16:40:26+00:00                                   3   \n",
      "\n",
      "   val_to_add  val_to_add_2  conv_rate  acc_rate  avg_daily_trips  \\\n",
      "0           1            10   0.229559  0.502340              613   \n",
      "1           2            20   0.697800  0.872357              891   \n",
      "2           3            30   0.933721  0.885730              781   \n",
      "\n",
      "   conv_rate_plus_val1  conv_rate_plus_val2  \n",
      "0             1.229559            10.229559  \n",
      "1             2.697800            20.697800  \n",
      "2             3.933721            30.933721  \n",
      "\n",
      "--- Fetching Historical Features for Batch Scoring ---\n",
      "Handling connection for 8081\n",
      "Successfully fetched batch scoring historical features:\n",
      "    driver_id                  event_timestamp  \\\n",
      "0       1001 2025-03-14 14:59:27.350638+00:00   \n",
      "1       1002 2025-03-14 14:59:27.350638+00:00   \n",
      "2       1003 2025-03-14 14:59:27.350638+00:00   \n",
      "\n",
      "   label_driver_reported_satisfaction  val_to_add  val_to_add_2  conv_rate  \\\n",
      "0                                   1           1            10   0.815464   \n",
      "1                                   5           2            20   0.445888   \n",
      "2                                   3           3            30   0.287972   \n",
      "\n",
      "   acc_rate  avg_daily_trips  conv_rate_plus_val1  conv_rate_plus_val2  \n",
      "0  0.251485              465             1.815464            10.815464  \n",
      "1  0.566106              646             2.445888            20.445888  \n",
      "2  0.948905              916             3.287972            30.287972  \n",
      "\n",
      "--- Write to Feature Store ---\n",
      "User has write access to the feature store.\n",
      "\n",
      "--- Fetching Online Features ---\n",
      "Handling connection for 8082\n",
      "Successfully fetched online features directly:\n",
      "\n",
      "acc_rate : [None, None]\n",
      "conv_rate_plus_val1 : [None, None]\n",
      "conv_rate_plus_val2 : [None, None]\n",
      "driver_id : [1001, 1002]\n",
      "\n",
      "--- Fetching Online Features via Feature Service ---\n",
      "Handling connection for 8082\n",
      "Successfully fetched online features via feature service:\n",
      "\n",
      "conv_rate : [None, None]\n",
      "conv_rate_plus_val1 : [None, None]\n",
      "conv_rate_plus_val2 : [None, None]\n",
      "driver_id : [1001, 1002]\n",
      "\n",
      "--- Fetching Online Features via Push Source ---\n",
      "Handling connection for 8082\n",
      "Successfully fetched online features via feature service:\n",
      "\n",
      "acc_rate : [None, None]\n",
      "avg_daily_trips : [None, None]\n",
      "conv_rate : [None, None]\n",
      "conv_rate_plus_val1 : [None, None]\n",
      "conv_rate_plus_val2 : [None, None]\n",
      "driver_id : [1001, 1002]\n",
      "\n",
      "--- Performing Push Source ---\n",
      "Unexpected error while pushing event: \n"
     ]
    }
   ],
   "execution_count": 28
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    " **Note:**\n",
    "**Currently, remote materialization not available in Feast when using the Remote Client**\n",
    "**Workaround: Consider using running it from pod like**\n",
    "  \n",
    " `kubectl exec deploy/feast-sample-kubernetes-auth -itc online -- bash -c 'feast materialize-incremental $(date -u +\"%Y-%m-%dT%H:%M:%S\")`\n"
   ],
   "id": "e451c30649630b2f"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Terminate the process",
   "id": "e88442b1bae2b327"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-03-14T14:59:48.327362Z",
     "start_time": "2025-03-14T14:59:48.324482Z"
    }
   },
   "cell_type": "code",
   "source": [
    "for name, process in port_forward_processes.items():\n",
    "    process.terminate()\n",
    "    print(f\"Stopped port forwarding for {name}\")"
   ],
   "id": "2984d62766da122a",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stopped port forwarding for offline_store\n",
      "Stopped port forwarding for online_store\n",
      "Stopped port forwarding for registry\n"
     ]
    }
   ],
   "execution_count": 29
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "[Next: Uninstall the Operator and all Feast objects](./04-uninstall.ipynb)",
   "id": "38c54e92643e0bda"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}