05.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1bd9ec5f-fbb8-4f28-b911-1f2b31927c67",
   "metadata": {},
   "source": [
    "# What is Transfer learning?\n",
    "\n",
    "Transfer learning is a technique in machine learning in which knowledge learned from a task is re-used in order to boost performance on a related task. For example, for image classification, knowledge gained while learning to recognize cars could be applied when trying to recognize trucks. Wikipedia\n",
    "\n",
    "## Why use transfer learning?\n",
    "1. Can leverage an existing neural network architecture proven to work on problems similar to our own\n",
    "2. Can leverage a woring network architecture which has already learned patterns on similar data to our own (so great performence with low data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1129b80d-f36f-46b9-9355-e9e2968c681f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('2.1.0.dev20230713', '0.16.0.dev20230713')"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "torch.__version__,torchvision.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80b49efd-6111-4b27-92d9-4ba282874484",
   "metadata": {},
   "source": [
    "Now we've got the versions of torch and torchvision, we're after, let's import the code we've writte in previous section "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c1d9b7cd-1612-41db-8745-0ddd226442d3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Continue with regular imports\n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n",
    "import torchvision\n",
    "\n",
    "from torch import nn\n",
    "from torchvision import transforms\n",
    "\n",
    "# Try to get torchinfo, install it if it doesn't work\n",
    "try:\n",
    "    from torchinfo import summary\n",
    "except:\n",
    "    print(\"[INFO] Couldn't find torchinfo... installing it.\")\n",
    "    !pip install -q torchinfo\n",
    "    from torchinfo import summary\n",
    "\n",
    "# Try to import the going_modular directory, download it from GitHub if it doesn't work\n",
    "try:\n",
    "    from going_modular.going_modular import data_setup, engine\n",
    "except:\n",
    "    # Get the going_modular scripts\n",
    "    print(\"[INFO] Couldn't find going_modular scripts... downloading them from GitHub.\")\n",
    "    !git clone https://github.com/mrdbourke/pytorch-deep-learning\n",
    "    !mv pytorch-deep-learning/going_modular .\n",
    "    !mv pytorch-deep-learning/data/pizza_steak_sushi_20_percent.zip ./data/05\n",
    "    !unzip ./data/05/pizza_steak_sushi_20_percent.zip\n",
    "    # !rm -rf pytorch-deep-learning\n",
    "    from going_modular.going_modular import data_setup, engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d91a1f33-88a5-41b1-81e2-ad03cde5b3a1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e844a820-6e66-4589-9424-06c8c57bec42",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sat Jul 15 10:35:01 2023       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|                               |                      |               MIG M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  NVIDIA GeForce ...  Off  | 00000000:26:00.0  On |                  N/A |\n",
      "| 63%   49C    P5    13W / 170W |    482MiB / 12288MiB |     22%      Default |\n",
      "|                               |                      |                  N/A |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                                  |\n",
      "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
      "|        ID   ID                                                   Usage      |\n",
      "|=============================================================================|\n",
      "|    0   N/A  N/A       990      G   /usr/lib/xorg/Xorg                192MiB |\n",
      "|    0   N/A  N/A      1279      G   /usr/bin/gnome-shell               42MiB |\n",
      "|    0   N/A  N/A      4567      G   ...AAAAAAAAA= --shared-files        7MiB |\n",
      "|    0   N/A  N/A      4855      G   ...411301003318091638,262144      156MiB |\n",
      "|    0   N/A  N/A      9175      G   ...features=BackForwardCache       76MiB |\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d096286-9403-4eba-93f8-8be7520ef5d0",
   "metadata": {
    "tags": []
   },
   "source": [
    "##  Get data\n",
    "\n",
    "We need our pizza, steak, sushi data to build a transfer learning model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "dd6e69cc-da20-4ecf-9c69-bd3ddeff770c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !unzip ./data/05/pizza_steak_sushi_20_percent.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a5cc7e00-627a-42ea-9d74-f59ddf3760b9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_dir = \"./data/05/train/\"\n",
    "test_dir = \"./data/05/test/\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07491e36-f20a-44da-b174-aa675f65643d",
   "metadata": {},
   "source": [
    "## Create Datasets and Dataloaders\n",
    "\n",
    "Now've got some data, now we wanna turn it into PyTorch DataLoaders.\n",
    "\n",
    "We can use `data_setup.py`and `create_dataloaders()`\n",
    "\n",
    "Methods of transformations:\n",
    "1. Manual\n",
    "2. Automatically - the transforms are picked by pretrained model\n",
    "\n",
    "When using a pretrained model, its important that the data that you pass through is transformed in the same way that the data was trained on"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "697fdfad-59ec-4e05-b255-d287ea922d4b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Pay attension when using pre trained models\n",
    "from going_modular.going_modular import *"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "221ca663-39ef-4c3a-9f15-19ad569f62ab",
   "metadata": {},
   "source": [
    "### Create transforms manually "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b95488f2-2584-45ea-8353-245b911336dc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from torchvision import transforms\n",
    "normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],\n",
    "                                 std=[0.229, 0.224, 0.225])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b887909f-4a8a-4877-8df3-32db6577cd96",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "manual_transforms = transforms.Compose([\n",
    "    transforms.Resize(224),\n",
    "    transforms.ToTensor(),\n",
    "    normalize\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7f8e2e69-ba06-4bd9-8170-f05b4db81c49",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_dataloader, test_dataloader, class_names = data_setup.create_dataloaders(train_dir,test_dir,manual_transforms,32)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8366adc-f855-4d25-82dd-b0c93edf5e5e",
   "metadata": {},
   "source": [
    "### Create transforms automatically"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ca507664-faf0-4de6-ae6f-23b2bf92c812",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import torchvision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "21125718-c63a-4fb5-9f9e-644e91695d10",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "weights = torchvision.models.EfficientNet_B1_Weights.DEFAULT # Default = best weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "44817c3c-bc99-4cdb-9bf9-04a156b12347",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "EfficientNet_B1_Weights.IMAGENET1K_V2"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "5eb953bb-f482-48c3-8888-c31988a79713",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Get the transforms used to create our pretrained weights\n",
    "auto_transforms= weights.transforms()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "11eb573b-3c1e-4320-9fc0-50676aae9eea",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ImageClassification(\n",
       "    crop_size=[240]\n",
       "    resize_size=[255]\n",
       "    mean=[0.485, 0.456, 0.406]\n",
       "    std=[0.229, 0.224, 0.225]\n",
       "    interpolation=InterpolationMode.BILINEAR\n",
       ")"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "auto_transforms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "bb919e05-8f25-4874-a38c-e81431e9ddbf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_dataloader, test_dataloader, class_names = data_setup.create_dataloaders(train_dir,test_dir,auto_transforms,32)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c85eb11-8c8f-4ce5-a932-7df53b85109a",
   "metadata": {},
   "source": [
    "## Getting a Pretrained model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "366fe480-dfd7-45dd-90f1-51a7d0a0b6ed",
   "metadata": {},
   "source": [
    "### Which pretrained model should you use?\n",
    "\n",
    "*Experiment, experiment, experiment!*\n",
    "\n",
    "The whole idea of transfer learning is to take an already well performing model from a problem space similar to your own and then customize to your own problem.\n",
    "\n",
    "Three things to consider:\n",
    "1. Speed - how fast does it run?\n",
    "2. Size - how big is it?\n",
    "3. Performance - how accuracte is it?\n",
    "\n",
    "Where would the model live?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cd6ecd6-5b9d-46c4-9aa0-1531543df012",
   "metadata": {},
   "source": [
    "### Setting up a pretrained model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "aee5626c-9651-4e63-ad22-adebd01cbf7d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/user/anaconda3/lib/python3.11/site-packages/torchvision/models/_utils.py:135: UserWarning: Using 'weights' as positional parameter(s) is deprecated since 0.13 and may be removed in the future. Please use keyword parameter(s) instead.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "model = torchvision.models.efficientnet_b1(weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1b82bcb1-5918-4a3c-91b2-8b2354d43183",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "6dfd9fe7-fc0d-4254-8ab1-5c2ebc8106c8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AdaptiveAvgPool2d(output_size=1)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.avgpool"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "977ab3bf-9b27-42f2-90ae-d2855256a588",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# model.classifier[1] = torch.nn.Linear(1280,len(class_names),bias=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1622b39-07ad-4678-ba0e-c1eabcc0d694",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Getting a summary of our model with torchinfo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "c5e5d1f6-860e-4cd8-a5a6-4de2e52ad63c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from torchinfo import summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "99b7b8e2-0023-4059-91e9-0213eb150bb5",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "============================================================================================================================================\n",
       "Layer (type (var_name))                                      Input Shape          Output Shape         Param #              Trainable\n",
       "============================================================================================================================================\n",
       "EfficientNet (EfficientNet)                                  [1, 3, 224, 224]     [1, 1000]            --                   True\n",
       "├─Sequential (features)                                      [1, 3, 224, 224]     [1, 1280, 7, 7]      --                   True\n",
       "│    └─Conv2dNormActivation (0)                              [1, 3, 224, 224]     [1, 32, 112, 112]    --                   True\n",
       "│    │    └─Conv2d (0)                                       [1, 3, 224, 224]     [1, 32, 112, 112]    864                  True\n",
       "│    │    └─BatchNorm2d (1)                                  [1, 32, 112, 112]    [1, 32, 112, 112]    64                   True\n",
       "│    │    └─SiLU (2)                                         [1, 32, 112, 112]    [1, 32, 112, 112]    --                   --\n",
       "│    └─Sequential (1)                                        [1, 32, 112, 112]    [1, 16, 112, 112]    --                   True\n",
       "│    │    └─MBConv (0)                                       [1, 32, 112, 112]    [1, 16, 112, 112]    1,448                True\n",
       "│    │    └─MBConv (1)                                       [1, 16, 112, 112]    [1, 16, 112, 112]    612                  True\n",
       "│    └─Sequential (2)                                        [1, 16, 112, 112]    [1, 24, 56, 56]      --                   True\n",
       "│    │    └─MBConv (0)                                       [1, 16, 112, 112]    [1, 24, 56, 56]      6,004                True\n",
       "│    │    └─MBConv (1)                                       [1, 24, 56, 56]      [1, 24, 56, 56]      10,710               True\n",
       "│    │    └─MBConv (2)                                       [1, 24, 56, 56]      [1, 24, 56, 56]      10,710               True\n",
       "│    └─Sequential (3)                                        [1, 24, 56, 56]      [1, 40, 28, 28]      --                   True\n",
       "│    │    └─MBConv (0)                                       [1, 24, 56, 56]      [1, 40, 28, 28]      15,350               True\n",
       "│    │    └─MBConv (1)                                       [1, 40, 28, 28]      [1, 40, 28, 28]      31,290               True\n",
       "│    │    └─MBConv (2)                                       [1, 40, 28, 28]      [1, 40, 28, 28]      31,290               True\n",
       "│    └─Sequential (4)                                        [1, 40, 28, 28]      [1, 80, 14, 14]      --                   True\n",
       "│    │    └─MBConv (0)                                       [1, 40, 28, 28]      [1, 80, 14, 14]      37,130               True\n",
       "│    │    └─MBConv (1)                                       [1, 80, 14, 14]      [1, 80, 14, 14]      102,900              True\n",
       "│    │    └─MBConv (2)                                       [1, 80, 14, 14]      [1, 80, 14, 14]      102,900              True\n",
       "│    │    └─MBConv (3)                                       [1, 80, 14, 14]      [1, 80, 14, 14]      102,900              True\n",
       "│    └─Sequential (5)                                        [1, 80, 14, 14]      [1, 112, 14, 14]     --                   True\n",
       "│    │    └─MBConv (0)                                       [1, 80, 14, 14]      [1, 112, 14, 14]     126,004              True\n",
       "│    │    └─MBConv (1)                                       [1, 112, 14, 14]     [1, 112, 14, 14]     208,572              True\n",
       "│    │    └─MBConv (2)                                       [1, 112, 14, 14]     [1, 112, 14, 14]     208,572              True\n",
       "│    │    └─MBConv (3)                                       [1, 112, 14, 14]     [1, 112, 14, 14]     208,572              True\n",
       "│    └─Sequential (6)                                        [1, 112, 14, 14]     [1, 192, 7, 7]       --                   True\n",
       "│    │    └─MBConv (0)                                       [1, 112, 14, 14]     [1, 192, 7, 7]       262,492              True\n",
       "│    │    └─MBConv (1)                                       [1, 192, 7, 7]       [1, 192, 7, 7]       587,952              True\n",
       "│    │    └─MBConv (2)                                       [1, 192, 7, 7]       [1, 192, 7, 7]       587,952              True\n",
       "│    │    └─MBConv (3)                                       [1, 192, 7, 7]       [1, 192, 7, 7]       587,952              True\n",
       "│    │    └─MBConv (4)                                       [1, 192, 7, 7]       [1, 192, 7, 7]       587,952              True\n",
       "│    └─Sequential (7)                                        [1, 192, 7, 7]       [1, 320, 7, 7]       --                   True\n",
       "│    │    └─MBConv (0)                                       [1, 192, 7, 7]       [1, 320, 7, 7]       717,232              True\n",
       "│    │    └─MBConv (1)                                       [1, 320, 7, 7]       [1, 320, 7, 7]       1,563,600            True\n",
       "│    └─Conv2dNormActivation (8)                              [1, 320, 7, 7]       [1, 1280, 7, 7]      --                   True\n",
       "│    │    └─Conv2d (0)                                       [1, 320, 7, 7]       [1, 1280, 7, 7]      409,600              True\n",
       "│    │    └─BatchNorm2d (1)                                  [1, 1280, 7, 7]      [1, 1280, 7, 7]      2,560                True\n",
       "│    │    └─SiLU (2)                                         [1, 1280, 7, 7]      [1, 1280, 7, 7]      --                   --\n",
       "├─AdaptiveAvgPool2d (avgpool)                                [1, 1280, 7, 7]      [1, 1280, 1, 1]      --                   --\n",
       "├─Sequential (classifier)                                    [1, 1280]            [1, 1000]            --                   True\n",
       "│    └─Dropout (0)                                           [1, 1280]            [1, 1280]            --                   --\n",
       "│    └─Linear (1)                                            [1, 1280]            [1, 1000]            1,281,000            True\n",
       "============================================================================================================================================\n",
       "Total params: 7,794,184\n",
       "Trainable params: 7,794,184\n",
       "Non-trainable params: 0\n",
       "Total mult-adds (Units.MEGABYTES): 569.73\n",
       "============================================================================================================================================\n",
       "Input size (MB): 0.60\n",
       "Forward/backward pass size (MB): 149.57\n",
       "Params size (MB): 31.18\n",
       "Estimated Total Size (MB): 181.35\n",
       "============================================================================================================================================"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "summary(model,input_size=(1,3,224,224),col_names=['input_size','output_size','num_params','trainable'],col_width=20,row_settings=['var_names'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19aa693b-5154-47b3-b59e-9c7a30ad43be",
   "metadata": {},
   "source": [
    "### Freezing the base model and changing the output layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "d2ef4d70-20e2-400d-ab6d-f81ba09eca90",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Freeze all the base layers in EffNet\n",
    "for param in model.features.parameters():\n",
    "    param.requires_grad = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "2831247e-63be-48bd-b9a0-5331c428c97d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model.classifier[1] = nn.Linear(1280,len(class_names),bias=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "51b0d5aa-20f0-4d9b-aa0f-529ed7ee7298",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "optimizer = torch.optim.Adam(model.parameters(),lr=0.001)\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "epochs = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "e51eca0a-9cba-456e-9bb7-23c72ee23e7f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from going_modular.going_modular import model_builder,engine"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9921869-4152-42e9-be67-4029d6663a50",
   "metadata": {},
   "source": [
    "## Train Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "0d9d75f2-75b9-4efd-8e89-fa1245aa85e8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "33e6fefdc6ba47eab4ecbf18acf348a5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/5 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 1 | train_loss: 0.9945 | train_acc: 0.6917 | test_loss: 0.8663 | test_acc: 0.8179\n",
      "Epoch: 2 | train_loss: 0.8045 | train_acc: 0.8583 | test_loss: 0.7087 | test_acc: 0.8965\n",
      "Epoch: 3 | train_loss: 0.7365 | train_acc: 0.8438 | test_loss: 0.6041 | test_acc: 0.9201\n",
      "Epoch: 4 | train_loss: 0.5921 | train_acc: 0.9187 | test_loss: 0.5439 | test_acc: 0.9514\n",
      "Epoch: 5 | train_loss: 0.5101 | train_acc: 0.9396 | test_loss: 0.4771 | test_acc: 0.9451\n"
     ]
    }
   ],
   "source": [
    "dict_metrics = engine.train(model, \n",
    "                     train_dataloader, \n",
    "                     test_dataloader, \n",
    "                     optimizer,\n",
    "                     criterion,\n",
    "                     epochs,\n",
    "                     device\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "4143ec7b-1f7d-411d-80a7-21b397fecd74",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "3397c3a3-f965-4dbb-a329-7922dccb4466",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dict_pd = pd.DataFrame(dict_metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "4cc79e1b-b7a8-422f-8e07-a352d9f55465",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dict_pd.to_csv('./save/efficientnet_v2_m_fe.csv',index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "67078bb0-dd34-4f33-a0f2-ceab98d163d0",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(   train_loss  train_acc  test_loss  test_acc\n",
       " 0    0.994501   0.691667   0.866298  0.817935\n",
       " 1    0.804522   0.858333   0.708745  0.896467\n",
       " 2    0.736538   0.843750   0.604063  0.920109\n",
       " 3    0.592058   0.918750   0.543901  0.951359\n",
       " 4    0.510064   0.939583   0.477088  0.945109,\n",
       "    train_loss  train_acc  test_loss  test_acc\n",
       " 0    0.617612   0.754167   0.381821  0.882880\n",
       " 1    0.133278   0.956250   0.251991  0.881250\n",
       " 2    0.098864   0.964583   0.102654  0.963859\n",
       " 3    0.044587   0.991667   0.072911  0.978804\n",
       " 4    0.017165   0.997917   0.047501  0.987500)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv('./save/efficientnet_v2_m_fe.csv'),pd.read_csv('./save/efficientnet_v2_m_ft.csv')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}