Dev container

Author: pamelafox

.devcontainer/devcontainer.json

@@ -0,0 +1,11 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
+// https://github.com/microsoft/vscode-dev-containers/tree/v0.245.0/containers/python-3
+{
+    "name": "Phi-3 Cookbook",
+    "image": "mcr.microsoft.com/devcontainers/python:3.12-bullseye",
+    "features": {
+        "ghcr.io/prulloac/devcontainer-features/ollama:1": {}
+    },
+    // Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
+    "remoteUser": "vscode"
+}

code/04.Finetuning/Phi_3_Inference_Finetuning.ipynb

@@ -59,7 +59,7 @@
       },
       "outputs": [],
       "source": [
-        "This command is run in a bash shell due to '%%bash' at the beginning.\n",
+        "# This command is run in a bash shell due to '%%bash' at the beginning.\n",
         "# 'pip install -qqq' is used to install Python packages with pip, Python's package installer, in a less verbose mode.\n",
         "# 'accelerate', 'transformers', 'auto-gptq', and 'optimum' are the packages being installed.\n",
         "# These packages are necessary for the fine-tuning and inference of the Phi-3 model.\n",
@@ -140,7 +140,8 @@
         "outputs = model.generate(**inputs,\n",
         "                         do_sample=True, max_new_tokens=120)\n",
         "\n",
-        "# Decode the generated tokens and remove any special tokens"
+        "# Decode the generated tokens and remove any special tokens\n",
+        "response = tokenizer.decode(outputs[0], skip_special_tokens=True)"
       ]
     },
     {

md/02.QuickStart/Ollama_QuickStart.md

@@ -4,7 +4,7 @@
 
 ## **1. Installation**
 
-Ollama supports running on Windows, macOS, and Linux. You can install Ollama through this link ([https://ollama.com/download](https://ollama.com/download)). After successful installation, you can directly use Ollama script to call Phi-3 through a terminal window. You can see all the [available libaries in Ollama.](https://ollama.com/library)
+Ollama supports running on Windows, macOS, and Linux. You can install Ollama through this link ([https://ollama.com/download](https://ollama.com/download)). After successful installation, you can directly use Ollama script to call Phi-3 through a terminal window. You can see all the [available libaries in Ollama](https://ollama.com/library). If you open this repository in a Codespace, it will already have Ollama installed.
 
 
 ```bash
@@ -26,7 +26,7 @@ If you want to call the Phi-3 API generated by ollama, you can use this command
 ollama serve
 
 ```
-***Note:***  If running MacOS or Linux, please note that you may encounter the following error <b>"Error: listen tcp 127.0.0.1:11434: bind: address already in use"</b> You may get this error when calling running the command. The solution for this problems is:
+***Note:***  If running MacOS or Linux, please note that you may encounter the following error <b>"Error: listen tcp 127.0.0.1:11434: bind: address already in use"</b> You may get this error when calling running the command. You can either ignore that error, since it typically indicates the server is already running, or you can stop the and restart Ollama:
 
 **macOS**
 
@@ -46,7 +46,7 @@ sudo systemctl stop ollama
 
 ```
 
-Ollama supports two API: generate and chat. You can call the model API provided by Ollama according to your needs. Local service port 11434. such as
+Ollama supports two API: generate and chat. You can call the model API provided by Ollama according to your needs, by sending requests to the local service running on port 11434.
 
 **Chat**
 
@@ -74,7 +74,7 @@ curl http://127.0.0.1:11434/api/chat -d '{
 This is the result in Postman
 
 
-![chat](../../imgs/02/Ollama/ollama_chat.png)
+![Screenshot of JSON results for chat request](../../imgs/02/Ollama/ollama_chat.png)
 
 
 ```bash
@@ -92,11 +92,11 @@ curl http://127.0.0.1:11434/api/generate -d '{
 This is the result in Postman
 
 
-![gen](../../imgs/02/Ollama/ollama_gen.png)
+![Screenshot of JSON results for generate request](../../imgs/02/Ollama/ollama_gen.png)
 
 # Additional Resources
 
-Check the list of available models in Ollama in [this link.](https://ollama.com/library)
+Check the list of available models in Ollama in [their library](https://ollama.com/library).
 
 Pull your model from the Ollama server using this command
 
@@ -113,17 +113,45 @@ ollama run phi3
 ***Note:*** Visit this link [https://github.com/ollama/ollama/blob/main/docs/api.md](https://github.com/ollama/ollama/blob/main/docs/api.md) to learn more
 
 
+## Calling Ollama from Python
+
+You can use `requests` or `urllib3` to make requests to the local server endpoints used above. However, a popular way to use Ollama in Python is via the [openai](https://pypi.org/project/openai/) SDK, since Ollama provides OpenAI-compatible server endpoints as well.
+
+Here is an example for phi3-mini:
+
+```python
+import openai
+
+client = openai.OpenAI(
+    base_url="http://localhost:11434/v1",
+    api_key="nokeyneeded",
+)
+
+response = client.chat.completions.create(
+    model="phi3"
+    temperature=0.7,
+    n=1,
+    messages=[
+        {"role": "system", "content": "You are a helpful assistant."},
+        {"role": "user", "content": "Write a haiku about a hungry cat"},
+    ],
+)
+
+print("Response:")
+print(response.choices[0].message.content)
+```
+
 ## Calling Ollama from JavaScript 
 
 ```javascript
-#Example of Summarize a file with Phi-3
+// Example of Summarize a file with Phi-3
 script({
     model: "ollama:phi3",
     title: "Summarize with Phi-3",
     system: ["system"],
 })
 
-#Example of summarize
+// Example of summarize
 const file = def("FILE", env.files)
 $`Summarize ${file} in a single paragraph.`
 ```
@@ -160,14 +188,3 @@ Run the app with the command:
 ```bash
 dotnet run
 ```
-
-
-
-
-
-
-
-
-
-
-