Model Distiller

Model Distiller#

Distillation examples for the 5, 15, 50 minutes guide of the finetuning guide.

import ollama
import json
from pprint import pprint

Get a list of simple recipes#

We want a list of recipes for finetuning. Let’s first get a list of things we can make.

response = ollama.generate(model='gemma:7b', prompt="What are 20 foods with less than 3 ingredients per recipe in bullet form?")["response"]

response

'Sure, here are 20 foods with less than 3 ingredients per recipe in bullet form:\n\n- Apple Slices with Cinnamon\n- Banana with Honey\n- Boiled Egg\n- Carrot and Celery with Olive Oil\n- Cherry Pie\n- Cream of Tomato Soup\n- Eggplant Fritters\n- Fruit Skewers\n- Green Bean Casserole\n- Ham and Cheese Sandwich\n- Lemon Curd\n- Mango with Mint\n- Orange Juice\n- Peach Cobbler\n- Pears with Apple Cider Vinegar\n- Quick Bread\n- Rice Pudding\n- Salmon with Lemon and Herbs\n- Scrambled Eggs with Vegetables\n- Simple Tomato Soup\n- Smoothies\n- Soufflé\n- Steamed Vegetables'

[r[2:] for r in response.split("\n")]

['re, here are 20 foods with less than 3 ingredients per recipe in bullet form:',
 '',
 'Apple Slices with Cinnamon',
 'Banana with Honey',
 'Boiled Egg',
 'Carrot and Celery with Olive Oil',
 'Cherry Pie',
 'Cream of Tomato Soup',
 'Eggplant Fritters',
 'Fruit Skewers',
 'Green Bean Casserole',
 'Ham and Cheese Sandwich',
 'Lemon Curd',
 'Mango with Mint',
 'Orange Juice',
 'Peach Cobbler',
 'Pears with Apple Cider Vinegar',
 'Quick Bread',
 'Rice Pudding',
 'Salmon with Lemon and Herbs',
 'Scrambled Eggs with Vegetables',
 'Simple Tomato Soup',
 'Smoothies',
 'Soufflé',
 'Steamed Vegetables']

Copy list for reproducibility#

Just for this tutorial copying the ones genereated on my computer for reproducibility

foods =[
'Apple Slices with Cinnamon',
 'Banana with Honey',
 'Boiled Eggs',
 'Brown Rice',
 'Cherry Pie',
 'Corn on the Cob',
 'Eggplant with Olive Oil',
 'Fruit Skewers',
 'Green Beans with Butter',
 'Guacamole',
 'Ham and Cheese Sandwich',
 'Hard-Boiled Shrimp',
 'Kale Salad',
 'Lemon Juice',
 'Mango Slices with Yogurt',
 'Muesli',
 'No-Bake Oats',
 'Orange Juice',
 'Peach Slices with Honey',
 'Pears with Apple Cider Vinegar',
 'Quick Oats',
 'Rice Pudding',
 'Salmon with Lemon Juice',
 'Simple Scrambled Eggs']

json_prompt = f"What is a recipe for {foods[0]} with the instructions written in a Shakespearean style in a json format?"

LLAMA 70B#

response = ollama.generate(model='llama2:70b', prompt=json_prompt)["response"]
print(response)

{
"name": "Apple Slices with Cinnamon",
"instructions": [
"Take thou three apples, fair and bright,
And slice them thin, with knife keen and tight.
Into a bowl, they shall be cast, anon,
With cinnamon, sugar, and a dash of fun.",
"Now, take a pan, and heat it on the fire,
Till butter melts, and bubbles do aspire.
Add thou the slices, and let them cook,
Till tender, and the cinnamon doth take root.",
" Remove from heat, and let cool to the touch,
Then serve, with joy, and a merry clutch.",
"Thou shalt enjoy, this sweet and tasty treat,
With friends, or alone, as thou dost eat."
],
"ingredients": [
"Apples",
"Cinnamon",
"Sugar",
"Butter"
]
}

This is a JSON object with the name of the recipe, "Apple Slices with Cinnamon", and a list of instructions written in a Shakespearean style. The instructions are broken down into four steps: slicing the apples, adding cinnamon, sugar, and butter to a pan, cooking the apple slices in the pan, and removing from heat and letting cool before serving. The ingredients list includes apples, cinnamon, sugar, and butter.

Gemma 7B#

response = ollama.generate(model='gemma:7b', prompt=json_prompt)["response"]
print(response)

```json
"apple_slices_with_cinnamon": {

  "ingredients": {
    "apples": 2,
    "cinnamon": 1/2 teaspoon,
    "butter": 1/4 cup,
    "sugar": 1/2 cup,
    "lemon_juice": 1 tablespoon
  },

  "instructions": {
    "1. Prepare the apples:** Slice the apples into thin strips.
    "2. Combine the cinnamon, butter, and sugar:** In a small bowl, mix the cinnamon, butter, and sugar. Rub the mixture between your fingers until it forms a fragrant paste.
    "3. Coat the apples:** Dip the apple slices into the cinnamon paste, coating them evenly.
    "4. Arrange on a baking sheet:** Spread the coated apple slices onto a baking sheet.
    "5. Bake:** Preheat the oven to 350 degrees Fahrenheit. Bake the apple slices for 10-12 minutes, or until golden brown.
    "6. Add lemon juice:** Once the apples are baked, squeeze the lemon juice over them.
    "7. Serve:** Serve the apple slices warm, with a sprinkle of additional cinnamon if desired.
  }
}
```

Mistral 7B#

response = ollama.generate(model='mistral:7b', prompt=json_prompt)["response"]
print(response)

 {
 "recipe": {
   "title": "Apple Slices with Cinnamon, a Delight for thee",
   "ingredients": [
     {
       "name": "Apples, ripe and red, 5",
       "quantity": ""
     },
     {
       "name": "Water, pure, in sufficient quantity, 1 cup",
       "quantity": ""
     },
     {
       "name": "Cinnamon, sweet and warming, a pinch or two",
       "quantity": ""
     },
     {
       "name": "Sweetener, of thine own choosing, to taste, 1 tbsp"
       "quantity": ""
     }
   ],
   "instructions": [
     {
       "step": "1. Gather, good apples, five in number, ripe and red.",
       "action": ""
     },
     {
       "step": "2. With a keen edge'd knife, slice each apple round and fair.",
       "action": "Slice the apples"
     },
     {
       "step": "3. In a pot of water, pure, let these slices gently swim.",
       "action": "Boil the apples in water"
     },
       {
         "step": "4. When tender they become, sweet cinnamon add, a pinch or two.",
         "action": "Add cinnamon"
     },
      {
        "step": "5. Taste the brew and add, as thou desirest, a sweetener's grace.",
        "action": "Add sweetener to taste"
      },
       {
         "step": "6. Simmer on gentle flame, till apples soften in thy pot.",
         "action": "Simmer the apple mixture"
       },
       {
         "step": "7. Serve up this delectable dish, to warm thy soul and heart.",
         "action": "Serve the apple slices with cinnamon"
       }
    ]
  }
}

Generate a Distillation Examples#

Loop through all models to generate 20 examples each.

for model in ('gemma:7b', 'mistral:7b', 'llama2:70b'):
    file_path = f"{model.split(':')[0]}responses_20.jsonl"

    with open(file_path, "w") as file:
        for i in range(20):
            print(f"{model} {i}")
            response = ollama.generate(model=model, prompt=json_prompt)["response"]
            rewrite_prompt = f"What is a recipe for {foods[i]}?"
    
            json.dump({"id":i,
                       "food": foods[i], 
                       "prompt":json_prompt,
                       "rewrite_prompt": rewrite_prompt,
                       "completion": response},
                       file)
            file.write("\n")

gemma:7b 0
gemma:7b 1
gemma:7b 2
gemma:7b 3
gemma:7b 4
gemma:7b 5
gemma:7b 6
gemma:7b 7
gemma:7b 8
gemma:7b 9
gemma:7b 10
gemma:7b 11
gemma:7b 12
gemma:7b 13
gemma:7b 14
gemma:7b 15
gemma:7b 16
gemma:7b 17
gemma:7b 18
gemma:7b 19
mistral:7b 0
mistral:7b 1
mistral:7b 2
mistral:7b 3
mistral:7b 4
mistral:7b 5
mistral:7b 6
mistral:7b 7
mistral:7b 8
mistral:7b 9
mistral:7b 10
mistral:7b 11
mistral:7b 12
mistral:7b 13
mistral:7b 14
mistral:7b 15
mistral:7b 16
mistral:7b 17
mistral:7b 18
mistral:7b 19
llama2:70b 0
llama2:70b 1
llama2:70b 2
llama2:70b 3
llama2:70b 4
llama2:70b 5
llama2:70b 6
llama2:70b 7
llama2:70b 8
llama2:70b 9
llama2:70b 10
llama2:70b 11
llama2:70b 12
llama2:70b 13
llama2:70b 14
llama2:70b 15
llama2:70b 16
llama2:70b 17
llama2:70b 18
llama2:70b 19

CSV Converter#

For use in the AI Studio Front End

import pandas as pd
file_path = "llama2responses_20.jsonl"

df = pd.read_json(file_path, lines=True)
df.to_csv(f"{file_path}.csv")
df
id food prompt rewrite_prompt completion
0 0 Apple Slices with Cinnamon What is a recipe for Apple Slices with Cinnamo... What is a recipe for Apple Slices with Cinnamon? {\n"name": "Apple Slices with Cinnamon",\n"ins...
1 1 Banana with Honey What is a recipe for Banana with Honey with th... What is a recipe for Banana with Honey? {\n"name": "Banana with Honey",\n"ingredients"...
2 2 Boiled Eggs What is a recipe for Boiled Eggs with the inst... What is a recipe for Boiled Eggs? {\n"name": "Boiled Eggs",\n"instructions": [\n...
3 3 Brown Rice What is a recipe for Brown Rice with the instr... What is a recipe for Brown Rice? ```\n{\n "name": "Brown Rice",\n "instructio...
4 4 Cherry Pie What is a recipe for Cherry Pie with the instr... What is a recipe for Cherry Pie? {\n"name": "Cherry Pie",\n"instructions": [\n"...
5 5 Corn on the Cob What is a recipe for Corn on the Cob with the ... What is a recipe for Corn on the Cob? \n{\n"name": "Corn on the Cob, Shakespearean S...
6 6 Eggplant with Olive Oil What is a recipe for Eggplant with Olive Oil w... What is a recipe for Eggplant with Olive Oil? {\n"ingredients": {\n"eggplant": 2,\n"olive oi...
7 7 Fruit Skewers What is a recipe for Fruit Skewers with the in... What is a recipe for Fruit Skewers? {\n"name": "Fruit Skewers",\n"instructions": [...
8 8 Green Beans with Butter What is a recipe for Green Beans with Butter w... What is a recipe for Green Beans with Butter? {\n"ingredients": [\n"green beans", \n"butter"...
9 9 Guacamole What is a recipe for Guacamole with the instru... What is a recipe for Guacamole? {\n"name": "Guacamole",\n"instructions": [\n"M...
10 10 Ham and Cheese Sandwich What is a recipe for Ham and Cheese Sandwich w... What is a recipe for Ham and Cheese Sandwich? {\n"name": "Ham and Cheese Sandwich",\n"instru...
11 11 Hard-Boiled Shrimp What is a recipe for Hard-Boiled Shrimp with t... What is a recipe for Hard-Boiled Shrimp? {\n"name": "Hard-Boiled Shrimp",\n"instruction...
12 12 Kale Salad What is a recipe for Kale Salad with the instr... What is a recipe for Kale Salad? ```\n{\n "ingredients": [\n {\n "name...
13 13 Lemon Juice What is a recipe for Lemon Juice with the inst... What is a recipe for Lemon Juice? {\n"title": "Lemon Juice, a Refreshing Draught...
14 14 Mango Slices with Yogurt What is a recipe for Mango Slices with Yogurt ... What is a recipe for Mango Slices with Yogurt? {\n"name": "Mango Slices with Yogurt",\n"instr...
15 15 Muesli What is a recipe for Muesli with the instructi... What is a recipe for Muesli? {\n"name": "Muesli",\n"instructions": [\n"Take...
16 16 No-Bake Oats What is a recipe for No-Bake Oats with the ins... What is a recipe for No-Bake Oats? {\n"name": "No-Bake Oats",\n"instructions": [\...
17 17 Orange Juice What is a recipe for Orange Juice with the ins... What is a recipe for Orange Juice? {\n"name": "Orange Juice",\n"instructions": [\...
18 18 Peach Slices with Honey What is a recipe for Peach Slices with Honey w... What is a recipe for Peach Slices with Honey? {\n"name": "Peach Slices with Honey",\n"instru...
19 19 Pears with Apple Cider Vinegar What is a recipe for Pears with Apple Cider Vi... What is a recipe for Pears with Apple Cider Vi... {\n"Pears with Apple Cider Vinegar": {\n"ingre...