Fun with local decision models

MacOS ships with a completely local LLM model made by Apple for (system) apps to use. Apfel makes this model accessible as a CLI, awesome! I use it for example for quick translations:

apfel "translate into English: De appel valt niet ver van de boom."

It is a small model though (but fast and doesn't burn up any tokens), according to the Apfel site it can take about 3000 words in context.

Another example, creating a commit message:

git status -v | apfel "write a commit message, one sentence"

Check out the Apfel site for more examples!

One other thing a small model like this can be useful for is quick decision making, like a local (naive) version of Jev.

First, define and save a schema:

{
  "type": "object",
  "properties": {
    "choice": { "enum": ["backend", "frontend", "database"] },
    "confidence": { "type": "number" }
  },
  "required": ["choice", "confidence"]
}

Then pipe any content into Apfel with this schema:

echo "Our checkout has returned 500 errors since 9am." | apfel --schema jev.json "Which label fits this issue?"

And voila!

{"confidence": 0.8, "choice": "backend"}