TIL that Apple provides on-device LLM models: https://developer.apple.com/documentation/foundationmodels On iOS you can interact with it via apps like Locally by LM Studio. On MacOS you can use `apfel`: https://apfel.franzai.com And you can serve an OpenAI-compatible endpoint with:

apfel --serve