Skip to main content
Full technical details in your project at /docs/modules/AI.md

What You Get

  • Proxy architecture - API keys server-side only (secure)
  • Streaming SSE - Real-time token-by-token responses
  • Config-gated - Auto-selects Proxy or Echo client from PROXY_BASE_URL
  • OpenRouter backend - One Edge Function reaches OpenRouter’s 500+ models; the shipped function allowlists a small set you expand
  • Cancellation support - Stop generation mid-stream (structured-concurrency cancellation)
  • Echo fallback - Test UI without API costs
Time saved: 24-40 hours of implementing streaming, proxy setup, error handling, and testing.
Module coupling. The LLMClient protocol and LLMMessage type live in the FeatureChat package, and the AI package re-exports FeatureChat (@_exported import FeatureChat). If you remove chat, move these two types first or Packages/AI will not compile. See App Store 4.3 hardening for the per-module removal steps.

What’s new in v2.2

  • ProxyLLMClient’s public API is unchanged. The request builder and SSE stream parser are standalone files under Sources/AI/Clients/Proxy/, so you can swap the wire format without rewriting the client.
  • Swift 6 strict concurrency throughout: protocol-typed properties use explicit any.

Key Components

LLMClient Protocol

The protocol is defined in FeatureChat (re-exported by the AI package). It has a single requirement that takes just the conversation history:
ProxyLLMClient additionally offers a fully-parameterised overload with model: and temperature: (the protocol method forwards to it with nil defaults):
LLMMessage is a simple value type — role is a String ("user", "assistant", "system"), not an enum:

Production Setup (From Real Code)

The boilerplate selects the LLM client in SwiftAIBoilerplatePro/Composition/LLMClientFactory.swift, reading the generated AppConfiguration (built from Config/Secrets.xcconfig). It returns EchoLLMClient until a real PROXY_BASE_URL is configured:
CompositionRoot wires it up with self.llmClient = createLLMClient(httpClient: httpClient). The default generated PROXY_PATH is /ai (the Edge Function route); EchoLLMClient itself lives in this same factory file, not in the AI package.

Why This Architecture

Proxy Pattern (Production):
  • ✅ API keys never in app (stored in Supabase secrets)
  • ✅ Authentication via JWT (automatic via AuthInterceptor)
  • ✅ Backend controls costs and rate limits
  • ✅ Can switch models without app update
Echo Fallback (Development):
  • ✅ Test UI without backend setup
  • ✅ No API costs during development
  • ✅ Perfect for rapid iteration
  • ✅ Auto-enabled until PROXY_BASE_URL is set to a real URL (placeholder/YOUR values still use Echo)

Streaming Pattern

Benefits:
  • ✅ Low latency (first token quickly)
  • ✅ Better UX (gradual appearance)
  • ✅ Cancellable (stop generation)
  • ✅ Memory efficient

Supported Models

OpenRouter exposes 500+ models. For cost control, the shipped Edge Function only forwards an explicit allowlist — anything else is rejected with 400 Invalid model:
To use Anthropic, Google, Meta, or any other OpenRouter model, add its slug to ALLOWED_MODELS, redeploy the function, then pass model: from the client:

Supabase Edge Function

The proxy keeps API keys server-side and enforces auth, entitlement, and rate limits before any upstream cost is incurred:
Deployment:

Customization Examples

Add a System Prompt

role is a plain String — use "system", "user", or "assistant":
The shipped Edge Function injects its own server-controlled SYSTEM_PROMPT and only accepts user/assistant roles from clients (ALLOWED_ROLES). Client-supplied system messages are only meaningful for a direct client you write yourself (the local EchoLLMClient ignores them and simply echoes the last user message). To change the production persona, edit SYSTEM_PROMPT in supabase/functions/ai/index.ts.

Add a Direct LLM Provider

The LLMClient protocol lives in FeatureChat (re-exported by import AI). Conform to it to bypass the proxy:

Security

API keys never in client:
  • ✅ Edge Function holds OpenRouter key
  • ✅ User auth required for proxy access
  • ✅ Rate limiting at Edge Function
  • ✅ No keys in client code
  • ✅ Messages not logged by proxy

Key Files

Dependencies

  • Core - Error handling, logging
  • Networking - HTTP client for proxy
  • FeatureChat - owns LLMClient / LLMMessage; the AI package re-exports it

Used By

  • App target - LLMClientFactory builds the proxy/echo client
  • Custom features - Your AI features
The boilerplate ships no LLM. The optional App Generator (a separate macOS product, coming soon) drives your own coding-agent CLI to scaffold a differentiated app from this template — it does not bundle a model either.

Best Practices

  • Use AsyncThrowingStream
  • Handle cancellation
  • Update UI incrementally
  • Show loading state
  • Map to AppError
  • Retry transient failures
  • Show user-friendly messages
  • Log technical details
  • Choose appropriate model
  • Use mini/flash for simple tasks
  • Cache system prompts
  • Limit message history

Learn More

Full Documentation

Complete AI guide

Supabase Setup

Deploy Edge Function

Feature Chat

See AI integration

Building Guide

Customize AI experience

Test Coverage

The AITests target covers the proxy client end to end — request building, SSE stream parsing, error mapping, and cancellation (Tests/AITests/Proxy/). It runs as part of the workspace suite: ~598 tests across 12 package test targets + the app test suites, in one Boilerplate.xctestplan run. Tests include:
  • Request building (headers, path, body)
  • SSE stream parsing and [DONE] termination
  • Error mapping to AppError
  • Cancellation