Skip to main content
Full technical details in your project at /docs/modules/AI.md

What You Get

  • Proxy architecture - API keys server-side only (secure)
  • Streaming SSE - Real-time token-by-token responses
  • Environment-gated - Auto-selects Proxy or Echo client
  • 500+ models - OpenAI, Anthropic, Google, Meta, DeepSeek
  • Cancellation support - Stop generation mid-stream
  • Echo fallback - Test UI without API costs
Time saved: 24-40 hours of implementing streaming, proxy setup, error handling, and testing.

Key Components

LLMClient Protocol

protocol LLMClient: Sendable {
    func streamResponse(
        messages: [LLMMessage],
        model: String?,
        temperature: Double?
    ) -> AsyncThrowingStream<String, Error>
}

Production Setup (From Real Code)

Here’s how the boilerplate actually creates the LLM client in CompositionRoot.swift:
// Environment-gated client selection
private func createLLMClient(httpClient: HTTPClient) -> LLMClient {
    let env = ProcessInfo.processInfo.environment
    let proxyBaseURL = env["PROXY_BASE_URL"]
    
    // If no proxy URL → use Echo client (testing)
    guard let baseURLString = proxyBaseURL,
          let baseURL = URL(string: baseURLString) else {
        AppLogger.info("LLM provider: EchoLLMClient", category: AppLogger.ai)
        return EchoLLMClient()
    }
    
    // Parse configuration
    let proxyPath = env["PROXY_PATH"] ?? "/v1/chat/stream"
    
    // Create production proxy client
    let proxyClient = ProxyLLMClient(
        baseURL: baseURL,
        httpClient: httpClient,  // Uses AuthInterceptor automatically
        path: proxyPath,
        defaultHeaders: [:]
    )
    
    AppLogger.info("LLM provider: ProxyLLMClient", category: AppLogger.ai)
    return proxyClient
}

// Usage in composition
self.llmClient = createLLMClient(httpClient: httpClient)

Why This Architecture

Proxy Pattern (Production):
  • ✅ API keys never in app (stored in Supabase secrets)
  • ✅ Authentication via JWT (automatic via AuthInterceptor)
  • ✅ Backend controls costs and rate limits
  • ✅ Can switch models without app update
Echo Fallback (Development):
  • ✅ Test UI without backend setup
  • ✅ No API costs during development
  • ✅ Perfect for rapid iteration
  • ✅ Auto-enabled when PROXY_BASE_URL empty

Streaming Pattern

// In ViewModel
for try await chunk in llmClient.streamResponse(messages: messages) {
    // Update UI with each token
    currentResponse += chunk
}
Benefits:
  • ✅ Low latency (first token quickly)
  • ✅ Better UX (gradual appearance)
  • ✅ Cancellable (stop generation)
  • ✅ Memory efficient

Supported Models

Access any model from openrouter.ai/models:

OpenAI

  • openai/gpt-4o ($2/1M tokens)
  • openai/gpt-4o-mini ($0.12/1M tokens)

Anthropic

  • anthropic/claude-3.7-sonnet ($2.50/1M)
  • anthropic/claude-3-5-haiku ($0.50/1M)

Google

  • google/gemini-2.5-pro ($1.25/1M)
  • google/gemini-2.0-flash ($0.40/1M)

Meta

  • meta-llama/llama-3.3-70b ($0.60/1M)
  • Open source, cost-effective

Supabase Edge Function

The proxy keeps API keys secure:
// supabase/functions/ai/index.ts
Deno.serve(async (req) => {
  // 1. Verify user auth
  // 2. Extract messages from request
  // 3. Call OpenRouter with server-side key
  // 4. Stream response back to client
})
Deployment:
supabase secrets set OPENROUTER_API_KEY=sk-or-v1-YOUR_KEY
supabase functions deploy ai

Customization Examples

Change AI Model

// In ChatViewModel or per-conversation
let stream = llmClient.streamResponse(
    messages: messages,
    model: "anthropic/claude-3-5-sonnet",  // Change model
    temperature: 0.7
)

Add System Prompt

// Add persona to conversation
let systemMessage = LLMMessage(
    role: .system,
    content: """
    You are a professional Swift developer.
    Provide clear, concise answers with code examples.
    """
)

let messages = [systemMessage] + conversationHistory + [userMessage]

Add Direct LLM Provider

// Want to call OpenAI directly?
class OpenAIClient: LLMClient {
    func streamResponse(messages: [LLMMessage]) 
        -> AsyncThrowingStream<String, Error> 
    {
        // Direct OpenAI API integration
        // No proxy needed
    }
}

// Update CompositionRoot
self.llmClient = OpenAIClient(apiKey: ...)

Security

API keys never in client:
  • ✅ Edge Function holds OpenRouter key
  • ✅ User auth required for proxy access
  • ✅ Rate limiting at Edge Function
  • ✅ No keys in client code
  • ✅ Messages not logged by proxy

Key Files

ComponentLocation
ProtocolPackages/AI/Sources/AI/LLMClient.swift
Proxy ClientPackages/AI/Sources/AI/Clients/ProxyLLMClient.swift
Echo ClientPackages/AI/Sources/AI/Clients/EchoLLMClient.swift
Edge Functionsupabase/functions/ai/index.ts

Dependencies

  • Core - Error handling, logging
  • Networking - HTTP client for proxy

Used By

  • FeatureChat - Chat ViewModels
  • Custom features - Your AI features

Best Practices

  • Use AsyncThrowingStream
  • Handle cancellation
  • Update UI incrementally
  • Show loading state
  • Map to AppError
  • Retry transient failures
  • Show user-friendly messages
  • Log technical details
  • Choose appropriate model
  • Use mini/flash for simple tasks
  • Cache system prompts
  • Limit message history

Learn More

Test Coverage

91%+ - Comprehensive testing Tests include:
  • Streaming responses
  • Error handling
  • Cancellation
  • Model selection
  • Timeout handling
I