Skip to main content

Kit: AI assistant Pro

@kit/ai gives a kit app an assistant that reads and changes the app's own data. Penny (the budget template) ships it: ask "Where did my money go this month?" and it answers with the real numbers and a chart card; ask it to move money into a goal and it shows an approval card first.

  • Any model, by env. AI_MODEL=provider:model. Changing providers never changes code.
  • Keys stay on the server. The app talks to one handler you mount on an Expo API route (the default), a Supabase Edge Function, a Cloudflare Worker or your own server. It checks the session, rate-limits, caps tokens and owns the system prompt and the tool list.
  • Tools run on the device, through the app's repository with the signed-in person's session, so the assistant can never do more than they can, on the mock, HTTP and Supabase backends alike.
  • Writes ask first. Anything that can't be undone waits on an approval card; a reversible change (a budget's limit) happens at once with Undo in a toast.
  • A mock that's the real thing. With the mock backend, a scripted model runs the same streamText, tools, cards and approvals on the device: no key, no server. The docs demos run on it.
  • On the device: the conversation (last 50 messages, per person), and a Settings switch to turn AI off or clear the conversations.

Set it up​

The kit app already has the packages (ai, @ai-sdk/react, zod, and the provider packages, imported only on the server). In patine.config.js:

ai: {
enabled: true, // false removes every way in
adapter: 'http', // 'mock' answers on the device; default: 'mock' on the mock backend, else 'http'
url: undefined, // default: /api/ai on the web; on a phone, the dev server's in development
provider: 'OpenRouter', // named in the first-use note: where questions go
fab: true, // a floating Ask button in the bottom end corner, above the tab bar on phones
},

EXPO_PUBLIC_AI=mock|http and EXPO_PUBLIC_AI_URL override it without editing the file. With adapter: 'http' the web build becomes a server build (web.output: 'server'), which EAS Hosting or the expo-server adapters serve; the mock stays a static page.

Then the server's env (never EXPO_PUBLIC_):

AI_MODEL=openrouter:openai/gpt-4o-mini
OPENROUTER_API_KEY=sk-or-…

Providers​

AI_MODELNeedsNotes
openrouter:<vendor/model>OPENROUTER_API_KEYOne key, every major model. The easy start
ollama:<model> (ollama:llama3.2)Ollama running locallyNo key. AI_BASE_URL if it isn't http://localhost:11434/v1. Pick a model that calls tools
openai:<model>OPENAI_API_KEY
anthropic:<model>ANTHROPIC_API_KEY
google:<model>GOOGLE_GENERATIVE_AI_API_KEY
mistral:<model>MISTRAL_API_KEY
compat:<model>AI_BASE_URL, AI_API_KEYAny OpenAI-compatible server: LiteLLM, vLLM, Groq, Together, a gateway

AI_MODEL_FAST (optional) runs one-shot tasks such as categorizing transactions. A missing key fails when the server starts, with the variable to set. Delete the providers you don't use from kit/core/src/ai/server/models.ts and their packages.

The server​

createAiHandler({ assistant }) is a (Request) => Response. The Expo API route is three lines (app/api/ai+api.ts):

import { pennySchema } from '@budget/ai/assistant';
import { createAiHandler } from '@kit/ai/server/createAiHandler';

export const POST = createAiHandler({ assistant: pennySchema });

A Supabase Edge Function or a Cloudflare Worker mounts the same handler; call its URL from the app with ai.url (Supabase's functions.invoke doesn't stream, the kit's fetch does):

// supabase/functions/ai/index.ts
const env = Deno.env.toObject();
Deno.serve(
createAiHandler({ assistant: pennySchema, env, verifyUser: verifyWithSupabase(env.SUPABASE_URL, env.SUPABASE_ANON_KEY) })
);

// Cloudflare Worker
export default { fetch: (request, env) => createAiHandler({ assistant: pennySchema, env })(request) };
EnvDefaultWhat
AI_RATE_PER_MIN10Requests a minute per person; over it answers 429 with when it resets
AI_DAILY_TOKENS200000Tokens a day per person, from each reply's usage
AI_MAX_OUTPUT_TOKENS1500Per reply
AI_ALLOW_MOCK_SESSIONSoff1 lets the mock backend's sessions in, to try a real model against the demo data locally. Never in production

(verifyWithSupabase and verifyWithBackend are in @kit/ai/server/verify.) Who's asking comes from the request's Authorization: Bearer <kit session>: Supabase Auth when EXPO_PUBLIC_SUPABASE_URL is set, else your REST backend's GET /me, or pass verifyUser. 401 signs the person out in the app; 429 shows the limit notice. The limits live in the process's memory: on serverless or several instances, keep them in KV, Redis or a table. Set a spending limit at the provider too.

Tools​

An assistant is two files, so the server never bundles the app:

  • src/ai/assistant.ts: the instructions and each tool's description, input (zod) and kind. Imports only zod. The server reads this one.
  • src/ai/assistant.client.ts: defineAssistant({ ...schema, tools: { name: { ...schema.tools.name, run } } }), plus the mock's scripts.
set_budget: {
...pennySchema.tools.set_budget, // kind: 'undoable'
run: async ({ category, limit }, ctx) => { /* ctx.repo.saveBudget(…) */ return { name, before, after } },
undo: (out, ctx) => ctx.repo.saveBudget(/* before */),
done: (out, ctx) => ctx.t('assistant.budgetSet', out), // the Undo toast
},
contribute_goal: {
...pennySchema.tools.contribute_goal, // kind: 'write'
summary: async ({ goal, amount }, ctx) => ({ title: 'Add $50 to Japan', changes: [{ label: 'Japan', before: '$3,420', after: '$3,470' }] }),
run: async ({ goal, amount }, ctx) => ctx.repo.contribute(…),
},

kind is 'read' (runs at once), 'undoable' (runs at once, Undo toast; needs undo) or 'write' (approval card from summary, runs on Approve). Tool outputs go back to the model, so return text it can read (formatted amounts) and only the fields it needs. Cards are React components by tool name, passed to AssistantProvider's cards; they draw from the tool's output or the app's own queries.

On wide screens (tablets, desktops) the floating button, the top bar's sparkle and ⌘K open a compact window in the button's corner (380 × 560, the card's radius and shadow, kept on screen) with no scrim, so the app beside it stays usable. Its header has Expand (the side sheet, same conversation), New chat and Close, which minimises it back to the button; Escape does too, focus goes to the message box as it opens and back to the button as it closes, it's a dialog named by the title, and it grows from its corner (instantly with reduce motion). On phones the button opens the full-screen sheet. The window is ChatWindow in @kit/ai, for any conversation. The button sits a theme spacing step in from the edges (xl on wide screens, lg on phones, plus the safe area and the tab bar).

The floating Ask button hides while the assistant is open; sheets and dialogs cover it, and a screen whose primary action sits at the bottom (a selection bar) calls useHideAskFab(active). The message box takes focus when the assistant opens on the web and on wide screens. The app's starter suggestions stay under every reply (the ones not asked yet) until the person types.

Wrap the app's routes in AssistantProvider (def, ctx, cards, title, suggestions); useAi() opens it, AskButton is the sparkle, and ?ask=1 (or ?ask=<suggestion key>) opens it from a link. useTask(def, 'categorize', ctx) runs a one-shot task with a typed answer.

The mock​

Each script matches the person's words, calls real tools on the mock repository, then fills its reply from their outputs, so the numbers match the screens:

{
match: /money go|לאן/i,
calls: () => [{ tool: 'spending_summary', input: { groupBy: 'category' } }],
reply: ([out], { t }) => t('assistant.mock.spending', { total: out.total, … }),
}

It types at a person's reading pace (MOCK_PACE: a 400–700 ms pause, then about 45 characters a second by words), and a card shows after its tool has run. Real providers stream through the SDK's smoothStream, word by word. Anything else gets the fallback. ?mock=error fails (Retry), ?mock=loading never answers, ?mock=empty makes the tools find nothing, ?mock=limit shows the daily-limit notice; Settings › Developer has the same. src/ai/assistant.test.ts runs every suggestion in English and Hebrew through the mock to its answer and checks the write stops on approval.

Security checklist​

  • Keys only in the server's env; a test fails if a provider package is imported outside ai/server or app/api.
  • The server owns the prompt and the tools; the client sends messages and a checked context (language, currency, time zone, date).
  • Writes run on the device after Approve, with the person's own session; reversible ones offer Undo.
  • Tool outputs are data: the instructions say so, and merchant names, notes and reviews are never instructions.
  • Rate and token limits per person; a spend limit at the provider.
  • Conversations stay on the device; Settings turns AI off and clears them; the first-use note names the provider.