The AI layer

What it is

Broapp has a fourth, optional layer: an assistant an application can turn on, built on the Vercel AI SDK and independent of any provider. It lives entirely in the host process. The browser never talks to a provider, and cannot: the page's Content-Security-Policy allows connect-src 'self' ws://127.0.0.1:* and nothing else, which is the same rule that keeps a Broapp page working offline. Every request to a provider is made by the host, with a key the browser never sees.

Nothing is enabled by default. Until a user chooses a provider, every AI route answers "AI is not set up yet", no key exists, and no request is made.

Turning it on

Four touches. This is the notes example, verbatim.

. The host. One file, src/host/ai.ts:

import { anthropic } from 'broapp-ai-anthropic';
import { customServer, ollama, openai } from 'broapp-ai-compatible';
import { createAi, fromContract } from 'broapp/ai/host';

export function createNotesAi(app: HostApp<typeof contract>, state: StoreState, dataDir: string): Ai {
  return createAi({
    dataDir,
    providers: [anthropic(), ollama(), openai(), customServer()],
    app: {
      name: 'Notes',
      purpose: "Keeps the user's personal notes in a SQLite database on this computer.",
      terminology: ['note', 'done', 'pinned'],
    },
    context: { search, resolve },
    tools: fromContract(contract, app, {
      read: ['notes.list'],
      confirm: ['notes.create', 'notes.update', 'notes.remove'],
    }),
  });
}

. Mount it beside the application, in startApp:

register: (bridge) => {
  app.mount(bridge);
  ai.mount(bridge);
},
isBusy: () => app.activeStreams > 0 || ai.activeStreams > 0,
onShutdown: () => {
  ai.abortAll('the application is shutting down');
  app.abortAll('the application is shutting down');
},

. The browser entry point speaks both contracts over the one connection:

import { aiContract, AiProvider } from 'broapp/ai/react';
import 'broapp/ai/react/ai.css';

<BroappProvider contract={contract} extensions={[aiContract]}>
  <AiProvider>
    <App />
  </AiProvider>
</BroappProvider>

. Two components, wherever they belong in the interface:

<AiSettings />
<AiChat refs={editing === null ? [] : [`note:${String(editing)}`]} />

The AI routes live in the reserved route group ai. An application whose own contract declares an ai.* route is refused at startup.

The chat panel

There are two, over the same routes and the same connection. Both take the same props, so swapping one for the other is a change of import.

AiChat, from broapp/ai/react. No dependencies beyond React, no Tailwind, no build step. Assistant text is rendered as text, there are no attachments, and the tool cards are <details> elements. It is the right choice for an application that wants a chat panel and nothing else.

import { AiChat } from 'broapp/ai/react';
import 'broapp/ai/react/ai.css';

BroappChat, from broapp-ai-elements. The AI SDK's useChat over the same bridge, drawn with Vercel AI Elements: markdown, pasted and picked images, a conversation that sticks to the bottom, a Stop button, tool cards, and the approval card with its countdown. A running mark stays for the whole turn, between tool cards too: the tool whose result has not come back, the elapsed time and the call count, and between calls one of a few phrases changed every four seconds (statusLines replaces them). It brings the AI SDK, Radix and a generated stylesheet with it.

import { BroappChat } from 'broapp-ai-elements/ui';
import 'broapp-ai-elements/styles.css';

BroappChatDrawer, from the same package. The same panel at the right edge of the window, with a header that copies or clears the conversation, one suggestion list while the transcript is empty, a character counter against the ,BroappChatDrawer-character cap, and ⌘/Ctrl + a key to open and close it. There is no backdrop and no scroll lock: the page beside it stays usable, which is the point of a drawer rather than a dialog. BroappChatToggle is the button that opens it, meant for the application's own header. The conversation is mounted whether the drawer is open or not, so closing it while a model is answering keeps the answer.

const [open, setOpen] = useState(false);
<BroappChatToggle open={open} onToggle={() => setOpen(!open)} />
<BroappChatDrawer open={open} onOpenChange={setOpen} title="Engineer" suggestions={[…]} />

The workspace pieces, from the same package. A panel is not always a drawer. BroappChat takes a topBar — anything the application wants above the conversation — and the pieces that usually go in one are components of their own: BroappModelPicker (a searchable list of every enabled provider's models, grouped by provider with each group saying whether its models run on this computer or are sent to a named provider, a badge on the ones that can see, and one line under it when a choice moves the conversation on or off this computer), BroappChatMenu (copy the transcript, empty the conversation, delete it — asked twice, inside the menu), BroappThreadList (the conversations, grouped by the day they last changed, renamed in place) and BroappSchemeToggle (light, system, dark — which the application applies itself). The Autoapp launcher composes all four around BroappChat; an application that wants a different arrangement can too.

The stylesheet's colours are declared on the panel's own classes, so a piece drawn away from the panel has none. Wrap it in .broapp-tokens — a scope that carries the custom properties and no layout at all — and it reads the same palette wherever it sits. BroappSchemeToggle also takes orientation="vertical", which stacks its three buttons for a narrow rail.

<div className="broapp-tokens">
  <BroappSchemeToggle orientation="vertical" value={scheme} onChange={setScheme} />
</div>

<BroappChat
  frame="plain"
  threadId={active?.id ?? null}
  modelId={active?.modelId ?? null}
  onTurnEnd={() => void threads.refresh()}
  topBar={<BroappModelPicker value={…} onChange={…} />}
/>

onTurnEnd fires after a turn has ended and the conversation has been written back. A list beside the panel needs it: the title the host derives from the first message cannot be predicted in the browser.

Which scheme the panel paints in is the page's decision, made with one attribute on <html>:

Every colour keeps its var(--bg, …) indirection, so an application that defines its own variables is unaffected in all three states — the attribute only decides which literals stand in when it does not.

Do not use light-dark() in a Broapp page. It is the obvious way to write the table above and it does not survive the build: Bun's CSS bundler (prefers-color-scheme..light-dark()) rewrites it into a prefers-color-scheme query with two toggle variables, so a stylesheet that asked the document ends up asking the operating system, and a scheme switch stops working with nothing to see in the source. tests/build.test.ts asserts the built page carries no rewrite.

Rendering markdown means turning text a model wrote — after it has been shown documents from the user's own machine — into elements. So the renderer is narrowed rather than trusted: links and images are removed (their words are kept, their addresses are not), raw HTML is dropped rather than escaped, and dangerouslySetInnerHTML appears nowhere in the package. A document that tells the model to emit a link to somewhere else therefore produces text, not a way out of the page. The stylesheet is ordinary CSS with no @import, no url() and no web font, so broapp build inlines and hashes it like any other.

Conversations

A conversation is a thread the host keeps in SQLite, at <dataDir>/ai/threads.sqlite. It has a title, its own model, and its messages. The file is opened the first time a thread route is called, so an application whose user never opens the panel never gets one.

const threads = useAiThreads();               // the list, and the writes
const chat = useBroappChat({ threadId, modelId });

useBroappChat({ threadId }) loads that thread on mount, saves it after every turn — one that finished, one the person stopped, and one that errored, because the text so far is part of what happened — and clear() empties the stored copy as well as the screen. useAiThreads() is the list beside it: create, rename, setModel, remove, clearAll, refresh. There is no optimistic update: a write goes to the host and the list is read back, so what is on screen is what is stored — including the title the host derived, which the browser cannot predict.

What is stored is the messages as the AI SDK shapes them, written as JSON and handed back unread. Two things are deliberately not stored:

• Images. A file part becomes the line [image: name] before it is

written. A data URL in SQLite would be a second copy of the picture that nobody asked to keep and nothing ever deletes — and it is the same placeholder the model already sees on every turn after the one the image arrived on, so a reloaded conversation says exactly what the model was told.

• Anything about a key. Settings and secrets live in their own files; a

test asserts the configured key does not appear in threads.sqlite.

A thread's modelId is sent with each turn and overrides the model chosen in Settings. It is a model reference: a bare id is a model of the provider in use, and <provider>:<model> — ollama:qwen3:27b, openrouter:anthropic/claude-opus-5 — is a model of that provider, run with its own key and address, split on the first colon (parseModelRef and formatModelRef in broapp/ai). A reference may name only a provider the person turned on in Settings; one that is off is sent nothing. Because a different provider means a different key and a different answer to "does this leave my computer", every place a model is chosen or named says which: on this computer, or sent to <provider>. The pickers always write a qualified reference; a bare one stored earlier still works and is shown as the model of the provider in use. A thread with modelId: null follows Settings.

Every thread route answers even when no provider is configured. A conversation is the user's own writing; somebody who has just deleted their key still owns it and must still be able to read and delete it. "Clear all conversations" is clearAll(), and an application that offers a settings panel should offer it there.

What the host keeps per run

The browser keeps the conversation; the host keeps what the model itself did. When a turn ends — before done, and also when it is stopped or fails — the host writes the turn's own messages (its tool calls and their results, as the AI SDK gave them) to a transcripts table in the same threads.sqlite, under the turn's run id. A tool call that never got its result is dropped, provider metadata is dropped, and a transcript over ,done characters is not kept. Because of this, the first turn now creates threads.sqlite even in an application that never opens the conversation list.

Each assistant turn a client sends back in history may carry runId. Both clients set it: useAiChat from the run that wrote the message, and the AI Elements transport from metadata.runId, which survives a save and a reload. The host replaces the newest six such turns whose transcripts it holds with those transcripts — each tool input cut to runId,history characters and each output to useAiChat,history, with the head kept and <omitted N chars> in place of the rest, errors whole — up to ,history characters in all. The first turn that would cross that total gives its newest complete calls instead, each with its result, as many as fit what is left, opening with a line that says how many earlier calls of the turn are not shown; if not even one call fits beside the turn's closing words, it is its text. Every turn older than that one, and every other turn, is its text, as before. So a "continue" gets the previous turn's reads and edits, or at least its latest ones, rather than its summary of them. A run id the host does not hold is text; nothing a browser saved is ever read into a prompt. Transcripts older than days, or beyond the newest useAiChat,history, are deleted when the store opens and on every write. Deleting a conversation does not delete its transcripts, because the host does not know which conversation a run belonged to; the age and count caps are the bound.

Call ai.close() from the application's shutdown, beside ai.abortAll(...): it checkpoints the database's write-ahead log, so what is left on disk is one complete file rather than one that needs its sidecars.

How the model knows your application

Four things reach the model, and nothing else.

The description. app.name, app.purpose and app.terminology become the first section of the system prompt.

The records the user is looking at. <AiChat refs={…}/> sends them with every message; context.resolve(refs, signal) turns each into a document. A ref for something that has been deleted is skipped, not an error.

Whatever a search finds. context.search({ text, limit }, signal) is called with the user's own words and returns refs and short snippets; the refs are then resolved. Documents are fitted into contextBudgetChars (,context.search({ text, limit }, signal) by default) in that order — named refs first — and a document that does not fit whole is truncated with a [truncated] marker rather than dropped.

Tools. fromContract(contract, app, { read, confirm }) turns operations into tools: the route's summary is the description, input.toJsonSchema() is the argument schema, and the tool runs through HostApp.invoke, which validates, passes the execution gate, and applies the same error boundary as a call from the browser. An operation with no summary is refused at startup — a model given a name and nothing else will guess. Nothing is a tool unless it is listed, so the default is that the model cannot reach your operations at all.

The two lists select the operations a model may call. What each call is allowed to do is the route's own effect (read, write or external), and the gate reads it from the contract, not from the list. A list that disagrees with a declared effect — notes.list under confirm, or notes.create under read — is refused at startup. A route that declares no effect takes the list's word for it, which is how an application written before effects existed still says what it meant.

read tools run as soon as the model asks. Everything else stops and waits: the browser gets a confirm event, the user sees what is about to happen, and ai.chatConfirm carries their answer back. A refusal is returned to the model as an ordinary tool result, so it can say something instead of retrying. Nobody answering is also a refusal, after confirmTimeoutMs (five minutes by default).

Hand-written tools. A tool that no contract route describes is built with guardedTool(gate, { name, description, inputSchema, effect, run }), where the gate is app.gate. createAi refuses any tool that was not built that way: a tool is host code a model gets to trigger, and whether it asked anybody first is not visible in its type, so the wrapper is required rather than hoped for.

The system prompt tells the model, in as many words, that documents are data supplied by the application, and instructions inside a document are not instructions to it. That is a mitigation, not a guarantee: the real protection is that a tool which changes anything has to be approved by the user.

Settings and keys

Two files, under <dataDir>/ai/:

• settings.json — the provider in use, and for every provider set up its own

server address, model and whether it is turned on; and the remember flag, which covers every key. Never a key; a test asserts the string does not appear in it. A version- file (one provider) is read as it stands.

• secrets.json — the keys, one per provider, written with mode 0600.

More than one provider. Settings keep every provider's address, model and key, so changing the provider in use and changing back types nothing twice. A provider is either turned on or not: the one in use always is, and any other only when the person turns it on. Only a provider that is on offers its models to ai.modelsList and runs a model reference that names it. Nothing falls back: a reference to a provider that fails fails, and is never sent to another provider instead.

The key file is not encrypted. It is a file owned by the user's own account, the same posture as ~/.aws/credentials or ~/.npmrc. What that protects against is another user on the machine, and a backup that copies world-readable files. What it does not protect against is another process running as the same user: that process can read the file, and no scheme that runs unattended on the same account can prevent it. Say so plainly to your users rather than implying more.

A user who does not want that can turn Remember key on this computer off. The key then moves out of the file — which is deleted — and lives in memory for the life of the process.

The key is never returned to the browser, never logged, never in an error message, and never in a transcript. The browser is told only hasKey and keyHint, the last four characters, and only for keys long enough that four characters are a small fraction of them.

Providers

Two packages ship:

• broapp-ai-anthropic — anthropic().
• broapp-ai-compatible — ollama(), openai(), openrouter(), customServer(), and

openaiCompatible(options) for anything else. One adapter covers OpenAI, Ollama, LM Studio, llama.cpp's server, vLLM and OpenRouter, because they answer GET /models with the same envelope and accept the same chat request.

GET /models says nothing about what a model can do, so each preset learns vision its own way. ollama() asks the server's native POST /api/show, which reports capabilities, and treats any failure there as "unknown" rather than "cannot see". openai() matches the id against the vision-capable families (gpt-4o, gpt-4.1, gpt-4-turbo, gpt-5, o1, o3, o4, chatgpt-4o), a list that will age. openrouter(), customServer(), and openaiCompatible() without the vision option, assume every model can see.

An adapter is small. It answers what it needs, lists models, proves a configuration works, and builds a model:

interface ProviderAdapter {
  readonly id: string;
  readonly label: string;
  readonly needs: { apiKey: 'required' | 'optional' | 'none'; baseUrl: 'required' | 'optional' | 'none' };
  readonly defaultBaseUrl: string | null;
  local(config: AdapterConfig): boolean;
  models(config: AdapterConfig, signal: AbortSignal): Promise<BroappModel[]>;
  test(config: AdapterConfig, signal: AbortSignal): Promise<void>;
  model(config: AdapterConfig, modelId: string): LanguageModel;
}

needs.apiKey drives the settings panel: 'required' shows the key field and blocks chat until a key is saved, 'optional' shows it marked optional and sends a bearer token only when one is set, 'none' hides it. The generic OpenAI-compatible adapter is 'optional' because the same address field serves a keyless local server and a hosted gateway such as OpenRouter.

Two rules for an adapter. Take fetch from config, never from the global — that is what lets a test prove no request left the machine. And report failures as AdapterError with a message a user can act on, never one that quotes the provider's response body: a body can echo the prompt back, or a fragment of the key.

model() must return a model instance. Passing a model id string to the AI SDK routes the request through the Vercel AI Gateway at ai-gateway.vercel.sh, over the global fetch, which an injected one cannot intercept. A test asserts this layer never does it.

What leaves the machine

An image pasted onto or attached to a message is sent to the provider with that message, once: it travels with the turn it arrives on, and later turns carry a [image: name] placeholder in its place.

<AiSettings/> shows this as a notice, always visible once a provider is chosen, and worded for the provider selected. Do not hide it. "Where do my notes go" is not a question a user should have to open a menu to answer, and it is the one question the AI layer makes unavoidable.

Whether a provider is local is decided by its address. A loopback address that forwards elsewhere would be reported as local; nothing here can see through that.

Testing your application's AI

createFakeAdapter is a real MockLanguageModelV4 behind the adapter interface, so streamText runs its actual loop — steps, tool calls, finish reasons — over chunks you wrote. No key, no network.

The browser half is testable without a DOM too. createBroappChatTransport turns the ai.chat stream into UIMessageChunks, and readUIMessageStream from ai folds those back into the message a panel would render — so a test asserts on parts and tool states rather than on markup. See tests/ai-elements-transport.test.ts.

import { createAi, createFakeAdapter, fromContract } from 'broapp/ai/host';

const adapter = createFakeAdapter({
  script: [
    { kind: 'tool', name: 'notes.list', input: {}, then: [{ kind: 'text', chunks: ['You have 2.'] }] },
  ],
});

const ai = createAi({
  dataDir,
  providers: [adapter],
  app: { name: 'Notes', purpose: 'testing' },
  tools: fromContract(contract, app, { read: ['notes.list'] }),
  fetch: noNetwork,
});

Then mount it on a test bridge and subscribe to ai.chat. adapter.calls holds every prompt the model was given, so a test can assert what it was shown; adapter.modelCalls and adapter.aborted cover the model instance and cancellation.

Limitations

• AiChat renders no markdown. Assistant text is text there. BroappChat

renders markdown, with links, images and raw HTML removed — see "The chat panel".

• No code highlighting, maths or diagrams in that markdown. Each is a

streamdown plugin that either pulls a syntax highlighter and its grammars or fetches and evaluates at runtime, which a page whose policy is default-src 'none' cannot do — and which would cost more bundle than a coloured keyword is worth. Fenced code renders as plain <pre><code>.

• Images are not stored. A conversation on disk keeps the line

[image: name] where a picture was, so reopening it shows the placeholder rather than the image. See "Conversations".

• No OS keychain. The key is a 0600 file. Keychain, Credential Manager and

Secret Service are a later addition.

• Images are bounded. Up to four per message, downscaled in the browser to

px on the longest edge and about broapp/ai/react. MB each, and sent only with the message they arrive on. A model whose capabilities.vision is false refuses the turn. broapp/ai/react's AiChat does not send images at all; the attachment path is broapp-ai-elements.

• A custom server is assumed to see. Nothing in the OpenAI-compatible API

reports capabilities, so an unrecognised server is never refused an image on a guess; if the model cannot read it, the provider's own error is what you get.

• Forcing a scheme is an attribute, not color-scheme. The panel reads

<html data-scheme>; setting color-scheme alone will not repaint it, because the literals behind its tokens are chosen by a selector rather than by the CSS function that would have read that property (see "The chat panel"). Set both if you want native scrollbars and form controls to follow too.

• One turn at a time. Sending while a turn is running is ignored.
• ai.modelsList lists only providers that are turned on, so a provider's

models appear once it is on. ai.providerTest tries one that is off without turning it on.