Streaming from a server

The reply crosses the network in the package's wire format: each BitAiEvent is one JSON object told apart by its type ("text.delta", "tool_call.started", ...), and the request is the transcript as the same message snapshots a saved conversation holds. BitAiWireSerializer reads and writes both, through metadata generated at compile time, so it is safe to trim and to compile ahead of time on both ends. A reader skips what it does not know - a property, or a whole event of a newer type - so a server can be updated before its clients.

1. The browser's session hands each request to a BitAiSseTransport, which posts it to the server and reads the events from the text/event-stream response as they arrive.
2. MapBitAiSse maps that endpoint. It reads the request back into live messages and streams whatever its handler produces - here StreamBitAiEventsAsync of the server's IChatClient, which turns the chat client's updates into events: text, reasoning, the function call and its result, the usage, the model id and the finish reason.
3. The function here runs on the server, in the chat client's FunctionInvokingChatClient, and the browser shows its call and its result. A tool registered in the browser (BitAiRequest.Tools) is declared to the model without being invocable on the server, so its call comes back to the session, which runs it and asks again with the result.

The request comes from the browser, so it never overrides the server: the chat options the server passes win, and the request only fills in what they leave unset. Pin the model and the limits on the server. A system prompt the browser sends is dropped, and so is a tool choice forcing a tool it did not declare: the instructions and the server's functions are the server's alone. The endpoints read a JSON body only, so a page of another site cannot post to them as a user signed in by cookie without passing the app's CORS policy. A failure on the server, or an event the wire format cannot write, ends the stream with a terminal error event and a generic message - the exception is logged, never sent. The model behind this page is scripted, so it costs nothing to call; an app registers its provider's chat client instead, and nothing else changes.

Each end installs only what it runs, so nothing of the server reaches the browser and nothing of the UI reaches the server:

• The Blazor app installs Bit.BlazorUI.AI, which brings Bit.BlazorUI.AI.Core with the SSE and WebSocket transports. Bit.BlazorUI.AI.SignalR adds the SignalR transport, and Bit.BlazorUI.AI.ChatClient an IChatClient called right where the components run (Blazor Server, Hybrid).
• The server installs Bit.BlazorUI.AI.Server - the SSE and WebSocket endpoints, the hub, the approval broker - and Bit.BlazorUI.AI.ChatClient to stream an IChatClient's reply. Neither has a component, a stylesheet or a script.

MapBitAiWebSocket maps the same handler on a WebSocket endpoint, and UseWebSocketTransport (or a BitAiWebSocketTransport) opens one socket per reply to it: the request goes up as the first message, the events come down one message each, and the server closes the socket after the last one. Stopping the reply closes the socket, which cancels the handler. The address has to be absolute - the transport takes the app's http or https base address and speaks ws or wss.

What it adds is the way back: a decision on a tool call the server holds goes up the very socket the call came down on, so it reaches the server instance holding the call without sticky sessions, and only that client can answer it. A browser sends its cookies with a WebSocket from any page and CORS does not apply, so an endpoint authenticated by cookie lists the origins it serves (BitAiWebSocketEndpointOptions.WebSockets).

The same reply streams over a SignalR connection the app already has: derive a hub from BitAiHub and produce the reply in its StreamAsync, then register UseSignalRTransport in the browser with the connection to it. The AiHub.cs tab shows the hub. The request and the events cross the hub as JSON values the package writes itself, so the hub protocol's settings change nothing.

All three transports implement IBitAiApprovalTransport: when the server streams a BitAiToolApprovalRequestedEvent for a call it runs, the user's decision is sent back - to the endpoint of MapBitAiApprovals, up the WebSocket, or to the hub - and handed to the stream waiting for it through a BitAiApprovalBroker (await approvals.WaitAsync(toolCallId, cancellationToken)), in the same reply. The endpoints and the hub tell the broker about the call before they send its event, so a decision that arrives before the reply gets to WaitAsync is kept for it. The WebSocket and the hub take a decision only on a call their own connection was asked about, and only once.

• BitAiChat - the chat surface over the session.
• BitAiToolCallView - the card of the call the server ran.
• BitAiReasoningPartView - the reasoning the chat client streamed.