Streaming

The transport answers with a stream of events, listed beside the chat as the session applies them: BitAiMessageStartedEvent; a BitAiMetadataEvent naming the model and its context window, merged into the reply's metadata (the model name under the reply comes from it); a BitAiTextDeltaEvent per chunk of the answer; a BitAiUsageUpdatedEvent with the tokens it used; and BitAiMessageCompletedEvent.

Each text delta is appended to the reply's text part (BitAiTextPart, which stays IsStreaming until the reply completes). The Markdown is parsed again on every delta, but only its tail is: everything before the last confirmed block boundary is kept as it was, so a long answer costs the same per token at its end as at its start.

What is still open is drawn so it never flickers: a fenced code block is left unhighlighted until its closing fence arrives, and an unmatched ** is hidden rather than shown as a stray asterisk until its partner comes.

Only the part the text goes into re-renders as it streams, and at most once per 50 ms (BitAiUiOptions.RenderInterval) however many deltas arrive in between; the rest of the transcript is not drawn again for them, and the chat around it renders as the turn starts and ends. Streaming faster than that changes nothing a reader can see, and on Blazor Server every render is a diff sent down the circuit, so it costs real bandwidth.

Providers deliver text in bursts: nothing for a moment, then a paragraph. Drawn as it comes, that reads as stutter. So a session paces it: BitAiChatSessionOptions.Pacing is on by default (BitAiTextPacingOptions.Default), buffering the text deltas and releasing them at a steady rate - a chunk every Interval (16 ms), sized to drain what is buffered over TargetDrain (150 ms), between MinCharsPerTick (1) and MaxCharsPerTick (48) characters. Every other event passes through in order, behind the text that came before it; a terminal error is let through at once. Pacing adds no delay when the model is slower than the pacer, which is the common case, and never holds the end of a reply back by more than one drain window.

The text deltas counted beside the chat are the paced ones. Set Pacing to null to render the deltas exactly as the transport produces them, change it for every session with ai.ConfigureSessions, or pace a stream consumed outside a session with the same operator, BitAiEventStreamExtensions.Paced (stream.Paced(options, cancellationToken)) - the Text pacing tab shows all three.

A message is an ordered list of parts, not a string: text, reasoning, code, images, files, tool calls, artifacts, structured data, each an IBitAiMessagePart with a PartId and a Kind that picks its view. A part that contributes words to the message's plain text - what copy and export use, and what a transport reads off the history with IBitAiMessage.GetText() - is an IBitAiTextualPart; one that grows while the reply streams is an IBitAiStreamablePart, whose IsStreaming lets its view skip expensive work (highlighting a half-written code block) until it is complete.

The events that fill a part are BitAiPartScopedEvents. A delta - text, reasoning, structured data - with no PartId, as this transport sends them, goes into the newest open part of its kind, created if there is none; a transport that interleaves several parts of one kind gives each delta the id of its part. BitAiPartStartedEvent(kind) opens a part explicitly, for what a delta cannot say: a code part's language and file name (BitAiMessagePartKinds.Code with the language and fileName Attributes), a Markdown part, a structured part's schema. BitAiPartCompletedEvent closes one, so it is drawn finished - a code part highlighted - before the whole reply is.

The other events each have a scenario of their own: reasoning (Reasoning), images and files (Attachments), citations and status lines (Citations), artifacts (Artifacts), structured data (Structured output), generated components (Generative UI), tool calls (Tool calling), agent runs (Multi-step agent) and errors (Errors and retry).

A session applies every event through its Processor, a BitAiConversationStreamProcessor: the reducer that turns events into parts, tool calls, artifacts and agent runs on the conversation, and the only code that knows the event vocabulary. Its EventApplied is raised for every event once it has been applied, on the thread that applied it - the log beside the chat is a component listening to it (AiEventLog.razor), drawn through a BitAiRenderScheduler, the same coalescer the AI components render with, so it renders at most once every 100 ms and the chat is not drawn again for it.

The reducer is usable on its own: new BitAiConversationStreamProcessor(conversation), then BeginGeneration(), Apply(evt) for each event and EndGeneration() replay recorded events into a conversation - to test a renderer, or to show a reply a server pushed rather than one a session asked for.

Information the framework has no event for travels as a BitAiCustomEvent(name, data), its name namespaced (acme:quota) so it never collides with another. The reducer applies nothing for it, but EventApplied still raises it, so the app handles it there - a quota shown in the page, or a custom part added to the reply (Processor.Current) and drawn by a renderer registered for its kind (see BitAiMessagePartView). The The reducer tab shows both.

• BitAiChat - the chat the answer streams into.
• BitAiTextPartView and BitAiStreamingMarkdown - the text part, re-parsed only at its tail while it grows.
• BitAiMarkdownView - the parsed document: headings, tables, quotes.
• BitAiCodeBlockView - the fenced code block, with its copy button.