Tool calling

The transport does not run the tool. On the first request it shows a status line while the model decides (BitAiStatusEvent, which the call then replaces), asks for the tool with BitAiToolCallStartedEvent, streams its arguments as BitAiToolCallDeltaEvents, closes them with BitAiToolCallCompletedEvent and finishes with BitAiFinishReason.ToolCalls. The session then resolves the tool from its registry, checks the approval policy, runs it, records the result on the call (BitAiToolCallPart.Result, a BitAiToolOutcome), and asks the model again. The transport reads the result off the message it is handed on that second request and answers from it, which is what lets an app decide where tools run without the transport knowing. The reply stays one message, busy from the first request to the last.

Arguments stream in as raw JSON and are parsed leniently on every delta, so an open card fills in key by key before the closing brace arrives rather than waiting for valid JSON. The cards here start collapsed (BitAiUiOptions.CollapseToolCalls) and these arguments arrive in a fifth of a second; it shows with a model writing a long argument, such as the content of a file.

A tool is an IBitAiTool: a Definition and an InvokeAsync(BitAiToolInvocation, CancellationToken) that returns a BitAiToolOutcome. The BitAiToolDefinition is what the model is told and what the card shows: the Name the model calls, a Description it reads when deciding to, the ParametersSchema of its arguments (a JSON Schema), and for the UI a Title, an Icon, its Risk and whether it RequiresApproval (Tool approval). One instance serves every conversation, so a tool keeps no state of its own. A failure is best returned as BitAiToolOutcome.Failure(...); a tool that throws anyway is logged, and its call gets a failed result whose message is the exception's, retryable, with the exception kept on the error.

BitAiTool.Create builds one from a delegate, where it is registered, deriving the schema from the argument type:

• Create<TArgs, TResult>(name, description, argumentsTypeInfo, resultTypeInfo, (args, token) => ..., configure) - the handler takes the arguments and returns a result, written back to the model as JSON. This page's get_weather.
• Create<TArgs>(name, description, argumentsTypeInfo, (args, invocation, token) => ..., configure) - the handler is also handed the BitAiToolInvocation and returns the whole BitAiToolOutcome: its display Text, its Citations, a structured failure (Custom tool UI's search_docs).
• The same two without the JsonTypeInfos read the types by reflection, so they are marked as unsafe to trim.
• Create(definition, (invocation, token) => ...) - nothing derived: the schema is whatever the definition holds, for arguments that are dynamic.

The JsonTypeInfo overloads take the metadata of the app's own JsonSerializerContext, so the schema is derived and the arguments are read without reflection, and a trimmed WebAssembly app - like this site - keeps working. configure adjusts the generated definition (definition => definition with { Title = ..., Risk = ... }). BitAiTool.CreateSchema derives a schema on its own (CreateSchema(typeInfo), safe to trim, or CreateSchema<T>() by reflection) for a definition written by hand.

A tool that needs services is a class of its own, registered with ai.AddTool<TTool>(), as the WeatherTool.cs tab shows. Each call hands it a BitAiToolInvocation: the ToolCallId and ToolName, the Arguments as JSON (GetArguments(typeInfo) reads them into the argument type, GetArguments<T>() by reflection), the ConversationId and MessageId it belongs to, the scoped Services to resolve dependencies from, Progress to report how far it has got (never null), and the app's own Context values.

The registered tools live in an IBitAiToolRegistry - a BitAiToolRegistry, a case-insensitive map from a tool's name to the tool, one for the app (its members are listed on BitAiToolCallView). It can change at run time (Register(tool), Unregister(name)), which is how an app offers a capability only to some users or conversations; a session reads it on every request. This page builds its session by hand, over a transport of its own, and hands it the registry (new BitAiChatSession(transport, tools: registry)); a session from BitAiChatSessionFactory gets the registry, the approval policy, the configured session options and the scoped services its tools resolve from without being asked - a session built by hand gets services only through its services argument.

Every request carries the definitions of those tools (BitAiRequest.Tools, what BitAiToolRegistry.GetDefinitions() returns), so a transport hands the model what it may call without knowing how the tools were registered, and the session's RequestOptions (BitAiRequestOptions) say how it may call them: ToolChoice is BitAiToolChoice.Auto (the model decides), None, Required (some tool), or Specific("get_weather") (that one). A transport maps the choice onto its provider's API, and is free to ignore what it cannot express.

A tool the provider runs itself - a web search on its side - is reported by the transport, not run by the session: BitAiToolCallStartedEvent with the complete Arguments, any number of BitAiToolProgressEvents while it runs (a BitAiToolProgress with a Fraction and a Message, drawn as the card's progress bar), then BitAiToolResultEvent with its outcome. The session never runs a call that already has a result, so local and provider tools can mix in one reply; a transport whose tools all run on its side turns BitAiChatSessionOptions.ExecuteTools off. The Multi-step agent scenario's search is one of these. The Request options tab shows a tool choice, and a provider tool reported with its progress.

• BitAiToolCallView - the card: what is called, its arguments as they stream, its progress and its result.
• BitAiJsonView - the arguments and the result inside it.
• BitAiThinkingIndicator - the status line shown while the model decides.
• BitAiChat - the chat over the session that runs the tool.