How Pi Abstracts OpenAI and Anthropic APIs

How Pi Abstracts OpenAI and Anthropic APIs

This analysis is based on pi commit 588915ec71714688cee8b7153339e8bdebb3e82e.

Pi supports multiple LLM APIs without forcing them into a single wire-level protocol. Its abstraction boundary is higher than the provider payload but lower than the agent itself.

The design can be summarized as:

Agent loop
  -> provider-neutral messages, tools, and stream events
  -> API-specific adapter
  -> native provider SDK or HTTP/SSE protocol

The agent does not know whether a model uses Chat Completions, Responses, or Anthropic Messages. The API adapter does.

What Are Pi and the Pi Coding Agent?

Pi is an open-source agent harness for building and running extensible AI agents. It provides reusable components for multi-provider LLM access, agent execution, tool calling, session state, authentication, extensions, and terminal interfaces.

The Pi coding agent is the project’s interactive terminal application for software development. It allows an LLM to inspect and modify local repositories, execute shell commands, use tools, maintain session context, and work with models from different providers. Underneath the CLI, Pi’s agent runtime manages the tool loop and state, while its AI layer translates provider-neutral requests into native APIs such as OpenAI Responses, OpenAI Chat Completions, and Anthropic Messages.

The Abstraction Boundary

Pi models APIs using a typed discriminator:

Model<"openai-completions">
Model<"openai-responses">
Model<"anthropic-messages">

The supported API identifiers are declared in types.ts.

Each API implementation exposes the same high-level streaming contract:

stream(model, context, options)
streamSimple(model, context, options)

The contract is defined in types.ts, while the common event protocol is defined in types.ts.

All adapters emit the same events:

text_start / text_delta / text_end
thinking_start / thinking_delta / thinking_end
toolcall_start / toolcall_delta / toolcall_end
done / error

The shared message model contains text, images, thinking, tool calls, tool results, usage, and stop reasons. See types.ts.

This is the important distinction:

Pi abstracts provider behavior at the semantic conversation and streaming level, but preserves native API structures inside each adapter.

A provider owns authentication and its model catalog. The API adapter owns wire-format conversion. Mixed-API providers can dispatch to different adapters based on model.api, as implemented in models.ts.

OpenAI Chat Completions: Compatibility First

The openai-completions adapter uses the OpenAI SDK’s Chat Completions API:

client.chat.completions.create(...)

The call is made in openai-completions.ts.

Pi converts its common message model into Chat Completions structures:

  • system or developer messages
  • user and assistant messages
  • image_url content
  • tool_calls
  • function tools
  • OpenAI custom grammar tools

The conversion logic is in openai-completions.ts.

The adapter then parses streamed Chat Completions chunks into Pi’s common event stream. It handles ordinary text deltas, tool argument deltas, usage, finish reasons, and several reasoning field conventions.

Compatibility is a first-class feature

Chat Completions is the most compatible API in Pi because many third-party services imitate it. The adapter therefore contains a large compatibility layer controlled by model.compat.

The compatibility options include:

  • whether developer messages are supported
  • whether reasoning_effort is supported
  • whether to use max_tokens or max_completion_tokens
  • whether streaming usage is supported
  • whether store is accepted
  • whether thinking must be converted to text
  • whether an empty reasoning_content field is required
  • whether strict tools are supported
  • whether grammar-constrained tools are supported
  • provider-specific reasoning formats
  • prompt-cache behavior
  • session-affinity headers
  • tool streaming flags
  • deferred-tool serialization

The full compatibility contract is defined in types.ts.

For example, the adapter emits reasoning_effort for ordinary OpenAI-compatible servers, but can instead emit formats such as:

reasoning: { effort }
thinking: { type }
enable_thinking
chat_template_kwargs
chat_template_args

The request construction is handled in openai-completions.ts.

This makes openai-completions more than a thin OpenAI wrapper. It is effectively a compatibility adapter for a large ecosystem of OpenAI-like APIs.

OpenAI Responses: Preserving Native Semantics

The openai-responses adapter is substantially different. It uses the Responses API’s item-oriented model rather than translating everything into Chat Completions messages.

The request looks conceptually like this:

{
  model,
  input,
  stream: true,
  tools?,
  reasoning?,
  include?,
  prompt_cache_key?,
  prompt_cache_retention?,
  store: false
}

The request is assembled in openai-responses.ts.

Responses input items

Pi maps its conversation into native Responses items:

  • message
  • reasoning
  • function_call
  • function_call_output
  • custom_tool_call
  • custom_tool_call_output
  • tool_search_call
  • tool_search_output

The conversion is implemented in openai-responses-shared.ts.

This allows Pi to preserve important Responses-specific identity information. Responses tool calls use both a call_id and an item ID. Pi stores them internally as a combined identifier such as:

call_id|item_id

When replaying history, the adapter normalizes or removes IDs when they are not valid for the current provider or model. That logic is in openai-responses-shared.ts.

Native reasoning continuity

Responses reasoning is not reduced to ordinary text.

When a reasoning item completes, Pi serializes the complete provider item and stores it in the common thinkingSignature field. During replay, it parses that value and reconstructs the original Responses reasoning item.

The replay path is visible in openai-responses-shared.ts.

The request can also ask OpenAI for encrypted reasoning content:

reasoning: {
  effort,
  summary
}

include: ["reasoning.encrypted_content"]

See openai-responses.ts.

The stream processor understands Responses-specific events such as:

response.output_item.added
response.reasoning_summary_text.delta
response.reasoning_text.delta
response.function_call_arguments.delta
response.output_item.done
response.completed
response.incomplete

See openai-responses-shared.ts.

This is one of the clearest examples of Pi preserving modern API behavior behind a common abstraction. The agent receives thinking_delta, but the adapter retains the underlying Responses reasoning item and its encrypted continuity data.

Responses-specific tools

The Responses adapter supports:

  • strict function tools
  • OpenAI custom grammar tools
  • client-executed deferred tool search

For deferred tools, Pi initially omits tools that have not yet been loaded. Later it emits native client-side tool search items:

tool_search_call
tool_search_output

The implementation is in openai-responses-shared.ts.

The capability is model-controlled through supportsToolSearch, declared in types.ts.

Prompt caching and service tiers

The Responses adapter also exposes several newer OpenAI request features:

  • prompt_cache_key
  • prompt_cache_retention
  • explicit prompt-cache mode
  • store: false
  • service_tier
  • service-tier-dependent cost calculation

See openai-responses.ts.

What Pi does not generalize

Pi’s common Tool type represents a client-executed function-style tool. The Responses adapter supports function tools, custom grammar tools, and client-side tool search.

I found no dedicated support in this adapter for other Responses server-side tools such as:

web_search
file_search
code_interpreter
computer_use

That means Pi preserves a meaningful subset of Responses-native behavior, especially reasoning and item identity, but does not expose every possible Responses feature through its generic agent tool abstraction.

Anthropic Messages: Native Content Blocks and Thinking

The anthropic-messages adapter uses Anthropic’s Messages API and its native SSE event model.

Requests contain:

  • system
  • messages
  • tools
  • thinking
  • stream: true

The request is sent in anthropic-messages.ts.

Pi maps the common message model into Anthropic content blocks:

  • text blocks
  • image blocks
  • thinking blocks
  • tool_use
  • tool_result

Anthropic tool results are placed inside user messages as tool_result blocks rather than using an OpenAI-style tool role. See anthropic-messages.ts.

The SSE parser converts Anthropic’s content-block events into Pi’s common stream events. See anthropic-messages.ts.

Adaptive thinking

Pi supports both Anthropic thinking generations.

For newer adaptive-thinking models it emits:

thinking: {
  type: "adaptive",
  display
}

output_config: {
  effort
}

For older models it emits:

thinking: {
  type: "enabled",
  budget_tokens
}

The branching logic is in anthropic-messages.ts.

Pi’s provider-neutral reasoning levels are mapped to Anthropic effort levels or token budgets by streamSimple, implemented in anthropic-messages.ts.

Thinking signatures and redaction

Anthropic thinking signatures are stored in the common thinkingSignature field. Redacted thinking is represented using redacted: true and replayed as an Anthropic redacted_thinking block.

See anthropic-messages.ts.

This allows Pi to preserve Anthropic’s multi-turn reasoning requirements without exposing Anthropic-specific content-block types to the agent loop.

Fine-grained tool streaming

Pi supports Anthropic’s newer tool streaming behavior.

When the provider supports eager tool input streaming, Pi adds:

eager_input_streaming: true

When it does not, Pi falls back to the legacy beta header:

fine-grained-tool-streaming-2025-05-14

The compatibility setting is defined in types.ts.

The actual tool conversion is implemented in anthropic-messages.ts.

Caching and deferred tools

Anthropic caching uses native cache_control markers on the system prompt, the final tool, and the latest conversation content.

Pi supports Anthropic’s longer one-hour cache retention and optional session-affinity routing.

For newer Anthropic models, deferred tools are represented with:

defer_loading: true

and later loaded through:

{
  type: "tool_reference",
  tool_name: "..."
}

See anthropic-messages.ts.

The capability is model-sensitive. The default rules enable it only for appropriate first-party Claude models, as implemented in anthropic-messages.ts.

Opaque Provider State

Pi’s common model deliberately has fields for provider-specific continuity:

ThinkingContent {
  thinking
  thinkingSignature?
  redacted?
}

TextContent {
  text
  textSignature?
}

AssistantMessage {
  responseId?
}

These fields are defined in types.ts.

The values are opaque to the agent loop:

  • Responses stores serialized reasoning items and message IDs.
  • Anthropic stores thinking signatures and redacted-thinking payloads.
  • Chat Completions stores provider-specific reasoning fields or reasoning details.

When history is replayed to the same provider and model, signatures are preserved. During cross-provider handoff, unsupported signatures are dropped and thinking may be converted into ordinary text. The transformation rules are implemented in transform-messages.ts.

Conclusion

Pi’s architecture is neither:

Everything becomes OpenAI Chat Completions

nor:

Every provider exposes its entire native API to the agent

Instead, it uses a layered compromise:

Common agent semantics
  + common streaming lifecycle
  + common text/thinking/tool-call model
  + opaque provider-specific continuity data
  + API-specific request and response adapters

The practical result is:

  • openai-completions emphasizes broad compatibility with OpenAI-like servers.
  • openai-responses preserves Responses-native items, reasoning continuity, encrypted reasoning, message phases, tool search, grammar tools, caching, and service tiers.
  • anthropic-messages preserves Anthropic content blocks, adaptive thinking, thinking signatures, redacted thinking, fine-grained tool streaming, cache controls, strict tools, and tool references.

The tradeoff is intentional. The agent loop stays portable, while modern features that have no cross-provider equivalent remain behind API-specific options and model-level compatibility metadata.

Comments

Post a Comment

Popular posts from this blog

Slang Terms About Money

Term Meaning Example Buck $1 “It costs 20 bucks.” K $1,000 “He makes 100K a year.” Dime $10 (or a tenth of a dollar) “I spent a dime on snacks.” Bill $100 “That’s a couple of bills.” C-note $100 “He handed me a C-note.” Benjamin $100 “I got paid in Benjamins.” Stack/Rack $1,000 “He dropped 5 racks on a new car.” G $1,000 “The job pays 50G a year.” Grand $1,000 “The car costs 20 grand.” Green Money in general “He’s got a lot of green.” Cheddar/Dough/Bread Money in general “Making bread is important.” Smackers/Clams/Bones Dollars (old-fashioned) “It cost me 50 smackers.”
Read more

Workaround for macOS Dictionary All Tab Issue

After installing custom dictionaries on macOS, you may encounter an issue where the All tab in the built-in Dictionary app does not display correctly. The main problem is that the CFBundleIdentifier key in the Info.plist file of the custom dictionaries is either missing or not unique. Here’s a step-by-step workaround I found to resolve the issue: Steps to Fix the Issue: Locate the ~/Library/Dictionaries directory. To do this, open Finder, press Cmd + Shift + G , and enter ~/Library/Dictionaries (the Library folder is hidden by default). Inside the Dictionaries folder, you will find a list of .dictionary files, which represent the custom dictionaries you have installed. To fix the issue, check the Info.plist file of each dictionary. Navigate to the .dictionary folder, then go to the Contents subfolder, where you will find the Info.plist file. Open the Info.plist file using a text editor or a plist editor. Look for the CFBundleIdentifier key in the ...
Read more

Mathematical Objects

A cheat sheet for mathematical objects and structures, useful for quick reference. Foundations of Sets and Relations Set A collection of distinct objects, often written as X = { x ∣ P ( x ) } X = \{x \mid P(x)\} X = { x ∣ P ( x )} , where P ( x ) P(x) P ( x ) is a property. No additional structure. Map / Function A mapping f : X → Y f: X \to Y f : X → Y assigns each x ∈ X x \in X x ∈ X to a unique y ∈ Y y \in Y y ∈ Y . Relation A subset R ⊆ X × X R \subseteq X \times X R ⊆ X × X , such as equivalence relations or partial orders. Basic Algebraic Structures Semigroup A set S S S with a binary operation ⋅ : S × S → S \cdot: S \times S \to S ⋅ : S × S → S such that: Associativity: For all a , b , c ∈ S a, b, c \in S a , b , c ∈ S , ( a ⋅ b ) ⋅ c = a ⋅ ( b ⋅ c ) (a \cdot b) \cdot c = a \cdot (b \cdot c) ( a ⋅ b ) ⋅ c = a ⋅ ( b ⋅ c ) . Monoid A semigroup with an identity element e ∈ M e \in M e ∈ M , satisfying: Identity: a ⋅ e = e ⋅ a = a ...
Read more

Essential Utilities for LaTeX Package and Class Development

Developing LaTeX packages or classes is not a common task for most users, who typically rely on pre-existing packages. According to the CTAN: Comprehensive TeX Archive Network , there are only 3,040 TeX developers worldwide (as of February 2025). As a result, online resources for LaTeX package and class development are relatively scarce. From my perspective, there isn’t currently a comprehensive guide to the utility commands essential for package development. So, I’ve compiled this blog post, gathering a set of useful commands and packages I’ve discovered or developed through experience. I hope it proves helpful for anyone who is developing—or planning to develop—LaTeX packages or classes. This post also serves as a personal reference for me to revisit the next time I work on a package. This post will be a dynamic document, and I’ll update it as I discover new commands. If you have any useful commands that I haven’t mentioned, please feel free to share them in the ...
Read more

Train PyTorch with Checkpoints

Training a PyTorch model with checkpoints is an important technique to ensure that the model is saved periodically during the training process. This helps to avoid losing all of the progress made during the training if there is a power failure, hardware failure or any other unexpected problem. To train a PyTorch model with checkpoints, you can follow these steps: Import the necessary libraries and packages. import torch import torch . nn as nn import torch . optim as optim from torch . utils . data import DataLoader Define your model architecture. class MyModel ( nn . Module ) : def __init__ ( self ) : super ( MyModel , self ) . __init__ ( ) self . fc1 = nn . Linear ( 784 , 128 ) self . fc2 = nn . Linear ( 128 , 64 ) self . fc3 = nn . Linear ( 64 , 10 ) def forward ( self , x ) : x = x . view ( x . size ( 0 ) , - 1 ) x = self . fc1 ( x ) x = nn . functional . relu ...
Read more