Provider Command And Event Flow Guide

This guide explains how provider-side command intents, next actions, and history events move through the codebase and reach an OpAMP client.

It is intended to serve two audiences:

• developers adding or refactoring a provider-side command or client-directed action
• tool-assisted workflows, including prompts for tools such as Codex

Scope

This guide covers:

• provider REST/UI command and client-action entry points
• in-memory state and history behavior
• response building and transport send behavior
• how standard commands, custom commands, remote config, identification, heartbeat changes, and other next actions differ
• developer expectations around history creation

This guide does not attempt to document consumer-side custom handler internals in detail. For that, see:

• Adding Your Own Custom Action
• Consumer Custom Handlers
• Command Process Implementation Note

Key Rule

Treat provider UI/API activity and MCP activity differently.

• Provider UI/API flows define the user-visible client history behavior.
• MCP activity should be excluded from the history rules described here.
• If an MCP path currently reuses a history-producing helper, treat that as an implementation detail or technical debt, not the desired contract for future work.

In other words: do not use MCP behavior as the reference model when deciding whether a provider-side client action should create a history entry.

High-Level Model

There are three related but distinct concepts in the provider:

1. CommandRecord Used for queued command intents such as restart, force resync, and custom commands. These live in ClientRecord.commands.
2. next_actions Used for provider-directed actions that are not stored as CommandRecord objects, such as applying remote config or changing connection settings.
3. EventHistory Used for user-visible history items that are not naturally represented as commands, or when the route adds history manually.

The key design consequence is that not every client-directed operation follows the same storage or history path.

Code Map

Primary files:

• provider/src/opamp_provider/app_routes_clients.py
• provider/src/opamp_provider/command_queue.py
• provider/src/opamp_provider/state.py
• provider/src/opamp_provider/app_response_builder.py
• provider/src/opamp_provider/app.py
• provider/src/opamp_provider/commands.py

Related docs:

• Command Process Implementation Note
• Adding Your Own Custom Action
• Provider Server Diagram (Mermaid Source)

Architecture Overview

flowchart LR
    UI["Provider UI / REST client"] --> ROUTES["app_routes_clients.py"]
    MCP["MCP tools"] -. excluded from history rules .-> CQ
    ROUTES --> CQ["command_queue.py"]
    ROUTES --> STORE["ClientStore in state.py"]
    CQ --> STORE
    STORE --> RESP["ServerToAgentResponseBuilder"]
    RESP --> HTTP["OpAMP HTTP response"]
    RESP --> WS["OpAMP WebSocket response"]
    HTTP --> AGENT["Agent"]
    WS --> AGENT

    STORE --> HIST["ClientRecord.events"]
    STORE --> CMDS["ClientRecord.commands"]
    STORE --> NEXT["ClientRecord.next_actions"]

Command And Event Surfaces

The provider currently uses several different surfaces to send data to a client.

This table is the core reason developers need to be explicit: only some paths create history as part of the state mutation itself.

The Preferred Pattern For True Commands

For provider UI/API initiated commands, the cleanest pattern is:

1. validate and normalize request payload
2. create a CommandRecord
3. append it to ClientRecord.commands
4. append the same intent to unified event history
5. build the outbound OpAMP payload on the next client check-in
6. mark the command sent once a command payload was actually emitted

That pattern is implemented by:

• queue_command_from_payload(...) in command_queue.py
• store.queue_command(...) in state.py

Standard/Custom Command Flow

sequenceDiagram
    participant UI as Provider UI/API
    participant Routes as app_routes_clients.py
    participant Queue as command_queue.py
    participant Store as state.py
    participant Builder as app_response_builder.py
    participant Agent as OpAMP Agent

    UI->>Routes: POST /api/clients/<id>/commands
    Routes->>Queue: queue_command_from_payload(...)
    Queue->>Store: queue_command(...)
    Note over Store: append CommandRecord to commands<br/>append same object to events
    Agent->>Builder: next AgentToServer check-in
    Builder->>Store: next_pending_command(...)
    Builder->>Builder: build command/custom payload
    Builder-->>Agent: ServerToAgent.command or custom_message or flags
    Builder->>Store: mark_command_sent(...)

What gets stored

store.queue_command(...) does two things at once:

• appends a CommandRecord to ClientRecord.commands
• appends that same command record to ClientRecord.events

That makes queue_command(...) the main history-producing API for command-like behavior.

What gets sent

ServerToAgentResponseBuilder._apply_command_intent(...) maps the queued command to an outbound payload:

• restart -> ServerToAgent.command
• force resync -> ServerToAgent.flags
• custom or custom_command -> ServerToAgent.custom_message

When the command is marked sent

The provider marks a queued command as sent only after a command payload was actually emitted:

• HTTP path: app.py calls mark_command_sent(...) after has_dispatched_command_payload(...)
• WebSocket path: same rule

History is therefore created at queue time, not at send time.

Next-Action Flow

Some provider actions are not modeled as CommandRecord.

Examples:

• remote config delivery
• package-availability placeholder behavior

These use ClientRecord.next_actions and are consumed separately from commands.

sequenceDiagram
    participant UI as Provider UI/API
    participant Routes as app_routes_clients.py
    participant Store as state.py
    participant Builder as app_response_builder.py
    participant Agent as OpAMP Agent

    UI->>Routes: route-specific POST/PUT
    Routes->>Store: set state needed for action
    Routes->>Store: enqueue_next_action(...)
    Note over Routes,Store: optional add_event(...)
    Agent->>Builder: HTTP AgentToServer check-in
    Builder->>Store: pop_next_action(...)
    Builder->>Builder: _apply_next_action(...)
    Builder-->>Agent: remote_config / connection_settings / error payload

Important transport detail

next_actions are currently popped only for the HTTP channel in build_response(...).

That means:

• queued commands can be dispatched on HTTP or WebSocket
• next_actions such as remote config apply are currently HTTP-only

This matters when you add a new client-directed action. Decide whether it is a true queued command or an HTTP-only next action before implementing it.

History-Producing APIs

The main store APIs and their history behavior are:

Current Provider Patterns

Unified command path

These already follow a clear command queue pattern:

• POST /api/clients/<client_id>/commands
• custom command submission from the provider UI through that same route

These are the best reference implementations for new command work.

Specialized but still history-safe

These do produce history, but not through queue_command(...):

• PUT /api/clients/<client_id>/heartbeat-frequency
• POST /api/clients/<client_id>/identify
• POST /api/clients/<client_id>/remote-config

Two different subpatterns exist here:

• heartbeat: the state API itself appends history
• identify and remote config: the route mutates state, then calls add_event(...)

One important implementation detail:

• the heartbeat route records state and history, but it does not currently enqueue change_connections on its own

Routes with no current history

These routes change client-directed state but do not currently add a history item:

• POST /api/clients/<client_id>/actions
• POST /api/clients/<client_id>/config

When modifying these areas, decide explicitly whether the absence of history is intentional or accidental.

MCP Exclusion

For this guide, MCP activity is excluded from the history contract.

Why:

• MCP is an automation and tooling surface, not the canonical UI-facing activity model.
• Developers should not infer user-visible history requirements from MCP helpers.
• Future command additions should preserve a clean distinction between provider UI/API history semantics and MCP automation semantics.

Practical rule

When you add or refactor a provider-side action:

• design history behavior based on the provider UI/API route
• do not treat MCP tool usage as evidence that the action should appear in the client history

Current-code caveat

Some MCP helpers currently reuse command queue helpers that append history automatically.

That does not change the design guidance above. If you are normalizing behavior in the future, prefer to preserve or restore the rule that MCP-originated activity should not drive UI history expectations.

How To Add A New Command

Use this decision tree first.

flowchart TD
    A["New provider-side client action"] --> B{"Is it a true queued command intent?"}
    B -->|Yes| C["Use /commands-style flow"]
    B -->|No| D{"Is it an HTTP-only next action?"}
    C --> E["Create/extend CommandObjectInterface implementation if needed"]
    C --> F["Queue with store.queue_command(...)"]
    C --> G["Ensure response builder emits command/custom_message/flags"]
    D -->|Yes| H["Store action-specific state"]
    H --> I["enqueue_next_action(...)"]
    I --> J{"Should UI history show it?"}
    J -->|Yes| K["Call add_event(...) or use a history-producing state API"]
    J -->|No| L["Leave as pure state mutation"]
    D -->|No| M["Re-evaluate design; maybe it is not a command surface"]

Step 1: Decide the semantic category

Ask these questions:

• Will the action be represented as a command intent that should appear in ClientRecord.commands?
• Should it be dispatchable on both HTTP and WebSocket?
• Is there a natural classifier and action pair?
• Should it appear in user-visible client history?

If the answer is yes to the first three, it usually belongs on the command queue path.

Step 2: For true commands, prefer the command queue path

Use or extend:

• POST /api/clients/<client_id>/commands
• queue_command_from_payload(...)
• store.queue_command(...)
• ServerToAgentResponseBuilder._apply_command_intent(...)

For custom commands:

• add a command implementation under provider/src/opamp_provider/command_implementations/
• rely on startup discovery in commands.py
• rely on wildcard custom routing rather than per-command builder registration

This is documented in more detail in Adding Your Own Custom Action.

Step 3: For next actions, be explicit about history

If the action is implemented with:

• enqueue_next_action(...)
• set_pending_remote_config(...)
• set_next_actions(...)
• set_agent_identification(...)

then history will not happen automatically unless you add it deliberately.

Do not assume that queuing something for the client automatically creates a timeline event.

Step 4: Confirm send-channel behavior

Before shipping a new action, verify:

• should it dispatch on HTTP only?
• should it dispatch on WebSocket too?
• is it consumed from commands, next_actions, or another state field?

If you use next_actions, the current implementation is HTTP-only.

Step 5: Add tests at the right layers

Recommended test layers:

• route test for validation and queueing behavior
• state/store test if new mutation/history behavior is introduced
• response-builder test for outbound protobuf shape
• end-to-end provider endpoint test covering queue and consume
• UI metadata test for custom commands if the command is user-selectable

History-specific assertions should verify the intended policy:

• command route -> history created automatically
• next-action route -> history only when explicitly added
• MCP-specific tests should not define UI-history expectations

Checklist For Developers

When adding or refactoring a provider-side command or client action:

• decide whether it is a CommandRecord flow or a next_actions flow
• choose the transport expectation: HTTP-only or HTTP plus WebSocket
• decide whether the action should create a user-visible history entry
• if it should, use a history-producing API or add store.add_event(...)
• if it is MCP-only or MCP-originated, do not use it to define history rules
• add or update tests for queueing, dispatch, and history behavior
• link the change back to the custom-command docs when relevant

Prompt Template For Codex Or Similar Tools

Use the following prompt as a starting point when asking a coding tool to add or refactor a provider-side command:

Implement a new provider-side client action in this repo.

Before editing, read:
- docs/dev/server(provider)/provider_command_and_event_flow_guide.md
- docs/dev/adding_your_own_custom_action.md
- docs/dev/command_process_implementation_note.md

Requirements:
- Treat provider UI/API command behavior as the source of truth.
- Exclude MCP activity from the history requirements for this task.
- Decide whether the new behavior belongs on the CommandRecord path or the next_actions path.
- If it is a true command, prefer the existing /api/clients/<client_id>/commands flow and store.queue_command(...).
- If it is a next action, be explicit about whether it should create UI-visible history.
- Do not assume enqueue_next_action(...) or other state setters create history automatically.
- Verify whether the action should dispatch over HTTP only or over HTTP and WebSocket.
- Add tests for validation, queueing, dispatch shape, and history behavior.

Deliverables:
- code changes
- test updates
- a short explanation of which state/history path was chosen and why

Recommended Reading Order

For a new contributor:

1. Read this guide first.
2. Read Command Process Implementation Note for the lower-level command queue details.
3. Read Adding Your Own Custom Action when the change involves a new custom command implementation.
4. Read Consumer Custom Handlers if a new consumer capability handler is needed.

Summary

The provider does not have a single universal path for all client-directed actions.

• queue_command(...) is the clean, history-producing path for true commands.
• next_actions and other state setters are separate mechanisms and usually require explicit history decisions.
• send behavior differs by channel: queued commands can go over HTTP or WebSocket, while next_actions are currently applied only on HTTP.
• MCP activity should be excluded when defining or reviewing provider UI history behavior.

If you follow those rules first, the implementation choices in this area become much easier to make consistently.