Agent Streaming

The streaming_result step type enables real-time token-by-token delivery of LLM output to clients via Server-Sent Events (SSE). This is ideal for chat-like interfaces where users should see the response being generated incrementally.

Requirements:

• The agent must use a dynamic_input or template_input trigger (the two trigger types with an SSE consumer)
• Runs must use priority: true (priority mode)
• Only one streaming_result step is allowed per agent definition

Setting Up Streaming

Workflow Structure

Place a streaming_result step as a direct child of a prompt_call step:

Step 1: prompt_call (model with streaming support)
  └── Step 2: streaming_result

The streaming_result step must be a direct child of prompt_call — no intermediate steps are allowed between them.

Prompt Call Requirements

The parent prompt_call step must use a streaming-capable model — the selected model must have the supports_streaming capability. Most major models from Anthropic, OpenAI, and Google support streaming. Check the model's capability badges in the model selector. Both simple format (plain text) and advanced/JSON template prompt calls support streaming.

If the model does not support streaming, the full response will be delivered as a single chunk when complete — the agent run still succeeds, but without real-time token delivery.

Side-Effect Steps

To perform side effects (save to memory, send email, publish content) alongside streaming, add those steps as sibling branches under the same prompt_call:

Step 1: prompt_call
  ├── Branch 1: streaming_result (real-time delivery)
  └── Branch 2: add_chat_turn (record assistant turn to conversation memory)

Sibling branches receive the full buffered output after streaming completes. They execute in parallel with normal scheduling.

Streaming plus a schema-validated result

Streaming does not have to be the agent's result. A streaming_result step publishes the run's output as soon as the stream ends, but it is only ever a progress channel: a display_result in a sibling branch takes precedence over it, whichever of the two finishes first. So a second branch can re-read the same buffered response, validate it, and deliver the checked payload as the agent's final output:

Step 1: prompt_call
  ├── Branch 1: streaming_result                (live progress channel)
  └── Branch 2: extract_content                 (expected_format: json + json_schema)
                └── display_result              (the run's final result)

The tokens give the UI something to render while the model works; the payload the client reads at the end has still been checked against the contract. If the response fails the schema, the extract_content step fails and the run fails — the invalid object never becomes the result.

Check the run's status before trusting its output. The streamed text is already the run's output the moment the stream ends, so a run that failed its validation step still carries that raw, unvalidated text as output. Only a completed run's output has been through the schema.

Two things to keep in mind:

• Clients that render stream_token events see the raw model output, including any JSON syntax, before validation runs. Treat the stream as progress, not as the payload — read the final result from the completed run (or the done event's run fetch).
• The validating branch is a sibling of streaming_result, not a step between it and the prompt_call. streaming_result must remain a direct child of the prompt call.

Consuming Streaming Events

SSE Event Types

When using the POST /agents/{'{agent_id}'}/runs/stream endpoint, the following streaming-specific events are emitted:

stream_token — An individual token from the LLM response:

event: stream_token
data: {"step_id": "streaming_step_1", "token": "Hello", "seq": 1}

stream_end — Streaming is complete for a step:

event: stream_end
data: {"step_id": "streaming_step_1"}

stream_reasoning — The model's reasoning while it thinks, sent only when the streaming result step enables show_reasoning and the request body sets "include_reasoning": true:

event: stream_reasoning
data: {"step_id": "streaming_step_1", "text": "The user asks about", "seq": 1}

Reasoning arrives in batches, and each batch is sent before the next answer token. It is never part of the run's output. It can restate the system prompt, retrieved content and tool results, and it is withheld whenever a governance policy redacts the step. Only models served through Together AI or Novita can stream their reasoning.

These events are interleaved with standard run events (step_completed, done, etc.).

JavaScript Client Example

This runs server-side. The API key must never reach a browser, and the Seclai API sends no CORS headers, so a page cannot call it directly. To stream into a web UI, consume the run on your server and relay your own SSE events to the browser — see Calling from a Browser.

const response = await fetch(
  `https://api.seclai.com/agents/${"{agentId}"}/runs/stream`,
  {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-API-Key": apiKey,
    },
    body: JSON.stringify({ input: "Tell me about climate change" }),
  }
);

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
let streamedText = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });

  let boundary;
  while ((boundary = buffer.indexOf("\\n\\n")) !== -1) {
    const message = buffer.slice(0, boundary);
    buffer = buffer.slice(boundary + 2);

    const eventMatch = message.match(/^event: (.+)$/m);
    const dataMatch = message.match(/^data: (.+)$/m);
    if (!eventMatch || !dataMatch) continue;

    const event = eventMatch[1];
    const data = JSON.parse(dataMatch[1]);

    if (event === "stream_token") {
      streamedText += data.token;
      updateUI(streamedText); // Update your UI incrementally
    } else if (event === "stream_end") {
      finalizeUI(); // Streaming complete
    } else if (event === "done") {
      // Run completed — data contains the full run response
      break;
    }
  }
}

Time to First Token

Streaming runs record a server-side time-to-first-token (TTFT) — the wall-clock elapsed time from when the run started to when the first SSE token was emitted by the server. TTFT captures the full responsiveness picture up to that point: input scan, governance gate, step dispatch, prompt assembly, model warm-up, and the first byte of generated output. Network buffering and round-trip time between the server and the client are not included; the Run Agent modal records a separate client-perceived TTFT for that view.

Where to see TTFT:

• Run Agent modal displays a TTFT badge next to the streaming output as soon as the first token arrives. This is the client-perceived value (includes network RTT to the SSE server).
• Trace details page surfaces the server-measured value in the run header (Time to first token: …) and on each streaming prompt_call step.
• GET /authenticated/agents/runs/{'{run_id}'} (polling) returns first_token_ms once the run records its first token. Useful for clients that want the authoritative server value.
• GET /authenticated/agents/runs/{'{run_id}'}/details (trace) returns both the run-level first_token_ms and a per-step first_token_ms on each streaming step.
• MCP get_agent_run_status returns first_token_ms (run-level) and per-step first_token_ms when include_step_outputs=true.
• Resource export (agent_traces) includes first_token_at on each run plus first_token_at on each attempt for offline analytics.

first_token_ms is null for non-streaming runs and for runs that errored before any token was emitted. The run-level value is set once, by the first streaming step that produces a token; multi-streaming-step runs (rare) capture only the earliest occurrence.

Fallback Behavior

Streaming falls back to non-streaming (single-chunk delivery) in these cases:

• Non-priority mode — Streaming requires priority: true; without it the full response is delivered as one chunk
• Model doesn't support streaming — The full response is delivered as one chunk
• Governance output policies — When blocking governance policies are active on the step, streaming is disabled to allow content screening before delivery
• Tool-calling prompts — The tool loop completes fully first, then the final LLM response is streamed
• MCP server — Streaming is not supported via MCP; falls back to normal execution