AI & Generation Steps

For common fields, string substitutions, metadata filters, caching, and execution order, see the Agent Steps Overview.

Prompt Call Step

Calls an AI model to generate a response. This is the most versatile step type, supporting 200+ models across multiple providers with features like structured JSON output, tool calling, and response formatting.

Fields

Model Lifecycle Controls (Prompt Call and Extract Data)

Both Prompt Call and Extract Data support optional per-step lifecycle overrides:

• auto_upgrade_strategy: one of none, early_adopter, middle_of_road, cautious_adopter
• auto_rollback_enabled: true, false, or null (inherit)
• auto_rollback_triggers: subset of agent_eval_fail, governance_flag, governance_block, agent_run_failed

If a step leaves these fields as null, it inherits the agent-level defaults.

How strategies behave:

• none: never auto-upgrade; always use the configured base model.
• early_adopter: upgrade to the successor as soon as one is configured.
• middle_of_road: upgrade only after a stability window following successor release.
• cautious_adopter: upgrade only after the current model is deprecated and the successor has been stable.

How rollback works:

When auto-rollback is enabled and the step actually ran on an auto-upgraded model, Seclai can automatically roll the branch definition back to a previously successful branch change when a configured trigger occurs.

Supported Models

Seclai supports 200+ models across 20+ providers — including OpenAI, Anthropic, Google, Amazon, DeepSeek, xAI, Meta, Mistral, Moonshot AI, Qwen, NVIDIA, and Cohere. See the Models page for the full, up-to-date list with capabilities and pricing.

Tools (Function Calling)

Prompt call steps can use tools — functions that the AI model can invoke during generation to retrieve additional information. When tools are configured, the model autonomously decides when to call them based on the conversation context. For example, when asked a question, the model might search a knowledge base, load a specific document, and then synthesize an answer.

Four tool groups are available: Web Tools (fetch pages, search the web), Knowledge Base Tools (semantic search, discovery), Content Tools (load documents, inspect content, write metadata and attachments), and Memory Bank Tools (read/write/search persistent memory).

For the full list of tools, parameters, and guidance on when to use tools vs. explicit steps, see the Tools page.

Formatting Cleanup

Apply automatic post-processing to the model's response:

File Attachments

Prompt Call steps accept file attachments carried on the agent's input (or emitted by an upstream step's output manifest). The runtime picks one of three paths per attachment, based on the resolved model's capabilities:

1. Native — when the model declares MIME support (e.g. Claude Haiku 4.5 + image/png, Gemini + audio/mpeg), the binary is wrapped in a vendor-specific media content block.
2. OCR / transcript demotion — when the model can't handle the MIME but a text counterpart exists, the extracted text is stitched onto the prompt body under an --- attachment text --- separator so the LLM still sees the content.
3. Dropped — no text counterpart available and the model can't handle the binary; an INFO log records the attachment, model, and reason.

A step "sees" attachments only when its template declares the dependency — implicit via {{input}} / {{agent.input}} / {{step.<id>.input|output}} references, or explicit via the {{attachments[…]}} family. Steps with no input reference no longer broadcast trigger uploads.

The toolbar's Input and Step Output menus surface attachment submenus (All / By index / By filename pattern) under each text-reference parent item, so the same affordance handles both the text reference and the attachment narrowing for that source.

Generating attachments

A Prompt Call doesn't only consume files — on an image-generation model (or any model with the built-in image_generation tool), it produces them. The generated images become the step's output as a multi-asset manifest, which downstream steps reference by step id: {{step.<this_step_id>.attachments}} (all), {{step.<id>.attachments[0]}} (one), or {{step.<id>.attachments[*.png]}} (by pattern). prompt_call and call_agent are the only two step types that generate attachments. Hand the result to a Send Email, Write to S3, or Display Result step — see the generate-and-deliver example.

See the Attachments guide for the high-level model and worked examples, and File Attachments for the full grammar, the per-step routing summary log, and per-step config behaviour.

Media generation tool (image / audio / video)

A Prompt Call can also let the orchestrating LLM decide to generate media mid-reasoning — the agentic counterpart to the one-shot Generate Image / Audio / Video steps. Enable it with a media_generation block on the step; the model then gets the matching built-in tools (seclai_generate_image / seclai_generate_audio / seclai_generate_video) and calls them with just a prompt plus light options. The generated assets land in the step's output manifest exactly like generated images above.

The author keeps cost control — the LLM never picks a model. Each enabled modality carries its own:

A run-level iterate flag (default off) feeds a generated image back into the model's next turn — when the orchestrating model accepts image input — so it can inspect and regenerate (generate-then-refine). It has no effect for audio/video or text-only models, which receive only the tool's JSON metadata.

When it calls a generator the model may also pass an optional seed (a non-negative integer) for reproducibility — omitting it (the default) gives fresh, varied output each call; passing a fixed value reproduces a prior result on models that support seeding.

Use the dedicated Generate Image / Audio / Video steps for deterministic, one-shot generation (you wire the step and pick the model); reach for the Prompt Call media_generation tool when the model should decide whether and how often to generate, or iterate on what it produced.

Streaming Output

Attaching a Streaming Result step immediately after a prompt call enables token-by-token streaming to the user via Server-Sent Events. The model's output is delivered as it is generated, rather than waiting for the full response.

Other steps can still run in parallel with the streaming result — for example, an Add Memory step to persist the prompt call's output, or a Webhook Call to notify an external system. The streaming result and all parallel siblings receive the same prompt call output and execute concurrently.

Input as Messages (Conversational Mode)

When use_step_input_as_messages is enabled (simple mode only), the step parses its input as a JSON array of chat messages and sends them directly to the model — instead of wrapping a single prompt template in a user message.

Expected input format:

[
  { "role": "user", "content": "Hello, how are you?" },
  { "role": "assistant", "content": "I'm doing well! How can I help you?" },
  { "role": "user", "content": "Can you summarize this document?" }
]

This is the exact format produced by the Load Memory step when configured with content_type: "application/json".

Why use this? Sending each message as a separate element (rather than embedding the entire history in a single prompt string) enables prompt caching on supported providers. The model can cache the stable prefix of earlier messages and only process the new ones, significantly reducing latency and cost for conversational workflows.

Recommended conversational workflow:

1. Write Memory (speaker: user) — save the incoming user message
2. Load Memory (order: oldest_first, content type: JSON) — load the full history as a messages array
3. Prompt Call (use_step_input_as_messages: true) — send messages to the model with a system prompt
4. Write Memory (speaker: assistant) — save the model's response
5. Streaming Result — stream the response to the user

Important: Use oldest_first order for prompt caching to work. The oldest messages form a stable prefix that the model can cache. Using newest_first would change the prefix on every turn, defeating the cache.

Adding additional context (RAG, preferences, metadata):

When your conversational agent also uses retrieval results, user preferences, or other contextual data, place that additional context in the system_template rather than in prompt_template (which must be null when using input-as-messages). Note that dynamic content in system_template (e.g. retrieval results that change per turn) will still change the cached prefix and can reduce prompt-caching effectiveness — the main caching benefit comes from keeping the conversation history as a stable message array via oldest_first ordering.

Example system_template with additional context:

You are a helpful assistant. Use the provided context to answer accurately.

Knowledge base results:
{{step.search-kb-retrieval.output}}

User preferences:
{{step.recall-prefs-search_memory.output}}

This keeps the conversation messages clean (each message is sent individually for caching) while still providing the model with all the context it needs via the system prompt.

When use_step_input_as_messages is enabled:

• The prompt_template field is ignored (hidden in the editor)
• The system_template is still used as the system prompt
• The step fails with a clear error if the input is not valid JSON or doesn't match the expected [{role, content}] schema
• Valid roles are: user, assistant, system

Simple vs Advanced Format

The simple_format field controls how the prompt is sent to the model:

Simple format (simple_format: true, default):

• Uses prompt_template (string) and system_template (string)
• Works like interactive chat interfaces (ChatGPT, Claude)
• Portable across model providers with little or no modification
• json_template must not be provided

Advanced format (simple_format: false):

• Uses json_template (JSON object) — the model's native prompt payload
• Gives access to provider-specific features and fine-grained control
• May require modifications when switching between model providers
• prompt_template must not be provided

Example advanced format JSON template:

{
  "messages": [
    {
      "role": "system",
      "content": "You are an expert analyst for {{organization.name}}."
    },
    {
      "role": "user",
      "content": "Analyze this: {{agent.input}}"
    }
  ]
}

String substitutions (e.g., {{agent.input}}, {{metadata.*}}) are supported inside both prompt_template and json_template.

Use Case Examples

Content summarization:

System: You are a concise content summarizer. Output a 2-3 sentence summary.

Prompt: Summarize this article:

{{agent.input}}

Question answering with context (RAG):

System: You are a helpful assistant for {{organization.name}}.
Answer questions using ONLY the provided context.
If the answer is not in the context, say "I don't have that information."

Prompt: Context:
{{step.retrieval.output}}

Question: {{agent.input}}

Structured data extraction:

System: Extract the following information from the text and return as JSON.

Prompt: {{agent.input}}

With json_template:

{
  "type": "object",
  "properties": {
    "company_name": { "type": "string" },
    "revenue": { "type": "string" },
    "employees": { "type": "integer" },
    "headquarters": { "type": "string" }
  }
}

Multi-language response:

System: You are a translator. Translate the input to {{metadata.target_language}}.
Maintain the original formatting and tone.

Prompt: {{agent.input}}

AI Assistant

The prompt call step includes an AI assistant that can generate a complete configuration based on a natural-language description of what you want the step to do. Click the AI Assistant button (requires a model to be selected first) to open the assistant modal.

The assistant can set all prompt call fields:

• Model and model variant — selects a cost-effective model that fits the task (e.g. tool use, structured output, long context)
• Mode — chooses between simple mode (prompt + system prompt) and JSON mode (vendor-native payload)
• Prompt template and system template — writes templates using substitution variables like {{input}} and {{step.<id>.output}}
• Temperature and max tokens — tunes generation parameters for the task
• Tools — enables tool use (e.g. web search) when the task requires it
• JSON template — builds vendor-specific payloads when JSON mode is appropriate
• Formatting cleanup — selects post-processing (Markdown → HTML, HTML → Markdown, plain text only)

For Prompt Call steps, the assistant follows these conventions:

• Use {{agent.input}} when referring to the user's original question/message
• Use {{input}} only for previous-step output/context
• Put core instruction logic in System Template
• In System Template, prefer this order: original question ({{agent.input}}) → instructions → previous-step data ({{input}} / {{step.<id>.output}})

The assistant is aware of the full agent workflow context — all steps, the trigger configuration, and the knowledge bases available — so it can reference outputs from earlier steps and tailor the prompt to the agent's purpose.

Each response also includes a purpose field (a brief description of the step's intent) and a note explaining what was changed or asking a follow-up question.

Generate Image / Audio / Video Steps

The dedicated generation steps — generate_image, generate_audio, and generate_video — produce media deterministically from a text prompt: you wire the step, pick a generation model, and it runs once. (For the agentic alternative, where the LLM decides whether and how often to generate, see the Prompt Call Media generation tool.)

Each calls a dedicated image/audio/video generation model (a catalog model that advertises an output modality and a per-unit price) and emits the asset(s) as the step's output manifest, referenced downstream by step id ({{step.<id>.attachments}}). Generation models are billed per unit — per image, per second, per 1,000 characters, or per 1,000 output tokens — shown on the model picker; unbillable models are hidden.

Shared fields

Generate Image

Generate Audio

Generate Video

Use Case Examples

Generate a marketing image: draft the prompt with a model, then render it.

Prompt Call: "Write a vivid image prompt for a hero banner about {{input}}."
  → Generate Image: { model: <image model>, prompt_template: {{step.prompt.output}}, size: 1536x1024 }
  → Display Result

Narrate a script to audio: write copy, then synthesize speech and archive it.

Prompt Call: "Write a 60-second narration script for {{input}}."
  → Generate Audio: { model: <tts model>, prompt_template: {{step.prompt.output}}, voice: <voice> }
  → Write to S3: archive the generated mp3

Animate a still into a clip: generate a key frame, then feed it to a video model.

Generate Image: { model: <image model>, prompt_template: "Product on a white studio backdrop" }
  → Generate Video: { model: <video model>, prompt_template: "Slow 360° rotation", input_attachments: [{{step.<image_id>.attachments}}], duration_s: 5 }
  → Display Result

← Back to Agent Steps Overview