# AI & Generation Steps For common fields, string substitutions, metadata filters, caching, and execution order, see the [Agent Steps Overview](https://seclai.com/docs/agent-steps). --- ## Prompt Call Step Calls an AI model to generate a response. This is the most versatile step type, supporting 200+ models across multiple providers with features like structured JSON output, tool calling, and response formatting. ### Fields | Field | Type | Required | Default | Description | | --------------------------------------- | ---------------- | -------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | model | enum | Yes | — | The AI model to use (see [supported models](#prompt-call-models) below) | | model_variant | string | No | null | Model variant identifier, if applicable | | auto_upgrade_strategy | enum | No | null (inherit) | Optional per-step model lifecycle strategy. When set, may automatically switch to a configured successor model before execution based on rollout policy. | | auto_rollback_enabled | boolean | No | null (inherit) | Optional per-step override to enable or disable automatic rollback after selected failure signals when an upgraded model is in use. | | auto_rollback_triggers | array | No | null (inherit) | Optional per-step rollback trigger list. Supported values: agent_eval_fail, governance_flag, governance_block, agent_run_failed. | | prompt_template | string or object | No | null | The prompt sent to the model. Supports substitutions. If not set, the step's input is used as the prompt. | | system_template | string | No | null | System instructions for the model. Sets the model's behavior and context. Supports substitutions. | | temperature | float | No | null (model default) | Controls randomness of the response. Range: 0.0 (deterministic) to 1.0 (creative). | | max_tokens | integer | No | null (model default) | Maximum number of tokens in the response | | simple_format | boolean | No | true | Controls the prompt input format. When true, uses a plain text prompt and system prompt (like ChatGPT/Claude). When false, uses an advanced JSON payload that leverages the model's native API features. | | json_template | object | No | null | The JSON payload sent to the model when simple_format is false. Uses the model's native prompt format, allowing access to provider-specific features. Supports substitutions. | | tools | array | No | {'[]'} | List of tool IDs the model can call during generation (see [Tools](https://seclai.com/docs/tools)) | | extended_caching_days | integer | No | null | Number of days to cache results (must be > 0) | | cache_all_minutes | integer | No | null | Cache all identical requests for N minutes (1–60) | | use_step_input_as_messages | boolean | No | false | Simple mode only. Parses step input as a chat messages array and sends directly to the model. See [Input as Messages](#prompt-call-input-as-messages). | ### Model Lifecycle Controls (Prompt Call and Extract Data) Both **Prompt Call** and **Extract Data** support optional per-step lifecycle overrides: - `auto_upgrade_strategy`: one of `none`, `early_adopter`, `middle_of_road`, `cautious_adopter` - `auto_rollback_enabled`: `true`, `false`, or `null` (inherit) - `auto_rollback_triggers`: subset of `agent_eval_fail`, `governance_flag`, `governance_block`, `agent_run_failed` If a step leaves these fields as `null`, it inherits the agent-level defaults. **How strategies behave:** - **none**: never auto-upgrade; always use the configured base model. - **early_adopter**: upgrade to the successor as soon as one is configured. - **middle_of_road**: upgrade only after a stability window following successor release. - **cautious_adopter**: upgrade only after the current model is deprecated and the successor has been stable. **How rollback works:** When auto-rollback is enabled and the step actually ran on an auto-upgraded model, Seclai can automatically roll the branch definition back to a previously successful branch change when a configured trigger occurs. ### Supported Models Seclai supports 200+ models across 20+ providers — including OpenAI, Anthropic, Google, Amazon, DeepSeek, xAI, Meta, Mistral, Moonshot AI, Qwen, NVIDIA, and Cohere. See the [Models](https://seclai.com/docs/models) page for the full, up-to-date list with capabilities and pricing. ### Tools (Function Calling) Prompt call steps can use **tools** — functions that the AI model can invoke during generation to retrieve additional information. When tools are configured, the model autonomously decides when to call them based on the conversation context. For example, when asked a question, the model might search a knowledge base, load a specific document, and then synthesize an answer. Four tool groups are available: **Web Tools** (fetch pages, search the web), **Knowledge Base Tools** (semantic search, discovery), **Content Tools** (load documents, inspect content, write metadata and attachments), and **Memory Bank Tools** (read/write/search persistent memory). For the full list of tools, parameters, and guidance on when to use tools vs. explicit steps, see the **[Tools](https://seclai.com/docs/tools)** page. ### Formatting Cleanup Apply automatic post-processing to the model's response: | Value | Description | | ------------------------------------- | ------------------------------------------- | | convert_markdown_to_html | Converts Markdown output to HTML | | convert_html_to_markdown | Converts HTML output to Markdown | | plain_text_only | Strips all formatting, returning plain text | ### File Attachments Prompt Call steps accept **file attachments** carried on the agent's input (or emitted by an upstream step's output manifest). The runtime picks one of three paths per attachment, based on the resolved model's capabilities: 1. **Native** — when the model declares MIME support (e.g. Claude Haiku 4.5 + `image/png`, Gemini + `audio/mpeg`), the binary is wrapped in a vendor-specific media content block. 2. **OCR / transcript demotion** — when the model can't handle the MIME but a text counterpart exists, the extracted text is stitched onto the prompt body under an --- attachment text --- separator so the LLM still sees the content. 3. **Dropped** — no text counterpart available and the model can't handle the binary; an INFO log records the attachment, model, and reason. A step "sees" attachments only when its template **declares the dependency** — implicit via {'{{input}}'} / {'{{agent.input}}'} / {'{{step..input|output}}'} references, or explicit via the {'{{attachments[…]}}'} family. Steps with no input reference no longer broadcast trigger uploads. The toolbar's **Input** and **Step Output** menus surface attachment submenus (All / By index / By filename pattern) under each text-reference parent item, so the same affordance handles both the text reference and the attachment narrowing for that source. #### Generating attachments A Prompt Call doesn't only consume files — on an **image-generation model** (or any model with the built-in `image_generation` tool), it **produces** them. The generated images become the step's output as a multi-asset manifest, which downstream steps reference by step id: {'{{step..attachments}}'} (all), {'{{step..attachments[0]}}'} (one), or {'{{step..attachments[*.png]}}'} (by pattern). `prompt_call` and `call_agent` are the only two step types that generate attachments. Hand the result to a [Send Email](https://seclai.com/docs/agent-steps/integration#send-email-step), [Write to S3](https://seclai.com/docs/agent-steps/integration#write-s3-step), or [Display Result](https://seclai.com/docs/agent-steps/output#display-result-step) step — see the [generate-and-deliver example](https://seclai.com/docs/agent-attachments#scenario-generate-image). See the [**Attachments guide**](https://seclai.com/docs/agent-attachments) for the high-level model and worked examples, and [File Attachments](https://seclai.com/docs/agent-steps#file-attachments) for the full grammar, the per-step routing summary log, and per-step config behaviour. #### Media generation tool (image / audio / video) A Prompt Call can also let the orchestrating LLM **decide** to generate media mid-reasoning — the agentic counterpart to the one-shot [Generate Image / Audio / Video](#generate-image-step) steps. Enable it with a media_generation block on the step; the model then gets the matching built-in tools (seclai_generate_image / seclai_generate_audio / seclai_generate_video) and calls them with just a prompt plus light options. The generated assets land in the step's output manifest exactly like generated images above. The author keeps cost control — the LLM never picks a model. Each enabled modality carries its **own**: | Field | Type | Default | Description | | ------------------------------------------- | ----------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | tier | fast / balanced / thorough | `balanced` | Quality/cost tier. The server routes the modality to that tier's generator (cheapest → flagship); the LLM supplies only the prompt and light params. | | max_generation_calls_per_run | integer (1–64) | `4` | Hard ceiling on how many times the model may call this generator in one run. Further calls are refused (the model is told, and reacts). | | max_generation_credits_per_run | integer (1–1,000,000) | `2000` | Cumulative credit ceiling. Each call's cost is **estimated before it runs** and refused if it would exceed the ceiling, so one expensive call can't blow past it. If generation pricing can't be verified, generation is refused (fail-closed) rather than running uncapped. | A run-level iterate flag (default off) feeds a generated **image** back into the model's next turn — when the orchestrating model accepts image input — so it can inspect and regenerate (generate-then-refine). It has no effect for audio/video or text-only models, which receive only the tool's JSON metadata. When it calls a generator the model may also pass an optional seed (a non-negative integer) for reproducibility — omitting it (the default) gives fresh, varied output each call; passing a fixed value reproduces a prior result on models that support seeding. Use the dedicated [Generate Image / Audio / Video](#generate-image-step) steps for deterministic, one-shot generation (you wire the step and pick the model); reach for the Prompt Call media_generation tool when the model should decide whether and how often to generate, or iterate on what it produced. ### Streaming Output Attaching a **[Streaming Result](https://seclai.com/docs/agent-steps/output#streaming-result-step)** step immediately after a prompt call enables token-by-token streaming to the user via Server-Sent Events. The model's output is delivered as it is generated, rather than waiting for the full response. Other steps can still run **in parallel** with the streaming result — for example, an [Add Memory](https://seclai.com/docs/agent-steps/memory#memory-add-step) step to persist the prompt call's output, or a [Webhook Call](https://seclai.com/docs/agent-steps/integration#webhook-step) to notify an external system. The streaming result and all parallel siblings receive the same prompt call output and execute concurrently. ### Input as Messages (Conversational Mode) When `use_step_input_as_messages` is enabled (simple mode only), the step parses its input as a JSON array of chat messages and sends them directly to the model — instead of wrapping a single prompt template in a user message. **Expected input format:** ```json [ { "role": "user", "content": "Hello, how are you?" }, { "role": "assistant", "content": "I'm doing well! How can I help you?" }, { "role": "user", "content": "Can you summarize this document?" } ] ``` This is the exact format produced by the **Load Memory** step when configured with `content_type: "application/json"`. **Why use this?** Sending each message as a separate element (rather than embedding the entire history in a single prompt string) enables **prompt caching** on supported providers. The model can cache the stable prefix of earlier messages and only process the new ones, significantly reducing latency and cost for conversational workflows. **Recommended conversational workflow:** 1. **Write Memory** (speaker: `user`) — save the incoming user message 2. **Load Memory** (order: `oldest_first`, content type: `JSON`) — load the full history as a messages array 3. **Prompt Call** (`use_step_input_as_messages: true`) — send messages to the model with a system prompt 4. **Write Memory** (speaker: `assistant`) — save the model's response 5. **Streaming Result** — stream the response to the user > **Important:** Use `oldest_first` order for prompt caching to work. The oldest messages form a stable prefix that the model can cache. Using `newest_first` would change the prefix on every turn, defeating the cache. **Adding additional context (RAG, preferences, metadata):** When your conversational agent also uses retrieval results, user preferences, or other contextual data, place that additional context in the `system_template` rather than in `prompt_template` (which must be `null` when using input-as-messages). Note that dynamic content in `system_template` (e.g. retrieval results that change per turn) will still change the cached prefix and can reduce prompt-caching effectiveness — the main caching benefit comes from keeping the conversation history as a stable message array via `oldest_first` ordering. Example `system_template` with additional context: ``` You are a helpful assistant. Use the provided context to answer accurately. Knowledge base results: {{step.search-kb-retrieval.output}} User preferences: {{step.recall-prefs-search_memory.output}} ``` This keeps the conversation messages clean (each message is sent individually for caching) while still providing the model with all the context it needs via the system prompt. When `use_step_input_as_messages` is enabled: - The `prompt_template` field is ignored (hidden in the editor) - The `system_template` is still used as the system prompt - The step fails with a clear error if the input is not valid JSON or doesn't match the expected `[{role, content}]` schema - Valid roles are: `user`, `assistant`, `system` ### Simple vs Advanced Format The `simple_format` field controls how the prompt is sent to the model: **Simple format** (`simple_format: true`, default): - Uses `prompt_template` (string) and `system_template` (string) - Works like interactive chat interfaces (ChatGPT, Claude) - Portable across model providers with little or no modification - `json_template` must **not** be provided **Advanced format** (`simple_format: false`): - Uses `json_template` (JSON object) — the model's native prompt payload - Gives access to provider-specific features and fine-grained control - May require modifications when switching between model providers - `prompt_template` must **not** be provided **Example advanced format JSON template:** ```json { "messages": [ { "role": "system", "content": "You are an expert analyst for {{organization.name}}." }, { "role": "user", "content": "Analyze this: {{agent.input}}" } ] } ``` String substitutions (e.g., `{{agent.input}}`, `{{metadata.*}}`) are supported inside both `prompt_template` and `json_template`. ### Use Case Examples **Content summarization:** ``` System: You are a concise content summarizer. Output a 2-3 sentence summary. Prompt: Summarize this article: {{agent.input}} ``` *Figure: Content summarization — the agent passes the input directly to a prompt call that produces a concise summary.* **Question answering with context (RAG):** ``` System: You are a helpful assistant for {{organization.name}}. Answer questions using ONLY the provided context. If the answer is not in the context, say "I don't have that information." Prompt: Context: {{step.retrieval.output}} Question: {{agent.input}} ``` *Figure: RAG pipeline: a Retrieval step searches the knowledge base, then a Prompt Call answers the question using the retrieved context.* **Structured data extraction:** ``` System: Extract the following information from the text and return as JSON. Prompt: {{agent.input}} ``` With `json_template`: ```json { "type": "object", "properties": { "company_name": { "type": "string" }, "revenue": { "type": "string" }, "employees": { "type": "integer" }, "headquarters": { "type": "string" } } } ``` *Figure: Structured extraction — a prompt call generates JSON, then an extract step parses it for downstream consumption.* **Multi-language response:** ``` System: You are a translator. Translate the input to {{metadata.target_language}}. Maintain the original formatting and tone. Prompt: {{agent.input}} ``` *Figure: Translation — the agent translates input to a target language specified in metadata.* ### AI Assistant The prompt call step includes an **AI assistant** that can generate a complete configuration based on a natural-language description of what you want the step to do. Click the **AI Assistant** button (requires a model to be selected first) to open the assistant modal. The assistant can set all prompt call fields: - **Model** and **model variant** — selects a cost-effective model that fits the task (e.g. tool use, structured output, long context) - **Mode** — chooses between simple mode (prompt + system prompt) and JSON mode (vendor-native payload) - **Prompt template** and **system template** — writes templates using substitution variables like `{{input}}` and `{{step..output}}` - **Temperature** and **max tokens** — tunes generation parameters for the task - **Tools** — enables tool use (e.g. web search) when the task requires it - **JSON template** — builds vendor-specific payloads when JSON mode is appropriate - **Formatting cleanup** — selects post-processing (Markdown → HTML, HTML → Markdown, plain text only) For Prompt Call steps, the assistant follows these conventions: - Use `{{agent.input}}` when referring to the user's original question/message - Use `{{input}}` only for previous-step output/context - Put core instruction logic in **System Template** - In **System Template**, prefer this order: original question (`{{agent.input}}`) → instructions → previous-step data (`{{input}}` / `{{step..output}}`) The assistant is aware of the full agent workflow context — all steps, the trigger configuration, and the knowledge bases available — so it can reference outputs from earlier steps and tailor the prompt to the agent's purpose. Each response also includes a **purpose** field (a brief description of the step's intent) and a **note** explaining what was changed or asking a follow-up question. --- ## Generate Image / Audio / Video Steps The dedicated generation steps — generate_image, generate_audio, and generate_video — produce media **deterministically** from a text prompt: you wire the step, pick a generation model, and it runs once. (For the agentic alternative, where the LLM decides whether and how often to generate, see the Prompt Call [Media generation tool](#prompt-call-media-generation).) Each calls a dedicated image/audio/video **generation model** (a catalog model that advertises an output modality and a per-unit price) and emits the asset(s) as the step's output manifest, referenced downstream by step id ({'{{step..attachments}}'}). Generation models are billed **per unit** — per image, per second, per 1,000 characters, or per 1,000 output tokens — shown on the model picker; unbillable models are hidden. ### Shared fields | Field | Type | Required | Description | | ----------------------------------- | ------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | model | enum | Yes | The generation model. Must produce this step's modality (image/audio/video). | | prompt_template | string | Yes | Describes the media to generate. Supports {'{{…}}'} substitutions. | | input_attachments | array | No | Optional reference image(s) — selector expressions (e.g. {'{{step..attachments}}'}) — for image **edit** (image-in → edited image) or **image-to-video** (animate a still). Only effective on models that accept image input. | | seed | integer | No | Optional reproducibility seed. Leave unset for fresh, varied output on every run; set a fixed value to reproduce the same result for the same prompt and options. Only honoured by models that support seeding. | | max_output_attachments | integer | No (64) | Safety ceiling on generated asset count — a backstop against a runaway generation, not a normal knob. | | max_output_total_bytes | integer | No (256 MB) | Safety ceiling on combined output size — high by default so large video never fails the step. | ### Generate Image | Field | Type | Default | Description | | ------------------------- | ------- | ------------- | --------------------------------------------------------------------------------------------------------------------- | | n | integer | `1` | Number of images to generate (capped by the model's max_n). Each image is billed separately. | | size | string | model default | Pixel size (e.g. `1024x1024`) — must be one of the model's supported sizes. | | aspect_ratio | string | — | Aspect ratio (e.g. `16:9`) — must be supported by the model; mutually exclusive with an explicit size on most models. | ### Generate Audio | Field | Type | Default | Description | | ------------------------- | ------ | ------------- | ------------------------------------------------------------------- | | voice | string | model default | Voice identifier — must be one of the model's supported voices. | | audio_format | string | model default | Output format (e.g. `mp3`, `wav`) — must be supported by the model. | ### Generate Video | Field | Type | Default | Description | | ----------------------- | ------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | resolution | string | model default | Video resolution (e.g. `720p`, `1080p`) — must be supported by the model. | | duration_s | integer | model default | Clip duration in seconds — must be one of the model's supported durations. Video is typically billed per second, so this drives cost. | | fps | integer | model default | Frames per second — must be one of the model's supported frame rates. Left at the model default when omitted. | ### Use Case Examples **Generate a marketing image:** draft the prompt with a model, then render it. ``` Prompt Call: "Write a vivid image prompt for a hero banner about {{input}}." → Generate Image: { model: , prompt_template: {{step.prompt.output}}, size: 1536x1024 } → Display Result ``` *Figure: Generate an image — a prompt call drafts the image prompt, the generate image step renders it, and the result is displayed.* **Narrate a script to audio:** write copy, then synthesize speech and archive it. ``` Prompt Call: "Write a 60-second narration script for {{input}}." → Generate Audio: { model: , prompt_template: {{step.prompt.output}}, voice: } → Write to S3: archive the generated mp3 ``` *Figure: Generate audio — a prompt call writes the script, the generate audio step narrates it, and the clip is archived to S3.* **Animate a still into a clip:** generate a key frame, then feed it to a video model. ``` Generate Image: { model: , prompt_template: "Product on a white studio backdrop" } → Generate Video: { model: