> For the complete documentation index, see [llms.txt](https://docs.apica.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.apica.io/flow/venn-and-pipeline-build.md). # Venn and Pipeline build Venn is your AI observability assistant in Apica Ascent. This guide shows how to use Venn to build and edit pipeline rules, what each rule kind does, and what changed in the pipeline builder layout. ### 1. What is Venn? Venn answers questions about your logs, metrics, traces, and anomalies — and can build pipeline rules for you from plain-language requests. **How to open Venn:** * **Floating bubble** — click the bubble in the corner of any page (tooltip **"Ask Venn"**). It opens the Venn drawer. * **Drawer** — a panel slides in from the right, titled **Venn**. You can pin it, expand it wider, start a **New chat**, or close it. * **Full page** — click on the Venn menu option from the top navbar for a full-screen chat. When the chat is empty you'll see **"Ask Venn anything"** with a few starter prompts. Type your question and press **Enter** to send (**Shift+Enter** for a new line).

*** ### 2. Building pipeline rules with Venn You do not need to know which rule type to choose. Describe your goal, and Venn determines the best option. **The flow:** 1. **Ask in plain language.** For example, *"Help me cut log volume from the payment app"* or *"Redact user name and IP address from access-logs."* 2. **Venn recommends and drafts.** If multiple namespaces or applications are available, Venn asks you to select them, then retrieves logs from the last hour, analyzes them, and drafts suitable rules. 3. **Review the suggestion card.** The card is headed **"Pipeline suggestion ready:"**. It shows the drafted rules and a short summary. Redaction suggestions also show a **"Redaction preview (before → after)"**. 4. **Click "Review."** The chat drawer closes and the **pipeline builder opens pre-filled** with the suggested rules. The **preview runs against real dataflow logs** so you can see the rules' effect before saving: 1. For **Forward** suggestions, the namespace/application Venn analyzed is **pre-selected** in the preview's Data Flows dropdowns and the preview runs against that flow automatically. 2. For other kinds (filter, rewrite, code, SIEM…), select the namespace and application on the **IN** tab of the **Pipeline Preview** section — the preview runs against whichever dataflow is selected. This selection only provides logs for the preview; it does not associate the pipeline with the dataflow. 5. **Validate with the DIFF tab.** Select the **DIFF** tab in **Pipeline Preview** to check the rule changes, using the left and right arrows to step through the previous and next logs. 6. **Save, then apply.** When you are satisfied with the preview, click **Save**. Then use **Apply Pipeline** to associate the pipeline with the namespace and application. The rules process incoming data only after you apply the pipeline. > **Nothing is saved automatically.** Suggested rules arrive as **draft** rules for you to review. Your own hand-authored rules are always kept — Venn's additions are grouped under headings like *Suggested Filters*, *Suggested Rewrites*, so you can tell them apart. Saving happens only when **you** press Save in the builder. **If a name is already taken:** when a suggested new pipeline's name already exists, Venn renames it and shows a banner (e.g. *A pipeline named "X" already exists, so the suggestion was named "Y". Edit the name above before saving…*). Change the name before saving if you'd like. **Don't want a suggestion?** Click **Dismiss** on the card. You'll be asked to confirm — dismissing discards the suggestion, and you'd need to ask Venn to regenerate it. #### Example: cut log volume from the payment app Ask: *"Help me cut log volume from the payment app."*

If multiple namespaces or applications are available, Venn asks you to select them. After you select a namespace and application, Venn retrieves logs from the last hour, analyzes them, and drafts pipeline rules to filter them.

Choose a suggestion, such as *"Suppress payment logs — keep 5 per minute, drop the rest."*

Venn may ask for a pipeline name. Choose a suggested name or enter your own.

After you provide a name, Venn presents the proposed pipeline and its rules.

Click **Review** to open the pipeline builder with the drafted rules. In the **Pipeline Preview** section, select the namespace and application on the **IN** tab. This selection provides logs for the preview. It does not associate the pipeline with the dataflow.

Select the **DIFF** tab in **Pipeline Preview** to validate the rule changes.

Use the left and right arrows to view the previous and next logs. This view shows which logs the pipeline filters. The following screenshot shows a filtered log.

When you are satisfied with the preview, click **Save**. Then use **Apply Pipeline** to associate the pipeline with the namespace and application. The rules process incoming data only after you apply the pipeline. #### Example: redact user name from access-logs Ask: *"Redact user name from access-logs."*

The suggested pipeline with the redaction review of the logs is shown below.

Click the **Review** button.

Use the **DIFF** view to validate the rule's output.

*** ### 3. The rule kinds Rules are added from the **Add Rule** menu in the builder. Ask Venn for any of these in plain language, or add them yourself.

#### Filter * **What it does:** Excludes whole events, or strips low-value labels off an event, to save storage and processing. * **When to use:** Cut ingest volume and cost by dropping noise. * **Ask Venn:** *"Drop debug-level logs from the checkout service."* #### Rewrite * **What it does:** Masks or replaces sensitive information using pattern matching — mainly PII redaction. * **When to use:** Redact emails, tokens, card numbers, etc. * **⚠️ Caveat:** Redaction is **permanent** — the original value is not recoverable once masked. Check the before → after preview first. * **Ask Venn:** *"Mask all email addresses in the auth logs."* #### Code * **What it does:** Runs a small custom script on each event, for logic a filter or rewrite can't express (parse, transform, enrich). * **When to use:** Advanced, one-off transformations. * **Ask Venn:** *"Write a rule that adds a field marking events over 500ms as slow."* #### Extract * **What it does:** Parses unstructured log text into structured fields using a pattern with named captures — each captured piece becomes its own field. * **When to use:** Turn raw log lines into queryable fields. * **Ask Venn:** *"Extract the status code and latency from my nginx logs."* #### Aggregate * **What it does:** Applies statistical functions (count, sum, average, min, max, median, percentiles, distinct…) over a time window. The result can be added back onto the event or turned into a metric. * **When to use:** Summarize high-volume events into counts or rates. * **Ask Venn:** *"Count errors per service every minute."* #### Forward * **What it does:** Chooses which events (and which fields) are forwarded to downstream destinations, to lower EPS and cost. The original is retained in storage. * **⚠️ Caveat:** Forward rules **cannot be mixed** with other rule types — a forward rule needs its **own pipeline**. * **Ask Venn:** *"Forward only error logs to the SIEM destination."* #### Stream * **What it does:** Redirects incoming logs by changing their Namespace and Application. By default it **copies** the events; if the originals are dropped it effectively **moves** them. * **When to use:** Route a subset of logs to a different namespace/app. * **Ask Venn:** *"Send audit logs to the security namespace."* #### SIEM * **What it does:** Matches known security signatures/patterns and turns matches into a real-time metric. Can optionally add a dashboard widget and an alert. * **⚠️ Note:** Any alert it creates starts **disabled** — enable it when you're ready. * **Ask Venn:** *"Detect failed SSH login attempts."* #### Tag * **What it does:** Identifies interesting events based on evaluation criteria and tracks them in real time. * **When to use:** Flag and follow events matching a condition. * **Ask Venn:** *"Tag every event mentioning 'OutOfMemory'."* > A pipeline with no rules shows: *"No rules have been configured for this pipeline. Select '+ Add Rule' to configure new rule."* *** ### 4. Editing an existing pipeline **Refine a suggestion:** with the builder open, just ask Venn again — *"make the filter also drop info logs"* — and the rules **refresh live** in the builder, replacing the previous AI suggestion. Your own rules stay put. **Structural changes** (without regenerating rule content) — Venn shows a card for each: | You ask | Card label | What happens | | ------------------------------------------ | ------------------- | ------------------------ | | Change rule order | **Reorder rules** | Execution order updated | | Remove rules | **Delete rules** | Named rules removed | | Rename the pipeline / edit its description | **Rename pipeline** | Name/description updated | Click **Review** on the card to apply it. These act on the pipeline **currently open in the builder** — if it isn't open you'll see *"Open the pipeline in the builder to apply this change."* Once applied, the card reads **"Applied to the pipeline."** All rule kinds are preserved — nothing gets regenerated or lost. #### Example: reorder the pipeline rules — swap rule #1 with rule #3

After reordering the rules:

*** ### 5. Deleting rules — you approve, always Deleting rules needs **your explicit approval**. Venn cannot approve a deletion on its own — you confirm it before anything is removed. This is a safety guard so an AI request can never quietly drop rules you meant to keep. #### Example: delete rule #3 from the pipeline