EXTRACT

Creating an EXTRACT Rule in Apica Ascent

To configure field extraction from log messages, follow the steps below to create an EXTRACT rule within a pipeline.

EXTRACT rule can help you convert unstructured logs into structured logs by using regex with re2 syntax.

1. Navigate to the Configure Pipeline

• Go to the pipeline where you want to apply the rule.

• Click on Configure Pipeline from the pipeline’s action menu.

2. Add a New EXTRACT Rule

• Hover over the + Add Rule button.

• Select EXTRACT from the rule type dropdown.

• A modal will open with a form organized into tabs.

From here, you can define the fields that you want to extract and configure the rule to match your specific use case.

When you select the EXTRACT rule, a new form will appear on the left side of the modal. The form is organized into tabs based on logical groupings.

3. Fill in Rule Details

In the Details tab:

• Specify the rule name, description, and other required fields.

• Click + Add More Parameters to define matching conditions.

Example:

Message =~ my-app

This ensures the rule is applied only to logs containing specific patterns.

4. Define the Extraction Pattern

• Enter a regular expression in RE2 syntax in the Pattern field that the rule will use to match and extract the desired fields from your log data. After filling in all required details, click Save to create the EXTRACT rule.

• This expression should match the log structure and include named capture groups to extract values. We recommend testing your regex in regex101.com. Additionaly you can validate the pattern by selecting the Validate button.

Example RE2 Pattern (Ingress Logs):

^(?P<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d+Z)\s+(?P<level>[A-Z]+)\s+\[(?P<app>[^\]]+)\]\s+\[pod=(?P<pod>[^\],]+),\s*namespace=(?P<namespace>[^\],]+),\s*environment=(?P<environment>[^\]]+)\]\s+(?P<message>.+)$

This pattern extracts the following fields:

• timestamp

• level

• app

• pod

• namespace

• environment

• message

5. Validate the Pattern

• In the modal, navigate to the right panel.

• Click Use Sample Logs → Custom Logs.

• Paste the following sample log data into the editor window and click Preview

Sample Raw logs:

[
    {
        "message": "2025-05-13T12:29:00.123456789Z INFO [my-app] [pod=my-app-abcd-12345, namespace=default, environment=production] Request received for delete rule"
    },
    {
        "message": "2025-05-13T14:29:00.123456789Z INFO [my-app] [pod=my-app-abcd-12345, namespace=default, environment=development] Request received for get rule"
    }
]

• Click Preview to confirm that fields are correctly extracted.

Verify the extracted fields from the raw message.

Change the expression not to match the logs, save pipeline and click preview

Message !~ my-app

Select any log from the logs and observe the extracted fields. No fields were extracted due to a non-matching expression.

6. Use JavaScript for Advanced Transformations

EXTRACT rules support inline JavaScript. Use the Event object to manipulate fields or define custom fields. Read Code Rule for more details. For example:

if (Event.AppName == "appERP") {
  Event.copyOfAppName = "appERPData";
}

Save pipeline and click preview to verify the new field added

8. Save and Apply the Rule

• Once all fields are configured and validated, click Save.

• The rule will be applied to incoming log lines that match the defined criteria.

• Extracted fields will be available for filtering, visualization, and alerting.

9. Use Pre-Built Rules (Optional)

Apica Ascent also provides pre-configured EXTRACT rules for commonly used log formats such as:

• IIS Logs

• Kubernetes Ingress Logs

• AWS VPC Flow Logs

These can help you get started quickly without writing custom patterns.