# Getting Started with Flow

## Quick Start Guide for Using Ascent's Flow Solution to Optimize Your Data Pipelines.

This guide will teach you how to use the Flow solution to optimize your data pipelines. You will learn how to create a processing pipeline that filters unnecessary data from your logs, reducing storage costs. Finally, you will learn how to route that streamlined data to a downstream observability platform, such as Datadog.

For a full video walkthrough, please refer to our video guide:

{% embed url="<https://www.youtube.com/watch?v=An4RCgvGcko>" %}

Let's begin.

Prerequisite: Make sure to have logs ingested into the Ascent platform before getting started.

### Step 1: Create A New Pipeline:

Go to -> Explore -> Pipeline

Click -> Actions -> Create Pipeline

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FM0IjBfLYtuG8kp4mSxb8%2Fimage.png?alt=media&#x26;token=44b76a3a-077c-44ed-9257-62e26321c95e" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FliAJHUQc7TH9OBgayx7j%2Fimage.png?alt=media&#x26;token=0382aecd-36bd-4ea9-8cd5-2ea3a886a3fd" alt=""><figcaption></figcaption></figure>

Enter a name for the new Pipeline and press "Create"

### Step 2: Create A Filter Rule:

Click on the 3 dotted lines menu for the Pipeline you created

* Click on "Configure Pipeline"

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FNAE3KYfKeUxTeGGHp2Eo%2Fimage.png?alt=media&#x26;token=5c70f175-f4bb-4964-a5fb-80930cbcb29e" alt=""><figcaption></figcaption></figure>

Click on "Add Rule" -> FILTER

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FgzAP0PDg8EXEicUMLFCk%2Fimage.png?alt=media&#x26;token=e99496a5-ffa8-4898-b770-a39d1b1fbe15" alt=""><figcaption></figcaption></figure>

Enter mandatory fields:

* Name / Group

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FwVdQkXZXiK4FgD7u6k5t%2Fimage.png?alt=media&#x26;token=819081c3-65d5-4b07-a4eb-07ea636822c5" alt=""><figcaption></figcaption></figure>

Click on "Drop Labels"

Then, click on "Data Flows"

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FBGMPuOOX0KcEOMPeuOkq%2Fimage.png?alt=media&#x26;token=56555f23-86bb-4cd8-81d4-35214c09fd48" alt=""><figcaption></figcaption></figure>

Next, you will select what labels you want to drop.

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FFFJZw2x57LMsGrdriIEn%2Fimage.png?alt=media&#x26;token=fd36f7a4-3547-4467-93a2-7d4110a57d9a" alt=""><figcaption></figcaption></figure>

Enter the labels you want to drop on the left hand side as shown below:

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FD0NU8sYywgQxZjuLl6Oi%2Fimage.png?alt=media&#x26;token=293ea3df-438b-45ec-96be-52b87a5f49b9" alt=""><figcaption></figcaption></figure>

To preview the changes, go to the right-hand side and click "Pipeline Preview" -> "Run Pipeline"

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FDrC2TDlunml493DWQ4JW%2Fimage.png?alt=media&#x26;token=3cf8307f-83de-4fd2-8cb0-f709dbda0f48" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FsXBWKcYa8rohi5otfUbt%2Fimage.png?alt=media&#x26;token=4a706641-c50a-4e2d-8b4c-8ceaaf67701c" alt=""><figcaption></figcaption></figure>

Click "Save Pipeline"

Next, "Apply Pipeline" by clicking on the 3 dot menuand clicking "Apply Pipeline"

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2F9hnM4FsqFEJVw1m6thQH%2Fimage.png?alt=media&#x26;token=6200294a-94cb-4d97-ab13-1f620d784974" alt=""><figcaption></figcaption></figure>

Then, select the namespace and logs you want to apply the new FILTER RULE ( in this case, we are applying it to our "OtelDemo" logs

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FTokESQ8B39PgYUH3cym0%2Fimage.png?alt=media&#x26;token=99c4b38f-aa3c-43f2-a186-90da8edf883d" alt=""><figcaption></figcaption></figure>

Click "Apply"

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2F5D6WBFtehDjL0tb1yd7r%2Fimage.png?alt=media&#x26;token=25fae608-6904-4c3e-b007-4de48746b882" alt=""><figcaption></figcaption></figure>

### Step 2: Create A Filter Rule:

Create a Forwarder (Datadog in this example), to push our FILTERED OTEL logs downstair to another Observability platform.

Click on "Integrations" -> "Forwarders"

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FCKFk8WadQPbMQqU629Vk%2Fimage.png?alt=media&#x26;token=ca2adda7-80fe-46ae-a078-62643c847777" alt=""><figcaption></figcaption></figure>

### Step 3: Create A Forwarder (DataDog in this example):

Click on "Add Forwarder" and select your destination (Datadog in our example)

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FUNfYsXxZkKRhAkvH5Uhy%2Fimage.png?alt=media&#x26;token=cb9ecbfa-4fbd-4864-83be-458877822600" alt=""><figcaption></figcaption></figure>

Then, copy over the "DataDog (JSON) configs as shown below:

Buffer\_size: 16000

Host: app.datadog.com

Tags: logs

Type: JSON

Name: Datadog Forwarder

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FYCZoWlINlqPMUhyQE09p%2Fimage.png?alt=media&#x26;token=14b1b180-0372-4d50-af23-ff6b81defe1b" alt=""><figcaption></figcaption></figure>

Click "Create"

### Step 4: Assign the Forwarder to the Logs:

Next, go back to "Pipelines" and

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FNRqyowtb58an4jOjpWmf%2Fimage.png?alt=media&#x26;token=07af3bdd-0921-4128-b401-4b26f51de8f7" alt=""><figcaption></figcaption></figure>

Click on "Map Forwarder" from the 3 dot menu:

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FyF7cyKLEM95fozzlod0C%2Fimage.png?alt=media&#x26;token=d8019fe7-3416-4238-a436-0dc634c7d4a0" alt=""><figcaption></figcaption></figure>

Select the "DataDog Forwarder" that you created and click OK:

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FVzvX6v5IDIQeJ1TyMCtq%2Fimage.png?alt=media&#x26;token=61b1ebfd-71df-47f6-84ee-e16ee51924c0" alt=""><figcaption></figcaption></figure>

### Step 5: Verify Data in Destination (Datadog in this example):

Go to your Datadog dashboard and verify data coming in as expected:

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FAIbLbqEzmv0h4N5u0MwU%2Fimage.png?alt=media&#x26;token=4ee99f4a-5c2d-4891-bbe4-0a3cf1aaa5f1" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2F1DClsAFP5t3Gqd3lW7iR%2Fimage.png?alt=media&#x26;token=c980b254-387e-4577-8ce5-d9b0af173912" alt=""><figcaption></figcaption></figure>

As you can see, data ingestion has decreased after the FILTER rule was applied:<br>

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FFNzjY02Bta7CEy5Ef1YR%2Fimage.png?alt=media&#x26;token=5b0908ef-cc20-4a19-83a5-c39cca0966cf" alt=""><figcaption></figcaption></figure>

### Conclusion:

By following this guide, you have learned how to successfully use Flow to manage and optimize your telemetry data. You now know how to build a data pipeline that filters unneeded fields, drops irrelevant log messages entirely, and forwards the clean, cost-effective data to a downstream platform like Datadog. Applying these techniques allows you to significantly reduce observability costs while maintaining cleaner and more efficient data pipelines.
