# Persistence Queue for Forwarder Buffers

### Overview

A persistence queue is a **file-based buffer** used by forwarders to temporarily store incoming data on disk before it is forwarded. This mechanism helps:

* Prevent data loss during backpressure or downstream outages
* Handle ingestion bursts by allowing larger queue sizes
* Control behavior when the queue becomes full

Persistence queues can be configured while **creating or updating** a forwarder.

#### Steps to create forwarder and configure persistence queue

* Click the **Integrations** tab from the navigation bar.
* Navigate to the **Forwarders** tab.
* Click **Add Forwarder** (top-right) to open the *Create Forwarder* modal.

  <figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FIxksfnjgXWKB70iPTfiz%2Fimage.png?alt=media&#x26;token=d41a78f0-f6a5-45c2-badb-0f843e8453c5" alt=""><figcaption></figcaption></figure>

### Persistence Queue Settings

You will find the following three settings related to the persistence queue:

1. **Buffer Size** : Maximum size (in bytes) of the persistent, file-based queue. Larger values use more disk and allow higher ingest bursts.
   1. **Available Space Indicator**

      1. The suffix shows the currently available disk space (in bytes).
      2. This is calculated as:

      **Total allowed space (1 GB) − Space used by all existing forwarders.**
2. **Persist Buffer** :&#x20;
   1. A toggle to **enable or disable persistence**.
   2. **Important Notes:**
      1. When persistence is **disabled**:
         1. The buffer size defaults to **10 MB**, even if a larger value is configured.
      2. To use the configured buffer size:
         1. Persistence **must be enabled**
3. **Queue policy**: Specifies the action to take when the buffer reaches its maximum size due to backpressure.

   Available options:

   **Drop Incoming Logs**

   * Drops all incoming logs in the request
   * Returns **HTTP 200**
   * Continues dropping logs until backpressure is relieved

   **Block Incoming Logs**

   * Blocks all incoming log requests
   * Returns **HTTP 429 (Too Many Requests)** with a retry signal
   * Continues blocking until backpressure is relieved

#### Finalizing Forwarder Creation

* Fill in the other details and click on create. On successful creating, forwarder will be shown on list.

  <figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FWlIVYLF9zVESt4cCBnhQ%2Fimage.png?alt=media&#x26;token=d02d5f9d-1361-4a80-b42b-17858c64d94b" alt=""><figcaption></figcaption></figure>

#### Map forwarder to data-flows

* Go to Explore -> Logs & Insights page via navbar.

  <figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FFfmoACjtncb0bb8uAbkE%2Fimage.png?alt=media&#x26;token=45b3b90f-b3c6-432b-8dd1-da7b7abd11e4" alt=""><figcaption></figcaption></figure>
* Click on a dataflow and click on the 3 dots on the top right of the table, and select Map Forwarder

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FttUu07p5TDJwsrlml2PA%2Fimage.png?alt=media&#x26;token=04561e35-8600-48db-af0f-8744b7923094" alt=""><figcaption></figcaption></figure>

* From the list Select your Forwarder and click on OK.
* Your Forwarder should show up in the list, and mapped to the flow you selected.

<figure><img src="https://2948796384-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LmzGprckLqwd5v6bs6m%2Fuploads%2FTBs5EvKCPdoQ6rcDw1jg%2Fimage.png?alt=media&#x26;token=a462a35d-ef7f-4319-a36a-fcc1650a7b50" alt=""><figcaption></figcaption></figure>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.apica.io/flow/persistence-queue-for-forwarder-buffers.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
