Using Fleet for Data Ingestion

Quick Start Guide for Using Ascent's Fleet Management to Get Up and Running with Ingesting Metrics and Logs into Ascent.

Step 1: Install Agent Manager

Step 2: Update Configuration File

Step 3: Apply Configuration File

Step 4: Verify Metrics/Logs are being Ingested

Step 1: Install Agent Manager:

Go to -> Explore -> Fleet

Click -> Install Manager

Select Platform: Linux

Select Agent Type: OpenTelemettry Collector

Click 'Proceed'

Click on "Download All"

• Open 'README' file for detailed instructions.

Go to Your Linux Terminal:

NOTE: Transfer 'Fleet Installation File' to the Linux host that you will collect data from.

• make sure file has permissions to allow to 'execute'

• Execute the following command to install the Agent Manager:

• $ sudo ./fleet-install.sh

Verify that the hostname is in the Fleet "Agents" UI tab:

Step 2: Update Your Configuration File:

Go to "Configurations" tab and search for:

• 'otelcol linux default config'

Then, click into the file to open the configuration file:

Copy the below code block of the Configuration file:

NOTE: You will have to insert your [ENV_URL_HERE]

• Your [ENV_URL_HERE] is your domain name:

Copy the below code block into the "Update Configuration" section in the UI:

receivers:
  hostmetrics:
    collection_interval: 60s
    scrapers:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
      memory:
        metrics:
          system.linux.memory.available:
            enabled: true
          system.memory.utilization:
            enabled: true
      disk:
      network:
      load:
      filesystem:
        include_virtual_filesystems: true
        metrics:
          system.filesystem.inodes.usage:
            enabled: true
          system.filesystem.usage:
            enabled: true
          system.filesystem.utilization:
            enabled: true
      paging:
      processes:
  filelog:
    include:
      - /var/log/syslog
      - /var/log/auth.log
    start_at: beginning
    operators:
      - type: add
        field: attributes.log_source
        value: ubuntu
      - type: move
        from: attributes["log_source"]
        to: resource["log_source"]
processors:
  attributes/os:
    actions:
      - key: ostype
        value: "linux"
        action: upsert
  attributes/host:
    actions:
      - key: hostname
        value: "{{$ .Agent.host_name }}"
        action: upsert
  batch:
    send_batch_size: 1000
    timeout: 5s
    
exporters:
  debug:
    verbosity: detailed
  prometheus:
    endpoint: 0.0.0.0:9464
  otlphttp/apicametrics:
    compression: gzip
    disable_keep_alives: true
    encoding: proto
    metrics_endpoint: "{{$ .Agent.secret.otelmetrics.endpoint }}"
    headers:
      Authorization: "Bearer {{$ .Agent.secret.otellogs.token }}"
    tls:
      insecure: false
      insecure_skip_verify: true
  otlphttp/logs:
    compression: gzip
    disable_keep_alives: true
    encoding: json
    logs_endpoint: "https://[ENV_URL_HERE]/v1/json_batch/otlplogs?namespace=Linux&application=otellogs"
    headers:
      Authorization: "Bearer {{$ .Agent.secret.otellogs.token }}"
    tls:
      insecure: false
      insecure_skip_verify: true
    sending_queue:
      queue_size: 10000
extensions:
service:
  extensions:
  pipelines:
    metrics/out:
      receivers: [hostmetrics]
      processors: [attributes/host, attributes/os]
      exporters: [otlphttp/apicametrics]
    logs/out:
      receivers: [filelog]
      processors: [attributes/host, batch]
      exporters: [otlphttp/logs]

NOTE: Currently, this configuration file is set up to collect syslogs. If you would like to collect different types of logs adjust the path to the logs you want to ingest:

   filelog:
    include:
      - /var/log/syslog
      - /var/log/auth.log

Step 3: Apply the Changes Made to the Configuration File:

Copy the below code block into the "Update Configuration" section in the UI:

Click "Update".

Then, go back to the "Agent" tab and click into the Linux hostname (where you'll be ingesting data from):

Step 4: Verify Metrics/Logs Are Being Ingested into Ascent:

Verify that logs are coming in and that it shows as "active":

Finally, go to Explore -> 'Logs & Insights' to verify datasource is there:

Optional* click on "Source Application" where you'll be able to see your logs: