Introduction

The digital landscape is evolving at an unprecedented pace. Enterprises are migrating to cloud-native architectures, embracing microservices, Kubernetes, and distributed applications to stay competitive. However, this shift introduces a new set of challenges—traditional monitoring tools are struggling to keep up with the scale, complexity, and velocity of modern applications.

This is where next-generation observability platforms, like Apica’s Ascent Platform, come in. Built on Kubernetes and OpenTelemetry, Apica delivers a scalable, AI-powered, cloud-native observability solution designed to handle billions of logs, metrics, and traces in real time.

More information on Kubernetes with OpenTelemetry

Why Traditional Monitoring is Failing in Cloud-Native Environments

In the past, traditional monitoring tools were sufficient for monolithic applications deployed on static infrastructure. However, modern applications are distributed, dynamic, and ephemeral.

Challenges with Traditional Monitoring:

• Data Silos – Logs, metrics, and traces are collected separately, making root-cause analysis slow and inefficient.

• Scalability Issues – Legacy tools struggle to handle high-cardinality telemetry data from microservices.

• Lack of Context – Traditional APM tools focus on isolated performance metrics, failing to provide full-stack observability.

• High Costs – Observability data grows exponentially, leading to excessive storage and retention costs.

• Manual Effort – Engineers spend too much time managing telemetry pipelines and analyzing fragmented data.

To address these challenges, enterprises must shift to a cloud-native observability approach that is scalable, cost-efficient, and AI-driven.

Apica’s Cloud-Native Observability: Built for the Future

Apica’s Ascent Platform is designed from the ground up to tackle modern observability challenges. Unlike traditional monitoring tools, it is built on Kubernetes, enabling infinite scalability and seamless multi-cloud deployments.

Key Advantages of Apica’s Cloud-Native Observability:

• Kubernetes-Powered – Dynamically scales observability pipelines, eliminating bottlenecks.

• Unified Data Store (InstaStore™) – Eliminates data silos by storing logs, metrics, traces, and events in a single repository.

• ZeroStorageTax Architecture – No more expensive tiered storage; data is stored in infinitely scalable object stores (AWS S3, Azure Blob, Ceph, etc.).

• AI-Driven Insights – Uses AI/ML anomaly detection, GenAI assistants, and automated root-cause analysis to accelerate issue resolution.

• Multi-Cloud & Hybrid Ready – Seamlessly integrates with AWS, Azure, GCP, and on-prem environments.

• Full OpenTelemetry Support – No proprietary agents needed—fully compatible with OpenTelemetry, Prometheus, Jaeger, and Loki.

As enterprises scale their applications, they need an observability platform that scales with them. Apica’s Kubernetes-native approach enables organizations to gain full-stack observability across highly distributed, multi-cloud environments.

Welcome to Apica Ascent

Apica Ascent is a powerful full-stack Telemetry Data Management and Observability platform designed to streamline and optimize your entire data life-cycle: Collect, Control, Store, and Observe.

Observability Data Lifecycle

The Apica Ascent platform consolidates observability data into a single platform, focusing on (M)etrics, (E)vents, (L)ogs, and (T)races, commonly known as MELT data. This integrated approach to MELT data is crucial for efficient root cause analysis. For example, if you encounter an API performance issue represented by latency metrics, being able to drill down to the API trace and accompanying logs becomes critical for faster root cause identification. Unlike traditional observability implementations, where data sits in separate silos that don't communicate, Apica Ascent ensures a cohesive view of all MELT data, leading to faster root cause outcomes.

This makes the Ascent platform a reliable first-mile solution for consolidating MELT data within your enterprise environments. Experience a seamless, fully integrated observability solution that enhances performance and efficiency across your infrastructure.

Capabilities

Apica Ascent employs a unified view of your enterprise, utilizing a full-stack approach to observability data life cycle management. By seamlessly integrating various capabilities, Apica Ascent facilitates a smoother and more effective root cause analysis process.

Communities and Compliance

Apica Ascent takes pride in its commitment to security and compliance. The platform adheres to SOC 2 Type II Compliance standards and is an esteemed member of the Cloud Native Computing Foundation (CNCF).

The Apica Ascent UI is your window to your IT data, logs, and metrics - ingested from all of your data sources and converged onto a single layer. The Apica Ascent UI enables you to perform a wide range of operations – from simple uptime monitoring and error troubleshooting to capacity planning, real-time forensics, performance studies, and many more.

You can access the Apica Ascent UI by logging into your Apica Ascent instance URL using your account credentials.

The navigation bar at the right side of the UI allows you to access your:

• Dashboards

• Queries

• Alerts

• Explore

• Events

• Rules

• Settings

The following sections in this article describe the various elements of the Apica Ascent UI and their purposes.

Dashboards

A dashboard is a collection of visualizations and queries that you've created against your log data. You could create dashboards to house visualizations and queries for specific as well as multiple data sources. Everything contained within a dashboard is updated in real-time.

The Dashboards page on the Apica Ascent UI lists all the dashboards you've created within Apica Ascent. Dashboards that you've favorited are marked with a yellow star icon and are also listed under the Dashboards dropdown menu for quick access in the navigation bar. The following images depict dashboards that you can create using Apica Ascent.

Queries

Apica Ascent enables you to write custom queries to analyze log data, display metrics and events, view and customize events and alerts, and create custom dashboards. The Queries page lists all of the queries you've created on Apica Ascent. You can mark some of them as favorites or archive the ones, not in use. Your favorite queries also appear in the drop-down menu of the Queries tab for quick access.

Alerts

Apica Ascent enables you to set alerts against events, data, or metrics of interest derived from your log data. The Alerts page on the UI lists all of the alerts you've configured on Apica Ascent. You can sort and display the list of alerts by their name, message, state, and the time they were last updated or created. Depending on your user permissions within Apica Ascent, you can click an alert to view more information or reconfigure the alert based on your need.

The following image depicts a typical Alerts page on the Apica Ascent UI.

Explore

The Explore page lists all of the log streams generated across your IT environment that are being ingested into Apica Ascent. The Explore page lists and categorizes logs based on Namespace, Application, ProcID, and when they were last updated. By default, logs are listed by the time they were ingested with the most recent applications appearing on the top. You can filter log streams by namespaces, applications, and ProcIDs. You can also filter them by custom time ranges.

You can also click into a specific application or ProcID to view logs in more detail and to search through or identify patterns within your log data.

The following image depicts a typical Explore page on the Apica Ascent UI.

Events

The Events page lists all the important events that have occurred in the Apica Ascent Platform. Events are listed by their Name, Message, and the time they were created. The Event page tracks important service notifications like service restarts, license expiry, etc...

Create

The Create dropdown menu enables you to create new reports, queries, dashboards, and alerts, as shown in the following image.

A function-specific modal appears based on what you select from this dropdown menu.

Why Kubernetes is a Game-Changer for Apica Ascent

As enterprises scale their cloud-native applications, they need an observability platform that can keep up with dynamic workloads, high-velocity data streams, and distributed architectures. Kubernetes has emerged as the foundation for modern observability platforms because it enables infinite scalability, automated resilience, and superior resource efficiency.

Apica’s Ascent Platform, built on Kubernetes, leverages these advantages to deliver next-generation observability—one that scales on demand, ensures high availability, and optimizes infrastructure resources efficiently.

Infinite Scale & Elastic Resource Management

Observability data is massive and continuously growing. Logs, metrics, traces, and events are ingested at an unprecedented scale, especially in high-throughput environments like fintech, telecom, and SaaS platforms.

Traditional monitoring solutions struggle because they rely on static infrastructure, making it difficult to scale on demand. Kubernetes, on the other hand, provides:

Horizontal Scalability – Automatically scales observability workloads based on real-time ingestion rates.

• Dynamic Resource Allocation – Ensures workloads receive the right amount of CPU, memory, and storage.

• Event-Driven Autoscaling – Uses Horizontal Pod Autoscaler (HPA) and Vertical Pod Autoscaler (VPA) to dynamically adjust observability workloads.

How Apica Leverages Kubernetes for Infinite Scale

• Auto-Scaling Observability Pipelines – Apica’s OpenTelemetry-based collectors automatically scale based on traffic volume, ensuring consistent performance.

• Seamless Multi-Cluster Deployment – Apica’s platform runs across Kubernetes clusters in AWS, Azure, GCP, and on-premise for global observability.

• Optimized Data Processing – High-throughput workloads are distributed across multiple nodes for maximum efficiency and minimal latency.

Result: Enterprises using Apica Ascent based on Kubernetes can ingest billions of logs, traces, and metrics in real time without worrying about infrastructure limitations.

Seamless Scaling for Any Environment

Modern enterprises operate in multi-cloud and hybrid environments, where observability data comes from Kubernetes clusters, virtual machines, serverless functions, and on-premises data centers.

Kubernetes removes infrastructure constraints by allowing observability workloads to be deployed across any cloud provider or on-prem environment, ensuring consistent visibility across all application layers.

• Run observability workloads anywhere – On-prem, hybrid cloud, or multi-cloud setups.

• Unified Observability Across Diverse Environments – Monitor Kubernetes, VMs, and serverless environments in a single platform.

• Zero Vendor Lock-In – Apica’s platform is built on open standards (OpenTelemetry, Prometheus, Jaeger) and deployable across AWS, Azure, GCP, Oracle Cloud, and private data centers.

How Apica Enables Seamless Scaling

• Scalable Resourcing – Kubernetes allows Apica to scale observability workloads for multiple customers or teams without resource contention.

• Cloud-Agnostic Deployment – Apica’s observability platform runs natively across any cloud provider or on-premises Kubernetes cluster.

• Unified Observability at Global Scale – Centralized data collection, analytics, and AI-driven insights across all Kubernetes environments.

Result: Enterprises gain full observability across all environments, whether running in AWS, Azure, GCP, or on-prem.

Resilience, Fault-Tolerance, and Self-Healing Architecture

One of Kubernetes' biggest advantages is its self-healing capabilities, ensuring that observability workloads remain highly available and fault-tolerant.

• Automatic Failover & Pod Recovery – Kubernetes automatically replaces failed observability agents and collectors, ensuring no gaps in monitoring.

• Load Balancing for Observability Workloads – Kubernetes evenly distributes data ingestion, preventing bottlenecks in observability pipelines.

• Multi-Region & Disaster Recovery Ready – Kubernetes automates failover between cloud regions, ensuring continued observability in case of outages.

How Apica Ensures High Availability with Kubernetes

• Redundant Observability Agents – Apica deploys multiple OpenTelemetry Collectors to prevent data loss during failures.

• AI-Driven Incident Recovery – Apica’s AI proactively detects infrastructure failures and triggers automated remediation workflows.

• Built-in Kubernetes Load Balancing – Ensures efficient routing of telemetry data to optimize performance.

Result: No single point of failure, ensuring continuous observability, even during infrastructure outages.

Security & Compliance at Scale

Observability data contains sensitive business insights, requiring enterprise-grade security and compliance measures. Kubernetes provides built-in security capabilities that make it ideal for running observability platforms at scale.

• RBAC (Role-Based Access Control) – Granular access control for observability workloads.

• End-to-End Encryption – TLS encryption for telemetry data in transit and at rest.

• Network Segmentation & Pod Security – Prevents unauthorized access to observability data.

• Multi-Tenant Observability with Isolation – Ensures customers or teams have secure, isolated access to their data.

How Apica Delivers Secure & Compliant Observability

• Secure Observability Pipelines – Data is encrypted at rest and in transit using TLS 1.2+ and AES encryption.

• Multi-Tenant Data Isolation – RBAC ensures fine-grained access control across teams and business units.

• Long-Term Retention for Compliance – Apica’s InstaStore™ data lake ensures observability data meets SOC 2, GDPR, HIPAA, and enterprise compliance standards.

Result: Enterprises gain observability at scale while ensuring full security and compliance.

Ascent Leverages Kubernetes Key Benefits and Best Practices

Kubernetes is redefining the observability landscape, enabling platforms like Apica’s Ascent to deliver:

• Infinite Scalability – Dynamically scales telemetry pipelines to handle billions of logs, traces, and metrics.

• Global Observability Across Any Environment – Runs on AWS, Azure, GCP, Oracle Cloud, and on-premises Kubernetes clusters.

• AI-Driven Automation & Self-Healing – Automatically detects and resolves failures, reducing operational overhead.

• Cost-Optimized & Storage-Efficient – Leverages object storage (S3, Azure Blob, Ceph) to eliminate unnecessary costs.

• Built-In Security & Compliance – Ensures full encryption, RBAC access control, and regulatory compliance.

Apica Ascent: Built on Kubernetes for Ultimate Scale & Performance

Observability isn’t just about collecting logs, metrics, and traces—it’s about ensuring real-time insights, high performance, and cost-efficiency at any scale. Traditional monitoring solutions often struggle with large-scale data ingestion, leading to performance bottlenecks, slow query times, and high storage costs.

Apica’s Ascent Platform, built on Kubernetes, solves these challenges by providing infinite scalability, AI-driven optimization, and seamless multi-cloud support. With a unified data store, OpenTelemetry-native architecture, and intelligent workload management, Apica delivers unparalleled observability performance while reducing operational complexity and costs.

More on Ascent Kubernetes Integration

InstaStore™: Kubernetes-Powered Unified Telemetry Data Lake

One of the biggest challenges in observability is data storage and retention. Traditional monitoring solutions rely on tiered storage models, leading to high costs, data fragmentation, and slow query times.

More information on Lake and InstaStore

Apica’s InstaStore™ data lake, built on Kubernetes, eliminates these limitations by providing:

• Infinite scalability – Stores billions of logs, traces, and metrics without performance degradation.

• ZeroStorageTax architecture – No more storage tiering, reducing storage costs by up to 60%.

• Real-time data indexing – Instant query access to historical and real-time telemetry data.

• Multi-cloud compatibility – Supports AWS S3, Azure Blob, Ceph, MinIO, and other object storage providers.

How InstaStore™ Enhances Observability Performance

• Single source of truth – Eliminates data silos by storing logs, metrics, traces, and events in a unified repository.

• On-demand query acceleration – Uses high-speed indexing for sub-second query response times.

• Long-term retention & compliance – SOC 2, GDPR, HIPAA-compliant storage for enterprise observability data.

Result: Enterprises can store, query, and analyze observability data instantly, at a fraction of the cost of traditional solutions.

Ascent Flow (Telemetry Pipeline): Data Flow Optimization at Scale

Observability pipelines must ingest, process, and export massive volumes of telemetry data while maintaining low latency and high efficiency. Without proper optimization, unstructured data overloads monitoring systems, leading to delays, noise, and unnecessary costs.

Apica’s Telemetry Pipeline, built on Kubernetes, solves this by:

• Filtering, enriching, and transforming telemetry data in real time.

• Automatically routing observability data to the most cost-effective storage backend.

• Optimizing ingestion rates to reduce infrastructure costs and enhance performance.

• Providing a drag-and-drop interface for managing data pipelines effortlessly.

More information on Flow

Fleet Management: Automated Deployment & Scaling of Observability Agents

The Challenge: Managing Observability Agents at Scale

In modern enterprise environments, observability data is collected from thousands of microservices, virtual machines, containers, and cloud functions. Manually deploying, configuring, and maintaining OpenTelemetry agents, Fluent-bit log collectors, and Prometheus exporters is resource-intensive and error-prone.

• Config drift leads to inconsistent telemetry data across environments.

• Manual agent updates result in security vulnerabilities and broken data pipelines.

• Lack of centralized management makes troubleshooting difficult.

Apica’s Fleet Management: Kubernetes-Powered Automation

Apica solves these challenges with Fleet Management, an automated system for managing OpenTelemetry collectors and other observability agents at enterprise scale.

• Automated Agent Deployment – Uses Kubernetes DaemonSets and StatefulSets to deploy and manage observability agents across clusters.

• Zero-Drift Configuration Management – Ensures all observability agents stay in sync with the latest configurations.

• Real-Time Health Monitoring – Continuously tracks agent status, performance, and data collection efficiency.

• Multi-Cloud & Hybrid Support – Deploys agents across AWS, Azure, GCP, on-prem environments, and edge locations.

Result: Enterprises eliminate manual observability agent management, ensuring consistent, reliable telemetry collection at scale.

More information on Fleet

Application Performance Monitoring (APM)

End-to-end distributed tracing for microservices: OpenTelemetry enables deep visibility into microservices interactions by capturing traces across service boundaries. This allows developers and operations teams to understand request flow, identify problematic dependencies, and detect failures in a distributed system. By leveraging OpenTelemetry’s context propagation, teams can follow a request from its origin to its termination, providing a clear picture of dependencies and bottlenecks. This improves troubleshooting efficiency, reduces the mean time to resolution (MTTR), and helps organizations build more resilient, scalable architectures. Additionally, OpenTelemetry supports integrations with distributed tracing backends such as Jaeger, Zipkin, and commercial solutions, ensuring flexibility in visualization and analysis.

Identifying latency bottlenecks in cloud-native environments: By collecting granular performance data, OpenTelemetry helps teams pinpoint where delays are occurring in an application. Whether it’s a slow database query, an overloaded service, or network latency, OpenTelemetry provides the data needed to optimize system responsiveness and improve user experience. With built-in support for metrics and histograms, OpenTelemetry allows teams to measure request duration, throughput, and error rates, enabling proactive performance tuning. Furthermore, OpenTelemetry facilitates real-time alerting on latency spikes, allowing DevOps teams to quickly diagnose and mitigate issues before they impact users. This level of insight is particularly beneficial for cloud-native applications where dynamic scaling and complex service interactions demand constant monitoring and optimization.

Infrastructure and Cloud Monitoring

Collecting host and container-level metrics: OpenTelemetry provides extensive support for collecting system-level and container-level metrics, including CPU, memory, disk usage, and network statistics. This enables teams to track resource consumption across distributed environments, identify performance anomalies, and optimize infrastructure utilization. By leveraging OpenTelemetry’s support for metric aggregation and real-time monitoring, organizations can ensure their applications remain resilient under varying workloads.

Monitoring Kubernetes clusters at scale: Kubernetes environments introduce unique challenges due to their dynamic and ephemeral nature. OpenTelemetry integrates seamlessly with Kubernetes to provide real-time visibility into cluster health, pod performance, and service-to-service communications. It enables DevOps teams to monitor workload scheduling efficiency, detect failing pods, and correlate application performance with underlying infrastructure issues. By centralizing observability across multiple clusters, OpenTelemetry empowers organizations to maintain high availability and reduce operational overhead in cloud-native environments.

Log Correlation with Traces and Metrics

Unified observability for root cause analysis: OpenTelemetry provides a comprehensive approach to observability by linking logs, metrics, and traces together, enabling teams to perform in-depth root cause analysis. By correlating log events with specific traces and spans, teams can identify exactly where failures occur within a distributed system, reducing the time spent diagnosing incidents and improving mean time to resolution (MTTR). This unified observability approach ensures that developers and operators have a complete understanding of system behavior, making debugging and performance optimization more efficient.

Enriching logs with trace and span context: OpenTelemetry enhances logging by automatically injecting trace and span identifiers into log messages, allowing for precise contextualization of events. This enrichment enables teams to follow an event from initiation through completion, offering clear insights into request flow and dependencies. Additionally, integrating log correlation with tracing helps detect patterns, anomalies, and dependencies that might not be immediately visible when logs are analyzed in isolation. This capability is especially beneficial in microservices architectures, where tracking down issues across multiple services can be complex without proper log-trace correlation.

Security and Compliance Observability

Capturing audit trails with OTEL logs and traces: OpenTelemetry enables organizations to create detailed audit trails by collecting logs and traces that capture user activity, API calls, and system interactions. These audit trails help organizations meet compliance requirements by providing clear, verifiable records of all system activities. By maintaining an immutable record of telemetry data, OpenTelemetry enhances accountability and security, ensuring that organizations can detect and investigate security incidents efficiently.

Detecting anomalies and unauthorized access patterns: OpenTelemetry’s advanced telemetry data collection allows security teams to analyze trends, detect anomalies, and identify unauthorized access attempts in real-time. By correlating logs, traces, and metrics, OpenTelemetry provides a holistic view of system behavior, helping teams recognize suspicious patterns, mitigate security threats, and prevent potential data breaches. This proactive security monitoring is essential for maintaining regulatory compliance and protecting sensitive data in distributed and cloud-native environments.

Business Analytics and SLO Monitoring

Defining Service Level Objectives (SLOs): Service Level Objectives (SLOs) are key performance indicators (KPIs) that define the desired reliability and performance targets for services. OpenTelemetry enables organizations to collect and analyze telemetry data that aligns with predefined SLOs, ensuring services meet business expectations. By leveraging OpenTelemetry metrics, organizations can measure service uptime, response times, and error rates, allowing teams to proactively address performance degradations before they impact end users. This approach fosters a culture of reliability engineering and helps teams adhere to Service Level Agreements (SLAs).

Analyzing user behavior and optimizing transactions: OpenTelemetry provides deep insights into user interactions and application workflows by capturing traces and metrics across distributed systems. By analyzing user journeys, organizations can identify friction points, optimize performance, and enhance user experience. OpenTelemetry allows businesses to track critical transactions, detect drop-offs, and correlate them with system behavior, ensuring continuous improvement. Additionally, businesses can leverage telemetry data to fine-tune application logic, allocate resources efficiently, and personalize user interactions based on real-time performance trends.

Solving Observability Fragmentation

Before OpenTelemetry, organizations struggled with multiple proprietary monitoring agents, each producing telemetry data in incompatible formats, leading to data silos, increased complexity, and a lack of correlation between logs, metrics, and traces. These challenges made it difficult to gain a comprehensive view of system health and troubleshoot issues efficiently.

OpenTelemetry unifies telemetry collection across all environments, ensuring seamless interoperability between tools and services, allowing teams to consolidate their observability strategy while reducing operational overhead and improving incident response times.

Reducing Vendor Lock-in

Traditional observability solutions force users into proprietary ecosystems, making migrations and integrations difficult. This often leads to increased costs, limited customization, and difficulty in adapting to evolving business needs.

OpenTelemetry decouples telemetry collection from backend storage and analysis, allowing organizations to switch observability platforms without re-instrumenting applications, ensuring greater flexibility, scalability, and long-term sustainability of their monitoring strategy.

Improving Performance and Cost Efficiency

With OpenTelemetry, organizations can eliminate redundant monitoring agents, reducing system overhead and lowering observability costs. By optimizing data collection and leveraging intelligent sampling and filtering, businesses can gain deep visibility without incurring excessive data ingestion costs. Additionally, OpenTelemetry provides the flexibility to fine-tune data collection strategies, ensuring that only the most valuable telemetry data is retained and processed.

This targeted approach not only enhances performance but also mitigates the risk of overwhelming storage and analytics systems with excessive data. By standardizing observability across an organization, OpenTelemetry helps engineering teams make data-driven decisions more effectively while controlling infrastructure spending.

Enhancing Developer and SRE Productivity

OpenTelemetry’s automatic instrumentation and standardized APIs make it easier for developers and SRE teams to implement observability across applications. By reducing the need for manual instrumentation, teams can accelerate deployment cycles and ensure that telemetry data is collected consistently and reliably.

This results in faster debugging, reduced MTTR (Mean Time to Resolution), and increased deployment confidence, while also enabling proactive issue detection and automated root cause analysis. With OpenTelemetry, organizations can shift from reactive troubleshooting to predictive monitoring, allowing engineering teams to optimize performance before issues escalate into major incidents.

Regulatory and Compliance Benefits

Many industries require strict compliance with security and auditing standards, including regulations such as GDPR, HIPAA, and SOC 2. OpenTelemetry provides structured telemetry data that simplifies compliance reporting by offering detailed, real-time insights into system activity, ensuring better traceability and transparency. By capturing rich metadata within traces, metrics, and logs, OpenTelemetry enhances auditability, enabling security teams to quickly detect and respond to anomalies.

Furthermore, OpenTelemetry’s vendor-neutral approach allows organizations to centralize security monitoring across multiple platforms, ensuring consistency in compliance efforts while reducing reliance on proprietary solutions.

What is OpenTelemetry (OTEL)?

OpenTelemetry (OTEL) is an open-source observability framework that provides a standardized approach to collecting, processing, and exporting telemetry data—including traces, metrics, and logs—from applications and infrastructure. It is a vendor-neutral solution designed to help organizations gain deep insights into the performance, health, and behavior of their distributed systems without being locked into proprietary observability tools.

By unifying telemetry collection across different platforms, programming languages, and monitoring solutions, OpenTelemetry simplifies instrumentation, reduces integration complexities, and enhances observability capabilities for modern cloud-native applications.

Definition and Purpose

At its core, OpenTelemetry serves the following primary purposes:

1. Standardization of Observability Data – OTEL defines a common set of APIs, libraries, and protocols for collecting and transmitting telemetry data, ensuring that observability data is structured and consistent across different environments.

2. Vendor-Neutral Telemetry Collection – Unlike proprietary solutions, OpenTelemetry is not tied to a single vendor, giving users the flexibility to export data to multiple observability platforms, including Prometheus, Jaeger, Zipkin, Elasticsearch, and various commercial solutions.

3. Comprehensive Observability for Distributed Systems – OTEL helps organizations monitor, trace, and analyze applications running in microservices architectures, Kubernetes clusters, serverless environments, and hybrid cloud infrastructures.

4. Simplified Instrumentation – Developers can use OpenTelemetry’s SDKs and automatic instrumentation to collect telemetry data without manually modifying large portions of their application code.

5. Better Troubleshooting and Performance Optimization – By correlating traces, metrics, and logs, OTEL enables teams to detect bottlenecks, troubleshoot incidents faster, and optimize system performance proactively.

Brief History and CNCF Involvement

OpenTelemetry originated as a merger of two popular open-source observability projects:

• OpenTracing – Focused on distributed tracing instrumentation.

• OpenCensus – Provided metrics collection and tracing capabilities.

Recognizing the need for a unified observability framework, the Cloud Native Computing Foundation (CNCF) merged OpenTracing and OpenCensus into OpenTelemetry in 2019, creating a single, industry-wide standard for telemetry data collection.

Key Milestones in Open Telemetry’s Evolution:

• 2016 – OpenTracing & OpenCensus emerge as separate projects to address distributed tracing and metrics collection.

• 2019 – CNCF consolidates both projects into OpenTelemetry to create a single, unified standard.

• 2021 – OpenTelemetry tracing reaches stable release, making it production-ready.

• 2022 – OpenTelemetry metrics reach general availability (GA), expanding beyond tracing.

• 2023-Present – Work continues on log correlation, profiling, and deeper integrations with various observability platforms.

CNCF’s Role in OpenTelemetry

The Cloud Native Computing Foundation (CNCF), a part of the Linux Foundation, serves as the governing body for OpenTelemetry. CNCF provides:

• Project oversight and funding to support OpenTelemetry’s development.

• Community-driven governance, ensuring OTEL remains an open and collaborative initiative.

Integration with other CNCF projects, such as Kubernetes, Prometheus, Fluentd, and Jaeger, to enhance observability capabilities for cloud-native workloads.

Why CNCF Backing Matters

CNCF’s involvement ensures OpenTelemetry remains a widely adopted, industry-backed, and continuously evolving framework. With support from major cloud providers (Google, Microsoft, AWS), observability vendors (Datadog, New Relic, Dynatrace), and enterprise technology companies, OpenTelemetry has become the de facto standard for open-source observability.

By adopting OpenTelemetry, organizations align with a future-proof, community-driven observability strategy, ensuring compatibility across cloud environments and monitoring solutions.

Choosing the Right OTEL Components

Understanding OTEL architecture: OpenTelemetry consists of multiple components, including APIs, SDKs, Collectors, and exporters. Selecting the right components depends on the architecture of your system and the telemetry data you need to collect. Organizations must assess whether they need distributed tracing, metrics, logs, or a combination of all three to achieve complete observability.

Deployment considerations: Choosing between an agent-based or sidecar deployment model affects resource utilization and scalability. OpenTelemetry provides flexible deployment options that integrate directly into microservices, Kubernetes clusters, and traditional monolithic applications.

Links for using OpenTelemetry with Apica Ascent:

• Getting Started with OpenTelemetry

• OpenTelemetry Integration for Ascent

OTEL SDKs: SPRING BOOT OPENTELEMETRY AND MORE

Language-specific SDKs: OpenTelemetry provides official SDKs for multiple programming languages, including Java, Python, JavaScript, Go, .NET, and more. Choosing the correct SDK ensures seamless instrumentation of applications to capture relevant telemetry data without requiring excessive code modifications.

Automatic vs. manual instrumentation: Many OpenTelemetry SDKs support automatic instrumentation, which simplifies the collection of telemetry data by automatically instrumenting common frameworks and libraries. Manual instrumentation, on the other hand, allows developers to capture more granular details specific to their business logic, providing richer observability insights.

Configuration and customization: Each OpenTelemetry SDK offers various configuration options, such as sampling rates, exporters, and resource attributes. Understanding these settings helps optimize observability while minimizing overhead on production systems.

Collector Setup for Telemetry Aggregation

Role of the OpenTelemetry Collector: The OpenTelemetry Collector acts as a central hub for processing, filtering, and exporting telemetry data. It eliminates the need to send data directly from applications to multiple backends, reducing the complexity of observability pipelines.

Collector pipeline configuration: OpenTelemetry Collectors support a pipeline model consisting of receivers (data ingestion), processors (data transformation), and exporters (data forwarding). Configuring these pipelines efficiently ensures that only relevant telemetry data is retained and sent to the appropriate monitoring backends.

Scalability and performance tuning: Organizations with high-volume telemetry data must optimize Collector performance using batching, compression, and load balancing techniques. Running multiple Collector instances or deploying Collectors at the edge can enhance data aggregation efficiency while minimizing network latency.

Instrumenting Applications with OTEL

Manual vs. Automatic Instrumentation

Understanding the differences: OpenTelemetry offers two approaches to instrumenting applications—automatic and manual instrumentation. Choosing the right approach depends on the level of detail required and the effort an organization is willing to invest.

Automatic Instrumentation: OpenTelemetry provides auto-instrumentation libraries that hook into commonly used frameworks (e.g., Spring Boot, Express, Flask, Django) to capture telemetry data without modifying application code. This is an easy way to get started and ensures coverage across key application functions with minimal effort. However, automatic instrumentation may not capture business-specific logic or custom events that organizations want to track.

Manual Instrumentation: With manual instrumentation, developers explicitly insert OpenTelemetry SDK calls into the application code. This provides precise control over what telemetry data is collected and allows capturing custom metrics, business transactions, and domain-specific spans. While more effort is required to implement, manual instrumentation results in richer observability data tailored to an organization’s needs.

Combining both approaches: Many organizations use a hybrid approach where auto-instrumentation provides baseline observability, and manual instrumentation is used to track critical business operations, unique workflows, or domain-specific logic.

Injecting Context Propagation Across Services

Why context propagation matters: In distributed systems, requests travel through multiple services, making it difficult to correlate logs, traces, and metrics. Context propagation ensures that telemetry data remains linked throughout an entire request lifecycle, enabling effective debugging and root cause analysis.

Using Trace Context and Baggage: OpenTelemetry follows the W3C Trace Context standard, which passes unique trace identifiers across service boundaries. Additionally, baggage propagation allows attaching custom metadata to traces, which can be used for debugging or business analytics.

Instrumentation strategies: Developers need to ensure that trace context is carried through HTTP requests, gRPC calls, and message queues. OpenTelemetry SDKs provide middleware and client libraries that handle this automatically for popular frameworks and protocols.

Ensuring compatibility across environments: Organizations using multiple tracing tools should verify that OpenTelemetry context propagation integrates well with existing logging and monitoring solutions, avoiding data fragmentation.

Exporting Telemetry Data

OTEL-native Exporters (OTLP, Prometheus, Jaeger, etc.)

OpenTelemetry Protocol (OTLP): OTLP is the native protocol for OpenTelemetry, offering a standardized and efficient way to transmit telemetry data. It supports traces, metrics, and logs in a unified format, ensuring compatibility with a broad range of observability backends. Organizations using OTLP benefit from reduced complexity and better performance, as the protocol is optimized for high-throughput data collection.

Prometheus Exporter: OpenTelemetry integrates seamlessly with Prometheus, a widely used open-source monitoring system. The Prometheus exporter allows applications instrumented with OpenTelemetry to send metrics to Prometheus, enabling real-time monitoring and alerting. This is particularly useful for organizations leveraging Prometheus as their primary observability backend.

Jaeger and Zipkin Exporters: OpenTelemetry supports both Jaeger and Zipkin, two popular distributed tracing backends. These exporters allow organizations to continue using their existing tracing infrastructure while benefiting from OpenTelemetry’s standardized instrumentation. By enabling these exporters, teams can visualize request flows and troubleshoot latency issues effectively.

Commercial Observability Platforms: Many commercial observability platforms, such as Datadog, New Relic, and Dynatrace, support OpenTelemetry exporters. This ensures that organizations adopting OpenTelemetry can seamlessly integrate their telemetry data into these platforms without vendor lock-in.

Integrating OTEL with Your Observability Backend (Your Platform)

Configuring Exporters for Seamless Data Ingestion: OpenTelemetry provides a flexible exporter configuration, allowing organizations to send telemetry data to multiple observability platforms simultaneously. This enables hybrid monitoring strategies where teams can leverage both open-source and commercial solutions for observability.

Optimizing Data Flow with the OpenTelemetry Collector: The OpenTelemetry Collector can be used as an intermediary layer to receive, process, and export telemetry data efficiently. By implementing batch processing, filtering, and data enrichment, organizations can optimize data flow while reducing unnecessary storage and processing costs.

Ensuring High Availability and Performance: When integrating OpenTelemetry with an observability backend, organizations should ensure that exporters and collectors are configured to handle high-volume telemetry data. Strategies such as load balancing, horizontal scaling, and adaptive sampling help maintain reliability while keeping infrastructure costs under control.

Security and Compliance Considerations: Organizations should implement encryption (e.g., TLS) and authentication mechanisms when exporting telemetry data to observability platforms. Ensuring secure transmission prevents unauthorized access and aligns with regulatory requirements.

Release Notes for recent software and platform updates across all products:

• Ascent 2.9.0

• Ascent 2.8.1

• Ascent 2.8.0

• Ascent 2.7.0

• Ascent 2.6.0

• Ascent 2.5.0

• Ascent 2.4.0

• Ascent 2.3.0

• Ascent 2.2.0

• Ascent 2.1.0

• Synthetic Monitoring (ASM)

• Loadtesting (ALT)

Standardizing Instrumentation Across Teams

Importance of consistency in observability: Ensuring that all teams follow a standardized approach to instrumentation is crucial for maintaining a reliable and actionable observability strategy. Without consistency, correlating telemetry data across services becomes challenging, leading to blind spots in monitoring and troubleshooting.

Collaborative approach to instrumentation: Organizations should establish cross-functional teams that include developers, SREs, and platform engineers to define and implement observability standards. This ensures alignment on best practices and reduces redundant or conflicting telemetry data collection.

Continuous improvement and governance: Standardization should not be a one-time effort. Organizations should regularly review and refine their observability practices to adapt to evolving business needs, new technologies, and OpenTelemetry updates.

Links for using OpenTelemetry with Apica Ascent:

• Getting Started with OpenTelemetry

• OpenTelemetry Integration for Ascent

Creating an Internal Instrumentation Policy

Defining clear guidelines for telemetry data collection: Organizations should document best practices for collecting, processing, and exporting telemetry data. This includes specifying which types of data (traces, metrics, logs) should be collected for different applications and environments.

Ensuring minimal performance impact: Instrumentation policies should balance comprehensive observability with system performance. Teams should implement sampling strategies, rate limiting, and filtering mechanisms to prevent excessive data collection from impacting application performance.

Establishing ownership and accountability: Clear guidelines should specify who is responsible for instrumenting different parts of the system. Assigning ownership ensures that observability is an integral part of the development and operational lifecycle rather than an afterthought.

Automating instrumentation where possible: Using automatic instrumentation libraries and OpenTelemetry’s SDKs can help enforce consistent observability standards with minimal manual effort. Automation reduces the likelihood of human errors and ensures that new services are consistently instrumented from day one.

Establishing Naming Conventions for Spans and Metrics

Consistent span naming for improved traceability: Using a structured and descriptive naming convention for spans ensures that distributed traces are easy to interpret. Naming should follow a hierarchical structure that includes service name, operation type, and key function details (e.g., order-service.db.query instead of queryDB).

Standardized metric naming for cross-team compatibility: Metric names should follow a standardized format that aligns with industry best practices. This includes using prefixes for different metric types (http_request_duration_seconds for latency metrics) and ensuring clear labels for filtering and aggregation.

Using semantic conventions: OpenTelemetry provides semantic conventions for naming spans, attributes, and metrics. Adhering to these standards improves interoperability and makes it easier to integrate OpenTelemetry data with third-party observability tools.

Documenting naming conventions for long-term consistency: Organizations should maintain a centralized documentation repository outlining agreed-upon naming conventions and examples. This ensures that new teams and developers can easily adopt and follow established best practices.

Optimizing OpenTelemetry Collector Performance

Managing Memory and CPU Overhead

Efficient resource allocation: OpenTelemetry Collectors process a large volume of telemetry data, making it essential to allocate adequate CPU and memory resources. Organizations should assess their workloads and set appropriate limits to prevent excessive resource consumption that could degrade system performance.

Using lightweight configurations: To optimize resource usage, organizations should enable only necessary receivers, processors, and exporters. Disabling unused components minimizes CPU and memory overhead, improving overall efficiency.

Load balancing Collectors: Deploying multiple Collector instances in a load-balanced configuration helps distribute processing across nodes, reducing bottlenecks and ensuring high availability. This is particularly important for large-scale deployments handling massive telemetry data volumes.

Monitoring Collector performance: Continuously tracking Collector resource usage through built-in metrics helps teams identify performance bottlenecks and optimize configurations. Organizations can set up alerts for CPU spikes, memory saturation, and dropped telemetry events to maintain system stability.

Implementing Batch Processing and Sampling Strategies

Batch processing for efficiency: Instead of sending individual telemetry events, OpenTelemetry Collectors support batch processing to aggregate and compress data before transmission. This reduces network overhead and optimizes performance while ensuring minimal data loss.

Adaptive sampling techniques: Organizations can use head-based and tail-based sampling techniques to limit the volume of telemetry data collected without losing critical observability insights. Tail-based sampling allows prioritizing high-value traces while discarding less useful data, improving cost efficiency.

Configuring sampling rates based on workload: Setting appropriate sampling rates based on application traffic patterns prevents excessive data ingestion while retaining sufficient observability coverage. Dynamic sampling strategies can adjust rates in real-time based on system health and alert conditions.

Ensuring data integrity with intelligent filtering: Organizations can filter and enrich telemetry data within OpenTelemetry Collectors, ensuring that only relevant metrics, logs, and traces are stored. This reduces storage costs and improves the relevance of observability data for troubleshooting and performance optimization.

Ensuring Data Security and Compliance

Masking Sensitive Data in Traces and Logs

Understanding the risks of exposed telemetry data: Logs and traces often contain sensitive information such as user credentials, API keys, personally identifiable information (PII), and payment details. If not properly handled, this data can be exposed in observability pipelines, leading to security breaches and compliance violations.

Implementing data masking and redaction: Organizations should establish policies for automatically redacting or masking sensitive data before it is ingested into logs or traces. OpenTelemetry allows for processors to be configured to scrub sensitive fields, ensuring that only anonymized data is transmitted.

Using attribute-based filtering: OpenTelemetry provides mechanisms to filter telemetry attributes before they reach a storage backend. By defining attribute allowlists and blocklists, teams can prevent the transmission of confidential information while preserving necessary observability data.

Enforcing encryption in transit and at rest: All telemetry data should be encrypted both in transit (e.g., using TLS) and at rest within storage systems. This ensures that intercepted data cannot be accessed by unauthorized entities.

Compliance with industry regulations: Many industries require specific security practices, such as GDPR's data minimization principle and HIPAA’s de-identification requirements. By implementing structured masking and redaction policies, organizations can align with these regulatory standards while maintaining robust observability.

Applying Role-Based Access Control (RBAC) for Telemetry Data

Defining access levels for different roles: Not all users need access to all telemetry data. Organizations should define clear RBAC policies that grant varying levels of access based on job responsibilities. For example, developers may only need application performance data, while security teams require access to audit logs.

Segmenting telemetry data by sensitivity: Logs, traces, and metrics can be categorized based on their sensitivity levels. By assigning access controls to different categories, organizations can prevent unauthorized personnel from accessing highly sensitive information.

Using authentication and authorization mechanisms: OpenTelemetry integrates with identity management systems to enforce authentication and authorization. Implementing Single Sign-On (SSO), multi-factor authentication (MFA), and API key restrictions ensures that only authorized users and services can access telemetry data.

Auditing and monitoring access logs: Continuous monitoring of who accesses telemetry data helps detect unauthorized access attempts. Audit logs should track all interactions with observability data, including user actions, query requests, and data exports.

Automating policy enforcement with infrastructure as code: RBAC policies should be defined in infrastructure as code (IaC) templates to ensure consistency across deployments. By automating role assignments and access restrictions, organizations can enforce security best practices at scale.

Avoiding Common Pitfalls

Over-instrumentation Leading to High Overhead

Understanding the risks of excessive instrumentation: Instrumenting every possible function, service, or transaction can introduce significant processing overhead, increasing CPU and memory consumption and impacting application performance. While observability is crucial, excessive instrumentation can slow down systems and lead to noise in telemetry data, making it harder to extract meaningful insights.

Implementing strategic instrumentation: Teams should focus on capturing key telemetry data that aligns with business and operational needs. Instead of collecting every possible trace or metric, organizations should define specific service-level objectives (SLOs) and monitor the most critical performance indicators, reducing unnecessary data collection.

Using adaptive sampling techniques: OpenTelemetry provides head-based and tail-based sampling, which allows teams to collect meaningful traces while reducing the data volume. Adaptive sampling dynamically adjusts based on traffic, ensuring visibility into important transactions without overwhelming observability pipelines.

Optimizing trace and metric retention policies: Organizations should implement retention policies that store only high-value telemetry data while discarding redundant or less critical information. This ensures that logs, traces, and metrics remain relevant and actionable while keeping storage costs manageable.

Regularly auditing telemetry data collection: Conduct periodic reviews of instrumentation policies and collected data to identify unnecessary metrics, spans, or logs that could be removed or optimized. Automating this audit process can help enforce efficient observability practices without human intervention.

Lack of Correlation Between Metrics, Logs, and Traces

The importance of unified observability: Metrics, logs, and traces serve different observability functions, but when analyzed in isolation, they provide an incomplete picture of system health. Ensuring proper correlation between these data types is critical for effective root cause analysis and performance optimization.

Implementing trace-log correlation: OpenTelemetry allows injecting trace and span identifiers into log messages, providing direct relationships between traces and log events. This makes it easier for engineers to investigate issues by linking logs to the specific traces that triggered them, reducing time spent on debugging.

Enriching metrics with trace and log context: By tagging metrics with trace identifiers and relevant log metadata, organizations can improve visibility into system-wide performance trends. This approach helps correlate spikes in error rates, latency fluctuations, and anomalous behaviors with specific transactions.

Leveraging OpenTelemetry semantic conventions: Using standardized naming conventions and attributes for spans, logs, and metrics ensures consistency across telemetry data. Following OpenTelemetry’s semantic conventions improves interoperability with various backends and enhances observability tool integrations.

Centralized observability dashboards: Organizations should aggregate and visualize logs, metrics, and traces in a unified observability platform. Tools like Grafana, Kibana, and OpenTelemetry-compatible backends enable cross-referencing telemetry data for more efficient troubleshooting and deeper insights.

Customer Release Notes

ASM 13.31.0

New Features & Enhancements

Automation for Checks and Alerts

• Added support for CI/CD pipeline to streamline check creation and maintenance through ASM APIs.

• Reduced manual efforts and ensured consistency across different environments through automation.

• Ability to perform CRUD operations for ZebraTester, Browser, and URL checks.

• Ability to create, upload, and assign ZebraTester and Browser scenarios for checks.

• Ability to create and assign Email, SMS, and Webhook alert targets or alert groups.

Chrome 130 Upgrade for Browser Checks

• All existing Browser checks have been upgraded from Chrome 115 to Chrome 130.

• All new Browser checks run on Chrome 130.

NG Private Locations

• Private locations/agents can be shared among sub-customer accounts to run checks.

• Users can utilize their own CA certificates for checks in Private locations to monitor internal applications.

Apica Grafana Plugin

• Upgraded the Apica Grafana plugin to version 2.0.11.

• Added support for page metrics, allowing users to analyze response time for specific pages instead of entire scenario metrics.

Bug Fixes

• Fixed the issue where invalid "acceptedCodes" were being accepted for URL checks in the POST /checks/url-v2 API.

Ascent 2.9.0

Ascent Synthetics (ASM+)

Check Management

• Introduced new check types: Browser, Postman, Traceroute, Mobile Device, and Compound.

• Added support for the full workflow of check management: Edit, Delete, Clone, and Run Check.

• Added support for Bulk Edit, Run, and Delete checks.

• Inclusion and exclusion periods can be added in the check schedule.

Private Location and Agent Management

• Introduced full self-service (Add, Edit, Delete, Reissue Certificate) for new check-type agnostic Private agents.

• Private locations can be added, edited, deleted, enabled, and disabled with the ability to associate Private repositories.

• A new "Private Locations" section in the UI allows easy navigation and management.

Check Analytics

• Enabled alerting and reporting on checks.

• Alerts and reports for a particular check can be created directly from the check page.

• Screenshots taken during Browser check execution can now be viewed in the Check Analysis page.

Bug Fixes

• Fixed an issue where filter criteria were not working correctly on the Checks page.

• Fixed a bug where some check results were missing on the Check Details page.

New Features and Enhancements

Fleet

• Fleet Agent Limits: Enforced license-based agent limits.

• Telemetry Enhancements: Added telemetry support for Fleet agents.

• Fleet UI Revamp: Major UI improvements, better agent configuration management, and pagination fixes.

• Fleet Summary Table: Redesigned the summary table for better usability.

• Kubernetes Agent Status: Fleet UI now displays Kubernetes agent statuses.

Observe

• Data Explorer Graph Enhancements: Enhanced GroupBy plotting with multiple Y-axis selection.

• Widgets Enhancements: Added delete functionality and improved widget load time.

• New Chart Type: Introduced Pie and HoneyComb charts for visualization.

• Grafana to Data Explorer: Added Grafana JSON conversion support in Data Explorer.

• GenAI Enhancements: Integrated "Log Explain" feature for enhanced log analysis in ALIVE.

• Data Explorer Enhancements: Improved metrics screen and query list support.

• Dashboard Optimization: Reduced load times for Data Explorer dashboards and preserved widget data across tabs.

• RCA Workbench: Introduced diagnostics and debugging features based on Data Explorer widgets.

• Dashboard Validation: Added validation for Data Explorer dashboard creation.

Authentication & Security Enhancements

• React Page Migration: Migrated Login, Setup, Signup, Reset, and Forgot Password pages to React (TSX) to reduce tech debt.

• Ascent Invitation Feature: Implemented user invitation functionality via Casdoor.

• Casdoor Sync: Synced Casdoor users and groups with the Ascent database.

• Port Management: Resolved open TCP/UDP port issues.

• Casdoor Integration: Enhanced authentication, session management, and email integration.

• API Key Support: Added API key support for Casdoor in Ascent.

• Casdoor Mail Service: Integrated Ascent mail service with Casdoor for email functionality.

• Casdoor Signing Certificates: Added support for Casdoor signing certificates to enhance security.

Ascent Bug Fixes

• GCP PubSub Plugin: Resolved file loading issues.

• ResizeObserver Compatibility: Fixed compatibility issues with the latest Chrome version.

• Alert Email Output: Truncated query output in triggered alert emails for better readability.

• Agent Sorting: Fixed sorting by "Last Modified" in Fleet UI.

• Incorrect Trace Volume: Fixed trace volume display on the Ascent landing page.

• Alert Bug Fix: Resolved discrepancies in triggered alert counts displayed in the navbar.

• Pipeline View: Fixed visual bugs in forwarder mapping and improved rule persistence.

• Fleet Improvements: Enhanced Fleet installer, improved Kubernetes token creation, and fixed pagination issues.

• Password Generation UI: Improved UI for password generation in Ascent.

• Query Save Fix: Resolved unknown error when saving queries in the Freemium tier.

• Moving Average Bug: Fixed AI-based query creation issues for Moving Average.

• Alert UNKNOWN Issue: Resolved alerts triggering with an UNKNOWN state.

• Alert Evaluation Fix: Fixed issues with alerts not evaluating after the first trigger.

• SNMP Source Bug: Fixed SNMP ingest source extension bugs.

• Fluent-Bit Installation: Addressed issues with Fluent-Bit post-agent manager installation.

• Dual Active Packages: Resolved the issue of showing two active packages in Fleet.

• Inclusion/Exclusion Fixes: Addressed syntax and period-saving issues.

• Certificate Upload: Fixed certificate upload issues and removed the feature from Freemium.

• Default Otel Configuration: Updated default Otel configuration for metric ingestion.

• Platform Validation: Enhanced platform validation in Fleet.

• Fleet Assign Package Error: Fixed package assignment issues.

• Disable Pattern Signature: Disabled pattern signature functionality in Freemium.

• Namespace Bug: Resolved incorrect namespace selection in Data Explorer.

• Fleet Advanced Search: Fixed and improved advanced search functionality.

• Dark Mode Fixes: Addressed UI inconsistencies, including Waterfall statistics and button styling.

• Fleet Installation: Resolved installation errors on Linux and Windows.

• Kubernetes Dropdown Fix: Fixed duplicate Kubernetes entries in Fleet dropdowns.

• Configuration Refresh: Addressed package reassignment and configuration refresh issues.

• Documentation Updates: Updated user and technical documentation.

For further details or inquiries, please refer to the official documentation or contact our support team.

Synthetic Monitoring (ASM 13.27.0) - SaaS

Features

1. NG Private Locations/Agents API Support: Added ASM API support for full self-serve new check-type agnostic Private Agents which can be grouped into Private Locations. Features include:

   • Creation and management of Private location.

   • Creation and management of Private agents.

   • Configuration of Private Container repositories for Private locations to use during check run.

2. Added API support for Timezone selection for Check Inclusion/Exclusion periods during UrlV2 check creation.

3. Extended the subscription page to include more check statistics per check type like Info, Warning, Error, and Fatal check counts.

4. Enhanced status updates for NG Private agents.

Bug Fixes

• Fixed the sporadic non-availability of agents in the Stockholm location issue when debugging a Selenium scenario.

• Fixed a bug with downloading scripts from http sources for Scripted and Postman checks.

• Fixed a bug where some block domain rules were not being respected in Browser checks.

• Fixed the issue where setLocation command was not working properly if it is not used at the start of a Selenium script for Browser checks.

Apica Data Fabric (ADF)

Features

1. Native Support for OTEL Logs.

   • Added native support for OTEL logs using the OTLP HTTP exporter.

2. Native Support for OTEL Traces.

   • Added native support for OTEL trace using the OTLP HTTP exporter.

3. STREAMS Rule Type

   • Introduced a new rule type for STREAMS.

4. Moving Average Improved

   • Enhanced moving average calculation using SMV (Simple Moving Average) and CMV (Cumulative Moving Average).

5. Pattern-Log Compare.

   • Feature to compare the logs and patterns side by side to different time ranges.

6. Improved ALIVE summary graph highlighting depending on table content to provide better data visualisation.

7. Data Explorer: Tabs Scrolling and Improvement

   • Added scrolling functionality and various improvements to the Data Explorer tabs for better navigation.

8. GPT-4o-mini and Limited Model Support

   • Introduced support for GPT-4o, GPT-4o-mini, GPT-3.5-Turbo.

9. API-Based Create Data-Explorer Dashboard

   • Added the ability to create Data-Explorer dashboards via API.

10. API-Based Create Sharable Dashboard

    • Enabled the creation of sharable dashboards through API.

11. Generic Implementation for Data Explorer Header

    • Made the Data Explorer header implementation generic and interdependent.

12. Check Management Map View

    • Introduced a map view for check management.

13. Check Management List View UI Changes

    • Updated the UI for the check management list view.

14. Data Explorer Header to Persist Data

    • Added functionality for the header of data explorer to persist data.

15. Automatically Create Splunk Universal Forwarder for Splunk S2S Proxy

    • Added automatic creation of Splunk universal forwarder for Splunk S2S Proxy.

16. Pipeline Tab in Search View

    • Added a new pipeline tab in the search view.

17. Code Rules Preview

    • Introduced a preview feature for code rules.

18. Health Check for Agents

    • Implemented a health check feature for agents.

Improvements

1. Trace/Default App Performance Improved

   • Enhanced the performance of the trace/default application.

2. New Algorithm for PS Compare and Anomalies Compare

   • Implemented a new algorithm for comparing architecture PS and detecting anomalies.

3. Widget Refresh Performance

   • Improved the performance of widget refresh operations.

4. Query API Performance for search

   • Enhanced the performance of the Query API for search.

5. Default Namespace for Logs for Syslog vs Per Host Namespaces

   • Enhanced default namespace handling for logs, distinguishing between syslog and per host namespaces.

6. UI Enhancements for Pipeline and Topology View

   • Improved UI for pipeline and topology views.

7. Agent Manager Improvements for Installation Scripts

   • Enhanced agent manager installation scripts.

8. Delete Agent Cleanup

   • Improved the cleanup process when deleting agents.

9. Remove Unsupported Agents

   • Enhanced the process to remove unsupported agents.

Bug Fixes

1. Y-Axis Overlapping on View

   • Fixed an issue where the Y-axis was overlapping on the view in the ALIVE application.

2. Gauge Widget Color Render Based on Zone

   • Fixed the rendering of gauge widget colors based on specified zones.

3. Group By for Data-Explorer

   • Fixed the group by functionality in the Data-Explorer.

4. Creating Alert Creates Panic

   • Resolved an issue where creating an alert caused a panic.

Release Notes - Ascent October 2024

Overview This release introduces a host of updates to enhance user experience, streamline operations, and address known issues across Fleet, Data Explorer, the Ascent platform, and ASM+. New features and improvements focus on usability, performance, and customization, while bug fixes enhance platform stability and reliability.

New Features and Enhancements

OpenTelemetry

OpenTelemetry Collectors can now be configured to use the standard ingest endpoints when pushing data to Apica Ascent

1. Traces - /v1/traces

2. Logs - /v1/logs

3. Metrics - /v1/metrics

Telemetry Pipelines

1. New forwarders added for Oracle Cloud

   • OCI Buckets

   • OCI Observability & Monitoring - Logs

Freemium Support

Experience Apica Ascent with the Freemium release. The Freemium is a FREE FOREVER release which includes all the capabilities of the Apica Ascent Intelligent Data Management Platform available as a convenient SaaS offering

1. Fleet Management

2. Telemetry Pipelines with support for platforms such as Splunk, Elasticsearch, Kafka, Datadog among others

3. Digital Experience Monitoring (Synthetic Monitoring for URL, Ping, Port and SSL Checks)

4. Log Management

5. Distributed Tracing

6. Infrastructure Monitoring

7. Enterprise ready with features such as SAML based SSO

8. ITOM integration with platforms such as PagerDuty, ServiceNow, and OpsGenie

Fleet Updates

1. Agent Management: - Introduced controls for managing agents within the Fleet UI for better administration. - A summary table was added to display agent statistics, providing quick insights. - Enabled rules for assigning configurations or packages to agents. - User-defined Fleet resource types (rules, alerts, agent_types, configurations, and packages) can now be imported via Git. - Fleet REST API search endpoints now support the ?summary query parameter for result summarization. - Expanded fleetctl CLI tool capabilities to manage Fleet API resources directly.

2. Advanced Search and Customization: - Users can save and retrieve advanced search queries in the Fleet Advanced Search Modal.

Data Explorer Enhancements

1. Improved Analytics Options: - Added support for PostgreSQL, expanding data integration capabilities. - Enhanced GroupBy functionality and a “Select All” label for better data analysis. - Enabled parameterized queries for dashboards, allowing dynamic user input for real-time customization. - Users can edit the dashboard header query and set the dropdown type (query, enum, or text) for customization.

2. Visualization Improvements: - Introduced a DenseStatusType chart to monitor active and inactive pods/instances in real time. - Added time zone customization for chart displays. - Optimized dark theme UI components with updated icons and design assets.

Ascent Platform Enhancements

1. ASM UI Enhancements: - Integrated repository and certificate management for streamlined admin controls. - Implemented a persistent last-view setting on the Check Management page.

2. General Improvements: - Enhanced navigation with streamlined redirection flows for faster page loads.

AI/ML and GenAI Enhancements

1. Pattern-Signature Processing: - Improved compaction with meaningful aliasing during pattern-signature (PS) merging. - Enhanced performance through PS coding representation for faster processing and UI responsiveness. - Fixed functionality for PS compaction at the backend.

2. GenAI Features: - GenAI document search functionality was added to the NavBar.

Bug Fixes

Fleet UI and Backend Fixes

1. UI and Agent Issues: - Resolved banner display inconsistencies during agent updates. - Fixed errors in anonymous report generation for Grafana Alloy. - Fixed agent-manager token refresh failures on Windows hosts.

2. Backend and API: - Fixed errors preventing default configuration/package assignments via the install endpoint. - Resolved OpAMP client failures and Windows socket exhaustion issues. - Corrected lookup errors for agents by instance ID during OpAMP registration.

Data Explorer Fixes

1. Performance and Stability: - Resolved crashes on the Data Explorer page. - Corrected schema issues and bugs affecting *-based queries and widget calculations. - Fixed default date-type inputs and adjusted other input defaults for smoother workflows.

2. UI Updates: - Fixed CSS and overflow issues in modals and alert render pages.

General UI and Usability Fixes

• Resolved usability regressions from the v3.11.2 update, improving input defaults and widget updates.

Miscellaneous Enhancements

1. Fleet-Specific Improvements: - Improved response times in Fleet views for queries involving large datasets.

2. Ascent Platform: - Resolved permission issues for non-admin users in the Namespace endpoint.

These updates reflect our commitment to delivering a robust and user-friendly platform. As always, we value your feedback to enhance our services further.

ASM

New Features & Enhancements:

1. ASM Private Location Management:

   • Introduced the ability for Customer Administrators to Add, Edit, and Delete private locations and private repositories, giving more control over location and data management.

   • Added a "Private Locations" section in the UI, allowing easy navigation and management of these locations.

   • Implemented endpoints to Enable/Disable Private Locations, retrieve lists of private locations and repositories, and associate repositories with specific private locations.

   • Included a Timezone selection feature for URL V2 endpoints, enhancing configuration flexibility for global deployments.

   • New options for managing Private Agents with functionalities such as Adding, Editing, and Deleting agents, as well as Reissuing Certificates for enhanced security.

2. Check Management Enhancements:

   • Integrated ZebraTester within Check Management, improving performance testing capabilities.

   • Enhanced the Check Analytics screen for a smoother experience, including a redesigned Schedule and Severity Handling screen supporting Dark Theme.

3. Improved API & Documentation:

   • Refined API Endpoints: Added support for handling advanced configuration for missing checks, private agent solutions, and new fields in the SSL Certificate Expiration Check.

   • Documentation Improvements: Updated ASM API documentation to include better descriptions, missing fields, and request/response formats for enhanced usability.

4. Canary Release Support:

   • Extended Deployment APIs to support Canary Releases, ensuring more robust testing and rollouts.

5. Performance Optimization:

   • Implemented pre-fetching of access groups to reduce database calls and improve the performance of core endpoints.

   • Optimized Sampling Interval for tables based on time duration to reduce load times.

6. Agent Status Monitoring:

   • Added visual indicators for the Enable/Disable Status of private locations, improving overall monitoring and management.

Bug Fixes:

1. Check Management:

   • Fixed inconsistencies in the Check Results Graph to ensure linear representation of data on the X-axis.

   • Addressed issues with timestamp formatting when clicked from different parts of the graph, which led to parsing errors.

2. Fleet Management:

   • Corrected the behavior of agent ID and customer GUIDs during initial state setup.

   • Resolved problems causing memory issues in multi-cluster environments.

3. UI & Visual Fixes:

   • Eliminated scroll issues when hovering over charts.

   • Adjusted the Date Picker to revert to its previous version for consistency and usability.

4. Multi-Cluster Stability:

   • Fixed degradation issues occurring when one of the single tenants in a multi-cluster environment was down.

   • Ensured smoother data loading and resolved UI lock-up issues when handling larger datasets.

5. Certificate Management:

   • Added validation checks and improved error handling for operations like adding, editing, and deleting SSL certificates and repositories.

Ascent

New Features & Enhancements

1. Fleet Management Improvements:

   • Fleet UI Enhancements: Redesigned Fleet management screens, including Agents and Configuration, with consolidated controls for improved usability and support for Dark Theme.

   • Kubernetes Environment Support: Introduced support for Kubernetes environments in Fleet, enabling better agent management and installation flexibility.

   • Fleet Agent Support for OpenTelemetry Collectors, Datadog and Grafana Alloy: Expanded the ecosystem of supported agents with compatibility for OpenTelemetry Collector, Datadog and Grafana Alloy agents.

   • Agent Liveness Status Metrics: Implemented new metrics to monitor the liveness status of each Fleet agent, ensuring better visibility and alerting.

   • Advanced Search for Fleet: Enhanced search capabilities with a new advanced search feature, making it easier to locate specific data and agents.

2. Data Explorer Enhancements:

   • Y-Axis Multi-Column Plotting: Enhanced Y-axis plotting, allowing for the selection and visualization of multiple columns, making complex data analysis simpler.

   • Time Range in Headers: Added time range indicators in the header, improving context and navigation during data exploration.

   • Custom Chart Integration: New customizable charts, such as Counters, are available for Data Explorer, providing enhanced visualization options.

   • Color Selection for Widgets: Users can now customize the colors of rendered data inside each widget on the Data Explorer page, making it easier to personalize and distinguish visual components.

     Performance & Optimization:

     • Lazy Loading Implementation: Optimized data explorer dashboards by implementing lazy loading, improving initial load times, and reducing resource consumption.

     • Custom Hooks for Skipping Component Mount Calls: Enhanced performance by introducing custom React hooks to skip unnecessary component mounts, minimizing UI lag.

3. UI/UX Improvements:

   • Dark Mode Icons & Design Adjustments: Optimized icon sets and UI components for a more consistent experience in dark mode.

   • New Toggle & Theme Options: Added a toggle for switching between Dark and Light modes in the navbar, giving users more control over their viewing experience.

4. Integration & API Updates:

   • Grafana Integration: Implemented a converter to transform Grafana JSON into Data Explorer JSON format, simplifying the migration of dashboards.

5. User Onboarding:

   • Improved Onboarding Experience: A dedicated onboarding screen for new users has been added to streamline the setup process and introduce key features.

Bug Fixes

1. Fleet Management:

   • Fixed issues where disconnected Fleet agents could not be deleted.

   • Resolved problems with log collection on Windows machines.

   • Addressed duplicate agent entries when reinstalling Fleet agents.

2. Data Explorer:

   • Corrected data inconsistency issues when switching between dashboards.

   • Fixed bugs related to alert tabs being incorrectly linked across dashboards.

   • Resolved intermittent behavior where data from one dashboard was erroneously stored in another.

3. ALIVE:

   • Improved the alignment and visualization of PS compare graphs and log comparisons.

   • Added zoom-in and enlarge options for better graph analysis.

   • Enhanced visual feedback for log loading during comparisons.

4. UI Bug Fixes:

   • Resolved AI button shadow and sizing issues for a more polished interface.

   • Corrected modal rendering in header dropdowns for persistent selections across tabs.

Discover the latest advancements and improvements of the Apica Ascent platform. This is your go-to destination for updates on platform enhancements and new features. Explore what's new to optimize your observability and data management strategies.

Synthetic Monitoring (ASM 13.25.0) - SaaS

• Features

  • We have added bulk GET support for the API endpoint /checks/config. Users can now request multiple check configurations in one go, preventing issues caused by rate limiting. This is especially beneficial for those automating the synchronization of their own versions of check configurations with the Ascent platform through the ASM API.

  • A user must be able to see the response body from a failed URL call in a ZebraTester checks, if available, to enable the identification of what error messages or content might be returned.

• Bugs Fixes:

  • We have eliminated the inconsistencies (spikes) in NG check result metrics previously impacted by infrastructure resource constraints. This has now been rolled out to all public and dedicated check locations available.

  • We have fixed the bug where the location API endpoint for Zebratester checks GET /checks/proxysniffer/locations was not returning all NG locations.

  • Expanding urls in check results for URLv2 check will display readable response content.

Synthetic Monitoring (ASM) - On-Prem

• Features:

  • Display response body for failed URL calls in a ZebraTester checks result.

• Bug Fixes:

  • We have fixed a bug that prevented new Browser check scenarios from syncing with the controlling agents effectively making them unavailable at time of check execution.

Loadtest (ALT)

• Bug Fixes:

  • Not all transaction names are available in ‘Edit Non-Functional Requirements (NFR)’.

ADF v3.7.7

• Features

  • We have added an OTel forwarder to be used in ADF/FLOW to send OTel data untouched downstream to external OTel collector.

• Bug Fixes:

  • ASM+ Pagination bug on Check Analytics

  • Email delivery bug

  • ASM+ check data ingest stability improvements

🗓️ 12th September 2024

Features

• React-grid-layout Integration: React-grid-layout has been integrated into Data Explorer for widget flexibility and condensed dashboards.

• Legend Component: A separate component for displaying legends in Data Explorer widgets was implemented, which shows statistics for the data that is being rendered in the widget.

• Port Management via UI: Added support for enabling and disabling k8s cluster ports via the UI.

• Ping Checks: Implemented Ping Checks in Check Management.

• Port Checks: Implemented Port Checks in Check Management.

• Logs as a Data Source: Apica logs can now be integrated as a data source for Data Explorer and users can create/run queries on top of logs. This also introduces a new way to set alerts on the platform using logs.

• File Compare Graph Y-axis Scale: The Y-axis of the File Compare graph now supports two modes: PS count and percentage.

• PS Compare Anomaly Marker: Added anomaly markers for better visualization in PS Compare.

• Dashboard Data Migration: Dashboard schemas are now formatted into Data Explorer format and moved from LogiqHub to ApicaHub Github Repositories.

• Legacy Dashboard Converter: A converter was implemented to convert legacy Dashboard JSON to Data Explorer JSON format.

• Data Explorer: Editing Controls and Breakpoints: Added editing controls and breakpoints in Data Explorer.

• Scatter Chart Support: Data Explorer now supports scatter chart visualizations.

• Dark Theme: Improved dark themes for multiple screens, including Logs & Insights, Dashboards, Topology, and Pipelines.

• Dashboard Import in Data Explorer Format: Frontend changes were implemented to import dashboards in Data Explorer format.

• Check Analytics Reports Integration: Enhanced check analytics by integrating it with reporting.

• FPR Checks Consolidated Metrics: Added the ability to enrich check data at time of ingestion using a new domain-specific language (DSL).

Improvements

• Check Status Widget: Added custom configuration options for the check status widget.

• Performance Improvements: Extended efforts to improve the performance of Data Explorer for smoother usage.

• Gauge Chart Design: Modified the Gauge chart design, providing more user-configurable options and units for charts.

• New Visualizations in Data Explorer: New widget types were added, including Check Status, Stat, Size, Date/Time, and Scatter chart visualizations.

• Statistical Data in Legends: Introduced statistical data to the new legend component in Data Explorer.

• Auto Gradient Colors: Implemented an automatic gradient color generator for area charts in Data Explorer.

• Grafana Dashboard Converter: Developed a converter for Grafana dashboards to be compatible with Data Explorer.

Bugs

• Invalid Log Timestamp: Fixed an issue where log timestamps were invalid.

• Tracing Volume Query Issue: Addressed an issue affecting tracing volume queries.

• File Compare Graph Display: Resolved issues with the display of the file compare graph summary.

• Data Explorer Page Crashing: Fixed errors causing the Data Explorer page to crash due to undefined values.

• Widgets Deletion Handling: Implemented proper handling for widget deletion to prevent crashes.

• Tab Loss on Reload: Resolved the issue where Data Explorer page tabs were lost on reload.

• Chart Label Issues: Fixed chart label issues and improved chart rendering.

Release Notes - Ascent January 2025

Overview

Introducing Apica Ascent Freemium—a FREE FOREVER version of our Intelligent Data Management Platform, now available as a convenient SaaS offering. This release democratizes intelligent observability, providing access to powerful features at no cost. Experience all the core capabilities of Ascent and take your telemetry data management to the next level.

New Features and Enhancements

Freemium Support

• Added Freemium support via the Freemium license, offering free access to Ascent's capabilities.

Core Features

1. Fleet Management

   • Efficiently manage data collection with support for up to 25 agents, including OpenTelemetry Collectors for Windows, Linux, and Kubernetes.

2. Telemetry Pipelines

   • Seamlessly integrate with popular platforms, including Splunk, Elasticsearch, Kafka, and Datadog, among others.

3. Digital Experience Monitoring

   • Leverage Synthetic Monitoring for URL, Ping, Port, and SSL checks to optimize the digital experience.

4. Log Management

   • Centralize log collection, analysis, and management for improved observability.

5. Distributed Tracing

   • Gain deep insights into application performance with distributed tracing capabilities.

6. Infrastructure Monitoring

   • Monitor and manage infrastructure performance to ensure optimal operations.

7. Enterprise-Ready Features

   • Enable SAML-based Single Sign-On (SSO) for enhanced security and ease of access.

8. ITOM Integration

   • Integrate seamlessly with IT operations management platforms such as PagerDuty, ServiceNow, and OpsGenie.

Key Benefits of Ascent Freemium

• Process up to 1TB/month of telemetry data, including logs, metrics, traces, events, and alerts.

• Unlimited users and dashboards for collaboration and real-time data visualization.

• No storage costs or credit card requirements.

• Built-in AI-driven insights to enhance troubleshooting and decision-making.

Browser Compatibility

Apica Ascent Freemium is available immediately. Sign up now at https://www.apica.io/freemium and start transforming your data management experience today.

Discover the latest advancements and improvements of the Apica Ascent platform. This is your go-to destination for updates on platform enhancements and new features. Explore what's new to optimize your observability and data management strategies.

Synthetic Monitoring (ASM 13.26.0) - SaaS

Features

• NG Private Locations/Agents: New check-type agnostic Private Agents can be grouped into Private Locations with full self-serve functionality in the ASM UI portal. *ASM API support for full self-server ability will be added during Q3.

  Features include the creation and management of Private Location and Agent along with Private Container Repositories for Private Agent use. Private Agent install packages (.rpm and .deb) will be available with support for RHEL v8+ and Debian v10+. Private locations can be set up to use either Docker or Podman driver for check execution.

• New Browser checks will automatically accept dialogs/modals that can pop up during a test such as alert/confirmation/prompts.

• New Browser checks will attach and include control of new tabs created by the target site. I.e. the chrome WebDriver will automatically attach to new tabs that are opened during check execution of a Browser check.

• Added SAN/SNI options to SSL Cert Expiration and Fingerprint Validation for URL checks.

• Compound check is available on NG locations.

• Extended the ability to append the custom message specified in _Apica_Message collection variable to Postman check result messages in case the Postman script fails.

Bug Fixes

• Screenshots for Browser checks were not working in new tabs or windows created by the check. This is fixed as part of the above feature that include control of created tabs and windows by the target site.

• Debug scenario of Browser checks from the Edit Check page will use the same location as the check does.

• Fixed the issue where ASM UI was throwing a 500 error from Ajax while adding target value for newly created Selenium scenarios.

• Fixed the sporadic non-availability of agents in the Stockholm location issue when debugging a Selenium scenario.

• Enhanced encryptapica feature in Scenarios for Browser checks. The target value of encryptapica prefixed store commands used in Selenium scenarios will be masked across all scenario commands in the Browser check results in case the specified target value appears in any other scenario commands (eg. echo command).

Synthetic Monitoring (ASM 13H.5) - OnPrem

Features

• The display response body for failed URL calls in a ZebraTester checks the result, if available, to enable the identification of what error messages or content might be returned.

• Added support for PUT API request to add or update URL v1 checks through ASM API.

Apica Data Fabric (ADF v3.9)

Features

• Dark Mode: A new dark mode option is now available, providing a dark-themed interface for users who prefer it.

• Code Rule Preview: Users can preview and compare the data after the code rule is applied.

• apicactl: Introduced a new command-line tool in Apica Github for API management.

• Bookmark date range: Users can now bookmark specific date ranges for quick access and reference.

• Data Explorer API endpoint: A new API endpoint has been added to support data explorer for Boomi OEM.

• Tabs are now scrollable: Improved usability by making the Tabs scrollable, ensuring better navigation and access.

• Pipeline tab inside search view: This enhances the search view and the user can see the pipeline of the selected flow.

• Pipeline application filter: While creating a new pipeline, users can filter which application to show in the pipeline view.

• Enhanced the Fleet agent manager installation.

Bug Fixes

• Inconsistent time range when moving from ALIVE to Facet Search page: Fixed the issue where the time range was inconsistent when moving from the ALIVE to the Facet Search page.

• Orphan tab from ALIVE: Resolved the issue of orphan tabs appearing from ALIVE.

• Alert page issue showing undefined value: Corrected the problem where the Alert view page was showing undefined values.

Getting Started

Data Management and Analytics

Observability

Discover the latest advancements and improvements of the Apica Ascent platform. This is your go-to destination for updates on platform enhancements and new features. Explore what's new to optimize your observability and data management strategies.

Data Fabric

Release v3.7 (February 11, 2023)

Welcome to the latest update of our product! We are excited to introduce several new features and improvements designed to enhance user experiences.

Refined User Interface:

• Introduced a refined User Interface across the app, enhancing user experience on the following pages:

  • Search

  • Data explorer

  • Topology

  • Pipeline

  • Dashboards

  • Query/Report editor

• Implemented dynamic quick date-time selection for granular control, empowering users to specify any date range they desire, not limited to predefined time ranges.

Infrastructure with Honeycomb View:

• This view offers users a bird's-eye view of all flow statuses on a single page.

• Users can customize group-by options like namespace, application, and severity to analyze the flow status of the entire stack.

• Flexible time range selection allows users to analyze their stack effectively.

Counter Widget in Explore Page

Added a new counter widget on the Explore page, enabling users to monitor ingested Trace volume across selected time ranges.

Query Snippets

Added Query Snippet templates, allowing users to create and insert query snippets from the settings page into the query editor using keyboard triggers/shortcuts.

ASM Plus

ASM Plus is a new offering enabling users to analyze their ASM synthetic check data in OpenTelemetry(OTel) format. Features include viewing check data as an Opentelemetry trace, page-level check execution details in a timeseries graph, check aggregator view with dynamic pivot table visualization, and check analysis view offering various visualizations like Waterfall chart, Flame graph, and Graph view.

• View checks data as a Opentelemetry trace in ASM plus.

• Check execution details (page level) view in a timeseries graph. Users can select different check attributes to analyze the check execution data.

• Check aggregator view

  • Provide a dynamic pivot table for visualizing the check data in different formats like Tabular, line chart, bar graph, etc. We have also added a feature where users can export their pivot table data in an excel format for further analysis.

  • Provides a timeseries graph for various kinds of service names.

• Check analysis view provides an option to view the check results data in the following visualizations:

  • Waterfall chart

  • Flamegraph

  • Graph view

New Forwarder for ServiceNow ITOM Event Management Connectors API:

• Added a new forwarder to facilitate integration with ServiceNow ITOM Event Management Connectors API.

New Query Parameter Type - Duration List:

• Introduced a new Query parameter type called Duration list, enabling users to create a dropdown of relative time durations in templatized queries.

Improved Dashboard Widgets Visualization:

• Enhanced dashboard widgets visualization by smoothing the data for better presentation.

Thank you for choosing our product! We hope you enjoy these new features and improvements. Should you have any questions or feedback, please do not hesitate to contact us.

Data Fabric Release v3.7.1 (March 11, 2024)

Bug Fixes:

ALIVE Graph and Summary Fixes: Corrected issues where the "select-all" function wasn't applying across all pages in the ALIVE graph and the pattern index and y-axis didn't match in the summary table.

ALIVE Page Navigation: The "psid log select-all" operation now correctly spans across all pages instead of just the current one.

Browser Compatibility: Resolved a bug where the Check analysis view was breaking specifically in old Firefox browsers.

UI and Display Fixes: Made improvements to various UI elements such as ensuring subject time intervals adhere strictly to different function screens and fixing issues with long horizontal content on the ALIVE summary page.

Query and Data Handling: Handled edge cases where errors in results could lead to spans having no data.

Performance and Functionality: Made improvements to several areas such as handling ingest ratelimiters more effectively, reducing open connections errors, and enhancing byte buffer pool performance.

Enhancements:

Dashboard Widget: Improved the overflow behavior for Alive Filter tags on the dashboard page for better visibility and usability.

User Experience: Enhanced the Add widget dialog by fixing issues related to selecting visualization types and restricting multiple API calls while using the "Add tag" feature.

Other Improvements:

Performance Optimization: Made improvements to several backend processes, including moving from ReadAll to io.Copy for better performance and memory benefits.

License Management: Fixed issues with licenses not syncing correctly and removed unknown fields from license display.

Code Maintenance: Made updates to code repositories for better version parity and improved rules page images display.

We're continuously working to enhance your experience with Apica Ascent Development, and we hope you find these updates valuable. If you have any questions or feedback, please don't hesitate to reach out to us. Thank you for choosing Apica!

Synthetic Monitoring

ASM 13.24 Public Release Notes (2024-04-12)

User Story Enhancements

• Added several new ASM commands to the ASM Manage Scenarios front end. See

for a complete list of supported Selenium IDE commands. Now, all of the commands listed in that article are available in the ASM Edit/Debug Scenarios page

Tasks

• ASM users now have the option to disable automatic page breaks when creating Browser checks:

Bug Fixes

• Fixed an issue in which checks were not correctly saved when an incorrect inclusion/exclusion period was used and the user was not notified of a reason. After the fix, users will be notified explicitly if their inclusion/exclusion period is incorrect.

• Fixed an issue which prevented custom DNS from being used on the latest infrastructure

• Fixed an issue which prevented an error message from being generated and displayed in the event that auto refresh fails to refresh a Dashboard.

• Fixed an issue with certain checks which prevented Request & Response Headers from showing correctly within the Check Details page:

• Fixed an issue which prevented API calls from returning correct responses when a new user’s time zone was not set

• Fixed an issue which prevented spaces in between the “accepted codes” field for a URLv2 check:

• Updated API documentation for URL, URLv2 checks to include acceptable "secureProtocolVersion" values

• Fixed an issue with Ad Hoc report generation for certain users

• Fixed issues which prevented Command checks from being created or fetched via the ASM API.

Epic

• Disabled the option to select "Firefox" on browser checks

• Disabled location information in the API for deprecated checks

• Disabled old Chrome versions when creating a Chrome check

• Disabled location information in the API for deprecated Chrome versions

• Disabled deprecated check types from the "create new check"

• Disabled deprecated check types from the integration wizard

• Disabled API endpoint for URLv1 checks

• Disabled API endpoint for Command v1 checks

• Disabled deprecated check types from /checks/command-v2/categories

• Disabled deprecated browser version from /AnalyzeUrl

• Replaced Firefox with Chrome when creating an iPhone, iPad, or Android Check in New Check Guide

• Removed deprecated check versions as options from the Edit Scenario page

• Disabled AppDynamics check types from the integration wizard

Synthetic Monitoring On Premise

On Premise ASM Patch 13H.4 Public Release Notes (2024-04-19)

User Story Enhancements

• Added the ability to add/edit “Accepted Codes”, “Port Number” and all “Secure Protocol Versions” for URLv1 checks via the ASM API. API documentation was updated to reflect the new functionality.

• Added SNI (Server Name Indication) support for URLv1 checks

Bug Fixes

• Fixed an issue which prevented Power Users with limited check editing permissions from saving checks after performing edits.

Advanced Scripting Engine

Major Release V7.5-B (Installation Kit dated April 17, 2024)

ZebraTester 7.5-B release contains the following new features.

• Support for Color Blindness: To improve support for vision impairments and color blindness adaptation we have added new themes to the GUI configuration section.

• Ability to change request method from the ZT GUI: This gives the users the ability to change request method from the ZT GUI. Depending on the request method the Request body field will be enabled & visible or not.

• Support user agent details from a file: Provides an option in ZT personal settings GUI settings area, where user can upload a JSON file, which have all the latest User-Agents details.

• Updated Browser Agent List: All the current and latest browser agent list has been updated. • Option to Disable Page Breaks: Option to comment/disable a page break in the recorded session.

• Variables as Page Break Names: Users can use variables when setting my page-breaks names to make scripts more dynamic.

• Add OR condition for content type validation: Logical OR condition against content type validation can be tested by users.

• ZebraTester Controller Pull file (.wri): User will be able to pull files from the execagent that have been written by the feature "writetofile". For this the files need to be pulled to the controller as any other out/err/result file.

• WebSocket Extension (MS1): WebSocket implementation capabilities of Zebra Tester, allowing users to conduct more comprehensive testing of WebSocket-based applications. A detailed how guide on how to use WebSocket extension is added in the documentation folder.

In addition, Zebra Tester V7.5-B release contains the following bug fixes / improvements:

• Bug Fix for XML extractor giving 500 internal error in ZT scripts.

• .Har file conversion issue.

• Conflict when using variables as Mime Type validation.

• Zebra Tester -Auto assign Fix

• Fix for time zone lists, shows the java standard supported time zones without the deprecated ones.

• Detailed Replay logs in ZT (extended logs)

• ALPN Protocol Negotiation

• Page Break - Threshold Breach (Trigger & Abort)

• Library Update (Update JGit library): Updated the JGit library to the latest version to leverage new features and improvements.

• Fix issues with JavaScript editor in ZT.

IRONdb

Release Version 1.2.0

NOTE: This release bumps the metric index version from 4 to 5. Upon restart, new indexes will be built and the old ones will be deleted. This process will use a significant amount of memory while the indexes are being rebuilt. It will also cause the first post-update boot to take longer than usual.

• Update index version from 4 to 5.

• Automatically clean up old index versions on startup to make sure outdated indexes don't clog the disk.

• Fix Ubuntu 20.04 specific bug where nodes could crash when trying to clean up status files when rolling up raw shards.

• Fix issue with level indexes where data was being lost when deleting metrics on levels where the metric has multiple tags.

• Fix issue where level indexes were incorrectly reporting that levels existed when all underlying metrics had been removed.

• Add new API endpoints, /compact_indexes and /invalidate_index_cache, that allow forcing compaction and cache invalidation for specific accounts, respectively.

• Fix rollup bug where raw shards could be prematurely deleted if a rollup was aborted due to corruption.

• Fix various potential memory corruption issues.

• Fix issue where jlog journal data could get corrupted.

Release v3.7.1 (March 11, 2024)

Bug Fixes:

ALIVE Graph and Summary Fixes: Corrected issues where the "select-all" function wasn't applying across all pages in the ALIVE graph and the pattern index and y-axis didn't match in the summary table.

ALIVE Page Navigation: The "psid log select-all" operation now correctly spans across all pages instead of just the current one.

Browser Compatibility: Resolved a bug where the Check analysis view was breaking specifically in old Firefox browsers.

UI and Display Fixes: Made improvements to various UI elements such as ensuring subject time intervals adhere strictly to different function screens and fixing issues with long horizontal content on the ALIVE summary page.

Query and Data Handling: Handled edge cases where errors in results could lead to spans having no data.

Performance and Functionality: Made improvements to several areas such as handling ingest ratelimiters more effectively, reducing open connections errors, and enhancing byte buffer pool performance.

Enhancements:

Dashboard Widget: Improved the overflow behavior for Alive Filter tags on the dashboard page for better visibility and usability.

User Experience: Enhanced the Add widget dialog by fixing issues related to selecting visualization types and restricting multiple API calls while using the "Add tag" feature.

Other Improvements:

Performance Optimization: Made improvements to several backend processes, including moving from ReadAll to io.Copy for better performance and memory benefits.

License Management: Fixed issues with licenses not syncing correctly and removed unknown fields from license display.

Code Maintenance: Made updates to code repositories for better version parity and improved rules page images display.

We're continuously working to enhance your experience with Apica Ascent Development, and we hope you find these updates valuable. If you have any questions or feedback, please don't hesitate to reach out to us. Thank you for choosing Apica!

Release v3.7 (February 11, 2023)

Welcome to the latest update of our product! We are excited to introduce several new features and improvements designed to enhance user experiences.

Refined User Interface:

• Introduced a refined User Interface across the app, enhancing user experience on the following pages:

  • Search

  • Data explorer

  • Topology

  • Pipeline

  • Dashboards

  • Query/Report editor

• Implemented dynamic quick date-time selection for granular control, empowering users to specify any date range they desire, not limited to predefined time ranges.

Infrastructure with Honeycomb View:

• This view offers users a bird's-eye view of all flow statuses on a single page.

• Users can customize group-by options like namespace, application, and severity to analyze the flow status of the entire stack.

• Flexible time range selection allows users to analyze their stack effectively.

Counter Widget in Explore Page

Added a new counter widget on the Explore page, enabling users to monitor ingested Trace volume across selected time ranges.

Query Snippets

Added Query Snippet templates, allowing users to create and insert query snippets from the settings page into the query editor using keyboard triggers/shortcuts.

ASM Plus

ASM Plus is a new offering enabling users to analyze their ASM synthetic check data in OpenTelemetry(OTel) format. Features include viewing check data as an Opentelemetry trace, page-level check execution details in a timeseries graph, check aggregator view with dynamic pivot table visualization, and check analysis view offering various visualizations like Waterfall chart, Flame graph, and Graph view.

• View checks data as a Opentelemetry trace in ASM plus.

• Check execution details (page level) view in a timeseries graph. Users can select different check attributes to analyze the check execution data.

• Check aggregator view

  • Provide a dynamic pivot table for visualizing the check data in different formats like Tabular, line chart, bar graph, etc. We have also added a feature where users can export their pivot table data in an excel format for further analysis.

  • Provides a timeseries graph for various kinds of service names.

• Check analysis view provides an option to view the check results data in the following visualizations:

  • Waterfall chart

  • Flamegraph

  • Graph view

New Forwarder for ServiceNow ITOM Event Management Connectors API:

• Added a new forwarder to facilitate integration with ServiceNow ITOM Event Management Connectors API.

New Query Parameter Type - Duration List:

• Introduced a new Query parameter type called Duration list, enabling users to create a dropdown of relative time durations in templatized queries.

Improved Dashboard Widgets Visualization:

• Enhanced dashboard widgets visualization by smoothing the data for better presentation.

Thank you for choosing our product! We hope you enjoy these new features and improvements. Should you have any questions or feedback, please do not hesitate to contact us.

Release v3.6 (September 11, 2023)

Welcome to Release v3.6. We're excited to introduce several new features and improvements designed to enhance user experience.

• Enhanced Log Analysis with Generative AI like ChatGPT and Azure OpenAI Service

  • We're excited to introduce the integration of Generative AI, including ChatGPT and Azure OpenAI Service, into the Explore feature. Now, you can easily select logs and engage in dynamic conversations with Generative AI to gain in-depth insights into your log data. Ask questions, request explanations, and explore your logs to gain deeper insights into your log data, making log analysis more informative and versatile.

• ALIVE (Autonomous Log Interaction Visual Explorer): ALIVE is a powerful interactive visualization tool designed to empower users in identifying issues and patterns within their applications. This innovative tool offers a rich and insightful representation of unstructured logs. Key features includes:

  • Autonomous Log Analysis: ALIVE autonomously analyzes logs, saving users time and effort.

  • Interactive Visualization: Enjoy an interactive and engaging experience when exploring log data.

  • Flow Representation: Understand the flow of log events with clear and intuitive visualizations.

  • Insightful Representation: Gain deep insights into your log data through meaningful visual representations.

  • Multivariate Analysis: Easily pinpoint issues across vast datasets at a glance.

  • Scalability: ALIVE scales effortlessly to accommodate your growing data needs, ensuring consistent performance.

  • Improved pattern compaction (PC) workload: With this release, we've enhanced the pattern compaction feature. Now, as the number of patterns increases, selected patterns can be further aggregated into the same group to prevent pattern count explosion. This process is called compaction. We've also added a button in the ingestion settings that allows you to disable or enable pattern compaction. Please note that this feature is not intended for use with static or small pattern sets, and excessive use of the PC action can result in pattern aliasing.

• Enhanced Onboarding Experience with App Tour: Introducing App Tour, designed to provide both new and returning users with a seamless introduction to our platform's key features. This guided tour ensures a smooth and intuitive navigation experience right from the start, helping you quickly become familiar with our app's functionalities and empowering you to make the most of our platform.

  • App tour coverage:

    • Explore (Data, Topology, Flows)

    • Dashboards

    • Source Extensions

    • Search

    • Rules

    • Forwarder

    • Queries

    • Create Rule

• Moved Source Extensions, Forwarders, Rule Packs and Import dashboards to new Integrations page.

• New Source Extensions

  • SNMP Source Extension

  • Apica Source Extension: The Apica Source Extension is a component designed to integrate with the Apica Synthetics and Load test platform. Its main purpose is to retrieve check results from the Apica platform and make them available for further processing or analysis within another system or tool.

• New Forwarders which can help users selectively send specific log data to downward destinations based on their filtering criteria, thereby reducing the amount of data stored and analyzed. This can lead to cost optimization as it allows users to focus their resources on the most relevant and important log data, rather than storing and processing unnecessary or redundant information.

  • Coralogix

  • GCP BigQuery

  • Azure Log Analytics

• Topology view Enhancements ✨ Recent enhancements in the topology view is the inclusion of the total events information. This improvement provides users with a clearer understanding of the overall event activity within the system or network.

• Pipeline Changes:

  • Pipeline Application Filtering Support

    • We're excited to introduce support for pipeline application filtering in this release. With this enhancement, users can efficiently filter log data when managing multiple applications, streamlining their data management processes.

  • Error Indicator

    • We've also added an error indicator to the Pipeline View. This indicator serves as a valuable visual cue when forwarding logs to destinations, helping users quickly identify and address any issues in their data flow to downward destinations.

• Faster Reports

  • In this release, we've significantly improved report generation speed by removing the 10-second polling delay.

Search

• Optimized Search

  • We've enhanced search performance by further optimizing the search query parallelism, ensuring quicker and more efficient results retrieval.

• Enhancement in Search feature by adding Regex for Extract.

  • Get a holistic taxonomy of logs by automatically categorizing them based on their content, context and other characteristics. This capability provides users with a way to extract and classify logs automatically, improving the speed and accuracy of log-analysis. This saves time and effort by automating the process of field extraction, eliminating the need for users to manually identify and extract fields .

Other Enhancements:

• Aggregate Settings Persistence

  • We've introduced the convenience of persistent aggregate settings. Now, when users select an aggregate, the system will remember their choice, ensuring that their selection remains consistent across sessions.

• Table View for Structured Data

  • We've deprecated the Tree view and introduced a more user-friendly Table View for structured data derived from log lines.

• Revamped Forwarder Selection UX

  • Experience an enhanced user interface when selecting forwarders during creation. Our redesigned forwarder selection process is more intuitive and efficient.

• Log-to-Traces Proxy

  • We've built a versatile proxy that can seamlessly convert logs into traces. This allows logs to be stitched into multiple spans, forming a comprehensive trace for improved monitoring and analysis.

• Multiple Widgets in Dashboards

  • Enhance your dashboards with ease. Users can now add multiple visualizations related to various queries in a single step, providing more flexibility in dashboard creation.

• Distribution Flow

  • We've deprecated the distribution flow feature from the forwarders page, making it even more accessible. Users can now access distribution flow directly from the explore page.

Bug Fixes

• Increased gRPC Recovery Limits

  • We've addressed an issue that could sometimes result in partial search results. By increasing the gRPC recovery limits, we've improved the reliability of search operations in our platform.

We Value Your Feedback

If you have any questions, encounter issues, or want to share your thoughts, please don't hesitate to contact our support team. Thank you for choosing Apica as your data fabric partner. We look forward to continuously improving your experience.

v3.5.9.1

📅 Fri, Mar 24, 2023

• Topology-powered root-cause analysis.

  Visualize your data streams as a topology with drill down to errors and warnings for faster root causes. Helps visualize the health of your applications. Users can quickly investigate the issues by clicking errors or alerts.

• Data flow Pipelines.

  The pipeline is a series of processes or stages through which data flow systematically and efficiently. Helps to visualize the flow between nodes, rules, and filters applied for the data flow. Shows the inflow and outflow information of data, and also helps in identifying the data loss or optimizing the data flow to forward destinations.

• Search results aggregates.

  Buit-in Pivot table makes it easy to analyze large data sets from search queries. Summarize or Visualize a set of data points for instant analysis. Some common examples of aggregation functions include(Count, Value, Sum, Count Unique Values, List Unique Values, Average, Median, Min, Max). Aggregation functions are used to summarize large datasets into a more manageable form for further analysis and visualization. And includes different types of visualizations (Table, Line chart, Area chart, Scatter chart, Dot chart, and Multiple pie chart).

• Re-designed Landing page.

  Instantly get access to valuable insights when you login into our redesigned Explore page. Users now log in and directly land on the Explore page with quick summaries at their fingertips.

  1. Introduced counter widgets for EPS, Total Flows, Total Events, Forwarders, and Source Extensions.

  2. Added a new Event Statistics column, which has counts of (Errors, Alerts, Critical, Emergency), (Warnings) and (Total) events.

• OSSEC HIDS Mappings

  Automatically map OSSEC HIDS event severity and log messages for Linux and Windows environments.

• Added support for exporting events and metrics from Apache Beam to Apica Ascent.

• OpenTelemetry otel.status_code Mapping

  Detect OpenTelemetry severity and level embeddings and map them into severity levels.

• Memory and performance improvements.

• Automated agent installation for Linux and Windows.

v3.5.8.4

📆 Thu, Dec 29, 2022

• Added support for Grafana dashboard import for:

  • Fluent Bit

  • Go

  • Kafka

  • Kubernetes

  • Node exporter

  • Postgres

  • Prometheus

  • Redis

• Added support for Large log messages up to 1Mb

• Added native support for Azure blob store for InstaStore

• Added new Ingest plugins for:

  • Kafka/Confluent

  • Azure EventHub

  • GCP PubSub

  • AWS Kinesis

  • S3 Compatible object storage

  • Splunk S2S cooked mode

  • IBM QRadar

  • OSSEC

• Added new Forwarder plugins for:

  • Splunk Metric index

  • Azure EventHub

  • AWS Kinesis

  • Google PubSub

  • S3 compatible object stores and Azure Blob store

• Added support for Renaming attributes of logs before forwarding data to the destination

• Added support for ingesting directly from Splunk UF (Universal Forwarder) and Splunk HF (Heavy Forwarder) using Splunk cooked mode

• Added support for ingesting Splunk Metrics

• Added support for Archiving alerts in InstaStore which will be available as an audit trail

• Added support for Archiving events that are older than 24 hours which will be available under events history

• Added new Severity Metrics to measure the logs levels within the time range.

• Made Search and UI performance enhancements ✨

3.0 - 2022-07-31

Changed

1. Distributed Tracing

   • Compatible with OpenTelemetry and Jaeger agents

   • Infinite Scale and retention on InstaStore

2. AnyTime - AI/ML Engine for any time-series database

   • Anomaly detection

   • Statistical data baselining and dynamic thresholds

   • Forecasting

   • Moving average

3. Anomaly and dynamic threshold-based alerts

4. Input Plugins

   • IBM QRadar

   • Splunk S2S

5. LogFlow Forwarders

   • Splunk Syslog, Splunk HEC, Splunk S2S

   • Datadog

   • Dynatrace

   • NewRelic

   • IBM QRadar

   • ArcSight

   • Syslog, Syslog CEF

6. New Data Sources

   • AWS CloudWatch Metrics ( YAML )

   • AWS CloudWatch Metrics (SQL Insights)

2.1.11.32 - 2021-06-11

Changed

• LD-35 multi-variate regEx in log2Metrics

2.1.11.31 - 2021-06-07

Changed

• A-4 Comparator operator results in lesser search results

2.1.11.30 - 2021-06-02

Changed

• LD-30 Custom Search Indices

2.1.11.26 - 2021-04-15

Changed

• Namespace level Log distribution graph in Search and Logs page

• Performance Improvements

2.1.11.25 - 2021-04-08

Fixed

• DEFECT#613 Query backend not honoring the startTime sent from UI

Changed

• PERF#614 Uploader Optimizations

2.1.11.24 - 2021-04-02

Changed

• PERF#603 Query Improvements

  • Bloom filter for faster search

  • Query interval skip improvements

• PERF#602 Metadata Improvements

  • Metadata uploader improvements

• UI#599 UI enhancements

  • Search and logs page optimizations

• FEATURE#580 AWS improvements

  • AWS ECS logging improvements

  • Customized AWS fargate 1.4 fluent driver image

  • AWS Cloudwatch exporter

2.1.11 - 2020-12-13

Log aggregation

• Support for and expressions in search

• Event rules designer support for && and || for individual parameters

• Performance and memory improvements

Data convergence

• Support for Apache DRUID connector

• Optionally deploy Grafana with the LOGIQ stack

UI

• Logs compare view to select 2 logs to be viewed side by side

• Easy toggle for activating/deactivating periodic queries

2.1.9 - 2020-11-28

Added

Log aggregation

• Support for full data/metadata recovery on service restarts

Log analytics

• Support for application and process/pod context in log and search views

Deployment and Infrastructure

• Support for node selectors. Both taints and node selectors are supported

• Support for using spot instances on EKS/AWS

• Support for using S3 compatible buckets directly without a caching gateway to optimize for region optimized deployments

UI

• Multi cluster support

• License management UI

• Ingest configuration settings exposed in the UI

• Logs page and search now have application and process/pod level contexts

2.1 - 2020-09-28

Added

Log aggregation

• Parquet with Snappy compression for data at REST

Log analytics

• Log view supports full JSON view for ingested log data like Search view

• Performance improvements for faster search, logs, and tailing

Monitoring

• Event deduplication can reduce event data by up to 1000x at peak data rates

• Deduplication of monitoring events at Namespace granularity

Deployment and Infrastructure

• Separation of LOGIQ server into microservices for data ingestion, ML/UI and S3/Metadata management

• Support for taints in HELM chart for more control over large-scale deployments e.g. schedule ingest pods on dedicated nodes etc.

• Log tailing infrastructure using Redis switches to diskless replication/no persistence

2.0 - 2020-08-12

Added

Log aggregation

• Support for AWS Fargate, Firelens, Fluent forward Protocol

• LOGIQ Fluent-bit daemon-set for K8S clusters

• Data extraction via Grok patterns, compatible with Logstash Grok patterns using the Grokky library

Log analytics

• Redesigned - Elastic/Kibana like search UI that scales to infinite data from S3 compatible object store

• Real-time alertable events and alerts from log data

• Real-time extraction of log data facets using Grok expressions

• 1-Click conversion of log data events to time series visualization

Logiqctl

• Logiqctl command-line toolkit

• Works with SAML users via API Key

Monitoring

• Prometheus alert manager integration into LOGIQ alerts for unified alerting across logs and metrics

• Built-in Logiq dashboard for LOGIQ cluster performance and health monitoring

LOGIQ Data platform

• Connect numerous popular data sources into the LOGIQ platforms such as Postgres, MySql, Elasticsearch, Athena, MongoDB, Prometheus, and more.

• JSON Data source for easily converting arbitrary JSON data into tables, widgets, and alerts

Role-based access control

• Namespace RBAC for log data from K8S namespaces

• SAML Integration for RBAC allowing SAML Attributes to map to RBAC groups

Security and Compliance

• Fully secured HELM deployment using Role, RoleBindings, ServiceAccounts and Pod Security policies for all service

• Cryptographically verified JWT token for API communication

• Built-in audit logging for the entire product and infrastructure

1.2.1 - 2020-05-11

Added

K8S

• Add support for ingress with http and optionally have https

• ServiceMonitor for ingesting server if prometheus is installed

UI

• Logs modal ordering to match how developers view logs from a file

• Highlight logline from search

Misc

• Bug fixes for performance, graceful failure handling/recovery

1.2.0 - 2020-04-28

Official GA of LOGIQ's complete Observability platform with support for metrics and log analytics

Added

K8S

• Scale-out and HA deployment for Kubernetes via HELM 3 chart ( https://github.com/logiqai/helm-charts )

UI

• Monitoring of time series metrics

• New Log viewer

• Log viewer integration with faceted search

• Log time machine to go back in time to log state

CLI

• logiqctl is now GA with support for log tailing, historical queries and search

1.1.0 - 2020-02-27

Fixed

• Fluentd sends error logs as info - fixed with grok patterns to extract proper severity strings from incoming messages

Added

• Anomaly detection via Eventing with built-in and custom rules

• Built-in UI Help with Intercom chat

• Expand and collapse search facets

Changed

• New AMI's for AWS marketplace

1.0.0 - 2020-01-21

Official GA of LOGIQ's Log Insights platform

Added

• AWS Marketplace AMI for all regions including Gov cloud regions

• AWS CloudFormation 1-click deployment

• Rsyslog, Syslog protocol support for data ingest via Rsyslogd, syslogd, syslog-ng, logstash, fluentd, fluentbit, docker logging syslog driver.

• Built-in UI with SQL Queries, Faceted search, Alerts, Dashboards

Release Version 1.2.0

NOTE: This release bumps the metric index version from 4 to 5. Upon restart, new indexes will be built and the old ones will be deleted. This process will use a significant amount of memory while the indexes are being rebuilt. It will also cause the first post-update boot to take longer than usual.

• Update index version from 4 to 5.

• Automatically clean up old index versions on startup to make sure outdated indexes don't clog the disk.

• Fix Ubuntu 20.04 specific bug where nodes could crash when trying to clean up status files when rolling up raw shards.

• Fix issue with level indexes where data was being lost when deleting metrics on levels where the metric has multiple tags.

• Fix issue where level indexes were incorrectly reporting that levels existed when all underlying metrics had been removed.

• Add new API endpoints, /compact_indexes and /invalidate_index_cache, that allow forcing compaction and cache invalidation for specific accounts, respectively.

• Fix rollup bug where raw shards could be prematurely deleted if a rollup was aborted due to corruption.

• Fix various potential memory corruption issues.

• Fix issue where jlog journal data could get corrupted.

• libmtev 2.7.1

Install Rsyslog

1. For Debian/Ubuntu:

   Copy

   sudo apt update
   sudo apt install rsyslog

2. For RHEL/CentOS:

   Copy

   sudo yum install rsyslog
   sudo systemctl enable rsyslog
   sudo systemctl start rsyslog

Verify that rsyslog is running:

sudo systemctl status rsyslog

Configure forwarding

Edit the rsyslog configuration file (usually /etc/rsyslog.conf or /etc/rsyslog.d/*.conf).

1. Open the configuration file:

   Copy

   sudo nano /etc/rsyslog.conf

2. Enable TCP forwarding by adding *.* @@remote-server-ip:514 to the config:

   Copy

   # /etc/rsyslog.conf configuration file for rsyslog
   #
   # For more information install rsyslog-doc and see
   # /usr/share/doc/rsyslog-doc/html/configuration/index.html
   #
   # Default logging rules can be found in /etc/rsyslog.d/50-default.conf
   
   
   #################
   #### MODULES ####
   #################
   
   *.* @@<YOUR-ASCENT-ENV>:514

3. Save your changes and restart rsyslog

   Copy

   sudo systemctl restart rsyslog

Verify ingestion in Ascent

On your server, use logger to log a custom message which you can track easily in order to verify ingestion has been successful.

1. Use the logger command to trigger a custom log entry:

   Copy

   logger "This is a test message from $(hostname)"

   It might take a slight moment for this entry to appear in the Ascent platform, so if it doesn’t show up immediately, give it a moment and check again.

2. In your Ascent platform, navigate to Explore > Logs & Insights

3. In the filter view, search for namespace default_namespace. Then look for your username which generated the custom log entry, and click on it.

4. This view should only display the custom log entry generated earlier

Install OpenTelemetry

1. Go to https://opentelemetry.io/docs/collector/installation/ or https://github.com/open-telemetry/opentelemetry-collector-releases/releases/ to find the package you want to install. At the point of writing this guide, 0.115.1 is the latest package so we’ll install otelcol-contrib_0.115.1_linux_amd64

2. On the machine you wish to collect metrics from, run the following 4 commands:

   1. Deb-based

      1. sudo apt-get update

      2. sudo apt-get -y install wget

      3. wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.115.1/otelcol-contrib_0.115.1_linux_amd64.deb

      4. sudo dpkg -i otelcol-contrib_0.115.1_linux_amd64.deb

   2. RHEL-based

      1. sudo dnf update -y

      2. sudo dnf install -y wget

      3. wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.115.1/otelcol-contrib_0.115.1_linux_amd64.rpm

      4. sudo rpm -ivh otelcol-contrib_0.115.1_linux_amd64.rpm

3. Navigate to /etc/otelcol-contrib/

4. Edit the file with your favourite file editor, for example: nano config.yaml

5. Paste the following into the config file overwriting it completely:

   1. Copy

      receivers:
        hostmetrics:
          collection_interval: 10s
          scrapers:
            cpu:
              metrics:
                system.cpu.utilization:
                  enabled: true
            load:
            memory:
            filesystem:
            network:
            disk:
            paging:
            processes:
      
      processors:
        batch:
          timeout: 5s
      
      exporters:
        debug:
          verbosity: detailed
        prometheusremotewrite:
          endpoint: https://<YOUR-ASCENT-ENV>/api/v1/receive
          headers:
            Authorization: Bearer <YOUR-INGEST-TOKEN>
          tls:
            insecure: false
            insecure_skip_verify: true
      
      service:
        pipelines:
          metrics:
            receivers: [hostmetrics]
            processors: [batch]
            exporters: [prometheusremotewrite, debug]

   2. Replace <YOUR-ASCENT-ENV>with your Ascent domain, e.g. company.apica.io

   3. Replace <YOUR-INGEST-TOKEN>with your Ascent Ingest Token, e.g. eyXXXXXXXXXXX...

      1. Follow this guide on how to obtain your ingest token - https://docs.apica.io/integrations/overview/generating-a-secure-ingest-token

6. When you’ve finished editing the config, save it and run otelcol-contrib validate --config=config.yaml

   1. If you get no error returned, the config file is valid.

7. Restart the service with sudo systemctl restart otelcol-contrib

8. Verify that the service is up and running correctly with sudo systemctl status otelcol-contrib

   1. A good result should look like this:

   2. Copy

      otelcol-contrib.service - OpenTelemetry Collector Contrib
           Loaded: loaded (/usr/lib/systemd/system/otelcol-contrib.service; enabled; preset: enabled)
           Active: active (running) since Tue 2024-11-19 15:29:59 UTC; 9s ago
         Main PID: 26248 (otelcol-contrib)
            Tasks: 8 (limit: 4630)
           Memory: 33.1M (peak: 33.7M)
              CPU: 98ms
           CGroup: /system.slice/otelcol-contrib.service
                   └─26248 /usr/bin/otelcol-contrib --config=/etc/otelcol-contrib/config.yaml
      
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]:      -> Description: Total number of created processes.
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]:      -> Unit: {processes}
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]:      -> DataType: Sum
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]:      -> IsMonotonic: true
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]:      -> AggregationTemporality: Cumulative
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]: NumberDataPoints #0
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]: StartTimestamp: 2024-11-18 10:25:54 +0000 UTC
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]: Timestamp: 2024-11-19 15:30:00.536392834 +0000 UTC
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]: Value: 26262
      Nov 19 15:30:04 otel-testing otelcol-contrib[26248]:         {"kind": "exporter", "data_type": "metrics", "name": "debug"}

   3. You can also view live logs using journalctl -u otelcol-contrib -f. With the above config you would see entries every 10 seconds.

Verify metrics in the Ascent platform

1. Click on the green “+ Create” button on the top navigation bar and select Query

2. In the dropdown menu on the left hand side, select Ascent Metrics

3. In the search bar, search for system_

   1. This will present all the different system metrics that is being scraped with your Otel configuration

   2. You can click any of the metrics directly to insert it into the query text, and hit execute to see the latest metrics.

Major Release V7.5-B (Installation Kit dated April 17, 2024)

ZebraTester 7.5-B release contains the following new features.

• Support for Color Blindness: To improve support for vision impairments and color blindness adaptation we have added new themes to the GUI configuration section.

• Ability to change request method from the ZT GUI: This gives the users the ability to change request method from the ZT GUI. Depending on the request method the Request body field will be enabled & visible or not.

• Support user agent details from a file: Provides an option in ZT personal settings GUI settings area, where user can upload a JSON file, which have all the latest User-Agents details.

• Updated Browser Agent List: All the current and latest browser agent list has been updated. • Option to Disable Page Breaks: Option to comment/disable a page break in the recorded session.

• Variables as Page Break Names: Users can use variables when setting my page-breaks names to make scripts more dynamic.

• Add OR condition for content type validation: Logical OR condition against content type validation can be tested by users.

• ZebraTester Controller Pull file (.wri): User will be able to pull files from the execagent that have been written by the feature "writetofile". For this the files need to be pulled to the controller as any other out/err/result file.

• WebSocket Extension (MS1): WebSocket implementation capabilities of Zebra Tester, allowing users to conduct more comprehensive testing of WebSocket-based applications. A detailed how guide on how to use WebSocket extension is added in the documentation folder.

In addition, Zebra Tester V7.5-B release contains the following bug fixes / improvements:

• Bug Fix for XML extractor giving 500 internal error in ZT scripts.

• .Har file conversion issue.

• Conflict when using variables as Mime Type validation.

• Zebra Tester -Auto assign Fix

• Fix for time zone lists, shows the java standard supported time zones without the deprecated ones.

• Detailed Replay logs in ZT (extended logs)

• ALPN Protocol Negotiation

• Page Break - Threshold Breach (Trigger & Abort)

• Library Update (Update JGit library): Updated the JGit library to the latest version to leverage new features and improvements.

• Fix issues with JavaScript editor in ZT.

Read previous Release notes,

Install otelcol-contrib

For DEB-based:

wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.120.0/otelcol-contrib_0.120.0_linux_amd64.deb

dpkg -i otelcol-contrib_0.121.0_linux_amd64.deb

For RHEL-based:

wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.121.0/otelcol-contrib_0.121.0_linux_amd64.rpm

rpm -ivh otelcol-contrib_0.121.0_linux_amd64.rpm

Configure Collector

Edit /etc/otelcol-contrib/config.yaml and replace the content with the below

receivers:
  filelog:
    include: ["<your_log_file_path>"]
    multiline:
      line_start_pattern: '^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3}'

processors: 
  batch:
    timeout: 5s

exporters:
  debug:
    verbosity: detailed   
  otlphttp:
    logs_endpoint: https://<your_domain>/v1/json_batch/otlplogs?namespace=<namespace>&application=<application>
    encoding: json
    compression: gzip
    headers:
      Authorization: "Bearer <your_token>"
    tls:
      insecure: false
      insecure_skip_verify: true

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [batch]
      exporters: [debug, otlphttp]

Replace the following values:

• <your_log_file_path>

  • Physical path to your log file

• <your_domain>

  • Hostname of your Apica environment (example.apica.io)

• <your_token>

  • Your ingest token, see how to obtain your ingest token

• <namespace>

  • A name for high-level grouping of logs, isolating different projects, environments, or teams.

• <application>

  • A name for logs generated by a specific service or process

• line_start_pattern

  • The above example uses a regex to match on the timestamp of a log entry to capture the entire entry. This needs to be adjusted to match the beginning of your log structure. See below example of entries that matches this pattern.

2000-00-00 00:00:00,000 INFO  [xxx] process1: message

2000-00-00 00:00:00,000 INFO  [xxx] process2: message

2000-00-00 00:00:00,000 ERROR [xxx] process3: Exception: xyz
java.lang.xxx: message
	at java.base
	at java.base
	at java.base
	at java.base
	at java.base
	at java.base
	at java.base
	
#### The entire stack trace will be captured as a single entry, based on the line_start_pattern

Validate and apply

When you're done with your edits, execute the below command to validate the config is valid (it should return nothing if everything is in order)

otelcol-contrib validate --config=/etc/otelcol-contrib/config.yaml

Restart OTel to apply your changes

systemctl restart otelcol-contrib

Ascent view

Assuming everything has been done correctly, your logs will start to appear in Explore > Logs & Insight on your Ascent environment. They will show up based on the namespace and application names that you set in your config.yaml file.

If you are using Apica Ascent online, this section is not relevant. Please use this section if you are setting up and configuring Apica Ascent that is either ON-PREMISE or in a PRIVATE CLOUD.

ASM 13.24 Public Release Notes (2024-04-12)

User Story Enhancements

• Updated the Compound Check type to run on the latest infrastructure

• Added a new supported Selenium IDE command, setLocation

• Added missing attributes to the response bodies of the /users and /users/{user_guid} API GET request endpoints

• Added several new ASM commands to the ASM Manage Scenarios front end. See

for a complete list of supported Selenium IDE commands. Now, all of the commands listed in that article are available in the ASM Edit/Debug Scenarios page

Tasks

• ASM users now have the option to disable automatic page breaks when creating Browser checks:

Bug Fixes

• Fixed an issue in which checks were not correctly saved when an incorrect inclusion/exclusion period was used and the user was not notified of a reason. After the fix, users will be notified explicitly if their inclusion/exclusion period is incorrect.

• Fixed an issue which prevented custom DNS from being used on the latest infrastructure

• Fixed an issue which prevented an error message from being generated and displayed in the event that auto refresh fails to refresh a Dashboard.

• Fixed an issue which prevented Power Users who had limited editing permissions from saving checks. For instance, Power Users who could edit only the name, description, and tags of a check could not save the check after doing so. The bug fix resolved this issue.

• Fixed the following API call: https://api-wpm.apicasystem.com/v3/Help/Route/GET-checks-proxysniffer-checkId-results-resultId-errorlog which was returning a 500 server error previously.

• Fixed an issue with certain checks which prevented Request & Response Headers from showing correctly within the Check Details page:

• Fixed an issue which prevented API calls from returning correct responses when a new user’s time zone was not set

• Fixed an issue which prevented spaces in between the “accepted codes” field for a URLv2 check:

• Updated API documentation for URL, URLv2 checks to include acceptable "secureProtocolVersion" values

• Fixed an issue with Ad Hoc report generation for certain users

• Fixed issues which prevented Command checks from being created or fetched via the ASM API.

Epic

• Disabled the option to select "Firefox" on browser checks

• Disabled location information in the API for deprecated checks

• Disabled old Chrome versions when creating a Chrome check

• Disabled location information in the API for deprecated Chrome versions

• Disabled deprecated check types from the "create new check"

• Disabled deprecated check types from the integration wizard

• Disabled API endpoint for URLv1 checks

• Disabled API endpoint for Command v1 checks

• Disabled deprecated check types from /checks/command-v2/categories

• Disabled deprecated browser version from /AnalyzeUrl

• Replaced Firefox with Chrome when creating an iPhone, iPad, or Android Check in New Check Guide

• Removed deprecated check versions as options from the Edit Scenario page

• Disabled AppDynamics check types from the integration wizard

Read previous Release Notes, go to: Knowledge Base

On Premise ASM Patch 13H.4 Public Release Notes (2024-04-19)

User Story Enhancements

• Added the ability to add/edit “Accepted Codes”, “Port Number” and all “Secure Protocol Versions” for URLv1 checks via the ASM API. API documentation was updated to reflect the new functionality.

• Added SNI (Server Name Indication) support for URLv1 checks

Bug Fixes

• Fixed an issue which prevented Power Users with limited check editing permissions from saving checks after performing edits.

Read previous Release Notes, go to: Knowledge Base

1 - Prerequisites

• Kubernetes 1.18, 1.19 or 1.20

• Helm 3.2.0+

• Dynamic PV provisioner support in the underlying infrastructure

• ReadWriteMany volumes for deployment scaling

Apica Ascent K8S components are made available as helm charts.

1.1 Add Apica Ascent helm repository

1.1.0 Adding Apica Ascent's helm repository to your HELM repositories

You can now run helm search repo apica-repo to see the available helm charts

1.1.1 Update your HELM chart

If you already added Apica Ascent's HELM repository in the past, you can get updated software releases using helm repo update

1.2 Create namespace where Apica Ascent will be deployed

This will create a namespace apica-ascent where we will deploy the Apica Ascent Log Insights stack.

1.3 Prepare your Values YAML file

Sample YAML files for small, medium, large cluster configurations can be downloaded at the following links.

These YAML files can be used for deployment with -f parameter as shown below in the description.

2. Install Apica Ascent

3 Customizing the deployment

3.1 Enabling https for the UI

3.1.1 Passing an ingress secret

If you want to pass your own ingress secret, you can do so when installing the HELM chart

3.2 Using an AWS S3 bucket

Depending on your requirements, you may want to host your storage in your own K8S cluster or create a bucket in a cloud provider like AWS.

3.2.1 Create an access/secret key pair for creating and managing your bucket

Go to AWS IAM console and create an access key and secret key that can be used to create your bucket and manage access to the bucket for writing and reading your log files

3.2.2 Deploy the Apica Ascent helm in gateway mode

Make sure to pass your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY and give a bucket name. The S3 gateway acts as a caching gateway and helps reduce API costs. Create a bucket in AWS s3 with a unique bucket name in the region where you plan to host the deployment.

3.3 Install Apica Ascent server certificates and Client CA [OPTIONAL]

Apica Ascent supports TLS for all ingest. We also enable non-TLS ports by default. It is however recommended that non-TLS ports not be used unless running in a secure VPC or cluster. The certificates can be provided to the cluster using K8S secrets. Replace the template sections below with your Base64 encoded secret files.

Save the secret file e.g. logiq-certs.yaml. Proceed to install the secret in the same namespace where you want to deploy Apica Ascent

The secret can now be passed into the Apica Ascent deployment

3.4 Changing the storage class

If you are planning on using a specific storage class for your volumes, you can customize it for the Apica Ascent deployment. By default, Apica Ascent uses the standard storage class

3.5 Using external AWS RDS Postgres database instance

To use external AWS RDS Postgres database for your Apica Ascent deployment, execute the following command.