In today’s interconnected world, a fundamental question plagues every network administrator and SRE: “Is my network running well?” The answer, often elusive, is precisely what synthetics aims to provide. By deploying a vast fleet of specialized probe agents, synthetics continuously monitors critical network health metrics, including latency, packet loss, jitter, and custom reachability checks, providing an unparalleled view into your network’s performance.
While the core concept of sending pings might seem simple, the magic and the complexity lie within the control plane of a robust and scalable synthetics system. This isn’t just about sending ICMP packets; it’s about orchestrating a distributed system of thousands of probes to deliver accurate, real-time insights across a large-scale network.
Designing the Control Plane: The Core Pillars
A well-designed synthetics control plane is the backbone of its effectiveness. It’s what transforms raw ping data into actionable intelligence. Let’s explore the key mechanisms that enable this sophisticated orchestration.
This section explores the control plane mechanisms necessary for managing a network of synthetics monitoring probes. Understanding these mechanisms offers valuable perspectives on how synthetics effectively handles distributed network monitoring:
Managing Large-Scale Agent Deployments:
Imagine deploying thousands of synthetics probes across various data centers, cloud regions, and remote offices. The control plane is your central hub for this monumental task. It facilitates automated deployment, upgrades, and health monitoring of these numerous probes, ensuring they are always running and reporting as expected. This involves sophisticated deployment strategies, version control, and continuous health checks to identify and address any agent-related issues proactively.
- Synthetics probe installation
To start with, the customers install a fleet of synthetics probes in their environment. Selector symthetics probes can be installed as Linux Debian/RPM packages or as Docker containers. The probes can be installed on Linux or Windows hosts, networking switches, and routers.
Customers often use Ansible or other fleet management tools to do the install. The artifacts to install are fetched from the Selector SaaS platform. All the agents auto-connect to the SaaS instance on startup to register. Connected probes are marked as registered but do not participate in synthetics yet until further action is taken.
- Synthetics inventory management
Synthetics is driven by inventory configuration on the Selector SaaS platform. All probes that register with the SaaS platform should be added to the synthetics inventory. Inventory for probes can be added even before the probes are installed.
- Health check, performance metrics, and logs from probes
Synthetics probes maintain a constant connection to the Selector SaaS platform, transmitting continuous health data to confirm their operational status. To minimize host system resource consumption, probes are designed to be lightweight. Additionally, they provide performance metrics for monitoring resource usage.
Given their deployment within customer environments, probes proactively send critical logs to the SaaS platform. This ensures the availability of necessary data for debugging and enhanced visibility during triage.
Example of issues that can be alerted on based on the metrics:
- The host system has a clock skew and is out of sync with NTP servers
- Ping iterations match the configured values
- State of agents registered but not operational
Implementing Pivot-Based Probes Grouping:
Raw data is only as good as its organization. The concept of pivot-based grouping is a powerful mechanism within the control plane that enables efficient organization and analysis of network monitoring data. Instead of just a flat list of probes, the control plane allows for the dynamic grouping of probes based on various “pivots”- geographical location, network segment, application served, or even custom tags. This will enable you to slice and dice your monitoring data to gain insights specific to certain parts of your infrastructure, enabling targeted troubleshooting and performance analysis. For example, you could quickly view all probes monitoring your e-commerce platform or all probes within a specific metropolitan area.
Pivots are configured based on the inventory columns. Customers can select one or more specific columns to define as the pivot. This creates mesh instances of probes participating in synthetics within their instance. A probes can participate in multiple mesh instances.
In the example below, 12 probes participate in multiple meshes based on pivot tags Tag1 and Tag2. The four meshes are:
- DC1 Mesh with four probes
- DC2 Mesh with four probes
- Cloud Mesh with four probes
- Overlay Mesh with three probes
Handling Configuration Management for Probes:
Consistency is key in a distributed system. The control plane plays a vital role in ensuring consistent and up-to-date configurations are applied to all monitoring probes. This includes managing which metrics to collect, the frequency of pings, target endpoints, and other related details.
Some of the example configurations that are supported and can be configured from Selector SaaS
- Ping frequency
- Packets per iteration
- Update source port and TOS values for ping packets
- Traceroute configuration such as max hop, first packet TTL, packet size and more
The configuration is synced from Selector SaaS to all the probes.
Summary
Fundamentally, the synthetics control plane serves as the essential operational mechanism, the sophisticated manager directing a complex system of network probes. It is the component that translates a basic concept into a robust, adaptable infrastructure for addressing the critical query: “Is network performance satisfactory?” By understanding these fundamental functionalities, one develops a more comprehensive understanding of the complex technical design that underpins efficient and thorough network observation.
To stay up-to-date with the latest news and blog posts from Selector, follow us on LinkedIn or X and subscribe to our YouTube channel.