At Caringo, we pride ourselves on making something complex—i.e., storing and accessing TB–PBs of data and billions of objects on heterogeneous hardware—
easy to manage.
The key to this is visibility into system status. We have had the option to use SNMP and Nagios for many years; however, we kept getting requests for an intuitive way to monitor Swarm object storage using more current monitoring and visualization platforms.
Integrate Elasticsearch into Swarm Object Storage
We took the first step in this direction in 2016 when we first integrated Elasticsearch into Swarm. Elasticsearch is an open-source RESTful search engine built upon Lucene. This provided us with a scalable way to index, view and search system and custom metadata.
Launch Prometheus Node Exporter for Swarm
The next step to make visual system status possible happened in April of 2019 when we launched the Prometheus node exporter for Swarm 10. Prometheus is a popular open-source monitoring solution. With the node exporter, we can export Swarm-specific metrics directly to Prometheus.
Using Grafana Open-Source Metrics Visualization Platform
The third and final step was using Grafana—an open-source metrics visualization platform. We were able to leverage some of the existing Grafana templates to quickly visualize Swarm metrics over a customizable period of time. Our technical staff has been using this internally for a few months to optimize Swarm cluster configuration with excellent results.
Demo: Monitoring Swarm Object Storage with Prometheus & Grafana
In our June 11 TechTuesday webinar (at 7am PT/10am ET), Monitoring Swarm Object Storage Using Prometheus Exporter & Grafana, John Bell, Senior Consultant, will host Abraham Felsenstein, System Integrator. Abraham will demonstrate how to import data from the Swarm Prometheus node exporter and view it via Grafana.
While object storage isn’t a panacea, it is an increasingly important storage technology that enables on-demand access for video workflows. More Details »
With the release of Swarm v10, a Prometheus exporter has been integrated. You may now monitor your object-based data storage with Grafana v6+ More Details »