StatsD

1. Introduction

StatsD is a tool based on UDP/TCP for aggregating and transmitting performance metrics. In a narrow sense, it is a daemon that listens on UDP or TCP, collects data sent by StatsD clients according to a simple protocol, aggregates it, and periodically pushes it to the backend. You only need to specify the domain name (or IP address) and corresponding port of the server running IoTDB_Adapter in the StatsD configuration file to seamlessly write StatsD data into IoTDB . This integration method directly leverages the core advantages of IoTDB for time-series data—including high-throughput writes, efficient compressed storage, fast time-range queries, and the scalable processing capabilities of distributed clusters for massive amounts of data—thereby providing technical support for long-term storage, complex analysis, and large-scale management of monitoring metrics.

2. Prerequisites

The following preparations are required to store data collected by StatsD in IoTDB :

• IoTDB >= V2.0.5.1 has been deployed and is running properly

• StatsD has been downloaded and started successfully

• IoTDB_Adapter is running properly

3. Installation and Configuration

3.1 IoTDB_Adapter Configuration

1. Contact Timecho staff to obtain the IoTDB_Adapter installation package

2. Extract the package

   # Take version 1.0 as an example
   unzip IoTDB_Adapter-1.0.zip

3. Modify the conf/application.properties configuration file to set the listening port, IoTDB node address, username, password, and path prefix.

   # Port on which the Adapter listens
   port=2003
   # IoTDB instance addresses; separate multiple nodeUrls with ;
   iotdb.nodeUrls=127.0.0.1:6667
   # Path prefix for data written to IoTDB, such as root.testdb (effective when sqlDialect is not table)
   iotdb.prefix=root.statsD
   # IoTDB username
   iotdb.user=root
   # IoTDB password
   iotdb.password=root
   # IoTDB SqlDialect (tree/table, default: tree)
   iotdb.sqlDialect=tree
   # IoTDB DataBase (effective when sqlDialect is table)
   iotdb.database=collect
   # IoTDB Table (effective when sqlDialect is table)
   iotdb.table=test
   # payload_formatter: supports statsd and opentsdb (default)
   payload.formatter=statsd
   # Source data precision (default: seconds; s|ms|us|ns)
   time.precision.source=s
   # Target IoTDB data precision (default: seconds)
   time.precision.iotdb=ms

4. Restart the IoTDB_Adapter service.

/sbin/start.sh

3.2 StatsD Configuration

1. Download and install StatsD

   • Source code: GitHub-StatsD

2. Create a config.js configuration file based on exampleConfig.js

3. Modify the config.js configuration file

   • Set the Adapter interface and transmission protocol.

   # StatsD server port
   port: 8125,
   # Transmission protocol
   repeaterProtocol: "tcp",
   # Destination port
   graphitePort: 2003,
   # Destination IP address
   graphiteHost: "127.0.0.1",
   backends: [ "./backends/graphite" ]

4. Start the StatsD service using the following command:

node stats.js config.js

3.3 Verification

Query the data through the IoTDB CLI to verify that it has been written successfully.

IoTDB> select * from root.statsD.stats.statsd.**
+-----------------------------+----------------------------------------+---------------------------------------+-----------------------------------------+-----------------------------------------------------+------------------------------------------------------+-------------------------------------------------+---------------------------------------------------+-------------------------------------------------+-----------------------------------------+---------------------------------------------+---------------------------+----------------------------------------------+------------------------------------------------+------------------------------------------------+
|                         Time|root.statsD.stats.statsd.processing_time|root.statsD.stats.statsd.bad_lines_seen|root.statsD.stats.statsd.packets_received|root.statsD.stats.statsd.graphiteStats.last_exception|root.statsD.stats.statsd.graphiteStats.calculationtime|root.statsD.stats.statsd.graphiteStats.flush_time|root.statsD.stats.statsd.graphiteStats.flush_length|root.statsD.stats.statsd.graphiteStats.last_flush|root.statsD.stats.statsd.metrics_received|root.statsD.stats.gauges.statsd.timestamp_lag|root.statsD.statsd.numStats|root.statsD.stats_counts.statsd.bad_lines_seen|root.statsD.stats_counts.statsd.packets_received|root.statsD.stats_counts.statsd.metrics_received|
+-----------------------------+----------------------------------------+---------------------------------------+-----------------------------------------+-----------------------------------------------------+------------------------------------------------------+-------------------------------------------------+---------------------------------------------------+-------------------------------------------------+-----------------------------------------+---------------------------------------------+---------------------------+----------------------------------------------+------------------------------------------------+------------------------------------------------+
|2024-07-17T12:17:08.000+08:00|                                     0.0|                                    0.0|                                      0.0|                                                  0.0|                                                   0.0|                                              0.0|                                              672.0|                                    1.721189808E9|                                      0.0|                                          NaN|                        4.0|                                           0.0|                                             0.0|                                             0.0|
|2024-07-17T12:17:18.000+08:00|                                     0.0|                                    0.0|                                      0.0|                                                  0.0|                                                   0.0|                                              1.0|                                              672.0|                                    1.721189828E9|                                      0.0|                                          NaN|                        4.0|                                           0.0|                                             0.0|                                             0.0|
|2024-07-17T12:17:28.000+08:00|                                     0.0|                                    0.0|                                      0.0|                                                  0.0|                                                   0.0|                                              0.0|                                              672.0|                                    1.721189838E9|                                      0.0|                                          NaN|                        4.0|                                           0.0|                                             0.0|                                             0.0|
+-----------------------------+----------------------------------------+---------------------------------------+-----------------------------------------+-----------------------------------------------------+------------------------------------------------------+-------------------------------------------------+---------------------------------------------------+-------------------------------------------------+-----------------------------------------+---------------------------------------------+---------------------------+----------------------------------------------+------------------------------------------------+------------------------------------------------+