Introduction
There are two essential components that should be defined when creating a Round-Robin Database (RRD) using rrdtool: the Data Source (DS) and the Round-Robin Archive (RRA). The Data Source definition dictates how the data is entered into the database and can provide rules for the interpretation, limits, and expected frequency of updates. The Round-Robin Archive definition dictates the storage of the data including how the data is consolidated, the time resolution, and the span of time the archive covers. A single RRD instance must contain at least one Data Source and Round-Robin Archive, but may contain more.
Data Source Definition
The general format for a DS specification is:
DS:Label:Type:Heartbeat:Min:Max
Label is the name of the Data Source. It may be from 1-19 characters long and consists of characters in the set [a-zA-Z0-9_]. The label should be descriptive of the data being collected. Example labels include “temperature”, “BytesIn”, “bytes_out”, etc.
Type is one of rrdtool’s defined data types: GAUGE, COUNTER, DERIVE, ABSOLUTE, or COMPUTE.
GAUGE is for storing simple measurements that are rate independent. Data types such as temperatures, prices, or system load are suitable for this data type.
COUNTER is for storing continuously incrementing counters. Data values input into this type of Data Source should never decrease except due to an overflow condition (which is handled by rrdtool). Data of this type is stored as a per-second rate. Example measurements that are suitable for this data type are velocity and throughput.
DERIVE stores the derivative of the line between the current and previous data points. This is useful for converting an input of simple measurements into a rate. For example, with an input of temperature readings, it will record the rate of temperature change instead of simply the raw temperature readings.
ABSOLUTE is used for counters which are reset upon reading. This behavior is often found in fast-moving counters that would otherwise overflow frequently. It is otherwise similar to the COUNTER type.
COMPUTE provides a means of applying a formula to other Data Sources in the RRD. As such, the DS specification for a COMPUTE type is differs than the general form:
DS:Label:COMPUTE:rpn-expression
The COMPUTE type can be thought of as a “virtual” Data Source that calculates its value based off of the values of other Data Sources.
Heartbeat defines the maximum number of seconds between two data updates before the value of the data is assumed to be unknown. This determines how rrdtool will interpolate data between readings.
Min and Max establish the expected range values for the data. Setting appropriate values provides additional safeguards against the accidental inclusion of invalid data. If the expected range values are unknown a value of “U” can be set instead.
Round Robin Archive Definition
The general format for a RRA specification is:
RRA:Consolidation Function:XFF:Steps:Rows
Consolidation Function (CF) is one of rrdtool’s defined functions: AVERAGE, MIN, MAX, and LAST. The different functions provide flexibility when aggregating Primary Data Points (PDP) into a RRA. Note that regardless of the CF chosen, aggregation will result in the loss of resolution for the data. For many applications, this is perfectly acceptable as maintaining fine-grained resolution on older data is often not necessary.
AVERAGE stores the average of all the Primary Data Points within the designated time range.
MIN stores the smallest value of all the Primary Data Pointswithinthe designated time range.
MAX stores the largest value of all the Primary Data Pointswithinthe designated time range.
LAST stores only the last (most recent) value of all the Primary Data Points within the designated time range.
XFF is the “X Files Factor”. This defines how many Primary Data Points may be unknown before the Consolidated Data Point is also defined as unknown. It is expressed as the ratio of the number of unknown PDP’s over the total number PDP’s for the aggregation period. As such, valid values are between 0 and 1. As an example, a XFF of 0.5 dictates that a CDP will be stored as unknown if the number of unknown PDP’s within the time period exceeds 50% of the total number of PDP’s.
Steps defines how many Primary Data Points will be aggregated into a Consolidated Data Point. The Consolidated Data Point is then recorded in the RRA. Each RRA Step is a multiple of the Step value associated with the Primary Data Point. For example, if the Primary Data Point Step has a resolution of 300 seconds and the RRA Step value is 12, then each Consolidated Data Point will be aggregated over 3600 seconds (12 x 300 seconds).
Rows defines how many Consolidated Data Point Steps are stored in the RRA. As each Row represents the time span defined by the RRA Step, it is a simple calculation to determine the maximum time represented in the archive. For example, if the Primary Data Point has a Step value of 300 seconds, and the RRA Step value is 12, and the Row value is 8760, then the RRA will be able to store 1 year’s worth of data. (300s x 12 x 8760 == 1 year)
rrdtool create
Before attempting to create a RRD, it is best to have fully defined the relevant Data Sources and Round Robin Archives the RRD will be storing. With those defined, there are only a couple other command line options to define in order to successfully create the RRD.
The general format for creating a RRD is as follows:
rrdtool create filename --start time --step secs \
DS:Label:Type:Heartbeat:Min:Max \
RRA:Consolidation Function:XFF:Steps:Rows
filename is any legitmate filename permitted by the filesystem. It is suggested that a suffix of “.rrd” be used to identify the file as a Round Robin Database.
start is the earliest time represented in the RRD. The RRD will not accept any times that predate (or equal) this value. The time is frequently presented as a standard Unix epoch value (number of seconds since Jan 1, 1970 UTC) although it it will also parse other formats including “now”, “now – 1 hour”, and more. (See the manual page for the at command for more details on the time format rrdtool supports.) The default value is the current time minus 10 seconds.
step is the number of seconds each Primary Data Point represents. The default value (300 seconds) provides a resolution of 1 PDP every 5 minutes.
Examples
Example 1
rrdtool create sysinfo.rrd --start now --step 300 \
DS:load:GAUGE:600:0:U \
RRA:AVERAGE:0.5:1:8928
This example creates a RRD with the filename of sysinfo.rrd. The starting time is “now” which rrdtool will determine at the time the command is executed. The step value for the Primary Data Points is 300 seconds.
The only Data Source defined in this RRD is labelled “load” and is of type GAUGE. It has a heartbeat value of 600 seconds, a minimum value of 0 and an unlimited (“U”) maximum value.
There is only one Round Robin Archive defined, and it uses the AVERAGE method to aggregate Primary Data Points into Consolidated Data Points. It has an XFF of 0.5, which means that no more than 50% of the PDP’s can be of the “unknown” type or the CDP will also be of “unknown” type. The RRA step value is 1, which in this case means there is a 1:1 correspondence between a PDP and a CDP. (i.e. no aggregation is actually performed) There are 8928 data rows in the RRA, so this archive represents a span of 31 days (8928 x 300 seconds).
Example 2
rrdtool create sysinfo.rrd --start now --step 300 \
DS:load:GAUGE:600:0:U \
RRA:AVERAGE:0.5:1:8928 \
RRA:AVERAGE:0.5:12:8760
This example is identical to Example 1 with the exception that it contains more than one RRA.
In this case, the first RRA defines a “high-resolution” view of the data but only for the last 31 days. In many applications, the value of the data is highest for the most recent data, and so maintaining it with a high fidelity is warranted.
The second RRA defines a “low-resolution” view of the data for a substantially longer period of time. In this example, the RRA step value is 12, so the AVERAGE Consolidation Function will be applied to 12 Primary Data Points to generate a single Consolidated Data Point. As each PDP represents 300 seconds, each step in this RRA will represent 3600 seconds, or 1 hour (300s x 12). With 8760 rows for this RRA, this archive represents a total time span of 1 year (365 days x 24 hours = 8760).
When generating graphs, rrdtool is smart enough to use the appropriate RRA(s) to provide the appropriate resolution for the time span requested.
Example 3
rrdtool create sysinfo.rrd --start now --step 300 \ DS:load:GAUGE:600:0:U \
DS:diskfree:DERIVE:600:U:U \
RRA:AVERAGE:0.5:1:8928 \
RRA:AVERAGE:0.5:12:8760
This example further extends the setup illustrated in Example 2. In this case, an additional Data Source has been defined to track the rate of change in the free disk space. As the amount of free disk space may increase or shrink, the DERIVE data type is required. Like the GAUGE Data Source, it has a heartbeat value of 600 seconds but both the minimum and maximum values are unknown for this DS. Note that the RRA’s apply to both of the Data Sources.