To begin, setup a chart that graphs the desired data in a typical manner before adding in the running average.  The following graph command creates a somewhat typical visualization of the last day’s network traffic as illustrated by the following graph:

rrdtool graph example.png \
  --title "Network Traffic" \
  --width=500 \
  --slope-mode \
  --end=now \
  --start=end-24h \
  --base=1000 \
  DEF:bytesIn=network.rrd:bytesIn:AVERAGE \
  DEF:bytesOut=network.rrd:bytesOut:AVERAGE \
  CDEF:bpsIn=bytesIn,8,* \
  CDEF:bpsOut=bytesOut,8,* \
  CDEF:bpsOutNeg=bpsOut,-1,* \
  VDEF:bpsInTot=bpsIn,TOTAL \
  VDEF:bpsOutTot=bpsOut,TOTAL \
  COMMENT:"              current     min        max      total\n" \
  AREA:bpsIn#FF880044:"Bits/s In " \
  GPRINT:bpsIn:LAST:"%6.1lf%s\t" \
  GPRINT:bpsIn:MIN:"%6.1lf%s\t" \
  GPRINT:bpsIn:MAX:"%6.1lf%s\t" \
  GPRINT:bpsInTot:"%6.1lf%s\n" \
  LINE1:bpsIn#FF8800CC \
  AREA:bpsOutNeg#44C80044:"Bits/s Out" \
  GPRINT:bpsOut:LAST:"%6.1lf%s\t" \
  GPRINT:bpsOut:MIN:"%6.1lf%s\t" \
  GPRINT:bpsOut:MAX:"%6.1lf%s\t" \
  GPRINT:bpsOutTot:"%6.1lf%s\n" \
  LINE1:bpsOutNeg#44C800CC \
  HRULE:0#000000

With the basic graph setup, adding the elements necessary for the running average can now begin.  In this example, it appears there is a roughly hourly cycle (Time Machine backups) that defines the period so a reasonable start is to work with a minimum average calculated over 60 minutes.

First, additional CDEF statements need to be declared applying the TREND operator to the desired data.  The format for this operation is straightforward:

CDEF:label=data_source,time_span,TREND

In this case, two new data sources are defined and are given the labels bspInTrend and bpsOutTrendNeg.  The data sources for these are the previously declared bpsIn and bpsOutNeg and the time span for them is 1 hour (3600 seconds).

  CDEF:bpsInTrend=bpsIn,3600,TREND \
  CDEF:bpsOutTrendNeg=bpsOutNeg,3600,TREND \

With the running averages now defined, it is possible to graph the data.  For the purposes of this example, the running average is graphed as a “shadow” on top of the base data.

  LINE2:bpsInTrend#000000CC \
  LINE2:bpsOutTrendNeg#000000CC \

The graph shown below illustrates the new running averages.

The running averages are now shown and it illustrates that despite the hourly peaks, the overall trend is fairly flat with the slight exception around 00:15.  However, the running averages also show a gap at the beginning of the graph. This gap is because it takes an hour of data before the hourly running average can be computed.  While this may be acceptable, it doesn’t look polished.

A minor tweak of the original DEF statements can provide “padding” necessary to eliminate the displayed gap in the running average lines.  This can be accomplished by extending the start time of the data sources by at least the same duration specified in the running averages.  Note that the display area is not affected by altering the start time of the DEF statements — that is controlled by the --start and --end options.  For the example data, the DEF statements need to extend the range of the data set by 1 hour so as to encompass 25 hours instead of the display area’s 24 hours:

  DEF:bytesIn=network.rrd:bytesIn:AVERAGE:start=end-25h \
  DEF:bytesOut=network.rrd:bytesOut:AVERAGE:start=end-25h \

Putting it together, the following command will pull in all the data necessary to calculate a running average for the extent of the display area, calculate the running averages, and display the results:

rrdtool graph example.png \
  --title "Network Traffic" \
  --width=500 \
  --slope-mode \
  --end=now \
  --start=end-24h \
  --base=1000 \
  DEF:bytesIn=network.rrd:bytesIn:AVERAGE:start=end-25h \
  DEF:bytesOut=network.rrd:bytesOut:AVERAGE:start=end-25h \
  CDEF:bpsIn=bytesIn,8,* \
  CDEF:bpsOut=bytesOut,8,* \
  CDEF:bpsOutNeg=bpsOut,-1,* \
  CDEF:bpsInTrend=bpsIn,3600,TREND \
  CDEF:bpsOutTrendNeg=bpsOutNeg,3600,TREND \
  VDEF:bpsInTot=bpsIn,TOTAL \
  VDEF:bpsOutTot=bpsOut,TOTAL \
  COMMENT:"              current     min        max      total\n" \
  AREA:bpsIn#FF880044:"Bits/s In " \
  GPRINT:bpsIn:LAST:"%6.1lf%s\t" \
  GPRINT:bpsIn:MIN:"%6.1lf%s\t" \
  GPRINT:bpsIn:MAX:"%6.1lf%s\t" \
  GPRINT:bpsInTot:"%6.1lf%s\n" \
  LINE1:bpsIn#FF8800CC \
  AREA:bpsOutNeg#44C80044:"Bits/s Out" \
  GPRINT:bpsOut:LAST:"%6.1lf%s\t" \
  GPRINT:bpsOut:MIN:"%6.1lf%s\t" \
  GPRINT:bpsOut:MAX:"%6.1lf%s\t" \
  GPRINT:bpsOutTot:"%6.1lf%s\n" \
  LINE1:bpsOutNeg#44C800CC \
  LINE2:bpsInTrend#000000CC \
  LINE2:bpsOutTrendNeg#000000CC \
  HRULE:0#000000

To begin, it helps to setup a chart that graphs the desired data in a typical manner before adding in the time shifted overlays.  The following graph command creates a somewhat typical visualization of the last hour’s network traffic as illustrated by Example 1 graph:

rrdtool graph example.png \
  --title "Network Traffic" \
  --width=500 \
  --slope-mode \
  --end=now \
  --start=end-1h \
  --base=1000 \
  DEF:bytesIn=network.rrd:bytesIn:AVERAGE \
  DEF:bytesOut=network.rrd:bytesOut:AVERAGE \
  CDEF:bpsIn=bytesIn,8,* \
  CDEF:bpsOut=bytesOut,8,* \
  CDEF:bpsOutNeg=bpsOut,-1,* \
  VDEF:bpsInTot=bpsIn,TOTAL \
  VDEF:bpsOutTot=bpsOut,TOTAL \
  COMMENT:"            current     min        max      total\n" \
  AREA:bpsIn#FF880044:"Bits/s In " \
  GPRINT:bpsIn:LAST:"%6.1lf%s\t" \
  GPRINT:bpsIn:MIN:"%6.1lf%s\t" \
  GPRINT:bpsIn:MAX:"%6.1lf%s\t" \
  GPRINT:bpsInTot:"%6.1lf%s\n" \
  LINE1:bpsIn#FF8800CC \
  AREA:bpsOutNeg#44C80044:"Bits/s Out" \
  GPRINT:bpsOut:LAST:"%6.1lf%s\t" \
  GPRINT:bpsOut:MIN:"%6.1lf%s\t" \
  GPRINT:bpsOut:MAX:"%6.1lf%s\t" \
  GPRINT:bpsOutTot:"%6.1lf%s\n" \
  LINE1:bpsOutNeg#44C800CC \
  HRULE:0#000000

With the basic chart setup as desired, now the command can be modified to support a time shifted overlay.

First, additional DEF statements need to be defined for time range for the source data to encompass the time span desired for comparison.  Note that this is not the same as the graph’s display range; those options (--start and --end) should remain unchanged.  The override start time should extend the time frame to the start of the time shifted data.  For example, the following would be suitable for comparison of the previous 1 hour:

  DEF:bytesInPrevHour=network.rrd:bytesIn:AVERAGE:start=end-2h \
  DEF:bytesOutPrevHour=network.rrd:bytesOut:AVERAGE:start=end-2h \

As another example, the following would be suitable for a comparison of the same hour, 1 day (24 hours) ago:

  DEF:bytesInPrevDay=network.rrd:bytesIn:AVERAGE:start=end-25h \
  DEF:bytesOutPrevDay=network.rrd:bytesOut:AVERAGE:start=end-25h \

Next, a new SHIFT statement needs to be introduced that shifts the data being displayed by the specified number of seconds.  The SHIFT statements must be specified after the DEF statements they are shifting.  The following statements would shift the newly inserted DEF statements by 1 hour (3600 seconds):

  SHIFT:bytesInPrevHour:3600 \
  SHIFT:bytesOutPrevHour:3600 \

Next, the CDEF and VDEF operations need to be supplied in order to transform the new data in a consistent manner with the original data.

  CDEF:bpsInPrev=bytesInPrevHour,8,* \
  CDEF:bpsOutPrev=bytesOutInPrevHour,8,* \
  CDEF:bpsOutPrevNeg=bpsOutPrev,-1,* \

Finally, the new elements are ready for graphing.  Care should be made to ensure the shifted data is displayed in a distinct manner from the current data so as to prevent visual confusion or obscuring the current data.

  LINE1:bpsInPrev#00000088 \ 
  LINE1:bpsOutPrevNeg#00000088 \

Pulling it all together, the following command will pull in the data from the extended time frame, shift it forward by the appropriate amount of time so that it aligns with the displayed time frame, perform the desired transformation options, and finally display the results:

rrdtool graph example.png \
  --title "Network Traffic" \
  --width=500 \
  --slope-mode \
  --end=now \
  --start=end-1h \
  --base=1000 \
  DEF:bytesIn=network.rrd:bytesIn:AVERAGE \
  DEF:bytesOut=network.rrd:bytesOut:AVERAGE \
  DEF:bytesInPrevHour=network.rrd:bytesIn:AVERAGE:start=end-2h \
  DEF:bytesOutPrevHour=network.rrd:bytesOut:AVERAGE:start=end-2h \
  SHIFT:bytesInPrevHour:3600 \
  SHIFT:bytesOutPrevHour:3600 \
  CDEF:bpsIn=bytesIn,8,* \
  CDEF:bpsOut=bytesOut,8,* \
  CDEF:bpsOutNeg=bpsOut,-1,* \
  CDEF:bpsInPrev=bytesInPrevHour,8,* \
  CDEF:bpsOutPrev=bytesOutInPrevHour,8,* \
  CDEF:bpsOutPrevNeg=bpsOutPrev,-1,* \
  VDEF:bpsInTot=bpsIn,TOTAL \
  VDEF:bpsOutTot=bpsOut,TOTAL \
  COMMENT:"            current     min        max      total\n" \
  AREA:bpsIn#FF880044:"Bits/s In " \
  GPRINT:bpsIn:LAST:"%6.1lf%s\t" \
  GPRINT:bpsIn:MIN:"%6.1lf%s\t" \
  GPRINT:bpsIn:MAX:"%6.1lf%s\t" \
  GPRINT:bpsInTot:"%6.1lf%s\n" \
  LINE1:bpsIn#FF8800CC \
  AREA:bpsOutNeg#44C80044:"Bits/s Out" \
  GPRINT:bpsOut:LAST:"%6.1lf%s\t" \
  GPRINT:bpsOut:MIN:"%6.1lf%s\t" \
  GPRINT:bpsOut:MAX:"%6.1lf%s\t" \
  GPRINT:bpsOutTot:"%6.1lf%s\n" \
  LINE1:bpsOutNeg#44C800CC \
  LINE1:bpsInPrev#00000088 \ 
  LINE1:bpsOutPrevNeg#00000088 \
  HRULE:0#000000

First, the /etc/named.conf file needs to be modified to specify both the statistics file and the appropriate control mechanisms.  As this is an essential configuration file, create a backup copy of the existing (working) file before editing in case something goes wrong.

In the named.conf file, ensure that there is an configuration block for the “controls”.  By default, it should already be present and look similar to the following:

controls  {
        inet 127.0.0.1 port 54 allow    {any;   }
        keys    { "rndc-key";    };
   };

This section defines the access control for the stats tool (among other things).  The essential elements for this purpose is that it defines the communications port as port 54 and it uses the key identified as “rndc-key” as its security token.  The key is actually defined in a separate file and read into the configuration via the include "/etc/rndc.key" statement earlier in the named.conf file.

There should also be a configuration block for the “options”.  A typical installation will look like the following:

options  {
        include "/etc/dns/options.conf.apple";
                /*
         * If there is a firewall between you and nameservers you want
         * to talk to, you might need to uncomment the query-source
         * directive below.  Previous versions of BIND always asked
         * questions using port 53, but BIND 8.1 uses an unprivileged
         * port by default.
         */
        // query-source address * port 53;
   };

This section requires some minor modification to specify the stats file.  In the block, insert a line specifying statistics-file "/full/path/to/the/named.stats";.  It is best to make the addition after the include statement so that the edited result looks similar to the following:

options  {
        include "/etc/dns/options.conf.apple";

        // Addition to enable statistics
        statistics-file "/var/log/named.stats";

                /*
         * If there is a firewall between you and nameservers you want
         * to talk to, you might need to uncomment the query-source
         * directive below.  Previous versions of BIND always asked
         * questions using port 53, but BIND 8.1 uses an unprivileged
         * port by default.
         */
        // query-source address * port 53;
   };

After the named.conf file has been configured properly, a new file must created for the configuration of the rndc utility.  Create the /etc/rndc.conf file and edit it with the following settings:

include "/etc/rndc.key";

options {
    default-key "rndc-key";
    default-server localhost;
    default-port 54;
};

The include statement must match the same included file specified in the named.conf file as the security key (“rndc-key”) must match or the rndc utility used to trigger the stats dump will not be authorized to connect.  The options block specifies the key name and the default server and port.  The assumption here is that the rndc tool will be run on the same host (localhost) as the DNS server.  The port should match the port specified in the controls block of the named.conf file.

At this point, the named daemon can be safely signaled to pick up the configuration changes.  This can be done easily via the Server Admin tool by restarting the DNS service or, for the more advanced sysadmin, a SIGHUP can be sent to the process.

To trigger a stats dump, simply call the rndc stats and look for the stats file specified in the named.conf file.  If the file isn’t found, there may be a authorization issue:  try running the command as an administrator or call the command via sudo (e.g. sudo rndc stats).

The stats can be triggered on an ongoing basis by specifying an appropriate entry for the launchd system.  A simple plist formatted file located in /Library/LaunchDaemons can serve to repeatedly call the stats file for ongoing monitoring.  (See launchd tips and tricks for more information.)

The following example would be named com.hostname.namedstats.plist and specifies that the stats should be generated every 5 minutes (300 seconds).

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.hostname.namedstats</string>
    <key>StartInterval</key>
    <integer>300</integer>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/sbin/rndc</string>
        <string>stats</string>
    </array>
</dict>
</plist>

Note that every time the stats are generated they are appended to the existing file which can quickly result in a very large, unwieldy file.  If recurring stats are desired, then ensuring the stats file is periodically rotated is essential.

In Linux, the typical log rotation utility is, appropriately enough, logrotate.  However, Mac OS X uses a different tool, newsyslog, to manage the log rotation responsibilities.  This utility is run automatically by the OS at frequently intervals and evaluates the rules defined in its configuration file — /etc/newsyslog.conf.  For the log files defined in the config, it can evaluate the need to rotate based on a number of criteria, such as file size, at a specific time, at a specific interval, etc.  It can also specify additional actions to take on the rotated log, including compression of the old log, maintaining a fixed number of previous log files, setting appropriate permissions, etc.  Full documentation is available in the man page for newsyslog.conf.

Each entry in the newsyslog.conf file is specified as a single line with space-delimited fields for the various options.  The general format is as follows (with [] surrounding the optional fields):

logfilename [owner:group] mode count size when flags [/pid_file] [sig_num]

An example entry for rotating the named.stats log file can be specified as follows:

/var/log/named.stats                    640  5     1000 *     J

In the above example, the full path is specified for the target log file — /var/log/named.stats.  Permissions (640) are specified in the typical Unix manner, in this case enabling read/write for the owner and read-only for the group.  This log file will maintain a maximum of 5 files, removing the oldest file as newer archived logs are created.  This log file is also set to rotate at a fixed size — 1000 kilobytes and can rotate at any time (“*”) this size threshold is exceeded.  The “J” flag specified calls for the the older log files to be compressed (further reducing used disk space) using the bzip2 utility.

A special note for binary (not text-based) logs:  by default, newsyslog will inject a line at the beginning of a file indicating when and why the log rotated.  For most text-based logs, this can be easily skipped over but for binary logs this might result in an unreadable file.  In this case, be sure to specify the “B” flag which suppresses the informational line from being injected.

This section covers some of the more advanced uses of color in a RRDTool graph.  These techniques can not only help in providing an additional bit of polish to an otherwise ordinary graph, but can also be useful in clarifying interpretation and providing additional information.

Despite the availability of the techniques outlined below, it is still essential that proper color conventions and patterns be observed.  For example, red typically denotes “hot” in the context of temperature or “severe” in a notification/aberration detection.  Using red to denote a cool temperature on a temperature graph or a “normal” operating condition is very likely to lead to viewer confusion.  For more information, please be sure to consult a good reference on color theory.

Examples

Example 1

This is a simple example of a “stock” graph.  It uses no special color treatments but it is still able to clearly convey the necessary information.  In this case, it simply presents a graph of the system temperature.

rrdtool graph "Example 1 Colors.png" \
--start "end-48 hours" --end "12am Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 1" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
AREA:temp#FF0000

Example 2

In this example, the use of an “alpha channel” in the color specification is introduced.  The easiest way to think of an alpha channel is as a transparency measure.  A alpha channel of “FF” would be completely opaque, while an alpha channel of “00″ would be completely transparent.  The alpha channel is specified in hexadecimal form at the end of the RGB color value.  In the example below, the color value of “FF000044″ has specified an alpha channel value of “44″.  If no alpha channel is specified, then a default value of “FF” (fully opaque) is used.

rrdtool graph "Example 2 Colors.png" \
--start "end-48 hours" --end "12am Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 2" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
AREA:temp#FF000044

Example 3

This example illustrates the use of the “layer cake” effect.  This technique can help to provide additional context to a graph, as the colored layers can help clearly delineate when a system is operating within tolerances or not.  Thus, the health of the system can be determined at a glance and is not dependent on the viewer being intimate with the operational thresholds.  This example breaks down the temperature readings into four layers (cold, cool, warm, and hot) of 50 degrees each, but it would be a trivial extension to increase/decrease the number of layers.

The CDEF for the middle layers can be somewhat intimidating for those who are not experts in Reverse Polish Notation.  In this example, each layer relies on being stacked and so the appropriate calculation is determining the portion of the temperature (if any) that makes up the layer.  The following breakdown may help make it more palatable:

cool=temp,50,GT,temp,100,GT,50,temp,50,-,IF,UNKN,IF
if (temp > 50) then
  if (temp > 100) then
    cool = 50
  else
    cool = temp - 50
else
  cool = UNKN

As each layer is a maximum of 50 degrees, the trick is to determine how much (if any) of a layer falls within the designation.  If the actual temperature exceeds that of the layer, then simply use the maximum value (50).  If the temperature falls within the layer, then the value should be the temperature less the total of any previous bands.  If the temperature is less than the minimal temperature for this layer, then simply return the “unknown” value to prevent any graphing.

rrdtool graph "Example 3 Colors.png" \
--start "end-48 hours" --end "12am Dec 5, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 3" \
--vertical-label "Temperature" \
--lower-limit 0 --rigid \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
CDEF:cold=temp,50,LE,temp,50,IF \
CDEF:cool=temp,50,GT,temp,100,GT,50,temp,50,-,IF,UNKN,IF \
CDEF:warm=temp,100,GT,temp,150,GT,50,temp,100,-,IF,UNKN,IF \
CDEF:hot=temp,150,GT,temp,150,-,UNKN,IF \
AREA:cold#0000FFAA:cold:STACK \
AREA:cool#0000FF44:cool:STACK \
AREA:warm#FF000044:warm:STACK \
AREA:hot#FF0000AA:hot:STACK

Example 4

There are several techniques for “feathering” the colors in a graph as shown in this example.  The technique illustrated in this example is suitable for a representing a single color palette with the gradient lightest at the top and darkest at the bottom.  It is achieved by simply overlaying the graph with the same color selection at selected proportions and relying on the alpha channel to “build up” as the layers overlap.  Care should be made when using this technique not too make the top layers so translucent they become difficult to discern.

This example simply maps the set of data values into sets for 1/4, 1/2 and 3/4 values and then overlays the original value graph.  Additional looks can also be achieved through the use of alternative data transformation maps/ratios.

rrdtool graph "Example 4 Colors.png" \
--start "end-48 hours" --end "12am Dec 5, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 4" \
--vertical-label "Temperature" \
--lower-limit 0 --rigid \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
CDEF:tier1=temp,4,/ \
CDEF:tier2=temp,2,/ \
CDEF:tier3=temp,4,/,3,* \
AREA:temp#FF000022: \
AREA:tier3#FF000022: \
AREA:tier2#FF000022: \
AREA:tier1#FF000022:

Example 5

This example illustrates another method of “feathering” the colors in the graph.  In this case, the color gradient is lightest at the bottom and darkest at the top.  In order to achieve this, the value is simply divided up and then each layer is stacked on top of the other while steadily increasing the alpha channel value.

rrdtool graph "Example 5 Colors.png" \
--start "end-48 hours" --end "12am Dec 5, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 5" \
--vertical-label "Temperature" \
--lower-limit 0 --rigid \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
CDEF:tier=temp,4,/ \
AREA:tier#FF000022::STACK \
AREA:tier#FF000044::STACK \
AREA:tier#FF000066::STACK \
AREA:tier#FF000088::STACK

Example 6

This example illustrates the use of highlights to clearly delineate the borders between stacked area graphs.  It allows the use of a softer color palette without having to resort to a clashing color scheme to define the borders.

The highlight lines should be specified after all the area graphs have been declared.  Each highlight should be specified in the same order as its corresponding area graph in order to ensure the proper color is “on top” should the data sets have any overlap.  It is typically easiest to maintain the same color scheme by using the same RGB value as the area graph but specifying high alpha channel value.

rrdtool graph "Example 6 Colors.png" \
--start "end-48 hours" --end "12am Jan 15, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 6" \
--vertical-label "Bytes" \
--lower-limit 0 --rigid \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
DEF:disk2=sysinfo.rrd:disk2_used:AVERAGE \
AREA:disk1#0000FF22:: \
AREA:disk2#00F00022::STACK \
LINE1:disk1#0000FFAA:"Disk 1" \
LINE1:disk2#00F000AA:"Disk 2":STACK

The VDEF directive provides a mechanism for applying mathematical operations on sets of data.  Unlike the CDEF directive, the result of a VDEF is a single value.  The VDEF can reference either a DEF or CDEF data set and can perform a number of operations including calculating averages, standard deviations, least squares lines, and more.  These values can then be further referenced in CDEFs for graphing or printed out.

There is often much confusion regarding the differences of the DEF, CDEF, and VDEF directives.  It may help to think of the different directives in the following manner:

• A DEF directive references a set of “raw” data as it is stored in a RRA
• A CDEF directive applies a function to each data point it references
• A VDEF directive applies a function to an aggregate of data points

The VDEF Directive

The basic format for a VDEF directive is as follows:

VDEF:Label=RPN Expression

Label is the name of the VDEF.  It may be referenced in other directives including HRULEs or CDEF calculations.  It may be from 1-19 characters long and consists of characters in the set [a-zA-Z0-9_].  Note that the label must be unique and cannot overlap with any labels assigned to other DEFs, CDEFs, or VDEFs.

RPN Expression is the mathematical or logical expression that is applied to a data set.  The expression uses Reverse Polish Notation to eliminate confusion or errors that may occur with the precedence rules required of traditional infix notation.  There are a number of mathematical, boolean and logical operators available for inclusion in a VDEF directive.

VDEF Examples

Example 1

This is a simple example which illustrates the use of VDEFs to reveal information about the set of data.  In this case, the MINIMUM, MAXIMUM, and AVERAGE operations are used to determine the respective values for the data set.  The resulting value for each operation is then used as the value for a HRULE providing a clear illustration in the resulting graph.  Note that the VDEF values are based only on the data referenced.  In this case, the data is bounded by the start and end time specified and so would likely differ for a different window into the original data set.

rrdtool graph "Example 1 VDEF.png" \
--start "end-48 hours" --end "12am Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 1" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
VDEF:min=temp,MINIMUM \
VDEF:max=temp,MAXIMUM \
VDEF:avg=temp,AVERAGE \
AREA:temp#FFEF00 \
HRULE:max#FF0000:"Max" \
HRULE:avg#000000:"Average" \
HRULE:min#0000FF:"Min"

Example 2

This example uses the average and standard deviation VDEF operations to identify temperature ranges that exceed the average temperature by more than one standard deviation.  The VDEF operations determine the appropriate values, which are then used in the CDEF operations to generate new data sets.  The original data set is graphed in the “hot” color with the “normal” and “cool” graphs overlayed on top of the appropriate sections.

The CDEF calculations may be difficult to parse for a novice to Reverse Polish syntax.  It may help to break it down as follows:

1. cool=temp,avg,stdev,-,LE,temp,UNKN,IF
2. cool=temp,(avg – stdev),LE,temp,UNKN,IF
3. cool=(temp <= (avg – stdev)),temp,UNKN,IF
4. cool=if (temp <= (avg – stdev)) then temp else UNKN

rrdtool graph "Example 2 VDEF.png" \
--start "end-48 hours" --end "12am Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 2" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
VDEF:avg=temp,AVERAGE \
VDEF:stdev=temp,STDEV \
CDEF:cool=temp,avg,stdev,-,LE,temp,UNKN,IF \
CDEF:medium=temp,avg,stdev,+,LE,temp,UNKN,IF \
AREA:temp#FF0000:"Hot" \
AREA:medium#FFEF00:"Normal" \
AREA:cool#0000FF:"Cool"

Example 3

This example illustrates how to use the least squares line VDEF operations to draw a trendline.  The LSLSLOPE operation can determine the slope of line and the LSLINT can provide the y-axis intercept value.  A trendline can then be generated using the classic "y=mx+b" formula (where m is the slope and b is the intercept).

The CDEF operation implements this formula using several “tricks” of rrdtool: a workaround for the CDEF requirement to reference a DEF or CDEF, and the use of the COUNT operation to increment a value for each data point in the graph set.  In the example, the CDEF reference requirement is satisfied by the “temp,POP” elements, which effectively puts a value from the temp DEF on the stack and then pops it back off, discarding it.

rrdtool graph "Example 3 VDEF.png" \
--start "end-48 hours" --end "12am Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 3" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
VDEF:slope=temp,LSLSLOPE \
VDEF:intercept=temp,LSLINT \
CDEF:trendline=temp,POP,COUNT,slope,*,intercept,+ \
AREA:temp#FFEF00 \
LINE2:trendline#000000

Example 4

This examples uses the PERCENTNAN operation in order to identify the 5% coolest and 5% hottest temperatures in the data set.  These values are then applied as part of the  CDEF calculations to generate the appropriate data sets for graphing.  Note that it is frequently best to use the PERCENTNAN operation instead of the PERCENT operation as the PERCENTNAN variant handles any gaps in the data in a more graceful manner.

rrdtool graph "Example 4 VDEF.png" \
--start "end-48 hours" --end "12am Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 4" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
VDEF:cool5=temp,5,PERCENTNAN \
VDEF:hot95=temp,95,PERCENTNAN \
CDEF:cool=temp,cool5,LE,temp,UNKN,IF \
CDEF:medium=temp,hot95,LE,temp,UNKN,IF \
AREA:temp#FF0000:"Hot" \
AREA:medium#FFEF00:"Normal" \
AREA:cool#0000FF:"Cool"

The CDEF directive provides a means for manipulating the “raw” data stored in a round-robin archive (RRA).  It is typically used to apply a mathematical function to each data point referenced by one or more DEF statements which results in an array of new values — each of remains associated with the respective time of the original “raw” data point.  This is an in-memory transformation only; the original data remains unchanged in the RRA.

There is often much confusion regarding the differences of the DEF, CDEF, and VDEF directives.  It may help to think of the different directives in the following manner:

• A DEF directive references a set of “raw” data as it is stored in a RRA
• A CDEF directive applies a function to each data point it references
• A VDEF directive applies a function to an aggregate of data points

The CDEF Directive

The basic format for a CDEF directive is as follows:

CDEF:Label=RPN Expression

Label is the name of the CDEF.  It may be referenced in other directives for inclusion in LINE, AREA charts or even other CDEF calculations.  It may be from 1-19 characters long and consists of characters in the set [a-zA-Z0-9_].  Note that the label must be unique and cannot overlap with any labels assigned to other DEFs, CDEFs, or VDEFs.

RPN Expression is the mathematical or logical expression that may be used to manipulate the raw data values as referenced by DEF or CDEF directives or even a pure mathematical function.  The expression uses Reverse Polish Notation to eliminate confusion or errors that may occur with the precedence rules required of traditional infix notation.  There are a number of mathematical, boolean and logical operators available for inclusion in a CDEF directive.

CDEF Examples

Example 1

This example illustrates the commonly used transformation of bytes to megabytes.  Disk and memory readings are often reported in bytes, but frequently this is not the most convenient unit for visualization.  In this case, the CDEF directive divides the “raw” data value as referenced by the DEF and divides it by 1048576 (1024 x 1024). Note that the AREA directive now references the CDEF label and that the vertical label has been updated to reflect the proper units.

rrdtool graph "Example 1 CDEF.png" \
--start "end-48 hours" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 1" \
--vertical-label "Megabytes" \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
CDEF:megadisk1=disk1,1048576,/ \
AREA:megadisk1#0000FF:"Disk 1"

Example 2

This example shows a slightly more complex instance of the reverse-polish math that may be referenced in a CDEF.  In this case, the CDEF first sums up the the “raw” values as referenced by the two DEF statements and then converts the sum to megabytes.

rrdtool graph "Example 2 CDEF.png" \
--start "end-48 hours" --end "Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 2" \
--vertical-label "Megabytes" \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
DEF:disk2=sysinfo.rrd:disk2_used:AVERAGE \
CDEF:megadisk=disk1,disk2,+,1048576,/ \
AREA:megadisk#0000FF:"Total Disk Used"

Example 3

This example illustrates a common technique for differentiating several types of related measurements.  It is a frequent graphing style for disk IO (reads vs. writes) as well as network IO (octets in vs. octets out).  It is achieved by simply negating the values of one of the operations and graphing the result.  In this example, a horizontal rule (HRULE) at the 0 point has also been added in order to highlight the baseline.  Note that the HRULE is specified as the last element to be drawn, which ensures that it will overlay the other graphed elements.

rrdtool graph "Example 3 CDEF.png" \
--start "end-48 hours" --end "Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 3" \
--vertical-label "Bytes" \
DEF:read=sysinfo.rrd:bytes_read:AVERAGE \
DEF:write=sysinfo.rrd:bytes_written:AVERAGE \
CDEF:negwrite=0,write,- \
AREA:read#0000FF:"Bytes Read" \
AREA:negwrite#00FF00:"Bytes Written" \
HRULE:0#000000

Example 4

This is a more complex example which illustrates the use of the IF operation as well as a more advanced graphing style useful to call out anomalous behaviors.  In this case, the temperature values (referenced by the DEF “temp”) are first assessed by the LE (less than or equal to) and GT (greater than) operations.  The values assigned in these CDEFs (iscool, ishot) will then be used in the CDEFs with the IF operations.  The IF operations evaluate iscool/ishot and if it is “true” (i.e. not zero), then the value for temp is returned.  Otherwise, the special constant “unknown”  is returned.  These values for the cool/hot CDEFs are then graphed and result in a clear demarcation where the system is over-heated.

rrdtool graph "Example 4 CDEF.png" \
--start "end-48 hours" --end "Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 4" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
CDEF:iscool=temp,175,LE \
CDEF:ishot=temp,175,GT \
CDEF:cool=iscool,temp,UNKN,IF \
CDEF:hot=ishot,temp,UNKN,IF \
AREA:cool#0000FF:"cool" \
AREA:hot#FF0000:"hot"

Example 5

This example illustrates the use of the LIMIT operation to achieve a similar effect for highlighting anomalous conditions.  In this case, the entire data set is initially graphed using the “hot” color and then the “cool” data set is overlayed on top of the appropriate sections.

rrdtool graph "Example 5 CDEF.png" \
--start "end-48 hours" --end "12am Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 5" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
CDEF:cool=temp,0,175,LIMIT \
AREA:temp#FF0000:"hot" \
AREA:cool#0000FF:"cool"

Example 6

This example shows the use of the MIN operation to provide a “layer cake effect” in the graph.  This is a popular graphing technique that can be used either to signify a state change above a given threshold or simply to provide a color gradient in the graph for extra polish. This particular example again relies on overlaying the “cool” graph to mask out the relevant sections of the data.

rrdtool graph "Example 6 CDEF.png" \
--start "end-48 hours" --end "12am Nov 1, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 6" \
--vertical-label "Temperature" \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
CDEF:cool=temp,175,MIN \
AREA:temp#FF0000:"hot" \
AREA:cool#0000FF:"cool"

This section covers some of the basic options and directives that can be used to personalize the graph.  There are a number of options available to control how the legends and other text are displayed, adjustments for the scaling behavior, as well as means for demarcating significant levels or marking points in time.

Rule Directives

Rules are simply straight lines that are drawn in the graphing area.  There are two variations:  Horizontal Rules (HRULEs) and Vertical Rules (VRULEs).  Typical usages of rules may include displaying a bar across the graph that may indicate an important threshold or a delimiter that reflects a system change.  Note that rules will not be drawn if they are not within the scope of the actual data ranges being displayed.  The general format for rules are as follows:

HRULE:Value#Color:Legend

VRULE:Time#Color:Legend

Value represents the value on the y-axis that will apply to a horizontal rule.  Note that horizontal rules can only be perfectly horizontal; it is not possible to supply a formula for a sloped line.  If the values of the data being displayed are not within the range of the horizontal line, then the line will not be displayed.  A frequent use of a HRULE is to demarcate a critical threshold in the data.

Time represents the value on the x-axis that will apply to a vertical rule.  The time must be presented as a standard Unix epoch value (number of seconds since Jan 1, 1970 UTC).  Note that if the time range of the data displayed does not contain the time specified for the VRULE, the rule will not be displayed.  A common use of a VRULE is to flag a “state change” in the measured systems, such as a new software release.

Color defines the color of the line or area graph.  This is expressed in the web-standard  RGB hexadecimal triplet and must be separated from the label by a ‘#’.  Example color definitions are: 333399 (blue), 33FF00 (bright green), and CC0000 (red).  If the color is not specified, then the area will be “invisible”.

Legend is the text associated with the legend entry for the graph.  It is an optional element, and if it is omitted the ‘:’ separator should also be omitted.

Rule Examples

Example 1

The following example is a simple example of the different rule types.  The HRULE specifies a red horizontal line drawn at the value of 750,000,000 on the y-axis.   The VRULE specifies a light green vertical line drawn at the value of 11:30am Dec 30, 2009 (1262201400 in Unix epoch format).

rrdtool graph "Example 1.png" \
--start "end-48 hours" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 1" \
--vertical-label "Bytes" \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
AREA:disk1#0000FF:"Disk 1" \
HRULE:750000000#FF0000:"750 MB Warning" \
VRULE:1262201400#00FF00:"Software Rollout"

Example 2

The following example illustrates use of a time value for the VRULE (Dec 18, 2009) that is not within the scope of the graphed data.  Note that the rule in this case is not displayed nor is there a legend element.  If the scope of the graphed data is changed to encompass the value of the VRULE, then the rule will again be displayed.  This behavior provides some utility in demarcating significant events without causing the y-axis to be “pinned” to a specified time.

rrdtool graph "Example 2 Rule.png" \
--start "end-48 hours" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 2" \
--vertical-label "Bytes" \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
AREA:disk1#0000FF:"Disk 1" \
HRULE:750000000#FF0000:"750 MB Warning" \
VRULE:1261201400#00FF00:"Software Rollout"

Legend and Title Options

--title specifies the text to be displayed above the graphing canvas.

--vertical-label specifies the text to be displayed next to the y-axis.  Typically, this indicates the units of the measurement.

--right-axis specifies an alternate y-axis scale that will be displayed on the right side of the graph.  This can be used to realign the units displayed which may have been adjusted to increase the legibility of the graph.  This parameter requires both a scale and offset (scale:offset) be specified.

--right-axis-label specifies the text to be displayed on the right axis.

--no-legend omits the legend information from being drawn.

--legend-position defines where the legend will be displayed in the graph.  Acceptable values are north, south, east and west.  The default value is south.

Legend and Title Examples

Example 1

This simple example illustrates the title, vertical-label, and legend-position parameters in use.  Note that the text for the label and title is quoted to ensure proper parsing.  The legend-position overrides the default value of south with east.

rrdtool graph "Example 1 Legend.png" \
--start "end-48 hours" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 1" \
--vertical-label "Bytes" \
--legend-position east \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
AREA:disk1#0000FF:"Disk 1"

Example 2

This example illustrates the use of the right-axis to specify an alternate scale.  In this case, the y-axis scale on the left (the default) reflects the CPU temperature measured in Celsius.  The y-axis scale on the right reflects the fan speed measured in RPM.  For this example, the scale has been set to 1 and the offset specified as 0.

rrdtool graph "Example 2 Legend.png" \
--start "end-48 hours" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 2" \
--vertical-label "Celsius" \
--right-axis 1:0 \
--right-axis-label RPM \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
DEF:fan=sysinfo.rrd:cpu_fan:AVERAGE \
AREA:temp#0000FF:"CPU Temperature" \
LINE1:fan#00FF00:"Fan"

Example 3

This example further refines the graph defined in Example 2 by adjusting the values of the fan so that they are closer to the range of the temperatures.  (This is performed by the CDEF directive, which will be further described in later examples.)  To keep the units correct, the scale for the right-label is adjusted to re-compensate.  Note that in this example the variations in temperature and RPM are now both clearly visible and the left and right scales on the y-axis reflect the different scales of the datatypes.

rrdtool graph "Example 3 Legend.png" \
--start "end-48 hours" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 3" \
--vertical-label "Celsius" \
--right-axis 20:0 \
--right-axis-label RPM \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
DEF:fan=sysinfo.rrd:cpu_fan:AVERAGE \
CDEF:fan20=fan,20,/ \
AREA:temp#0000FF:"CPU Temperature" \
LINE1:fan20#00FF00:"Fan"

Scale Options

--upper-limit provides an override of the default auto-scaling behavior by setting an explicit upper limit value for the y-axis.  Note that the value provided by this option will continue to be overridden if an actual data value in the graph exceeds the limit specified.

--lower-limit provides an override of the default auto-scaling behavior by setting an explicit lower limit value for the y-axis.  Note that the value provided by this option will continue to be overridden if an actual data value in the graph is lower than the limit specified.

--rigid is used in conjunction with the upper-limit and lower-limit options to provide a definitive maximum and minimum y-axis value.  With this option specified, the auto-scaling behavior will not adjust the scale even if a data value would exceed (or fall below) the upper or lower limits specified.

--logarithmic changes the y-axis to use a logarithmic scale instead of the default linear scale.  This can be useful to visualize fine-grain patterns in the data that may otherwise be obscured if the values are wide-ranging.

--units-exponent sets the exponent expressed in the y-axis scale to a fixed value.  For example, setting this to 3 would lead to the scale to consistently use units of 1000 (10^3).

--units-length defines how many characters rrdtool should assume are present in the y-axis scale labels.  This may be necessary to specify when deviating from the default values for the expression of the units (via the units-exponent, logarithmic, or units=si options) in order to prevent rrdtool from overlapping the vertical label and scale labels.

--units=si overrides the exponential notation with the standard SI unit symbols (k, M, etc.)  Note that the exponential notation is the default only for logarithmic graphs; linear graphs already use the SI notation.

Scale Examples

Example 1

This example illustrates the use of the upper and lower limit options.  Note that in this case the specified lower-limit is ignored, as an actual data value is lower than the value specified and so the auto-scaler adjusts the lower range of the scale to compensate.

rrdtool graph "Example 1 Scale.png" \
--start "end-1 month" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 1" \
--vertical-label "Bytes" \
--upper-limit 1000000000 \
--lower-limit 500000000 \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
AREA:disk1#0000FF:"Disk"

Example 2

This example demonstrates how the rigid option can be used to enforce the specified upper and lower limits.  Note that this strict adherence to the specified limits may prevent data that is out of range from being displayed.

rrdtool graph "Example 2 Scale.png" \
--start "end-1 month" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 2" \
--vertical-label "Bytes" \
--upper-limit 1000000000 \
--lower-limit 500000000 \
--rigid \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
AREA:disk1#0000FF:"Disk"

Example 3

In this example, the logarithmic option is specified to alter the y-axis scaling behavior.  The units-exponent is also specified which maintains the expression of the scale at the fixed rate of 10^3 (1000′s).

rrdtool graph "Example 3 Scale.png" \
--start "end-1 month" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 3" \
--vertical-label "kilobytes" \
--logarithmic \
--units-exponent 3 \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
AREA:disk1#0000FF:"Disk"

Example 4

This is an alternate view of the previous graph using the SI notation instead of the default exponential notation for a logarithmic graph.  In addition, the units-length option is specified to facilitate the alignment of the axis label and scale units.

rrdtool graph "Example 4 Scale.png" \
--start "end-1 month" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 4" \
--vertical-label "Bytes" \
--logarithmic \
--units=si \
--units-length 5 \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
AREA:disk1#0000FF:"Disk"

This post describes some basic techniques for generating area charts using RRD tool.  It builds upon the previous Line Graphs posting which describes DEFs and other basic graphing options.  The data used for the examples was generated using the tool specified in the RRD Example Data posting.

Area Chart Graphing Directives

As specified in the Line Graphs posting, the data definitions (DEFs) must be declared prior to the display directives.  The general format for an area directive is as follows:

AREA:Label#Color:Legend:STACK

Label refers to the label given to a previously defined data definition (DEF).  This defines the source of the data to be graphed.

Color defines the color of the line or area graph.  This is expressed in the web-standard  RGB hexadecimal triplet and must be separated from the label by a ‘#’.  Example color definitions are: 333399 (blue), 33FF00 (bright green), and CC0000 (red).  If the color is not specified, then the area will be “invisible”.

Legend is the text associated with the legend entry for the graph.  It is an optional element, and if it is omitted the ‘:’ separator should also be omitted.

STACK specifies that the results in the graph element be offset from the top of the previous display output instead of the normal 0-based offset.  It is case-sensitive and should always be specified in all-caps.  It is an optional element, and if it is omitted the ‘:’ separator should also be omitted.  To specify a stacked element without a legend, simply omit any entry for the legend but remember the colon.  (e.g. AREA:label#00FFAA::STACK)

Examples

Example 1

This is a simple example of an area graph.  In this case, there is only one data element defined and displayed (“disk1″).

rrdtool graph "Example 1.png" \
--start "end-28 hours" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 1" \
--vertical-label "Bytes" \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
AREA:disk1#0000FF:"Disk 1"

Example 2

This example illustrates more than one display element.  In this case, the elements are not stacked and so both use the default 0-based offset.  Note that the order of display directives also defines the drawing order, which may result in some (or all) of an area graph in being hidden.

rrdtool graph "Example 2.png" \
--start "end-48 hours" --end "Jan 4, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 2" \
--vertical-label "Bytes" \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
DEF:disk2=sysinfo.rrd:disk2_used:AVERAGE \
AREA:disk1#0000FF:"Disk 1" \
AREA:disk2#FF0000:"Disk 2"

Example 3

This example illustrates the use of the STACK option for the AREA directive.  Remember that the STACK offsets the data display to start from the top of the previously displayed element.  If there is no previous element, then the STACK option uses the default 0-based offset.  Any number of display directives can use the STACK option.  A frequent use of the STACK option is to illustrate a “summation” of a set of servers or services.

Compare this graph with that of Example 2, which covers the same set of data.

rrdtool graph "Example 3.png" \
--start "end-48 hours" --end "Jan 4, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 3" \
--vertical-label "Bytes" \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
DEF:disk2=sysinfo.rrd:disk2_used:AVERAGE \
AREA:disk1#0000FF:"Disk 1":STACK \
AREA:disk2#FF0000:"Disk 2":STACK

Example 4

This example illustrates the technique of making an “invisible” element.  In this case, the “disk1″ directive has no color or legend specified.  However, despite it not being displayed, it is still present as applicable to the STACK option for the “disk2″ element.

rrdtool graph "Example 4.png" \
--start "end-48 hours" --end "Jan 4, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 4" \
--vertical-label "Bytes" \
DEF:disk1=sysinfo.rrd:disk_used:AVERAGE \
DEF:disk2=sysinfo.rrd:disk2_used:AVERAGE \
AREA:disk1::STACK \
AREA:disk2#FF0000:"Disk 2":STACK

This post covers some of the basic techniques for generating graphs using RRD tool.  The demonstration examples assumes that the data has been created as described in the RRD Example Data posting.  It should provide a basic understanding of how to specify data sources and transform them into basic line graphs using some of RRDTool’s built-in graphing options.

Data Definitions

The data definition is the source of the data to be graphed.  It provides a label that is referenced by the graphing directives.  There can be multiple data definitions in a single graph and the data may be manipulated extensively by other operations (CDEFs and VDEFs).  In addition, it is possible to specify data from multiple files to be combined in a single graph.

The basic format for a Data Definition (DEF) is as follows:

DEF:Label=RRD File:Data Source:Consolidation Function

Label is the name of the data definition. It may be from 1-19 characters long and consists of characters in the set [a-zA-Z0-9_].  The label should be descriptive of the data being graphed and may be the same label as used in the original Data Source.  Example labels include “temperature”, “BytesIn”, “bytes_out”, etc.

RRD File is the path and name of the data file.  If the full path is not specified, then rrdtool assumes the current working directory.

Data Source is the name of the data source (DS) within the RRD file.  See the posting on Creating a RRD File for a reference of the DS definition if necessary.

Consolidation Function specifies how rrdtool may consolidate data prior to display.  This does not affect the actual data stored in the RRD file; it is used to aggregate data points if the resolution of the graph is too low to display each data point individually.  There available consolidation functions are MIN, MAX, AVERAGE, and LAST.

MIN uses only the smallest value of the Primary Data Points within the resolution’s time span.

MAX uses only the largest value of the Primary Data Points within the resolution’s time span.

AVERAGE uses the average of the all Primary Data Points within the resolution’s time span.

LAST uses only the last (most recent) Primary Data Point within the resolution’s time span.

Graphing Directive for Lines

With the data definitions established, graphing directives can be used to specify how the data will be displayed.  Note that the graphing directives should be specified only after the data definitions (DEF) have been declared.  The general format for a line directive is as follows:

LINE[Width]:Label#Color:Legend

Width is an optional element for line graphs.  It defines the width of the line used to draw the graph.

Label refers to the label given to a previously defined data definition (DEF).  This defines the source of the data to be graphed.

Color defines the color of the line or area graph.  This is expressed in the web-standard  RGB hexadecimal triplet and must be separated from the label by a ‘#’.  Example color definitions are: 333399 (blue), 33FF00 (bright green), and CC0000 (red).  If the color is not specified, then the line will be “invisible”.

Legend is the text associated with the legend entry for the graph.  It is an optional element, and if it is omitted the ‘:’ separator should also be omitted.

Display Options

There are a number of options available that define how the graph is rendered.  These options can define the size, format, time span, legend, title, etc. of the graph.  None of them are explicitly required (default values will be used), but they help provide some basic customization of the graphs.

--width specifies the width (in pixels) of the graphing canvas.  The actual image width may be wider to accommodate any border or label elements.

--height specifies the height (in pixels) of the graphing canvas.  The actual image height may be taller to accommodate any border, title, or legend elements.

--imgformat specifies the desired output format of the graph.  Available options include: PNG, SVG, EPS, and PDF.

--start specifies the start time (in absolute or relative terms) for the data displayed on the graphing canvas.  Frequently, the start time is a relative offset of the end time (e.g. “end-24 hours”, “end-1 month”, “end-2 weeks”)

--end specifies the end time (in absolute or relative terms) for the data displayed on the graphing canvas.  The most frequently used value for the end time is “now”, which results in a graph showing the latest data.

--title specifies the text to be displayed above the graphing canvas.

--vertical-label specifies the text to be displayed next to the y-axis.  Typically, this indicates the units of the measurement.

Examples

Example 1

This basic example creates a graph with the filename “Example 1.png”.  As specified by the start and end dates, it encompasses the entire year of 2009.  The resulting PNG file will have a data canvas of 500 x 120 pixels, although the full size of the image will be larger to accommodate the legend and border.  This example uses only one data definition (DEF) which references the sysinfo.rrd file’s “load” data element (but note that the label for the DEF itself is called “sysload”).  The line directive (LINE1) in turn references the “sysload” label to draw a 1 pixel wide blue line.

rrdtool graph "Example 1.png" \
--start="Jan 1, 2009" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
DEF:sysload=sysinfo.rrd:load:AVERAGE \
LINE1:sysload#0000FF

Example 2

This example is similar to the first with the exception that it illustrates the use of relative dates to define the graphing period.  In this case, the start time is defined as 2 weeks less than the end time.  A frequent use of relative dates is specifying the end time as “now” and the start time relative to that (such as “end-24 hours” or “end-1 week”).  Note also RRDTool adjusts the x-axis legend to reflect the different time span.

rrdtool graph "Example 2.png" \
--start="end-2 weeks" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
DEF:load=sysinfo.rrd:load:AVERAGE \
LINE1:load#0000FF

Example 3

This example extends the previous example by adding an additional data definition and line.  Note that the lines are drawn in the order they are specified, which means that where lines overlap, the line specified latest in the graph command will be on top.

rrdtool graph "Example 3.png" \
--start="end-2 weeks" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
DEF:load=sysinfo.rrd:load:AVERAGE \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
LINE1:load#0000FF \
LINE1:temp#FF0000

Example 4

Extending the example further, this iteration introduces some basic legend and title information.  This example creates a title for the graph (“Example 4″) and a label for the vertical axis (“Units”).  In addition, each of the line directives are amended to include a legend specification.  RRDTool will create the appropriate “color squares” for the legend entries.

rrdtool graph "Example 4.png" \
--start="end-2 weeks" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 4" \
--vertical-label "Units" \
DEF:load=sysinfo.rrd:load:AVERAGE \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
LINE1:load#0000FF:"System Load" \
LINE1:temp#FF0000:"CPU Temperature"

Example 5

This example simply illustrates a variation of the width of the line element.  In this case, the line representing the temperature has been specified as being 3 pixels wide (LINE3).  Line width can be specified as 1, 2 or 3 pixels wide.

rrdtool graph "Example 5.png" \
--start="end-2 weeks" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 5" \
--vertical-label "Units" \
DEF:load=sysinfo.rrd:load:AVERAGE \
DEF:temp=sysinfo.rrd:temperature:AVERAGE \
LINE1:load#0000FF:"System Load" \
LINE3:temp#FF0000:"CPU Temperature"

Example 6

This final example illustrates the use of different round-robin archives (RRAs) within a graph definition.  In this case, the different RRAs use a different consolidation function (CF) in order to record different aspects of the data patterns.  Specifying the MIN and MAX consolidation functions in the DEFs allow us to retrieve and plot the min and max temperature readings.

rrdtool graph "Example 6.png" \
--start="end-30 days" --end "Dec 31, 2009" \
--imgformat PNG --width 500 --height 120 \
--title "Example 6" \
--vertical-label "Celsius" \
DEF:mintemp=sysinfo.rrd:temperature:MIN \
DEF:maxtemp=sysinfo.rrd:temperature:MAX \
LINE1:mintemp#0000FF:"Minimum Temperature" \
LINE1:maxtemp#FF0000:"Maximum Temperature"