Circles of Archimedes

Background

PHP is a very popular language for web development.  Providing bindings for directly calling rrdtool within PHP provides a simple and efficient means of providing a high-level scripting interface into the data management and display capabilities of rrdtool.  The stock installation of rrdtool does not provide any support for PHP so bindings must be created for it manually.

Setup

1. Download the php_rrdtool archive from the contributions directory of the RRDTool site.
2. Extract the archive contents into the /usr/include/php/ext directory.

   sudo tar xzvf php_rrdtool.tar.gz -C /usr/include/php/ext

3. Change the owner and group to “root” and “wheel”.

   sudo chown -R root:wheel /usr/include/php/ext/rrdtool

4. Navigate to the newly created rrdtool directory.

   cd /usr/include/php/ext/rrdtool

5. Generate the configuration files for the PHP bindings.

   sudo phpize

6. Execute the newly-generated configuration script for the PHP bindings.  Note that the --with-rrdtool argument takes as its value the “root” directory where the various rrdtool files are located.  If installed using the default values supplied by the MacPorts utility, the “root” directory would be /opt/local.

   sudo ./configure CFLAGS="-fnested-functions" --with-php-config=/usr/bin/php-config --with-rrdtool=/opt/local

7. Make the php rrdtool extension.

   sudo make

8. Install the php rrdtool extension.  Make a note of the directory into which the extension was deployed.

   sudo make install

9. Make a backup copy of the /etc/php.ini file for safety.  If the file does not yet exist, then copy the /etc/php.ini.default instead.

   sudo cp php.ini php.ini.orig

   or

   sudo cp php.ini.default php.ini

10. Edit the /etc/php.ini file and alter the value of the extension_dir variable to the deployment directory (noted in Step 8).

    extension_dir = "/usr/lib/php/extensions/no-debug-non-zts-20060613/"

11. Edit the /etc/php.ini file and a new variable under the Dynamic Extensions section specifying the rrdtool extension.

    extension=rrdtool.so

12. Restart the apache server to load the new configuration information.

    sudo apachectl restart

13. Verify the rrdtool extension can be loaded from within PHP.

    php -m

Special Notice for Mac OS X Users

With the evolution from 32-bit to 64-bit architectures in progress, it may arise that the rrdtool module is accessible through some mechanisms and not through others.  For example, it has been observed on some hardware/operating systems that the module will load properly when referenced directly by the PHP interpreter (i.e. php -m), but is not accessible when referenced within a web page served by Apache (i.e. not listed as part of the phpinfo() output).  This situation is most likely due to Apache running in 64-bit mode while the PHP interpreter and associated extensions are running in 32-bit mode.  If  this situation arises, then the suggested solution is to force Apache to run in 32-bit mode.

Background

With the evolution of Apple’s hardware from 32-bit to 64-bit compatibility, there are bound to be some difficulties.  They have tried to alleviate much of the complexity of running in this environment through their use of universal binaries, but sometimes there are unforeseen difficulties that require “undoing” some of Apple’s handiwork.

As an example, the standard Apache web server is actually made of 4 different executables:  32-bit and 64-bit versions for the PowerPC architecture, and 32-bit and 64-bit versions for the Intel (x86) architecture.  Normally, the operating system runs the “most appropriate” version depending on the nature of the hardware.  However, this may cause problems when trying to use Apache modules which may not support the “most appropriate” version.  A common occurrence of this is when using the PHP module and some of its extensions.

There are two possible methods of alleviating this problem:  recompile every module and extension so that it is in the supported architecture or strip the 64-bit binary from the Apache universal binary which forces the operating system to run it in 32-bit mode.  As the more expedient solution is simply to force Apache into 32-bit mode, that is the route explained below.

Special Notice

This procedure alters the installed Apache web server.  The changed binary may be over-written by a future update which may require this procedure to be repeated.  This procedure has been tested only on Leopard running on an Intel-based Mac Pro.  Older PowerPC-based systems or systems running Snow Leopard may not require this process.

Setup

1. List the supported architectures in the Apache binary.

   file /usr/sbin/httpd

2. Verify that “Mach-O 64-bit executable x86_64″ is listed as part of the universal binary.
3. Copy the original executable to a backup location for safety.

   sudo cp /usr/sbin/httpd /usr/sbin/httpd.orig

4. Stop the web server.

   sudo apachectl stop

5. Remove the 64-bit executable from the Apache universal binary.

   sudo lipo /usr/sbin/httpd -remove x86_64 -output /usr/sbin/httpd

6. List the supported architectures now in the Apache binary.

   file /usr/sbin/httpd

7. Verify the “Mach-O 64-bit executable x86_64″ is no longer listed as part of the universal binary.
8. Restart the web server.

   sudo apachectl start

Background

Bugzilla is a very popular open-source bug-tracking system.  It is used by many companies (large and small) to not only track defects and their remediation, but also to track various other development tasks, including new feature work, enhancement requests, etc.  It can even be used as part of a change-control process (e.g. SAS-70, SOx, etc.).

Because Bugzilla is an open-source project, it is actively enhanced and can be installed on a number of different systems.  It is easily customized in both appearance and functionality which provides it with the flexibility to fit into the existing development process of almost any team.

The project’s site (http://www.bugzilla.org) has a plethora of documentation, example usages, forums, as well as the latest releases.  The latest stable release can be found on their download page.  This post contains specific instructions for installing Bugzilla on a system running Mac OS X Server (Leopard).

Preliminary Preparations

Perl

An appropriate version of perl should already be available as part of the normal operating system installation.  Bugzilla requires a minimum of version 5.8.1 and version 5.8.8 should already be installed (as of OS X 10.5.8).  The version of perl can be verified with the following command:

/usr/bin/perl --version

MySQL

Bugzilla can utilize several different databases to store its data, but Mac OS X Server already has MySQL included in its normal installation so it makes sense to use that option.  However, the MySQL installed by default with Mac OS X Server does not include the necessary files to build a MySQL application so these must be installed manually.

To install the MySQL build files:

1. Go to the Apple open source download page and select the appropriate version of the operating system the installation is running.
2. Download the binary version of the MySQL package (for 10.5.8, the correct package is MySQL-45).
3. Decompress the packaged binary.

   tar xzf MySQL-45.binaries.tar.gz

4. Navigate to the uncompressed package directory.

   cd MySQL-45.binaries

5. Decompress the internal package into the destination directory.

   sudo tar -xzvf MySQL-45.root.tar.gz -C /

To enable the MySQL service:

1. Open the Server Admin application and connect to the server.
2. Select the server name and click on the “Settings” icon and then the “Services” tab.
3. Enable the MySQL service by clicking on the appropriate checkbox.
4. Click on the “Save” button to save the new configuration.

To start the MySQL service:

1. Open the Server Admin application and connect to the server.
2. Select the MySQL service from the list of available services.
3. Click on the “Settings” tab.
4. If necessary, set the root password for the MySQL service.  The default database location of /var/mysql may also be overridden if desired.
5. Click on the “Save” button to save the new configuration.
6. Click on the “Start MySQL” to startup the new service.

Apache

Bugzilla can utilize a wide variety of web servers, but Apache is already installed with Mac OS X Server so it makes sense to use that option.  For this installation, Bugzilla will be configured in its own subdomain (e.g. bugzilla.example.com) and not as a directory of another domain (e.g. www.example.com/bugzilla).

To setup the bugzilla domain:

1. Open the Server Admin application and connect to the server hosting DNS services.
2. Select the DNS service from the list of available services.
3. Select the “Zones” icon.
4. Select the Primary Zone for the site (e.g. example.com).
5. Click on the “Add Record” drop-down and select the “Add Alias (CNAME)” item.
6. Enter the Alias Name for the bugzilla subdomain.  The alias name should be fully qualified. This means the full domain name should be given and terminated with a period.

   bugzilla.example.com.

7. Enter the Destination for the bugzilla alias.  (i.e. the name of the server running Apache).  The domain name should be fully qualified. This means the full domain name should be given and terminated with a period.

   www.example.com.

8. Click on the “Save” button to save the new configuration.
9. Restart the DNS service to activate the new domain name.

To setup the bugzilla web site:

1. Open the Server Admin application and connect to the server hosting the web server.
2. Select the Web service from the list of available services.
3. Select the “Sites” icon.
4. Click on the “+” button to create a new site entry.
5. Click on the “General” tab and fill out the information for the new site entry:
   

   Domain Name: bugzilla.example.com
   Host Description: Bugzilla
   IP Address: any
   Port: 80
   Web Folder: /opt/local/www/bugzilla
   Default Index Files: index.cgi
   Error Document: /error.html
   Administrator Email: admin@example.com

6. Select the “Options” tab and enable the “CGI Execution” and “Allow All Overrides” options.
7. Select the “Logging” tab and setup the logging information:
   

   Enable Access Log: enabled
   Archive every: 7 days
   Location: /var/log/apache2/bugzilla_access_log
   Error Log Archive every: 7 days
   Error Log Location: /var/log/apache2/bugzilla_error_log
   Error Log Level: Warnings

8. Select the “Web Services” tab and disable any additional services selected.
9. Enable the new site by clicking the “Enabled” checkbox next to the site entry.
10. Click on the “Save” button to save the new configuration.
11. Restart the Web service to activate the new site.

Bugzilla

In this installation, Bugzilla is installed in the /opt/local/src directory so as to keep it isolated from the “stock” operating system tools and utilities.  This makes upgrading either the operating system or Bugzilla easier and less likely to result in conflicts.  In order to further simplify future upgrades, a symbolic link (/opt/local/www/bugzilla) is used as the root web folder instead of directly utilizing the actual Bugzilla source directory.

1. Download the latest Bugzilla source tarball.
2. Copy the tarball to source directory.

   sudo cp bugzilla-3.4.2.tar.gz /opt/local/src

3. Unpack the source tarball.

   cd /opt/local/src
sudo tar xzf bugzilla-3.4.2.tar.gz

4. Change the owner and group to the default web server user.

   sudo chown -R _www:_www bugzilla-3.4.2

5. Create an alias in the web folder to the active Bugzilla source.

   sudo ln -s /opt/local/src/bugzilla-3.4.2 /opt/local/www/bugzilla

Perl Modules

There are a number of perl modules (both required and optional) that Bugzilla requires for operating.  Fortunately, there are several utilities included in the Bugzilla directory that can be used to determine what needs installation as well as actually installing the necessary modules.  For these installations, it is strongly suggested that you run the commands in a root shell.

To install the perl modules:

1. Create a root shell.

   sudo bash

2. Execute the check-modules.pl script.

   /opt/local/src/bugzilla-3.4.2/checksetup.pl --check_modules

3. Install all the required perl modules identified by the check-modules.pl script.
4. Install the appropriate database binding module (DBD:mysql)

   /usr/bin/perl install-module.pl DBD::mysql

5. Install any optional modules desired.
6. Exit from the root shell.

   exit

Mail

Mac OS X Server already comes with a compatible mail transfer agent (Postfix) so no additional installation is necessary.  A working mail service is necessary for Bugzilla to issue event alerts and notifications.

Configuration

localconfig

The localconfig file is an auto-generated file that contains many of the core Bugzilla configuration variables.  Generation of the file is done by running the checksetup.pl script (with no additional parameters).  The resulting file is not immediately usable as it requires a couple quick updates to properly configure the database access.

1. Execute the checksetup.pl script to generate the localconfig file.

   cd /opt/local/src/bugzilla-3.4.2
./checksetup.pl

2. Change the $db_driver variable in the localconfig file.

   $db_driver = 'mysql'

3. Change the $db_pass variable in the localconfig file to the password for ‘bugs’ user of the database.  (The ‘bugs’ user will be created shortly.)

   $db_pass = '********'

4. Change the $webservergroup variable in the localconfig file.

   $webservergroup = '_www'

MySQL

In order to provide a better working experience with Bugzilla, it is necessary to update some of the MySQL configuration variables.  In addition, a new database user must be created.

1. Edit the /etc/my.cnf file to permit large attachments and many comments

   max_allowed_packet=4M

2. Edit the /etc/my.cnf file to allow small words in full-text indexes

   ft_min_word_len=2

3. Add the ‘bugs’ database user with associated privileges.  Be sure to specify the same password for the ‘bugs’ users as in the configuration of the localconfig file.

   mysql -u root -p
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES, CREATE TEMPORARY TABLES, DROP, REFERENCES on bugs.* to bugs@localhost IDENTIFIED BY '********';
mysql> FLUSH PRIVILEGES;
mysql> commit;
mysql> exit;

4. Run the checksetup.pl script again to create the database tables, groups, users, and indexes.  Be sure to note the created administrator account and password for future reference.

Bugzilla

Bugzilla should now be accessible to a web browser at the domain specified earlier.  (http://bugzilla.example.com)  While it is usable in this state, there are some additional configuration settings that should be made with the administrator’s account.

1. Login as the administrator using the bugzilla administrator’s email address and password previously set.
2. Click on the “Administration” link.
3. Click on the “Parameters” link.
4. Set the email address of the maintainer.

   bugadmin@example.com

5. Set the URL base of the installation.

   http://bugzilla.example.com/

6. Set the cookie domain for the service.

   bugzilla.example.com

7. Click on the “Save Changes” button to save the new parameters.

Related Links

• Configuring RADIUS support for Bugzilla

Background

The robots.txt file provides guidance to the search agents (from Google, Yahoo, MSN, etc.) on how to search your web site.  Well-behaving search agents (aka “bots”, “crawlers”, or “spiders”) look for a robots.txt file in the root directory of the web site in order to determine permissions, restrictions, or hints for properly crawling and indexing the site.  Observance of the controls expressed in a robots.txt file is strictly voluntary on the part of the search agents and is not a substitution  for appropriate security measures on any sensitive information.

The robots.txt file is intended to be read and processed only by the automated search agents.  The rules expressed in the file do not apply to users visiting the site with a typical web browser (e.g. Internet Explorer, FireFox, Safari, etc.).

File Format

The format of a robots.txt file consists of a minimum of one record.  Each record consists of a series of directives with one or more User-agent directives followed by one or more Disallow directives.  There can be only one directive specified per line, and it should be in the format of <directive>: <value>.  Comments are specified by a leading “#” character.

While there are a number of other possible directives that can be specified within a record, the minimum required are a User-agent and a Disallow directive.

Location

The search agent looks for the robots.txt file at the root of the web site.  If the server is running more than one site, then one file should be created for each site.  The exact location may vary depending on the site configuration, but this can be checked in the Server Admin application.  (Select the “Web” service, click on the “Sites” icon, and note the “Web Folder” value for each Domain Name.)

Basic Directives

User-agent

The User-agent directive consists of either the name of a search agent or the wildcard character (“*”).  The wildcard character indicates the rule set applies to all search agents and can only be specified once in the robots.txt file.

Disallow

The Disallow directive follows the User-agent directive and specifies a path (either partial or full) that should not be accessed by the search agent.  Multiple paths may be specified by using multiple Disallow directives within a record.  Multiple paths should not be specified within a single Disallow directive.  The value of the directive may be left blank to indicate there are no agent restrictions.

Additional Directives

There are a number of additional directives that may be specified in a record, but not all of them are recognized by the various search agents.  The only directives that seem to be universally parsed are the basic directives — User-agent and Disallow.

Crawl-delay

The Crawl-delay directive specifies a delay (in seconds) between requests by a search agent.  The delay can be used to help prevent an aggressive search agent from overloading the site by issuing requests too rapidly.

Visit-time

The Visit-time directive specifies the acceptable time range for a search agent to crawl the site.  This can be used to defer any search traffic that might otherwise occur during peak traffic times.  The times specified are in 24 hour (military) format and are based on GMT.  (e.g. “2200-0530″ would advise search agents to crawl the site from 10:00pm to 5:30am GMT.)

Request-rate

The Request-rate directive is similar in effect to the Crawl-delay directive however instead of specifying an absolute time between requests a rate is used.  For example, a rate of 1/30 indicates a maximum of 1 request should be issued every 30 seconds.  In addition, time of day restrictions can also be specified which can be used to provide a more flexible policy with different rates for different times of day.

Comment

The Comment directive can be used instead of the comment delimiter (“#”).  The Comment directive is intended to be returned to the administrator of the search agent and should contain a statement of the site’s crawl policy.

Allow

The Allow directive provides an exception into an otherwise disallowed area of the site.  This may provide a means whereby a search agent can bypass restrictions in order to access deeply-nested content.

Robot-version

The Robot-version directive specifies which version of the Robot Exclusion Standard the rule set observes.  It is intended primarily for site documentation purposes.

Examples

Open Access

The wildcard User-agent indicates the record applies to all search agents and the blank Disallow value means there are no restrictions.

User-agent: *
Disallow:

Blocked Access

The wildcard User-agent indicates the record applies to all search agents and the specification of the root directory (“/”) restricts the search agent from all content.

User-agent: *
Disallow: /

Multiple Restrictions

The wildcard User-agent indicates the record applies to all search agents and the Disallow directives restrict the search agents from content in the /doc/private and /cgi-bin directories.

User-agent: *
Disallow: /doc/private
Disallow: /cgi-bin

Timed Access

The wildcard User-agent indicates the record applies to all search agents and the Disallow directive restricts the search agents from content in the /doc/private directory.  The Visit-time directive advises the search agent to only attempt to crawl the site betwen 11:00pm and 4:30am (GMT).

User-agent: *
Disallow: /doc/private
Visit-time: 2300-0430