<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Circles of Archimedes &#187; web</title>
	<atom:link href="http://wiki.springsurprise.com/tag/web/feed/" rel="self" type="application/rss+xml" />
	<link>http://wiki.springsurprise.com</link>
	<description>Technical scribbles in the sand</description>
	<lastBuildDate>Mon, 24 Oct 2011 01:00:15 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3</generator>
		<item>
		<title>Creating PHP bindings for RRDTool</title>
		<link>http://wiki.springsurprise.com/2009/10/18/creating-php-bindings-for-rrdtool/</link>
		<comments>http://wiki.springsurprise.com/2009/10/18/creating-php-bindings-for-rrdtool/#comments</comments>
		<pubDate>Sun, 18 Oct 2009 15:00:42 +0000</pubDate>
		<dc:creator>steven</dc:creator>
				<category><![CDATA[Graphing Solutions]]></category>
		<category><![CDATA[Technical Tidbits]]></category>
		<category><![CDATA[apache]]></category>
		<category><![CDATA[graph]]></category>
		<category><![CDATA[php]]></category>
		<category><![CDATA[rrdtool]]></category>
		<category><![CDATA[web]]></category>

		<guid isPermaLink="false">http://wiki.springsurprise.com/?p=363</guid>
		<description><![CDATA[Background PHP is a very popular language for web development.  Providing bindings for directly calling rrdtool within PHP provides a simple and efficient means of providing a high-level scripting interface into the data management and display capabilities of rrdtool.  The &#8230; <a href="http://wiki.springsurprise.com/2009/10/18/creating-php-bindings-for-rrdtool/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<h3>Background</h3>
<p>PHP is a very popular language for web development.  Providing bindings for directly calling rrdtool within PHP provides a simple and efficient means of providing a high-level scripting interface into the data management and display capabilities of rrdtool.  The stock installation of rrdtool does not provide any support for PHP so bindings must be created for it manually.</p>
<h3>Setup</h3>
<ol>
<li>Download the <a href="http://oss.oetiker.ch/rrdtool/pub/contrib/php_rrdtool.tar.gz">php_rrdtool</a> archive from the <a href="http://oss.oetiker.ch/rrdtool/pub/contrib/">contributions directory</a> of the RRDTool site.</li>
<li>Extract the archive contents into the <code>/usr/include/php/ext</code> directory.
<p style="padding-left: 30px;"><code>sudo tar xzvf php_rrdtool.tar.gz -C /usr/include/php/ext</code></p>
</li>
<li>Change the owner and group to &#8220;root&#8221; and &#8220;wheel&#8221;.
<p style="padding-left: 30px;"><code>sudo chown -R root:wheel /usr/include/php/ext/rrdtool</code></p>
</li>
<li>Navigate to the newly created rrdtool directory.
<p style="padding-left: 30px;"><code>cd /usr/include/php/ext/rrdtool</code></p>
</li>
<li>Generate the configuration files for the PHP bindings.
<p style="padding-left: 30px;"><code>sudo phpize</code></p>
</li>
<li>Execute the newly-generated configuration script for the PHP bindings.  Note that the <code>--with-rrdtool</code> argument takes as its value the &#8220;root&#8221; directory where the various rrdtool files are located.  If installed using the default values supplied by the MacPorts utility, the &#8220;root&#8221; directory would be <code>/opt/local</code>.
<p style="padding-left: 30px;"><code>sudo ./configure CFLAGS="-fnested-functions" --with-php-config=/usr/bin/php-config --with-rrdtool=/opt/local</code></p>
</li>
<li>Make the php rrdtool extension.
<p style="padding-left: 30px;"><code>sudo make</code></p>
</li>
<li>Install the php rrdtool extension.  Make a note of the directory into which the extension was deployed.
<p style="padding-left: 30px;"><code>sudo make install</code></p>
</li>
<li>Make a backup copy of the <code>/etc/php.ini</code> file for safety.  If the file does not yet exist, then copy the <code>/etc/php.ini.default</code> instead.
<p style="padding-left: 30px;"><code>sudo cp php.ini php.ini.orig</code></p>
<p style="padding-left: 60px;"><em>or</em></p>
<p style="padding-left: 30px;"><code>sudo cp php.ini.default php.ini</code></p>
</li>
<li>Edit the <code>/etc/php.ini</code> file and alter the value of the <code>extension_dir</code> variable to the deployment directory (noted in Step 8).
<p style="padding-left: 30px;"><code>extension_dir = "/usr/lib/php/extensions/no-debug-non-zts-20060613/"</code></p>
</li>
<li>Edit the <code>/etc/php.ini</code> file and a new variable under the Dynamic Extensions section specifying the rrdtool extension.
<p style="padding-left: 30px;"><code>extension=rrdtool.so</code></p>
</li>
<li>Restart the apache server to load the new configuration information.
<p style="padding-left: 30px;"><code>sudo apachectl restart</code></p>
</li>
<li>Verify the rrdtool extension can be loaded from within PHP.
<p style="padding-left: 30px;"><code>php -m</code></p>
</li>
</ol>
<h4><span style="color: #993300;">Special Notice for Mac OS X Users</span></h4>
<p>With the evolution from 32-bit to 64-bit architectures in progress, it may arise that the rrdtool module is accessible through some mechanisms and not through others.  For example, it has been observed on some hardware/operating systems that the module will load properly when referenced directly by the PHP interpreter (i.e. <code>php -m</code>), but is not accessible when referenced within a web page served by Apache (i.e. not listed as part of the <code>phpinfo()</code> output).  This situation is most likely due to Apache running in 64-bit mode while the PHP interpreter and associated extensions are running in 32-bit mode.  If  this situation arises, then the suggested solution is to <a href="http://wiki.springsurprise.com/2009/10/17/running-apache-in-32-bit-mode-on-leopard/">force Apache to run in 32-bit mode</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://wiki.springsurprise.com/2009/10/18/creating-php-bindings-for-rrdtool/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Running Apache in 32-bit mode on Leopard</title>
		<link>http://wiki.springsurprise.com/2009/10/17/running-apache-in-32-bit-mode-on-leopard/</link>
		<comments>http://wiki.springsurprise.com/2009/10/17/running-apache-in-32-bit-mode-on-leopard/#comments</comments>
		<pubDate>Sun, 18 Oct 2009 03:35:34 +0000</pubDate>
		<dc:creator>steven</dc:creator>
				<category><![CDATA[Technical Tidbits]]></category>
		<category><![CDATA[apache]]></category>
		<category><![CDATA[http]]></category>
		<category><![CDATA[leopard]]></category>
		<category><![CDATA[leopard server]]></category>
		<category><![CDATA[web]]></category>

		<guid isPermaLink="false">http://wiki.springsurprise.com/?p=371</guid>
		<description><![CDATA[Background With the evolution of Apple&#8217;s hardware from 32-bit to 64-bit compatibility, there are bound to be some difficulties.  They have tried to alleviate much of the complexity of running in this environment through their use of universal binaries, but &#8230; <a href="http://wiki.springsurprise.com/2009/10/17/running-apache-in-32-bit-mode-on-leopard/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<h3>Background</h3>
<p>With the evolution of Apple&#8217;s hardware from 32-bit to 64-bit compatibility, there are bound to be some difficulties.  They have tried to alleviate much of the complexity of running in this environment through their use of universal binaries, but sometimes there are unforeseen difficulties that require &#8220;undoing&#8221; some of Apple&#8217;s handiwork.</p>
<p>As an example, the standard Apache web server is actually made of 4 different executables:  32-bit and 64-bit versions for the PowerPC architecture, and 32-bit and 64-bit versions for the Intel (x86) architecture.  Normally, the operating system runs the &#8220;most appropriate&#8221; version depending on the nature of the hardware.  However, this may cause problems when trying to use Apache modules which may not support the &#8220;most appropriate&#8221; version.  A common occurrence of this is when using the PHP module and some of its extensions.</p>
<p>There are two possible methods of alleviating this problem:  recompile every module and extension so that it is in the supported architecture or strip the 64-bit binary from the Apache universal binary which forces the operating system to run it in 32-bit mode.  As the more expedient solution is simply to force Apache into 32-bit mode, that is the route explained below.</p>
<h4><span style="color: #993300;">Special Notice</span></h4>
<p>This procedure alters the installed Apache web server.  The changed binary may be over-written by a future update which may require this procedure to be repeated.  This procedure has been tested only on Leopard running on an Intel-based Mac Pro.  Older PowerPC-based systems or systems running Snow Leopard may not require this process.</p>
<h3>Setup</h3>
<ol>
<li>List the supported architectures in the Apache binary.
<p style="padding-left: 30px;"><code>file /usr/sbin/httpd</code></p>
</li>
<li>Verify that &#8220;Mach-O 64-bit executable x86_64&#8243; is listed as part of the universal binary.</li>
<li>Copy the original executable to a backup location for safety.
<p style="padding-left: 30px;"><code>sudo cp /usr/sbin/httpd /usr/sbin/httpd.orig</code></p>
</li>
<li>Stop the web server.
<p style="padding-left: 30px;"><code>sudo apachectl stop</code></p>
</li>
<li>Remove the 64-bit executable from the Apache universal binary.
<p style="padding-left: 30px;"><code>sudo lipo /usr/sbin/httpd -remove x86_64 -output /usr/sbin/httpd</code></p>
</li>
<li>List the supported architectures now in the Apache binary.
<p style="padding-left: 30px;"><code>file /usr/sbin/httpd</code></p>
</li>
<li>Verify the &#8220;Mach-O 64-bit executable x86_64&#8243; is no longer listed as part of the universal binary.</li>
<li>Restart the web server.
<p style="padding-left: 30px;"><code>sudo apachectl start</code></p>
</li>
</ol>
]]></content:encoded>
			<wfw:commentRss>http://wiki.springsurprise.com/2009/10/17/running-apache-in-32-bit-mode-on-leopard/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Installing Bugzilla on Mac OS X Server</title>
		<link>http://wiki.springsurprise.com/2009/09/30/installing-bugzilla-on-mac-os-x-server/</link>
		<comments>http://wiki.springsurprise.com/2009/09/30/installing-bugzilla-on-mac-os-x-server/#comments</comments>
		<pubDate>Wed, 30 Sep 2009 23:48:09 +0000</pubDate>
		<dc:creator>steven</dc:creator>
				<category><![CDATA[Server Solutions]]></category>
		<category><![CDATA[Software Development]]></category>
		<category><![CDATA[web]]></category>

		<guid isPermaLink="false">http://wiki.springsurprise.com/?p=237</guid>
		<description><![CDATA[Background Bugzilla is a very popular open-source bug-tracking system.  It is used by many companies (large and small) to not only track defects and their remediation, but also to track various other development tasks, including new feature work, enhancement requests, &#8230; <a href="http://wiki.springsurprise.com/2009/09/30/installing-bugzilla-on-mac-os-x-server/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<h3>Background</h3>
<p>Bugzilla is a very popular open-source bug-tracking system.  It is used by many companies (large and small) to not only track defects and their remediation, but also to track various other development tasks, including new feature work, enhancement requests, etc.  It can even be used as part of a change-control process (e.g. SAS-70, SOx, etc.).</p>
<p>Because Bugzilla is an open-source project, it is actively enhanced and can be installed on a number of different systems.  It is easily customized in both appearance and functionality which provides it with the flexibility to fit into the existing development process of almost any team.</p>
<p>The project&#8217;s site (<a href="http://www.bugzilla.org">http://www.bugzilla.org</a>) has a plethora of documentation, example usages, forums, as well as the latest releases.  The latest stable release can be found on their <a href="http://www.bugzilla.org/download/">download</a> page.  This post contains specific instructions for installing Bugzilla on a system running Mac OS X Server (Leopard).</p>
<h3>Preliminary Preparations</h3>
<h4>Perl</h4>
<p>An appropriate version of perl should already be available as part of the normal operating system installation.  Bugzilla requires a minimum of version 5.8.1 and version 5.8.8 should already be installed (as of OS X 10.5.8).  The version of perl can be verified with the following command:</p>
<p style="padding-left: 30px;"><code>/usr/bin/perl --version</code></p>
<h4>MySQL</h4>
<p>Bugzilla can utilize several different databases to store its data, but Mac OS X Server already has MySQL included in its normal installation so it makes sense to use that option.  However, the MySQL installed by default with Mac OS X Server does <em>not</em> include the necessary files to build a MySQL application so these must be installed manually.</p>
<p>To install the MySQL build files:</p>
<ol>
<li>Go to the Apple <a href="http://www.opensource.apple.com/">open source download page</a> and select the appropriate version of the operating system the installation is running.</li>
<li>Download the binary version of the MySQL package (for 10.5.8, the correct package is MySQL-45).</li>
<li>Decompress the packaged binary.
<p style="padding-left: 30px;"><code>tar xzf MySQL-45.binaries.tar.gz</code></p>
</li>
<li>Navigate to the uncompressed package directory.
<p style="padding-left: 30px;"><code>cd MySQL-45.binaries</code></p>
</li>
<li>Decompress the <em>internal</em> package into the destination directory.
<p style="padding-left: 30px;"><code>sudo tar -xzvf MySQL-45.root.tar.gz -C /</code></p>
</li>
</ol>
<p>To enable the MySQL service:</p>
<ol>
<li>Open the <strong>Server Admin</strong> application and connect to the server.</li>
<li>Select the server name and click on the &#8220;Settings&#8221; icon and then the &#8220;Services&#8221; tab.</li>
<li>Enable the MySQL service by clicking on the appropriate checkbox.</li>
<li>Click on the &#8220;Save&#8221; button to save the new configuration.</li>
</ol>
<p>To start the MySQL service:</p>
<ol>
<li>Open the <strong>Server Admin</strong> application and connect to the server.</li>
<li>Select the <strong>MySQL</strong> service from the list of available services.</li>
<li>Click on the &#8220;Settings&#8221; tab.</li>
<li>If necessary, set the root password for the MySQL service.  The default database location of <code>/var/mysql</code> may also be overridden if desired.</li>
<li>Click on the &#8220;Save&#8221; button to save the new configuration.</li>
<li>Click on the &#8220;Start MySQL&#8221; to startup the new service.</li>
</ol>
<h4>Apache</h4>
<p>Bugzilla can utilize a wide variety of web servers, but Apache is already installed with Mac OS X Server so it makes sense to use that option.  For this installation, Bugzilla will be configured in its own subdomain (e.g. bugzilla.example.com) and not as a directory of another domain (e.g. www.example.com/bugzilla).</p>
<p>To setup the bugzilla domain:</p>
<ol>
<li>Open the <strong>Server Admin</strong> application and connect to the server hosting DNS services.</li>
<li>Select the <strong>DNS</strong> service from the list of available services.</li>
<li>Select the &#8220;Zones&#8221; icon.</li>
<li>Select the Primary Zone for the site (e.g. example.com).</li>
<li>Click on the &#8220;Add Record&#8221; drop-down and select the &#8220;Add Alias (CNAME)&#8221; item.</li>
<li>Enter the Alias Name for the bugzilla subdomain.  The alias name should be <em>fully qualified</em>. This means the full domain name should be given and terminated with a period.
<p style="padding-left: 30px;"><code>bugzilla.example.com.</code></p>
</li>
<li>Enter the Destination for the bugzilla alias.  (i.e. the name of the server running Apache).  The domain name should be <em>fully qualified</em>. This means the full domain name should be given and terminated with a period.
<p style="padding-left: 30px;"><code>www.example.com.</code></p>
</li>
<li>Click on the &#8220;Save&#8221; button to save the new configuration.</li>
<li>Restart the DNS service to activate the new domain name.</li>
</ol>
<p>To setup the bugzilla web site:</p>
<ol>
<li>Open the <strong>Server Admin</strong> application and connect to the server hosting the web server.</li>
<li>Select the <strong>Web</strong> service from the list of available services.</li>
<li>Select the &#8220;Sites&#8221; icon.</li>
<li>Click on the &#8220;+&#8221; button to create a new site entry.</li>
<li>Click on the &#8220;General&#8221; tab and fill out the information for the new site entry:<br />
<blockquote><p>Domain Name: <em> bugzilla.example.com</em><br />
Host Description: <em>Bugzilla</em><br />
IP Address: <em>any</em><br />
Port: <em>80</em><br />
Web Folder:<em> /opt/local/www/bugzilla</em><br />
Default Index Files: <em> index.cgi</em><br />
Error Document: <em>/error.html</em><br />
Administrator Email: <em> admin@example.com</em></p></blockquote>
</li>
<li>Select the &#8220;Options&#8221; tab and enable the &#8220;CGI Execution&#8221; and &#8220;Allow All Overrides&#8221; options.</li>
<li>Select the &#8220;Logging&#8221; tab and setup the logging information:<br />
<blockquote><p>Enable Access Log: <em>enabled</em><br />
Archive every: <em>7 days</em><br />
Location: <em>/var/log/apache2/bugzilla_access_log</em><br />
Error Log Archive every: <em>7 days</em><br />
Error Log Location: <em>/var/log/apache2/bugzilla_error_log</em><br />
Error Log Level: <em>Warnings</em></p></blockquote>
</li>
<li>Select the &#8220;Web Services&#8221; tab and disable any additional services selected.</li>
<li>Enable the new site by clicking the &#8220;Enabled&#8221; checkbox next to the site entry.</li>
<li>Click on the &#8220;Save&#8221; button to save the new configuration.</li>
<li>Restart the Web service to activate the new site.</li>
</ol>
<h4>Bugzilla</h4>
<p>In this installation, Bugzilla is installed in the <code>/opt/local/src</code> directory so as to keep it isolated from the &#8220;stock&#8221; operating system tools and utilities.  This makes upgrading either the operating system or Bugzilla easier and less likely to result in conflicts.  In order to further simplify future upgrades, a symbolic link (<code>/opt/local/www/bugzilla</code>) is used as the root web folder instead of directly utilizing the actual Bugzilla source directory.</p>
<ol>
<li>Download the latest Bugzilla source tarball.</li>
<li>Copy the tarball to source directory.
<p style="padding-left: 30px;"><code>sudo cp bugzilla-3.4.2.tar.gz /opt/local/src</code></p>
</li>
<li>Unpack the source tarball.
<p style="padding-left: 30px;"><code>cd /opt/local/src<br />
sudo tar xzf bugzilla-3.4.2.tar.gz</code></li>
<li>Change the owner and group to the default web server user.
<p style="padding-left: 30px;"><code>sudo chown -R _www:_www bugzilla-3.4.2</code></p>
</li>
<li>Create an alias in the web folder to the active Bugzilla source.
<p style="padding-left: 30px;"><code>sudo ln -s /opt/local/src/bugzilla-3.4.2 /opt/local/www/bugzilla</code></p>
</li>
</ol>
<h4>Perl Modules</h4>
<p>There are a number of perl modules (both required and optional) that Bugzilla requires for operating.  Fortunately, there are several utilities included in the Bugzilla directory that can be used to determine what needs installation as well as actually installing the necessary modules.  For these installations, it is strongly suggested that you run the commands in a root shell.</p>
<p>To install the perl modules:</p>
<ol>
<li>Create a root shell.
<p style="padding-left: 30px;"><code>sudo bash</code></p>
</li>
<li>Execute the check-modules.pl script.
<p style="padding-left: 30px;"><code>/opt/local/src/bugzilla-3.4.2/checksetup.pl --check_modules</code></p>
</li>
<li>Install all the <em>required</em> perl modules identified by the check-modules.pl script.</li>
<li>Install the appropriate database binding module (DBD:mysql)
<p style="padding-left: 30px;"><code>/usr/bin/perl install-module.pl DBD::mysql</code></p>
</li>
<li>Install any <em>optional</em> modules desired.</li>
<li>Exit from the root shell.
<p style="padding-left: 30px;"><code>exit</code></p>
</li>
</ol>
<h4>Mail</h4>
<p>Mac OS X Server already comes with a compatible mail transfer agent (Postfix) so no additional installation is necessary.  A working mail service is necessary for Bugzilla to issue event alerts and notifications.</p>
<h3>Configuration</h3>
<h4>localconfig</h4>
<p>The <code>localconfig</code> file is an auto-generated file that contains many of the core Bugzilla configuration variables.  Generation of the file is done by running the <code>checksetup.pl</code> script (with no additional parameters).  The resulting file is <em>not</em> immediately usable as it requires a couple quick updates to properly configure the database access.</p>
<ol>
<li>Execute the <code>checksetup.pl</code> script to generate the <code>localconfig</code> file.
<p style="padding-left: 30px;"><code>cd /opt/local/src/bugzilla-3.4.2<br />
./checksetup.pl</code></li>
<li>Change the $db_driver variable in the <code>localconfig</code> file.
<p style="padding-left: 30px;"><code>$db_driver = 'mysql'</code></p>
</li>
<li>Change the $db_pass variable in the <code>localconfig</code> file to the password for &#8216;bugs&#8217; user of the database.  (The &#8216;bugs&#8217; user will be created shortly.)
<p style="padding-left: 30px;"><code>$db_pass = '********'</code></p>
</li>
<li>Change the $webservergroup variable in the <code>localconfig</code> file.
<p style="padding-left: 30px;"><code>$webservergroup = '_www'</code></p>
</li>
</ol>
<h4>MySQL</h4>
<p>In order to provide a better working experience with Bugzilla, it is necessary to update some of the MySQL configuration variables.  In addition, a new database user must be created.</p>
<ol>
<li>Edit the /etc/my.cnf file to permit large attachments and many comments
<p style="padding-left: 30px;"><code>max_allowed_packet=4M</code></p>
</li>
<li>Edit the /etc/my.cnf file to allow small words in full-text indexes
<p style="padding-left: 30px;"><code>ft_min_word_len=2</code></p>
</li>
<li>Add the &#8216;bugs&#8217; database user with associated privileges.  Be sure to specify the same password for the &#8216;bugs&#8217; users as in the configuration of the <code>localconfig</code> file.
<p style="padding-left: 30px;"><code>mysql -u root -p<br />
mysql&gt; GRANT SELECT, INSERT, UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES, CREATE TEMPORARY TABLES, DROP, REFERENCES on bugs.* to bugs@localhost IDENTIFIED BY '********';<br />
mysql&gt; FLUSH PRIVILEGES;<br />
mysql&gt; commit;<br />
mysql&gt; exit;</code></li>
<li>Run the checksetup.pl script again to create the database tables, groups, users, and indexes.  Be sure to note the created administrator account and password for future reference.</li>
</ol>
<h4>Bugzilla</h4>
<p>Bugzilla should now be accessible to a web browser at the domain specified earlier.  (http://bugzilla.example.com)  While it is usable in this state, there are some additional configuration settings that should be made with the administrator&#8217;s account.</p>
<ol>
<li>Login as the administrator using the bugzilla administrator&#8217;s email address and password previously set.</li>
<li>Click on the &#8220;Administration&#8221; link.</li>
<li>Click on the &#8220;Parameters&#8221; link.</li>
<li>Set the email address of the maintainer.
<p style="padding-left: 30px;"><code>bugadmin@example.com</code></p>
</li>
<li>Set the URL base of the installation.
<p style="padding-left: 30px;"><code>http://bugzilla.example.com/</code></p>
</li>
<li>Set the cookie domain for the service.
<p style="padding-left: 30px;"><code>bugzilla.example.com</code></p>
</li>
<li>Click on the &#8220;Save Changes&#8221; button to save the new parameters.</li>
</ol>
<h3>Related Links</h3>
<ul>
<li><a href="http://wiki.springsurprise.com/2009/10/06/configuring-radius-for-bugzilla-on-mac-os-x-server/">Configuring RADIUS support for Bugzilla</a></li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://wiki.springsurprise.com/2009/09/30/installing-bugzilla-on-mac-os-x-server/feed/</wfw:commentRss>
		<slash:comments>7</slash:comments>
		</item>
		<item>
		<title>Refining Search Traffic with Robots.txt</title>
		<link>http://wiki.springsurprise.com/2009/07/13/refining-search-traffic-with-robots-txt/</link>
		<comments>http://wiki.springsurprise.com/2009/07/13/refining-search-traffic-with-robots-txt/#comments</comments>
		<pubDate>Mon, 13 Jul 2009 17:29:24 +0000</pubDate>
		<dc:creator>steven</dc:creator>
				<category><![CDATA[Server Solutions]]></category>
		<category><![CDATA[robots]]></category>
		<category><![CDATA[search]]></category>
		<category><![CDATA[SEO]]></category>
		<category><![CDATA[web]]></category>

		<guid isPermaLink="false">http://wiki.springsurprise.com/?p=94</guid>
		<description><![CDATA[Background The robots.txt file provides guidance to the search agents (from Google, Yahoo, MSN, etc.) on how to search your web site.  Well-behaving search agents (aka &#8220;bots&#8221;, &#8220;crawlers&#8221;, or &#8220;spiders&#8221;) look for a robots.txt file in the root directory of &#8230; <a href="http://wiki.springsurprise.com/2009/07/13/refining-search-traffic-with-robots-txt/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<h3>Background</h3>
<p>The <code>robots.txt</code> file provides guidance to the search agents (from Google, Yahoo, MSN, etc.) on how to search your web site.  Well-behaving search agents (aka &#8220;bots&#8221;, &#8220;crawlers&#8221;, or &#8220;spiders&#8221;) look for a <code>robots.txt</code> file in the root directory of the web site in order to determine permissions, restrictions, or hints for properly crawling and indexing the site.  <em>Observance of the controls expressed in a robots.txt file is strictly voluntary on the part of the search agents and is <strong>not</strong> a substitution  for appropriate security measures on any sensitive information.</em></p>
<p>The <code>robots.txt</code> file is intended to be read and processed only by the automated search agents.  The rules expressed in the file do not apply to users visiting the site with a typical web browser (e.g. Internet Explorer, FireFox, Safari, etc.).</p>
<h3>File Format</h3>
<p>The format of a <code>robots.txt</code> file consists of a minimum of one <strong>record</strong>.  Each record consists of a series of <strong>directives </strong>with one or more User-agent directives followed by one or more Disallow directives.  There can be only one directive specified per line, and it should be in the format of &lt;<code>directive</code>&gt;<code>:</code> &lt;<code>value</code>&gt;.  Comments are specified by a leading &#8220;#&#8221; character.</p>
<p>While there are a number of other possible directives that can be specified within a record, the minimum required are a User-agent and a Disallow directive.</p>
<h3>Location</h3>
<p>The search agent looks for the <code>robots.txt</code> file at the root of the web site.  If the server is running more than one site, then one file should be created for each site.  The exact location may vary depending on the site configuration, but this can be checked in the <strong>Server Admin</strong> application.  (Select the &#8220;Web&#8221; service, click on the &#8220;Sites&#8221; icon, and note the &#8220;Web Folder&#8221; value for each Domain Name.)</p>
<h3><strong>Basic Directives</strong></h3>
<h4>User-agent</h4>
<p>The User-agent directive consists of either the name of a search agent or the wildcard character (&#8220;*&#8221;).  The wildcard character indicates the rule set applies to all search agents and can only be specified once in the robots.txt file.</p>
<h4>Disallow</h4>
<p>The Disallow directive follows the User-agent directive and specifies a path (either partial or full) that should not be accessed by the search agent.  Multiple paths may be specified by using multiple Disallow directives within a record.  Multiple paths should <em>not</em> be specified within a single Disallow directive.  The value of the directive may be left blank to indicate there are no agent restrictions.</p>
<h3>Additional Directives</h3>
<p>There are a number of additional directives that may be specified in a record, but not all of them are recognized by the various search agents.  The only directives that seem to be universally parsed are the basic directives &#8212; User-agent and Disallow.</p>
<h4>Crawl-delay</h4>
<p>The Crawl-delay directive specifies a delay (in seconds) between requests by a search agent.  The delay can be used to help prevent an aggressive search agent from overloading the site by issuing requests too rapidly.</p>
<h4>Visit-time</h4>
<p>The Visit-time directive specifies the acceptable time range for a search agent to crawl the site.  This can be used to defer any search traffic that might otherwise occur during peak traffic times.  The times specified are in 24 hour (military) format and are based on GMT.  (e.g. &#8220;2200-0530&#8243; would advise search agents to crawl the site from 10:00pm to 5:30am GMT.)</p>
<h4>Request-rate</h4>
<p>The Request-rate directive is similar in effect to the Crawl-delay directive however instead of specifying an absolute time between requests a rate is used.  For example, a rate of 1/30 indicates a maximum of 1 request should be issued every 30 seconds.  In addition, time of day restrictions can also be specified which can be used to provide a more flexible policy with different rates for different times of day.</p>
<h4>Comment</h4>
<p>The Comment directive can be used instead of the comment delimiter (&#8220;#&#8221;).  The Comment directive is intended to be returned to the administrator of the search agent and should contain a statement of the site&#8217;s crawl policy.</p>
<h4>Allow</h4>
<p>The Allow directive provides an exception into an otherwise disallowed area of the site.  This may provide a means whereby a search agent can bypass restrictions in order to access deeply-nested content.</p>
<h4>Robot-version</h4>
<p>The Robot-version directive specifies which version of the Robot Exclusion Standard the rule set observes.  It is intended primarily for site documentation purposes.</p>
<h3>Examples</h3>
<h4>Open Access</h4>
<p>The wildcard User-agent indicates the record applies to all search agents and the blank Disallow value means there are no restrictions.</p>
<p><code>User-agent: *<br />
Disallow:</code></p>
<h4>Blocked Access</h4>
<p>The wildcard User-agent indicates the record applies to all search agents and the specification of the root directory (&#8220;/&#8221;) restricts the search agent from all content.</p>
<p><code>User-agent: *<br />
Disallow: /</code></p>
<h4>Multiple Restrictions</h4>
<p>The wildcard User-agent indicates the record applies to all search agents and the Disallow directives restrict the search agents from content in the <code>/doc/private</code> and <code>/cgi-bin</code> directories.</p>
<p><code>User-agent: *<br />
Disallow: /doc/private<br />
Disallow: /cgi-bin</code></p>
<h4>Timed Access</h4>
<p>The wildcard User-agent indicates the record applies to all search agents and the Disallow directive restricts the search agents from content in the <code>/doc/private</code> directory.  The Visit-time directive advises the search agent to only attempt to crawl the site betwen 11:00pm and 4:30am (GMT).</p>
<p><code>User-agent: *<br />
Disallow: /doc/private<br />
Visit-time: 2300-0430</code></p>
]]></content:encoded>
			<wfw:commentRss>http://wiki.springsurprise.com/2009/07/13/refining-search-traffic-with-robots-txt/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
	</channel>
</rss>

