Submitted by: Crispin Stutzman (cstutzman@pprc.org)
Originally Published: 2/3/99
Standard Adopted: 6/20/00

BACKGROUND:

The proposed Standard provides guidance to P2Rx regional pollution prevention information centers regarding minimum suggested web site performance data which should be collected in order to facilitate effective aggregate measurement of the web site activity of regional centers. The document also suggests optional web measures for each center to consider.

RATIONALE:

There is an increasing emphasis within EPA and other governmental agencies on the use of objective numerical measures of program effectiveness. This trend has been brought about by (among other factors) the Government Performance and Results Act (GPRA). In response to this need for objective metrics, the P2Rx group has developed minimal standard for web site measurement that can be adopted by all participating centers. This proposed standard supports compliance with the GPRA by providing a common reporting standard that will greatly facilitate the aggregation of web site statistics from the participating regional centers. It also provides individual centers with a starting point for their own measurement protocols, and encourages informal benchmarking and information sharing among centers.

Under this proposal, two measures are required of compliant centers. These metrics are: (1) the number of user sessions, and (2) the number of page impressions. The two required measures are relatively easy to gather, lend themselves to aggregation, and can act as a proving ground for the aggregations of P2Rx web use.

This proposed standard is accompanied by the following caveat: it is recognized that each regional information center is unique in terms of audience served, maturity of program, information model, and a number of other factors. These differences make direct comparison of centers based solely or primarily on their web site statistics of questionable value. Thus it should be emphasized that the primary intent of this standard is to facilitate aggregate performance reporting of all centers.

P2Rx AGGREGATE MEASURES

Each P2Rx center should collect the following information about the use of its web site. This information will be aggregated by the National Coordinator on a P2Rx-wide basis every six months.

Number of User Sessions (sometimes called "Visits"). User sessions are defined as consecutive hits to a web site coming from a single user (as identified by IP address or cookie) within a "sliding" time window. The time window is defined as a period of time beginning with the first "hit" to the web site from a given user, and ending after a predetermined time interval has passed without additional activity. Typically, the length of this time interval is 30 minutes. For purposes of aggregate reporting by the Regional centers, the 30 minute interval should be used unless specific conditions suggest otherwise. Any center which uses a longer time interval should note this as a footnote to their user session statistics.

Number of Page Impressions. Page impressions are normally defined as a "GET" request for a HTML document - e.g., only the main document requests, excluding requests for graphics files or other supporting files. The intent of this metric is to provide a more meaningful measurement of total pages viewed than the number of "hits" to the web site. For the purposes of aggregate reporting by the Regional Centers, page impressions should include requests for any document file, including Portable Document Format (.pdf) files.

SCOPE OF MEASUREMENT

Significant differences exist among centers regarding internal use of their web sites - e.g., as an in-house resource for responding to clients requests for information. The issue of which users should be included during web site measurement is left to each Center. In general, Centers should make an effort to exclude hits to the site that originate from web site maintenance activities and search engine "spiders." Internal use of the web site by Center staff and affiliates should be counted if that use is directly related to serving the Center's clients.

ADDITONAL COMMONENTS ON STANDARD MEASUREMENTS

Normally, each center will provide data from the previous six months to the national coordinator. In addition, to simplify the tracking of reporting periods, beginning with FY00, the measurement periods will be established to coincide with federal fiscal year quarters. To permit data collection to begin as soon as possible, the first measurement period will begin July 1, 1999 and end on September 30, 1999. This data should be submitted to the National Coordinator by October 15, 1999. The second measurement period will begin October 1, 1999 (start of first quarter FY00) and end on March 31, 2000 (end of second quarter FY00). Statistics from this initial period will be submitted to the national coordinator by April 17, 2000.

Since our centers use a variety of measurement tools that have varying measurement capabilities, not all centers will have perfect data. (For example, a center might gather information about page impressions on only a selected portion of its web site. Or it might not have the tools to filter out its own web site use.)

P2Rx recognizes our differences, and each center will strive to gather the best data possible given its software, access to log files, etc.

Each center will probably collect additional web measurement data individually. The web development subgroup has suggested additional measures (below) that each center might find illuminating.

SUGGESTED SUPPLEMENTAL WEB MEASURES FOR A REGIONAL CENTER

In addition to the standard aggregate measures of web site use, the web development subgroup suggests that each center might find the following measures helpful in managing its web site. These metrics are not required, but participating centers are strongly urged to consider their use. Many of the web subgroup members are already working to collect this information about the use of their own web sites.

Top 10-50 Pages Requested or Files Downloaded (# TBD by each center). This is a simple measurement of which resources (documents) on your site are most useful to your users.

Percent Error Rate on Requested Hits. This measurement is derived from the web log by measuring all requests for information that result in an error code (e.g., a "404-Page Not Found error"). This is a useful measure of overall site integrity and can often be useful as a tool for identifying outdated or incorrect links to a site.

Number Repeat Visits. This is often an indirect measurement but seeks to determine what percentage of your users have visited your site previously. This is one indication of the value of the site to your users (based on the assumption that repeat visits indicate some degree of satisfaction with the site). Methods for determining repeat visits include the use of cookies, registration forms, or by indirect measurement of "Probable Repeat Visits" as detailed in Greg Geyer's web site (http://www.eitec.org/oecajuly/)

Top Referring Sites. A referring site is the web site URL last visited by the user prior to coming to your site. This can be a very useful measurement for determining how users are finding your site. Implementation of this metric will require the use of extended log format, including Referrer URL, by the web servers.

Search engines as a % of total users. This is similar to the Referring sites measurement but focuses specifically on referrals from web search engines. This measurement is provided as an option on several of the commercial web log analysis applications. To the fullest extent possible, each center should filter out its own se of its web site and search engine spiders, and report only statistics that are originating from outside the center.

SOFTWARE FOR WEB LOG ANALYSIS

Because each Center is faced with different challenges in obtaining access to web logs and web support services, a standard software package for web log analysis is not being required at this time. Nonetheless, the web subgroup strongly recommends the centers acquire software that permits them to track these suggested measures. There are many software packages available; Web Trends is an example of a package that several of the regional centers are using. It's a relatively inexpensive tool (under $400) that's very powerful. If you're unfamiliar with the reports that Web Trends can generate, take a look at http://www.webtrends.com/products/reports/log/default.htm for a sample.

Back to P2Rx Administrative Information

TOP