P2Rx Web Site Measures
Revised standard adopted 9/26/02
Revision proposed 8/22/02
Standard Adopted: 6/20/00
Originally Submitted by: Crispin Stutzman
Originally Published: 2/3/99

BACKGROUND:

The web measures Standard provides guidance to P2Rx Regional Pollution Prevention Information Centers regarding web site performance data which should be collected in order to facilitate effective aggregate measurement and management of web site activity.

RATIONALE:

There is an increasing emphasis within EPA and other governmental agencies on the use of objective numerical measures of program effectiveness. This trend has been brought about by (among other factors) the Government Performance and Results Act (GPRA). In response to this need for objective metrics, the P2Rx group has developed a standard for web site measurement that can be adopted by all participating Centers. This standard supports compliance with the GPRA by providing common reporting criteria that will facilitate aggregation of consistent web site statistics from participating regional Centers.

It also provides individual Centers with data that can be used to manage their web sites and product offerings.

Under this proposal, several measures are required of compliant Centers. These metrics are:
(1) number of user sessions,
(2) number of page impressions,
(3) number of visitors that visit 10 or more times,
(4) number of user sessions for P2Rx products. Specifically,
     a. Topic Hubs
     b. P2 Programs Directory
     c. News
     d. Additional Resources
     e. Libraries
     f. Innovative Technologies
     g. Other Products - aggregatenumber for all other P2Rx products, such as:
         i. Research Projects Database
         ii. Requests for Proposals Database
         iii. Mercury Reduction programs
         iv. Homes Across America

This proposed standard is accompanied by the following caveat: It is recognized that each regional information Center is unique in terms of audience served, maturity of program, information model, and a number of other factors. These differences make direct comparison of Centers based solely or primarily on their web site statistics of questionable value. Thus it should be emphasized that the primary intent of this standard is to facilitate aggregate performance reporting of all Centers and to provide management information regarding products and services to Center directors.

By looking at the trend of aggregate activity of premier P2Rx products, topic hubs and P2 Programs Directory, Center directors will see how much these products are being used and what the relative proportion of use is for these products. Other products under consideration for more national exposure can be tracked, helping determine their potential for more exposure. It is recognized that web activity will be related to marketing effort, and that actual historical activity is only one measure to consider when making decisions about future projects.

P2Rx AGGREGATE MEASURES

Each P2Rx Center should collect the following information every quarter about the use of its web site. This information will be aggregated by the National Coordinator on a P2Rx-wide basis every six months. The National Coordinator will report actual numbers and the trend of web activity using quarterly aggregate data including all historic data available that meets requirements of this standard. All reports shall include the caveat, “the P2Rx centers provide services and engage in many interdependent activities that are funded through the P2Rx grant and other leveraged fund sources. The activity and web measures identified in this report are wholly or partially funded by the P2Rx grant.”

Number of User Sessions (sometimes called “Visits”). User sessions are defined as the number of unique users who visited a web site during a certain time window. The time window is defined as a period of time beginning with the first “hit” to the web site from a given user, and ending after a predetermined time interval has passed without additional activity. Typically, the length of this time interval is 30 minutes. For purposes of aggregate reporting by the Regional Centers, the 30 minute interval should be used unless specific conditions suggest otherwise. Any Center which uses a different time interval should note this as a footnote to their user session statistics. Since P2Rx does not issue “cookies” to users, nor require passwords, users are counted by IP address.

Number of Page Impressions. Page impressions are normally defined as a “GET” request for a HTML document – e.g., only the main document requests, excluding requests for graphics files or other supporting files. The intent of this metric is to provide a more meaningful measurement of total pages viewed than the number of “hits” to the web site. For the purposes of aggregate reporting by the Regional Centers, page impressions should include requests for any document file, including Portable Document Format (.pdf) files.

Number of user sessions for P2Rx products. Aggregate numbers will be reported to reflect use of P2Rx products. Specifically, use of
· Topic hubs
· P2 programs directory
· News (news web pages or electronic versions of newsletters)
· Additional resources (sector resources)
· Libraries
· Innovative technologies
· Other P2Rx products – all other P2Rx products will be lumped together and reported here. Other products include, but are not limited to, Research Projects Database, Requests for Proposals database, Mercury reduction programs database, and Homes Across America database

SCOPE OF MEASUREMENT

Significant differences exist among Centers regarding internal use of their web sites – e.g., as an in-house resource for responding to clients requests for information. In order to be conservative in reporting, Centers should exclude activity on the site that originates from Center staff and affiliates, web site maintenance, and search engine “spiders.”

ADDITONAL COMMENTS ON STANDARD MEASUREMENTS

Each Center will provide quarterly web data from the previous six months to the national coordinator. In addition, to simplify tracking of reporting periods, beginning with FY00, measurement periods will be established to coincide with federal fiscal year quarters.

P2Rx recognizes our differences, and each Center will strive to gather the best data possible given its software, access to log files, etc.

Each Center will probably collect additional web measurement data individually. The web development subgroup has suggested additional measures (below) that each Center might find illuminating.

SUGGESTED SUPPLEMENTAL WEB MEASURES FOR A REGIONAL CENTER

In addition to the standard aggregate measures of web site use, the web development subgroup suggests that each Center might find the following measures helpful in managing its web site. These metrics are not required, but participating Centers are strongly urged to consider their use. Many of the web subgroup members are already working to collect this information about the use of their own web sites.

Top 10-50 Pages Requested or Files Downloaded (# TBD by each Center). This is a simple measurement of which resources (documents) on your site are most useful to your users.

Percent Error Rate on Requested Hits. This measurement is derived from the web log by measuring all requests for information that result in an error code (e.g., a “404-Page Not Found error”). This is a useful measure of overall site integrity and can often be useful as a tool for identifying outdated or incorrect links to a site.

Number Repeat Visits. This is often an indirect measurement but seeks to determine what percentage of users have visited your site previously. This is one indication of the value of the site to your users (based on the assumption that repeat visits indicate some degree of satisfaction with the site). Methods for determining repeat visits include the use of cookies, registration forms, or by indirect measurement of “Probable Repeat Visits”.

Top Referring Sites. A referring site is the web site URL last visited by the user prior to coming to your site. This can be a very useful measurement for determining how users are finding your site. Implementation of this metric will require the use of extended log format, including Referrer URL, by web servers.

Search engines as a % of total users. This is similar to the Referring sites measurement but focuses specifically on referrals from web search engines. This measurement is provided as an option on several of the commercial web log analysis applications. To the fullest extent possible, each Center should filter out its own use of its web site and search engine spiders, and report only statistics that are originating from outside the Center.

Back to P2Rx Administrative Information

TOP