P2Rx Admin Group Conference Call

Wednesday September 13, 2006

Attendees: Andy Bray and Rachel Colella (NEWMOA); Gary Hunt and John Calcagni (WRRC); Bob Iverson, Carolyn Knepp, Joy Scrogum, and Deb Jacobson, (GLRPPR); Thomas Vinson-Peng (Zerowaste); Rick Yoder (P2RIC); Mike Vogel and Elizabeth Bird (Peaks to Prairies); Kevin Dick (WRPPN); Chris Wiley (PPRC); Rob Guillemin (EPA R1); Phil Kaplan (EPA R5); Javier Balli (EPA R6); Chet McLaughlin (EPA R7); and Beth Anderson and Tom Tillman (EPA HQ).

Action Items:

• Beth – email listserv to summarize points from the call regarding the revised proposals (i.e. page limit, due date, budget issue, etc).
• Beth – call or email centers individually to discuss specific problems in their proposals.
• Centers – review individual proposals to prepare for group discussion on September 18^th.

Discussion:  Beth started off the by stressing that EPA HQ would like the Centers to revise their proposals to increase the focus on measuring outcomes.  Beth stated that the proposed work plans needed to be revised to more so reflect how outcomes would be measured and steps Centers would take to make progress in this area. 

Beth suggested that Centers use the ICR-approved OECA survey, making a few minor modifications to make it more P2-related.  Beth noted that the survey modifications would need to first be approved by OMB, but that if the changes were not great and the revised survey could be submitted for review under the existing ICR that it would have a much shorter review period.  Beth also suggested that the survey be kept short and include questions that focused on measuring the impact of P2Rx services.  Andy raised concern about using the existing survey and whether or not the scope of that survey would adequately answer the questions EPA and the Centers are asking about the efficacy of P2Rx services.  Beth urged the Centers to use the OECA survey in the short-term because it is readily available and can be put in the field much sooner than any new survey could be.  A new survey would require a full OMB review which could take upwards of nine months.

Andy reported on initial literature review done by NEWMOA to explore how other information/networking services and organizations have gone about measuring behavior change resulting from their services.   Early indications are that the connection between specific pieces of information and behavior changes is complex. 

Beth stated that EPA will not fund basic research into behavior change.  She stated that EPA is interested in behavior change as a result of knowledge gained and that to obtain this information Centers may have to survey the same people multiple times to connect the information provided to environmental outcomes.  Beth also stated that EPA is not interested in the Centers just making information more accessible and that Centers may need to modify their work to be able to more effectively measure the resulting outcomes.

The bottom line is that EPA is looking for Centers to link information provided to increased customer knowledge to environmental outcomes.  Rob stated that there should be some flexibility in how Centers go about demonstrating this link.  He suggested that Centers talk with state level P2 staff and solicit their help in conducting a survey or providing outreach. 

As far as revisions for the proposal, Beth stated that Centers needed to expand their work plans to further explain how they were going to conduct surveys.  Beth also stated that EPA wants to see more specific information, such as what types of questions will be in the survey, what behavior changes will be measured, and what outcomes the survey is designed to show.

Kevin stated that the objective of P2Rx is to make information accessible to other P2 programs and facilities so that they can make changes in their behavior to improve environmental outcomes.  He offered that ultimately the survey should be designed to measure the success in meeting that objective.

Rick said he knows there is a behavior change as a result of P2Rx, but is concerned over how to word any survey tool to document this change to EPA’s satisfaction.  Beth stated that Centers need to use instances where they have direct contact with their customers and suggested surveying people at regional meetings and other interactions as well as following-up with meeting attendees to better document outcomes. 

Elizabeth suggested that Centers survey clients of direct services such as “Rapid Response.” 

Chet suggested that Centers work with partners to devise a strategy to show causality between information provided and outcomes achieved and find a way for partners to give credit to the Centers for their contributions.  There was significant discussion on the difficulties of assessing this type of credit.  It was pointed out that it is, above other things, politically unpopular for Centers to claim any credit for the environmental outcomes when they are not the ones implementing the project.  Elizabeth asks if we could show correlation instead of causation.  Beth and Tom agree that this is probably sufficient. 

Chris offered that we could measure correlation by adding a survey question to the module that says “did you receive information from a P2Rx Center.”  Beth agrees that this is a good idea, but cautioned that customer satisfaction questions such as “did you like this website” or “did you find this useful” will not be helpful in showing impact.

Andy raised concerns about the potential for P2Rx services to be unfairly evaluated solely on a dollars spent/pounds reduced basis and that doing ignored the complexity of the behavior change continuum.  Tom suggested that we try to take advantage of services that have direct contact with end users when trying to measure behavior change.  Phil stated that unfortunately information service providers are being held at less value under the PART review of EPA programs.  In the long term he expects funding for information services to diminish.  He agreed that information sharing is a good thing and that P2Rx Centers should not abruptly change their role to doing direct assistance, but that Centers need to go beyond an information role and that conducting a survey is a good start towards determining what that changing role might be.

There was further discussion of the need for Centers to cut back on other tasks to allow for more resources to be committed to measurement.  Beth requested that Centers attribute a dollar amount or percentage to each task in the proposal so that EPA can make more informed funding decisions regarding the revised work plans.  Phil stated that EPA just wants to make sure that the costs are appropriate and reasonable.  Rob offered that it might be difficult for Centers to arrive at an estimate of the level of resources to commit to measurement because it is a new project for Centers.  Beth reported that in the past other programs have dedicated approximately 10-20% of their budget towards measurement. 

Beth also stated that all Centers should remove the “Calculator Clearinghouse” task from their proposals because it was going to be funded in a separate grant to NPPR.  Beth offered that Centers could commit these resources to spending more time on other outcome measurement-related tasks.

Beth questioned the need for each Center to include the “Regional Summary Report” task in their proposal and suggested having only one Center perform that task.  Andy offered that PPRC could generate the programming that would pull the necessary numbers from the Regional Modules of the Results Data System but that the proposed task also included some explanatory narrative to accompany the regional numbers and that portion of the work was better suited for individual Centers. 

Beth also raised concern about the need for the proposed “Best References” resource given the existing Topic Hubs.  Andy explained that these would be two different products geared towards different audiences and that the Best References resource is being proposed as a way to better serve the regulated community be streamlining access to the resources that go into building the Topic Hubs.  Beth stated that the distinction was not clear and that the Centers need to better explain these differences. 

Centers were urged to show that their web-based products (i.e. Topic Hubs, News, etc.) are having an environmental impact  are worth the investment.  Elizabeth suggested conducting a web-based survey for every P2Rx user.  The survey would automatically pop-up when users were logged on and they could choose whether or not to complete it.  Beth agreed that this is a good idea.  There was additional discussion and some skepticism among the group of achieving meaningful response rates.

Because EPA is asking Centers to expand certain areas of their proposals, Beth said the revised proposals could be up to 15 pages in length.  The revised proposals will be due on October 11^th and if a grant is awarded, the money should be available by the end of December.  Centers should submit the revised proposal to Beth via email.

Beth mentioned that not all Centers referred to EPA Strategic Goal #5 in their proposal and they need to make sure to do so when they re-submit.  Beth also noted that Centers need to review their budgets.  Right now the proposals are $24,000 over budget and therefore some Centers may have tasks cut.  Beth will call each center individually to discuss their budget proposals.

Andy asked about the end date for the grant money --- in the RFP it stated September 30, 2007 as the date for all work to be completed.  Chet stated that the RFP should be revised because the end date will depend on when the grants are awarded.  The grant will last for a 12-month period beginning on the date that the grant is awarded.

P2Rx Center Discussion: After the EPA Coordinators got off the line, the P2Rx Center Directors continued a brief discussion about how they are going to complete the requested revisions.  The group agreed that all Centers should get equal funding.  Andy stated that for the Centers to get equal grant money, each center should request a budget of $104,250 in their proposals.  This is based on the EPA’s budget of $722,000 divided evenly among the Centers seven ways; Zero Waste is not included in this because they are on a different funding cycle. 

The group also agreed that Centers should bundle tasks together when individual tasks don’t require a lot of money (i.e. <5% of effort).  For example, Mike stated that other Centers’ contributions to “Homes Across America” project was only about 1% of their budget and Gary said same thing was true for Centers’ contributions to the “National Library” at WRRC.

Next Call: The P2Rx Center Directors will have a follow-up call on Monday, September 18^th at 2:15pm ET to brainstorm and discuss Center revisions for their proposals.