P2Rx Admin Group Conference Call

Wednesday, October 10, 2007

Attendees: Andy Bray and Rachel Colella (NEWMOA); Gary Hunt and Curt McCracken (WRRC); Bob Iverson (GLRPPR); Thomas Vinson and Lynn Turner (Zero Waste); Rick Yoder (P2 RIC); Mike Vogel and Elizabeth Bird (Peaks to Prairies); Ed Gonzalez (WRPPN); Chris Wiley (PPRC); Jean Waters (2008 National Coordinator); Laura Bollinger, Andrea Hramits, and Julie Shannon (EPA); and Jocelyn Segal and Lissa Lynch (Abt).

Action Items:

• All Centers – be prepared to launch the surveys on Monday, October 15, 2007.
• All Centers – submit write-ups for 2008 Calendar ASAP.

P2Rx Program Evaluation:  EPA discussed their progress on the evaluation for the P2Rx program.  EPA’s Pollution Prevention Division (PPD) and Office of Policy, Economics, and Innovation (OPEI), along with EPA’s contractor have finalized the questions for this evaluation.

Final Evaluation Questions – Andrea previously set up a system in Survey Monkey to allow P2Rx Coordinators and EPA stakeholders to rank the evaluation questions according to priority.  Although Andrea could not see who ranked these questions, 10 people participated.  Questions identified in the final evaluation as high priority were those questions which had 50% or more of the participants ranking them as high priority.

Thomas was surprised that Question #9, which was related to the grant process, was not ranked as a high priority question – he thought that the Centers would have ranked this question as important.  Andrea stated that 40% of the participants ranked this question as high and 20% as medium-priority.  Some of the comments noted by participants in Survey Monkey were that this question was closely related to Question #1 and they could possibly be combined.  Jocelyn noted that the reason these questions are listed separately is because Question #9 is directly focused on the grant itself, while Question #1 is more process-oriented.

Note: during the Centers-only portion of the call, only four P2Rx Centers reported participating in the ranking of the evaluation questions on Survey Monkey, therefore, over half of the responses were from EPA.

Matrix – Jocelyn reviewed the draft matrix, which identifies the evaluation questions (left side) and possible data sources (top) that they can use when evaluating the P2Rx program.  She stated that she was hoping to get some additional information from the P2Rx Centers, particularly any meeting minutes that include discussions related to the types of questions in the evaluation; local surveys or other methods that Centers use to assess clients’ needs; and semi-annual activity and web measures broken down by Center.

Thomas asked if the data in the P2 Results Data System would also be helpful.  However, Elizabeth noted that we are not able to link state data to P2Rx activities when evaluating the program – the data is currently aggregated and would need to be separated out to evaluate the specific P2Rx activities.  Furthermore, Apt does not have direct access to this data.  At first Jocelyn felt that Centers would be able to include data from the P2 Results System if they felt it was helpful.  She noted that Apt will finish collecting the data in April and start analyzing it in May – as long as we know the data measures from P2 Results Data System now, Apt create a “place-holder” and can incorporate them into the report.  Andy stated that although we are able to know the data points now, we will not be able to know the volume of data that we will have by April.  Chris noted that many Centers are only starting to collect data now (October 1^st) and therefore, may not have a full data set in only 6 months.  Gary stated that WRRC is not posed to start collecting data until January 1^st and therefore, they would have even less data to use.  After some discussion, Jocelyn stated that she was reluctant to use this data because it is brand new and the data quality would not be great – she also noted that the difference between Centers would affect the analysis.

Because the data among the Centers is variable, Jocelyn stated that she would like to find out from each Center individually, the data sources and documentation that they have available.  She considers this an interim step that will also help develop the matrix.  She also noted that she would be doing interviews with Centers in the upcoming weeks to find out more information.

P2Rx and Center Surveys:  Andy stated that Center surveys are now ready to be launched – all revisions to the survey have been made and the Web Group is prepped and ready to post it on the Center websites.  Although the placement of the survey will vary slightly between the websites (top banner, left hand navigation, etc), the link text and graphic will be the same across all Centers’ sites.  The survey will be launched on Monday, October 15, 2007.   

Survey Distribution Log – Gary stated that he was concerned about the success of the web survey because in past surveys, WRRC has not had sufficient responses.  Andrea hoped that putting the survey on every page the website would increase the response rate.  She also noted that Centers are allowed to solicit survey responses other ways, as long as they fill out the survey distribution log to identify any potential bias.   

Because the survey needs to stay anonymous, the hard-copy versions that are handed out at meetings and/or training events should be submitted to Andrea.  However, Andy stated that because the surveys are on NEWMOA’s Survey Monkey account, we are the only Center who has access to the survey data – Andrea will not be able to enter hard-copy data into the system.  Andrea stated that she would enter the hard-copy data into an EPA database and then combine their database with NEWMOA’s database.  However, Elizabeth also noted that every survey is slightly different – Centers were allowed to list additional products or services for one of the questions.  Therefore, there are 9 distinct surveys and databases in Survey Monkey.  Andy and Andrea agreed to talk about these issues further offline.

Introductory “Script” – The purpose of the survey “script” that EPA developed is that it is a uniform way to present the survey, which helps reduce bias by framing the introduction so as not to influence people’s responses.  Andrea recommended that Centers introduce themselves at the beginning of the event where they are handing the survey out at and give people enough time at the end of the event to fill out the survey without distraction.

Andrea stated that EPA has been developing evaluation questions to present to focus groups at the upcoming WRPPN conference.  Ed stated that there will be 3 focus groups, with no more than 9 people in each.

Andy stated that the new survey “script” has also been added to the online versions of the survey and that the links to the survey will remain the same.  Rachel sent the revised hard-copy version of the survey to Centers via email earlier today.

Centers

Next P2Rx Coordinators Meeting:  The Centers agreed that the January timeframe is appropriate for the next face-to-face meeting.  Since it will be winter, a few people suggested we hold the meeting in a warm climate, such as Florida or Las Vegas.  Others suggested that we chose a location near a ski resort, such as Yosemite, Reno (near Lake Tahoe), or Bozeman, MT.  Centers will email location ideas to NEWMOA and we will vote on it.

2008 P2Rx Calendar:  Most of the Centers do not think that they will make the October 15^th deadline for their submissions.  Obviously, the sooner the write-ups are submitted, the sooner the calendar can be completed and supplied to our customers.