Learning, Doing and Catalyzing: The National Indicators Team 2014 Highlights

Introduction

The National CRED Indicators Workgroup, a multi-regional team, self-organized in May 2012 to explore the nature and interest in impact indicators and evaluation occurring at multi-state regional levels. The CRED program leaders in the North Central, Southern, and Northeastern regions were at various stages of identifying, reporting and amalgamating the collective outcomes of community economic development programs. All four regions recognized the need for sharing compelling stories about the collective impacts of our community and economic development work, at local, state, regional and national levels.

The Team

Mary Simon Leuci, Walt Whitmer, Charlie French, Paul Lachapelle, Susan Jakes, Deborah Tootle, Scott Chazdon , Rachel Wellborn, Aida Balsano, and Allison Nichols.

"We as a state [New Hampshire] focus a lot of energy on attracting new industry and sometimes forget that more jobs are created by existing businesses than by those that relocate here. Sharing the impacts of your Business Retention and Expansion Program (BR&E) really tells the story of how important it is to help existing businesses retain and grow jobs."

Benefits of National Collaboration

Community and Economic Development programming continues to be diverse and relatively small component of Extension nationally. The national group's efforts is reinvigorating regional efforts that will help us, locally, regionally and nationally better speak to both the goals and impact of the program. We have worked and learned together, we have each increased our capacity to evaluate and report, we have raised the credibility of CRED, we have managed to keep this alive through changes in leadership for extension directors and program leaders, and we have gained attention from our extension directors and nationally. While several of the regions are not to the point of regional indicator collection that is ready to be reported, the process has been helpful in talking and thinking more about the indicators of our work.

2014-2015

Northeast Regional Update

• Identification of 7 high-potential impact indicators (drawing from 2013 survey that was conducted of 40 NE staff to identify each state's evaluation capacity, areas of focus, and indicators for which they currently collect data on, or would like to collect data on).
• Several Northeast states received training from by Dr. Scott Chazdon to implement Ripple Effect Mapping (REM), which is a qualitative, visual approach to capturing the perceived direct and indirect impacts of a specific program on the target audience.
  Hiring of the inaugural NERCRD Evaluation Fellow
• Re-invigoration of NE Evaluation working group and initiation of pilot indicators project with 6 states (PA, VT, NY, NH, WV, ME).

North Central Regional Update

• We are currently compiling the 6th year (FY2014) of collective impacts report for the region, to be released by June 2015. For the past several years all 12 states have reported on at least one or two indicators. Our effort is joint effort between CRED program leaders and NCRCRD.
• State extension directors continue to support an aggregated report
• State extension directors are supporting use of vignettes we develop and quantitate impacts as part of narrative development for entries in Excellence in Extension Impacts.
• The North Central Report is being used within NIFA and with Congress to build support for CRED programming as well as for RRDCs.

Southern Regional Update

• Has does extensive refinement of indicators since first year trial data gathering
• Has again sent our request of data across region
• 29 universities due to both 18662 and 1890 LGUs in south including University of Puerto Rico and University of Virgin Islands - Very large, diverse region
• Have limited indicators to ones we feel are real impacts and more quantitative, but have also developed vignette instructions to give greater detail and nuance

Western Region Update

• New survey instrument developed and distributed to 36 active NACDEP members in Western Region
• Team of 5 agents and specialists formed to collect, analyze and refine common indicators.
• Western Rural Development Center (WRDC) assisting with coordination of common indicators.
• Focus on collecting Public Value Statements that reflect diverse and unique Western Region cultural and natural characteristics

In Pilot Grove, Missouri, access to a grocery store was 20 miles away; so 23 investors put up $320,000 to form the limited liability corporation that built a store. Tyler's Market opened in November 2013, employs four local residents and serves as a new social hub in the community. "It's heaven sent for us, because at our age it's very difficult to get out and get groceries," says longtime Pilot Grove resident Pat Dill, 81.

Regional Challenges

• Northeast: Multi-state collaboration and coordination
• North Central: We are discussing how and if we can use the data from recent years as a starting point for probing and understanding where specific states are having success, then using that to spur cross-state transfer of impactful programs. We have had a number of program leaders change since we began our initial work in 2007 and we need more non-economic impact indicators.
• South: With 29 different Universities and very few FTE, we are struggling with many different reporting systems that do not collect the data we need. Some states with little to no county Community development staff are only collecting state level work.
• West: How to work with and share info with colleagues in the other regions.

Strategies

• Northeast: Our pilot project is designed to address coordination. We want to continue to learn from each other's experiences refining key indicators, exploring tools to better assess CRED impact, and developing processes and agreements for aggregating across states and regions.
• North Central: We are setting up a working face-to-face meeting to delve into these issues that will include learning from National CRED Indicators Work Group for and looking at best practices from other regions and from our evaluators. We also invite an evaluator or two to join our meetings.
• South: The more tools that can be developed the better. Working toward a national list - even though some regions may not report on something is a good idea. Little variations in wording can mean very different collection.
• West: The Western Team will be updating the website with info and samples of indicators and evaluative tools used in states across the US, developing a national survey, and discussing the long and short-term goals and objectives with national team.

Developing indicators

Lessons Learned

• States define and program CRED very differently across the region (and country for that matter) and therefore finding consistent and widely shared metrics is a slow process requiring a substantial time commitment and buy-in from the 'right' people in each state
• Varied processes and systems for collecting impact data of all kinds across states makes aggregating impact data a challenge that we will have to continue to work through
• There remains limited common understanding of the language, tools and methods for collecting long term impact data in the field and over time across the region. There is very little in the way of a widely shared clearinghouse of information and best practices
• We have learned you don't wait until you have it perfect, you collect what you can and it's better to stand firm on numbers you have, knowing they are low and underreported.
• We recognized we raise the tide for all of us by preparing a collective report, and it's appropriate to indicate how many states reported a specific indicator.
• There are similar challenges across regions
• We do need national dialogue about what are the most appropriate indicators of social and community change beyond economics, where is the research base and how might we create measures that work for assembling meta data.
• We need to work around limited capacity to do something, not matter how small.

Hal Chase summarizing the outcomes after a process of CES led strategic planning for the Fort Des Moines Museum and Education Center (Iowa), the first Officer Training site for African-American soldiers in the United States.