PHASE TWO: ACTION
Step 03
Planning and Focusing your Evaluation

So you’ve made it through your pre-evaluation check up and have a better sense of your program’s logic model. You have discovered your evaluation assets and have figured out your capacity to embark on an evaluation journey, but wait – your adventure to evaluation legacy does not stop there.

In this step, you will begin to gather your stakeholders and develop a road map for your evaluation and provide your organization with a clear sense of the “who-what-how-when” of your program’s evaluation plan.

An evaluation plan is a roadmap that outlines your program evaluation activities, and is based on your program priorities, skills, resources, and time. It includes a description of program activities and goals, a description of the purpose of the evaluation, a list of key stakeholders, evaluation questions (process and outcome), a description of the evaluation methodology and proposed timeline, data collection plan, analysis and interpretation plan, as well as use and sharing plans.

Who will manage the evaluation and collect the data?
What stakeholders need to be involved in your evaluation?
How will you analyze the report and communicate your findings?
When should important steps in your evaluation take place?

TAKEAWAY FROM STEP 03

An evaluation plan is a roadmap for your evaluation journey.

Key Actions

01

Focus Your Evaluation

Here you will set the evaluation focus by establishing priorities for your evaluation by determining what aspects of your program need to be evaluated and who will use your results. In order to do this, you need to ask yourself the following questions. These questions have been found to be effective for evaluating grassroots youth sector programs:

Focus 1
Focus 2

02

Begin to think about the evaluation questions you need to answer, and clearly consider what your program needs to know from the evaluation to help narrow the focus.

Process Evaluation: Process indicators concern how a program is operating.

Some examples of process evaluation questions are: To what degree were program activities accomplished? What was the quality of program components? How well were program activities implemented? Were staff adequately trained? Was the target population reached? Were participants satisfied with the program activities? Was the location and timing of the program accessible for participants? How did external factors influence program delivery? Were there barriers/ common problems to program implementation? How were barriers addressed? Was the program well supported (financially and otherwise)? Does the program differ from the original plan? What program changes were made?

Outcome Evaluation: Outcome indicators concern the changes in the youth as a result of their participation in the program, and as noted in the logic model, can be broken down into short-term outcomes (6-12 months; immediate changes in knowledge, awareness, attitudes, skills) and medium-term outcomes (1-2 years; changes in behaviour) to long-term outcomes (3-5 years; changes in the broader community, population).

Some examples of outcome evaluation questions are: To what extent did participants’ level of knowledge improve? Did their attitudes change? Did they develop new skills? To what extent did participants change their behaviour or actions? Has this new behaviour been maintained and for how long? To what extent has the target population’s condition or situation improved?

03

Plan for Data Collection

Creating a plan for how you will collect your evaluation data is a critical component of your evaluation journey. At this step, you will need to begin to determine what information sources you need, what data is available, and what type of methodology you will need to use. Here are some questions you can ask to help create your data collection plan.

  1. What type of evaluation am I doing (process, outcome, or both)?
  2. What are my evaluation questions?
  3. What type of evaluation design and methodology should I use?
  4. What data collection methods should I use?
  5. What are my specific outcome indicators?
  6. How will I analyze my data?

You can also use an evaluation plan methods grid to help create a more visual representation of your data collection plan.

03

Plan for Conclusions and to Share Lessons

The planning stage is a good time to start figuring out how the data will be contextualized to draw conclusions and how your program intends to use what they learn from the evaluation to improve their program and inform their stakeholders.

Questions to consider:

  • Who within the organization will be involved in the conversation about evaluation results?
  • When, and how often, will results be shared internally?
  • Who will decide how programs should be changed as a result of the evaluation?
  • With which stakeholders (e.g., youth, community, funders) will the organization share results?
  • Which setting and method (e.g., written report, focus group, community presentation, video, etc.) will be the most appropriate forum in which to share results with each stakeholder group
  • How will stakeholder feedback be incorporated into decisions about changes to the program?

04

Plan for Conclusions and to Share Lessons

The planning stage is a good time to start figuring out how the data will be contextualized to draw conclusions and how your program intends to use what they learn from the evaluation to improve their program and inform their stakeholders. Questions to consider:

  • Who within the organization will be involved in the conversation about evaluation results?
  • When, and how often, will results be shared internally?
  • Who will decide how programs should be changed as a result of the evaluation?
  • With which stakeholders (e.g., youth, community, funders) will the organization share results?
  • Which setting and method (e.g., written report, focus group, community presentation, video, etc.) will be the most appropriate forum in which to share results with each stakeholder group
  • How will stakeholder feedback be incorporated into decisions about changes to the program?
Hot Tips

There are different methodologies for analyzing quantitative and qualitative data, so be sure to be mindful of your program’s capacities when selecting/designing your data collection tools.

Q&A

Why should I use an evaluation plan?

With an evaluation plan, you can ensure that everyone in your program is aware of what is being evaluated and is informed on the “who-what-how-when” of your evaluation journey. It also helps foster clear direction on program priorities while keeping your stakeholders in the loop on the steps you are taking to ensure that your program endeavours are accomplished and the needs of your program participants are being met. Through an evaluation plan, you can create a shared sense of purpose within your team on program objectives and it will also help evaluation processes to continue during times of transition and staff change.

Who should be involved in the evaluation of my youth program?

Representatives from many stakeholder groups can collaborate to support an evaluation: board members, management, staff, parents, youth, volunteers and community members. Involving stakeholders generates buy-in for evaluation efforts and will help focus the evaluation on the program’s goals and activities. Stakeholders should have the opportunity to express their goals for the project and understand how program outcomes and decision-making are connected to the evaluation. It may not be feasible or productive to have all stakeholders attend every meeting. As an evaluation team, you will need to determine the critical players and get the right people to the table as you plan meetings and activities.

Tools / Templates / Checklists

Reflect and Improve: A Tool Kit for Engaging Youth and Adults as Partners in Program Evaluation

This tool provides resources to engage young people and adults in the evaluation of community-building initiatives. It was developed from a participatory, empowerment-oriented perspective.

Source: Innovation Center for Community & Youth Development
DOWNLOAD THIS TOOL

A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions

This guide aims to assist evaluators and their clients in the process of engaging stakeholders.

Source: Robert Wood Johnson Foundation Evaluation Series
DOWNLOAD THIS CHECKLIST

Elevator Activity

Use this activity to determine how you would describe your program to a potential stakeholder.

Source: Atlantic Centre of Excellence for Women’s Health
DOWNLOAD THIS ACTIVITY

YouthREX's 10 Step Evaluation Plan

To be added in

Source: YouthREX
DOWNLOAD THIS TOOL
Learn More...

Watch: Developing an Evaluation Plan

Read: Evaluating a Strategy: Asking Good Questions

Read: Good Evaluation Questions: A Checklist to Help Focus Your Evaluation

CHECK IT OUT

“Having a written evaluation plan will foster transparency and ensure that stakeholders are on the same page with regards to purpose, use, and users of the evaluation results. Moreover, use of evaluation results is not something that can be helped or wished for but must be planned, directed and intentional.”

Developing an Effective Evaluation Plan (National Center for Chronic Disease Prevention and Health Promotion)