PHASE ONE: DISCOVERY
Step 01
Pre-Evaluation Check Up

Assess your organization’s capacity to embark on its evaluation journey.

The Pre-Evaluation Check Up will capture information about your organization, the context that it is operating in, and staff capacity for evaluation. This includes taking inventory of existing organizational evaluation assets and resources (e.g., an existing intake survey, a staff member who is familiar with evaluation), and noting what additional supports and resources are needed in order to undertake an evaluation. 

The Check Up will also help your organization identify the right areas to focus on for your evaluation, and potential evaluation goals.

What are your evaluation assets?
What resources do you need to successfully complete this journey?
What is your capacity to undertake this journey?
Where is your organization on your evaluation journey?

TAKEAWAY FROM STEP 01

Step 01 will help your organization co-create a shared understanding of its capacity to engage in program evaluation, select key stakeholders, and decide whether an internal or external evaluator should be used.

Key Actions

01

Start with an Evaluation Capacity Review to understand what assets your program has in place and what resources it needs to support your evaluation. Check out the Tools / Templates / Checklists section for resources for doing this.

02

Identify which of your stakeholders you should involve in the evaluation process. Check out the Stakeholder Identification and Analysis technique in the Tool/Templates /Checklist section.

03

Determine appropriate strategies for engaging with individual stakeholders.

04

Remember, meaningfully involving youth participants strengthens evaluations of youth programs.

Hot Tips

Complete a stakeholder analysis before you begin your stakeholder engagement process.

Pay attention to their interests and expectations, their powers, their interrelationships, and the various roles they might play. Don’t forget that youth are an important stakeholder group!

Chat with your staff and take inventory of existing organizational evaluation assets and resources – you may have more capacity than you realize!

Q&A

Who are our program stakeholders?

Stakeholders are individuals, groups, or organizations that can impact or may be impacted by the evaluation process or findings from an evaluation. The term “Key Stakeholders” is used to refer to individuals and groups who will be using the evaluation findings to make decisions.

An exhaustive stakeholder list includes organization staff, program participants, past participants, parents/family, informal supports (e.g., faith leaders), teachers/principals/school board, police/criminal justice, youth advisory groups, partner agencies/groups, funders, policy makers, any other stakeholders staff recommend, as well as the evaluation team. Although it would be great to include all of these unique voices, it is not always possible – or appropriate – to solicit input from all potential stakeholders in a program evaluation.

Why should we engage stakeholders?

Stakeholders provide unique perspectives and program insights, and can be engaged with at different points in an evaluation for different reasons. For example, seasoned program staff can likely articulate the broader social and cultural context that a program is operating in, while youth participants could potentially be able to provide feedback on the appropriateness of a particular outcome measure.

Overall, engaging different stakeholders in your evaluation..:

  • Contributes to the development of an accurate and rich understanding of a program through logic modelling
  • Enhances evaluation planning, design, and implementation through a variety of perspectives
  • Facilitates accurate interpretation of results

Contribution to the evaluation by stakeholders also increases the likelihood that evaluation findings will be used.

Why should we engage youth participants in our evaluation?

Engaging youth in your program evaluation can provide direct benefits to your evaluation, as well as their own professional and emotional development. For youth, it can enhance their skills (e.g., research, public speaking), social competencies (e.g., empathy, teamwork), self-confidence, civic competencies, and social capital (i.e., connections and relationships).

For the evaluation itself, youth involvement can:

  • Improve the overall quality of the research and evaluation process (e.g., evaluation tools)
  • Facilitate accurate interpretation of results/youth voice
  • Contribute to validating the diverse experiences of youth
  • Promote power sharing power between youth and adults
  • Improve programming through youth input/feedback

Should we use an internal or external evaluator?

To decide whether internal or external evaluator or both, ask yourself these questions:

  • Does your program have funds designated for evaluation purposes?
  • Have you successfully conducted evaluations of similar programs or services in the past?
  • Are existing program practices and information collection forms useful for evaluation purposes?
  • Can you collect evaluation information as part of your regular program operations (at intake, program end)?
  • Are there program staff who have training and experience in evaluation-related tasks?
  • Are there advisory board members who have training and experience in evaluation-related tasks?
  • Are you able to objectively and ethically research your own program?
  • Are you able to design a program evaluation that will yield usable results?

If you answered no to some or most of these questions, you may want to consider using an external evaluator.

Tools / Templates / Checklists

Assessing Your Readiness for Evaluation

This tool can be used to elicit and address beliefs and attitudes about evaluation. If you are concerned that many participants may hold negative beliefs and attitudes, you can address these from the start.

Source: Evaluation Toolkit by Magnet Schools
DOWNLOAD THIS TOOL

Assessing your Evaluation Capacity

The following is a means for establishing an aggregate inventory of technical resources available and self reported capacity of staff to perform a range of standard evaluation related tasks.

Source: JDC Partnerships
DOWNLOAD THIS CHECKLIST

Readiness for Evaluation Metaphor Activity

You can use the “Metaphor Activity” table to show how evaluation can support magnet program implementation.

Source: Adapted from a template by the Evaluation Toolkit by Magnet Schools
DOWNLOAD THIS ACTIVITY

Identifying Stakeholders and their Roles Worksheet

This worksheet is intended to guide evaluators and program personnel in reflection and decision making around involving stakeholders in an evaluation.

Source: The Evaluation Center, Western Michigan University
DOWNLOAD THIS TEMPLATE

Creating Gracious Spaces in which Youth and Adults are Valued

These questions can begin the discussion (either as a large group or in pairs) about what it means to be a young person and how to best create partnerships based on equal voice.

Source: Innovation Center for Community & Youth Development
DOWNLOAD THIS ACTIVITY
Learn More...

Report: Building Evaluation Capacity in Human Service Organizations

by James Bell Associates

DOWNLOAD HERE
CHECK IT OUT

“Youth are not merely respondents of methods created by adults nor are they consultants on adult-led evaluations. Instead they are creators of knowledge, shaping their own methods, analyzing the data gathered, and reporting their findings.”

Dr. Kim Sabo Flores
Youth Engagement in Evaluation: A Continuum from collaboration to Youth-Led