Self-evaluation and improvement
In this page
- The evaluation process
- Initial planning
- What is evidence?
- Developing effective questions
Evaluation is the process of asking questions to understand the difference youth work is making for young people. We evaluate to help us to make our youth work as effective as it can be for young people.
An outcome is the change or difference you are trying to make.
An indicator is the thing you measure that helps you to understand a young person is achieving the outcome.
Evidence is the information you gather to help you understand if the indicator is happening.
Evaluation Support Scotland outlines a simple four-stage evaluation pathway.
Stage 1: Setting outcomes and indicators - the Youth Work Outcomes and Indicators help you do this.
Stage 2: Collecting evidence – this toolkit provides lots of ideas for way you can collect data and links to other toolkits and websites.
Stage 3: Analysing and Reporting – making sense of the information you have gathered, looking for trends common themes and explanations for why certain outcomes or changes have (or haven’t) happened.
Stage 4: Acting on your learning – this is a critical stage in the evaluation pathway but is sometimes forgotten. An evaluation is only valuable if it helps you to change and improve your practice.
You can find out more about the evaluation pathway on the Evaluation Support Scotland website. You can also download our 'Golden rules of evaluation' guide here.
Evaluation should be included in your initial planning. Think of evaluation as a core part of the planning cycle. The findings from an evaluation should be used to inform future practice.
Including evaluation in initial planning means that information can be gathered from young people at the start of their participation in youth work. This is called a baseline.
Why is a baseline important?
If you want to understand how young people have changed and developed through youth work, then you need to know where they were at the start of their involvement. For specific projects you would gather information at the start of the project, end of the project and depending on the length of the project, you might want to collect data at various other points in between. If you want to understand the long-term impact of a youth work project, you can gather information from young people months or even years after the project has ended.
For youth work that doesn’t have a specific start and end, track young people at regular intervals throughout their youth work journey. If you didn’t ask young people to do a baseline measurement at the start of their engagement, you can ask them to reflect back on how they would have rated themselves at the start. It's not as accurate as actually asking them at the start but can still be very helpful for young people to reflect on their journey.
Evidence is the information you gather to help you understand if the indicator is happening. Young people can choose to evidence their learning through youth work in a whole range of ways. Evaluation needs to be receptive and sensitive to this.
Evidence might be:
- Quantitative - quantitative information can tell us what has happened. It is usually concerned with numbers. How many? How often?
- Qualitative – qualitative information is often concerned with why things happened. Qualitative information helps tell the story behind the numbers. It is concerned with words, thoughts, opinions and explanations. Qualitative information can take more time to analyse and interpret but is a critical part of evaluation. Evaluation Support Scotland has produced a guide on using qualitative data in evaluations.
- The records or outputs of young people’s engagement. This could be photos, films, blogs, other media or changes to policies/services to name but a few.
How to gather evidence?
Indicators help us to understand how well we are progressing towards the outcome. There are different ways to gather evidence which would help us know if the indicator is happening, such as:
- Gathering statistics put together by other departments or organisations
- Questionnaires for young people
- 1-2-1 sessions or focus groups/workshops with young people
- Diaries completed by young people
- Questionnaires for youth workers or other partners
- Structured observations made by youth workers
- External observations and assessments (external evaluation)
- Outputs of activity
In your self-evaluation, the questions you ask as are as important as, if not more important than, the way you choose to gather the information. It’s worth spending the time working and testing your questions and thinking about the type of information they're likely to give you.
Questions to avoid
- Long questions: a young person may only remember one part of the question and respond only to that part.
- Multiple questions, like "what do you feel about the situation of drug-taking amongst young people today compared with that of five years ago?" The solution is to break it down into a series of simple questions. Each question should only ask one thing.
- Ambiguous questions. For example, ‘drug’ could be interpreted in many ways - prescriptions, over-the-counter medicines, illegal substances and coffee are all drugs. So distinguish between different types of drugs.
- Questions containing jargon or complicated language. For example, "Would you consider yourself as having a NIMBY attitude?" would not be an appropriate question.
- Leading questions. Try to make sure that the respondent can give any answer without feeling they are giving the wrong answer. "Do you agree or disagree with the government’s harsh welfare policy?" is an example of a leading question.
- Save the Children have a helpful resource on creating questions for children and young people.