Ensuring media training has the required impact

The evaluation process is set in motion the moment you are engaged by the media organisation you are being asked to help.

The first questions you ask in your research, or discovery phase, will form the framework for your evaluation of the intervention. Keeping accurate notes is essential if you are to do the job well.

You will have been invited to help a media organisation because senior managers have a desired outcome and see training as the way to achieve it.

They will have their objectives and, usually, a clear ROI (return on investment).

You need to understand those because they will form some of the measurements you will use to evaluate the project. But there will be more.

As set out in the module ‘Researching a media training project’ your job is to design training that will deliver what the business wants, but also to question presumptions and assumptions and, where appropriate, introduce new perspectives.

These, too, if accepted by the key stakeholders, will form an important part of your evaluation process.

With evaluation you are trying to establish:

• Validation: was it the right choice.
• Impact: was it effective?
• Benefits: what was the result for the business? ROI: was it time, effort and money well spent in terms of return on investment?

To help you evaluate, it is useful to have three measurements:

1. the beginning of the project.
2. the middle of the project.
3. the end of the project.

For a training course you may only have two of these, the beginning and the end, although the feedback forms in point 3 (below) will give you some mid-way measurements.

1: Evaluation starts when you are researching the event

As mentioned above, your research phase will produce valuable information about what is expected and what needs to change.

You will have the initial management brief, with, hopefully, clear deliverables, and you will have any added value you bring to the project.

The main points, listed in the research module, include:

• Understanding the needs of the business.
• Clarifying the importance of your intervention.
• Ensuring all key stakeholders agree the business objectives.
• Watching, listening and absorbing the business from management to newsroom and in the field.
• Challenging all assumptions at all levels.
• Reviewing and assessing what already exists.
• Agreeing and organising a skills audit and questionnaire.
• Talking to business competitors and the media organisation’s audience.

2: Evaluation elements are contained in course design

All these steps listed above will have produced a framework that you will have taken to the design phase, see the module ‘Designing a successful media training plan’.This proposed course design will then have been signed off by key stakeholders. This is where you set out how the agreed deliverables will be approached and how you plan to prepare staff to be able to deliver.

Each training module will have an objective such as ‘by the end of this session you will be able to ….’. This is another important piece of information for your course evaluation.

3: Feedback forms offer some level of evaluation

The course feedback forms, which you handed out at the end of the event, provide valuable evaluation of how the training was received – but not about how it will be implemented.

It’s important to analyse these forms and offer the main findings to the stakeholders. They will need to ensure that there are the opportunities available for course attendees to put into practice the training you have given them.

If they don’t enable this to happen it will be hard to evaluate the success. That is why the agreement reached in point 1 (above) is so important. This feedback can help you with some mid-way evaluation because you can see how trainees had progressed after they were introduced to the concepts and before they implemented them.

4: Changes in the workplace

You then need to see what changes your training has brought to the workplace. This will involve careful monitoring, talking to staff, unit managers, senior managers, and those promoting new business for the organisation.

You will be looking for the positive and negative impact on the following:

• Changes in staff behaviour.
• The impact on workflows and output.
• What this means for unit and organisational results.

You need to spend enough time examining how changes have impacted the business – it may take several years.