Review of the Danish Evaluation Institute

Download 241 Kb.
Size241 Kb.
1   ...   57   58   59   60   61   62   63   64   ...   86

The evaluations

The EVA Act and the EVA Ordinance lay down certain parameters for the implementation of action plan evaluations: they must include self-evaluation and information about the implementation of the evaluations must be provided for all institutions taking part. An external expert panel is to be responsible for appraisal of the self-evaluation reports and for the contents of the official evaluation report. The institutions evaluated are to be provided with an opportunity to react to the report before it is published.

During the site visit several members of EVA’s staff considered that these methodological requirements were an unnecessary drain on EVA’s resources. Others considered it positive that the act laid down certain methodological requirements. These function as a guarantee of quality and help to enhance EVA’s legitimacy.

The expert panel can affirm that EVA’s evaluations are carried out in accordance with international practice. It finds it difficult, however, to understand why the evaluation methodology should be stipulated by law. This limits EVA’s scope to “develop and renew methods for evaluation and quality development” as laid down in its mandate.

Site visits are not obligatory but are nevertheless included in all of the evaluations and often pilot studies and end-user surveys are conducted as well. EVA is not responsible for follow-up. Here responsibilities lie with the ministries and/or the individual educational institutions.

Differentiation between the evaluation methods for the various educational sectors has taken place. In normal cases an evaluation takes about one year to complete.

Pilot studies

In general a pilot study is undertaken in a field before the terms of reference for a project are determined by EVA’s board. One permanent feature in these studies consists of meetings with stakeholders in which the project team probes which specific issues need to be spotlighted and what the political agenda in the area looks like. Experiences from these meetings were described in the self-evaluation as very positive.

Expert panels

An expert panel is then appointed, comprising 4-5 experts per evaluation. Altogether about 50 experts are engaged each year. The panels are put together on the basis of a requirement profile drawn up in collaboration between the project team and EVA’s management and which is included in the terms of reference for the project. Certain requirements apply for all projects, for instance that panels must include women and men, as well as Nordic representatives and employers representatives.

As a rule there are no student or pupil representatives and this is a shortcoming in the opinion of the expert panel. The questions posed by students and pupils differ from those of established teachers and researchers and therefore help to widen the perspective of the evaluations.

According to the self-evaluation, the recruitment of experts is a long process that requires a great deal of resources and is very dependent on the knowledge and contact networks of the consultants. During the site visit both EVA’s staff and the representatives of the institutions described the composition of the panels as a decisive factor if the evaluation model is to function. On the whole the results are described as successful.

One proposal from the expert panel is that the institutions to be evaluated could also be involved in the process of appointing experts. Appointing the experts on the basis of recommendations from the institutions rather than merely presenting their names would offer a different kind of guarantee of their legitimacy. Involving the institutions in the recruitment process would also mean that they would become more active partners in the evaluation. They would share responsibility for the completion and outcome of the process.

Share with your friends:
1   ...   57   58   59   60   61   62   63   64   ...   86

The database is protected by copyright © 2020
send message

    Main page