10. We reviewed the annual efficiency reports supporting a sample of 12 efficiency projects to determine whether robust systems and processes are in place to record and report claimed savings. The sample covered 86 per cent of the £442 million reported gains in 2005/06 and covered a range of projects of different sizes although we focused more on higher value projects.
11. We found that the annual returns provided clear evidence of a commitment to improve efficiency across the public sector. But further action is necessary to provide assurance on the level of savings reported and their impact on service delivery. In particular, we found that:
• Most of our sample of projects appeared to have established robust baselines but some did not. Robust baselines which represent the situation before a project to improve efficiency begins are important because they provide a ’line in the sand’ against which improvements can be measured.
• More needs to be done to develop measures of output quality to ensure that service quality is maintained and reported gains do not simply result in a cut in services. Attempts have been made with some of the projects we examined to identify measures which would allow the quality of services to be measured but in other projects we had concerns about the quality of output measures in place.
• In some cases it is acceptable to claim savings gross of development costs if, for example, the project was introduced to meet a number of business objectives and not solely to generate efficiency gains. In one of the projects examined, however, we were concerned that relevant development costs should have been deducted from the reported savings.
• Methods for calculating efficiency gains may not always be robust. In two projects we examined, the calculation of reported efficiency gains was based on extrapolation of a sample of savings. This approach casts doubts as to the accuracy of reported gains. However, in one of these projects, local government efficiency savings, further work has been commissioned to explore how efficiency gains can be better measured.
• Most projects we examined have a clear audit trail for reporting claimed efficiency gains. But in at least two of the projects examined, reported gains were not subject to detailed validation by EGDD or the Scottish Executive Health Department.