The National Endowment for Science Technology and the Arts (NESTA) is the lead agency in the development of public sector innovation measurement in the UK. In 2008/09 it commissioned six organisations to undertake scoping/exploratory projects. See Appendix 1 for details of the six projects.
In March 2008 the then Department for Innovation, Universities and Skills (DIUS) published the White Paper ‘Innovation Nation’, which stated that NESTA would:
“…develop a new Innovation Index to measure UK innovation drawing on input and expertise from partners such as the ONS, DIUS, BERR, TSB, AIM, the Design Council, CBI and others. A pilot Index will be published in 2009 with a fuller system in place by 2010” (page 49).
To develop the UK public sector innovation index, NESTA, as the lead agency, proposed to develop:
A set of input, output, adoption and outcome indicators that measure some or all dimensions of public sector innovation processes;
A methodology piloted by NESTA for capturing public sector innovation data that can be later adopted by others; and
Additional outputs such as work on framework conditions and public sector productivity.
The conceptual framework shows aspects of innovation ‘Inputs’, ‘Outputs of innovation activity’, ‘Adoption of innovation’, Outcomes’. Each aspect flows through to the next and wider conditions impact on each area.
Some potential indicators are shown in the table below:
Aspect of Innovation
Some potential indicators
Investment in innovation or capability to innovate: spending; procurement & commissioning strategies; sourcing; support for innovators
Adoption of new to organisation products, services processes, procedures, delivery models; mechanisms for sharing learning and encouraging adoptions across and between organisations
Cost-benefit analysis and other assessments of ‘value added’ of innovation; customer and staff satisfaction measures; productivity and quality measures; performance measures
After receiving the reports from the six exploratory projects, NESTA concluded a survey of innovation activity and results by public sector organisations to be the most effective approach in providing a valuable index of levels of innovation in the public sector.
Such an index would:
Create a valuable source of data showing how innovative different public sector organisations are on a number of dimensions, showing levels of variation and areas where the public sector or different parts of it are weak or strong;
Allow organisational innovation performance to be related to organisational outcomes as measured in other surveys and by audit bodies.
Provide benefits to organisations completing the survey. Each participant would receive a report that benchmarked their performance on different questions against their peers, as well as analysis relating answers to other measurements of organisational performance.
There are two challenges that need to be addressed. Firstly, if the project were to piggyback onto an existing Community Innovation Survey (CIS), subjectivity becomes an issue. Scrutiny by central government makes it difficult for organisations to answer subjective questions without considering how their answers will reflect on them. It is possible to counter-balance this by asking questions whose answers are not entirely subjective and can be confirmed. Anonymity would also help allay concerns from respondents that the data would be used to used for individual performance measurement.
Secondly, if the questionnaire is closely aligned to the private sector CIS, it will not measure innovation open and hidden innovation well. This is particularly important for public sector innovation. NESTA proposes to develop a UK survey instrument enhanced to incorporate questions specifically tailored that will measure the levels of hidden, process and open innovation. These questions will be developed and tested with representative organisations to ensure alignment with the key individual sector drivers. The main body of the survey will be similar to the CIS approach to enable comparability with surveys undertaken in other countries.
To support the Index, NESTA proposes to provide a suite of tools to meet user demand for an organisational and a project scorecard. The demand to provide a performance measurement tool for organisations and an evaluation tool for projects will be addressed by incorporating, and if necessary adapting, the Innovation Capability Measurement tool being developed by the UK Department for Business, Innovation and Skills (or other comparable existing tools), and the Department of Health’s project evaluation tool.
Alongside the survey work NESTA will also prepare a technical paper explaining how other types of measurement of public sector innovation could be accomplished, in particular, what actions need to be undertaken by others before it would be possible to design a metric that is directionally correct. This is particularly important for the growth accounting approach to public sector innovation, which NESTA believes will be feasible in the future when public sector productivity data and information on specific government innovation projects improve.
By August 2010 a framework for the Index had been developed and a survey instrument to measure innovative activities and capabilities in selected public sector areas had been prepared and tested. The survey was piloted across 200 local authorities and NHS trusts in England.
In March 2011, the NESTA released its final report of the project – Innovation in Public Sector Organisations: A pilot survey for measuring innovation across the public sector4 The report presents the findings, lessons and conclusions, and measurement framework of the pilot study. These are summarised below.
A pressing need for innovation in the public sector was identified. The rationale for the project was such that an innovation index should enable this. The pilot study found that a survey-based index can provide practical insight into innovation in the public sector.
The pilot project found that:
The overall index scores suggested that innovation is stifled
Innovation efforts focus on efficiency and localism
Mechanisms to encourage effective sharing of ideas should be improved
Short-termism and restructuring hinder organisations from innovating
The overall conclusion was that the survey tool should be developed further and made available for use across the public sector.
This report has clearly been provided to assist further development of measurement of innovation in the UK. Most useful is the appendix with real examples of individual organisational benchmark scorecards.