There are certain limitations to this study. Given the time limit for gathering the information, some officials who are currently involved with the project and others, who participated or were involved in past evaluations, were not available for interview during the data collection period. Their viewpoints could have contributed to a more comprehensive analysis of evaluations carried out so far. Additionally, the study is not a longitudinal study and respondents were asked about events taking place over a considerable period of time.
Moreover, evaluation is inherently subjective (Angell and Smithson, 1991; Hirschheim and Smithson, 1998) and the criteria used to evaluate telecentres and to analyse past evaluations might not provide accurate measures. Different evaluators can view the same situation from their own perspective and because this study is “an evaluation of evaluations” it is not free of this problem of perception (Mabry, 2002). Finally, various dominant perspectives in evaluation have been equally represented in the framework built for this analysis. It is possible however that other methodologies and frameworks could have been overlooked and that the framework used here, though comprehensive, may not be entirely so.
Conclusions and further research
This paper set out to offer a comprehensive analysis of evaluation practices carried out by the Venezuelan government when assessing the Infocentros programme. IS, Development and Telecentre evaluation literature has been used to help establish an analysis framework when examining the evaluation initiatives. Despite the limitations detailed at the outset, this study has been conducted with rigour and offers insight into the nature of the evaluation of telecentre practices in the specific instance of Venezuela.
One of the central conclusions that can be drawn from the foregoing discussion is that a universal model of telecentre evaluation does not exist. While the implementation of a specific model of telecentre depends on a country’s specific circumstances, so too does the evaluation of these initiatives, thus each country develops a model suitable to its needs or in line with what the evaluation seeks to demonstrate.
As development potential is not yet fully realised in Venezuela, and more time is needed before final outcomes and overall achievement of benefits and impacts of telecentros on society are observed with regard to policies and on the overall governance of developing countries, the findings of current evaluation and monitoring initiatives may prove unsatisfactory or unconvincing in the achievement of an exhaustive evaluation of the telecentre programmes as a whole. Yet there remains a core validity in respect of the particular projects evaluated in Venezuela.
Research on and the carrying out of evaluation implies also an increase in evaluation budgets, time devoted, and evaluation skills. Yet little money or will for this exists in developing countries. These remain uninterested in evaluation initiatives (unless they provide some political benefit) nor in learning exercises for future evaluations, because developing countries continue to be subject to problems of administration. Moreover, evaluations can be difficult, political and subjective, but it is better evaluate than do nothing, even if the methodology, motivations and context are imperfect. Comprehensive evaluation has to be a learning process, which each evaluation providing lessons for the next, and as a consequence of the overall process itself.
There is a plethora of methodologies and frameworks for telecentre evaluation offered by the literature but in practice, they are not strictly followed or even worse, not considered. Evaluations are carried out following specific political interests and as a means that contribute to political discourse, - an excuse for re-election or as a populist tool. Despite more comprehensive frameworks and methodologies being known of by officials, evaluations are performed using political criteria and positivist approaches, which tends to show a deterministic use of technology which assumes development because people have internet access.
For the moment, it appears that there is a favourable climate for improvement in Venezuela as a consequence of the changing context in the political and socio-economic realities, yet it is too early to provide definitive verdicts. As in many other developing countries, Venezuelan officials are attempting to create at least the appearance that government is concerned about less favoured citizens and they are ceaselessly creating social programmes in order to benefit them.
Further research is needed to verify the proposed model for examining evaluation of telecentres and to propose other models to be applied in evaluation of other ICT and development projects both in Venezuela and in other developing countries. Taking into account that the project evaluation for 2004 is at the planning stage, further research will be needed in order to show to what extent the planned evaluation strategy is followed, the quality of that evaluation and whether officials and evaluation team expectations have been satisfied. Moreover, further research will allow the completion of this analysis by including in 2004, additional evaluation exercises, and thus securing a more comprehensive picture of the evolution of evaluation procedures in Venezuela.
Further research is also necessary in evaluation of other development programmes which introduce ICTs for development in Venezuela, in order to compare findings and conclusions and thus provide a richer picture of the current problems of practice in evaluation of development programmes. Finally, according to Hirschheim and Smithson (1998), there is a need for further research in the “understanding zone”; that the functions and nature of evaluations should be understood, similarly with limitations and problems of any evaluation process. This paper attempts to contribute to the challenging area of evaluation. It is hoped these results may benefit other Venezuelan developmental initiatives.