Beth Sprunt, Nossal Institute for Global Health, The University of Melbourne.
Technical notes prepared for the UN DESA/UNESCO expert group meeting “Disability data and statistics, monitoring and evaluation: the way forward, a disability inclusive development agenda towards 2015 and beyond”, 8 to 10 July 2014, Paris.
This paper provides a case study of Fiji’s efforts to achieve meaningful and feasible disability-disaggregation of the Fiji Education Management Information System (FEMIS). The Ministry of Education is working with a bilateral education sector program to expand access to disability-inclusive education and to build and strengthen FEMIS, amongst other priorities. By providing insight into the context, challenges, questions and actions being taken in one country, this paper aims to support grounded and practical planning for data in the disability inclusive development agenda.
Disability-inclusive education in Fiji – a snapshot
Fiji is a nation of approximately 300 islands in the South Pacific Ocean, one third of which are inhabited. The July 2014 population is estimated as 903,207 (Central Intelligence Agency, 2014). Fiji’s approach to education of children with disabilities is at a dynamic point in its history. The approach has been based predominantly on education in special schools, with a small number of notable exceptions, but a distinct shift to inclusive education has begun. The Ministry of Education’s Policy in Effective Implementation of Inclusive Education in Fiji became effective in November 2011 (Fiji Ministry of Education, 2011). In January 2013 under the Access to Quality Education Program (AQEP), funded by the Australian government aid program, children with disabilities commenced enrolling in five Inclusive Education Demonstration Schools located in four rural areas and a squatter settlement in the capital Suva. Subsequently, at the beginning of 2014 the Ministry of Education began trialling ‘inclusion cluster schools’ which are mainstream primary schools located near some of the special schools. The cluster schools receive support from the special schools and children are being transitioned from the special schools to the cluster schools, with teacher aides relocating where required and possible. A large capacity development program is being rolled out across Fiji by the Ministry of Education together with AQEP to build capacity of head teachers and ‘Inclusion Coordinators’, who are teachers in mainstream schools selected as focal points for disability inclusion in the school.
AQEP is a 5 year program which commenced in August 2011, valued at $AUD 50 million, the goal of which is to improve the ability of children from very poor communities, including those with a disability, to access a quality school education. The substantive components of the program are: social protection, infrastructure, building education support structures and systems, and an additional work stream for emergency response. The AQEP Disability Inclusion Strategy is a cross-cutting strategy, working within all components. The Strategy aims to achieve 3 outcomes for children with disabilities: increased access (measured by enrolment and attendance), increased retention and completion rates, and improved learning outcomes.
In recognition of efforts in the disability inclusion sector in the Pacific (and elsewhere) to strengthen disability-inclusive budgeting, information on the AQEP Disability Inclusion Strategy is provided here based on the budget lines, which include: (i) funding for Inclusive Education Demonstration Schools (predominantly teacher aide salaries); (ii) capacity development of Head Teachers, teachers, Inclusion Coordinators, teacher aides, specialists in Braille and sign language, district based MoE staff and Special Schools; (iii) small grants to cover disability-specific costs such as assistive devices and equipment or access to medical and rehabilitation services; (iv) involvement of Disabled Persons Organisations (DPO) for training, awareness-raising, networks, support and referrals; (v) development and implementation of an Early Childhood Development strategy; (vi) support for national consultations and development of the long-term implementation strategy of the national inclusive education policy; (vii) monitoring and mentoring trips; and (viii) research for systems strengthening. It is within these last two activity streams that the Disability Inclusion Strategy has a focus on disability-disaggregation of Fiji’s Educational Management Information System.
Fiji Education Management Information System (FEMIS)
The Fiji Education Management Information System (FEMIS) was introduced by the Ministry of Education with technical and financial support from AQEP. The previously existing system is the Fiji School Information Management System (SIMS), which officially closed down in 2012 and was replaced by FEMIS in 2013. SIMS, like many other education management information systems (EMIS) in the Pacific, was based on a school census and provided school-level aggregate information. FEMIS is an online system and contains individual student data entered at the school. Each child’s record includes a large variety of data items, such as: student ID number, registered birth number, parent details, gender, ethnicity, date of birth, home situation (eg. household income, electricity, employment), school attendance, record of school fees, and financial assistance accessed. In addition, FEMIS has built in a ‘Health’ section on each child’s record, which is under construction and awaiting finalisation of the National School Health Policy, and a ‘Disability’ section, which is discussed in greater detail below.
At the school level, information is entered into FEMIS on a range of items which are common to other country EMIS’, such as: locality description (eg. urban/rural/remote), governance and management arrangements, utilities, buildings, equipment, staffing, and a range of data that supports financial management. FEMIS is linked to the national teacher data system and the national Literacy and Numeracy Assessment (LANA) database. Together with the fact that FEMIS is based on individual student records, these linkages provide unique opportunities to analyse information in relation to a multitude of questions, such as: which children with disabilities, in which settings, under what circumstances, are achieving what educational outcomes? Or, are teacher aides with specialist skills such as sign language located in schools where they are needed? Or, which teachers with what type of training are creating environments that result in good learning outcomes for children with disabilities?
Disability disaggregation is already possible within FEMIS, however the approach that has been used in the ‘Disability’ section on the student record has some inherent problems. The instruction simply states: “Please indicate disabilities for this student” and tick-boxes are next to each of the following descriptors: No disability; Hearing; Sight; Speech; Intellectual; Physical; Reading; Others. Multiple categories can be selected, unless ‘no disability’ is chosen. There are no further instructions and no training has been provided to the head teacher or teachers to assist in determining which categories are relevant. The Ministry of Education acknowledges that these categories were provided to the FEMIS database programmers with limited time for consultation. This has resulted in data which is very difficult to interpret. For example, as of May 25th 2014, there were 12,897 student records in which the disability section had been completed (of an expected approximate 200,000 students nationally). Of the 12,897, there were 2,597 children for whom teachers had ticked one of the disability categories. Of these, 47.5% were “Reading”, 12.5% “Intellectual”, 12.5% “Speech”, 11.5% “Sight”, 4.9% “Hearing”, and 4.1% “Physical”. Initial investigation of the figures from a small number of schools with which AQEP is familiar shows that the data does not reflect the real situation. Interpretation of the figures is difficult, in light of the questionable validity and reliability of the approach to gathering the information, and the lack of detail about the degree of difficulty experienced or whether the students learning needs have been met already. This means that planning for services based on this information is very restricted and changes in the figures over time will be difficult to interpret.
Which indicators require EMIS disability disaggregation?
Fiji is one of four case countries in an ongoing research program titled “Developing and testing indicators for the education of children with disability in the Pacific” (April 2013-2016) funded by the Australian Government through the Department of Foreign Affairs and Trade’s Australian Development Research Awards Scheme. The research, being undertaken in partnership by Monash University, the CBM-Nossal Institute Partnership for Disability Inclusive Development, Pacific Disability Forum and the Pacific Islands Forum Secretariat, will result in indicators and guidelines for implementation. Having been developed through a regional survey and qualitative methods in the four case countries, the emergent indicators are currently under review through a Delphi process of international experts and are yet to be trialled in Fiji, Samoa, Vanuatu and the Solomon Islands. At this stage, the emerging indicators cover a wide spectrum of information needs. They range from measures of value to families, DPO representatives and teachers at the school level in terms of classroom and playground experiences, to indicators of policy and legislative status. Included are indicators useful in measuring regional and global frameworks, such as the Pacific Education Development Framework (Pacific Islands Forum Secretariat, 2009), the Incheon Strategy (UNESCAP, 2012) and the UN Convention on the Rights of Persons with Disabilities (United Nations Committee on the Rights of Persons with Disabilities, 2009).
The call for disability-disaggregation of education management information systems has been growing in many quarters globally (Mitra, 2013; Pacific Islands Forum Secretariat, 2012; Robson, 2005; Task Force on the Global Partnership for Education (GPE), 2013; UNESCO Regional Bureau of Education for Latin America and the Caribbean, 2011). Several indicators arising within the ADRAS Pacific research and the indicators within the AQEP monitoring and evaluation framework rely on a valid and reliable means of disaggregating the national EMIS for disability. A few examples of constructs within indicators which require disability-disaggregated EMIS are: enrolment, transition, completion, ‘dropout’ rates, and achievement. There are many other indicators that measure environmental changes related to disability-inclusive education which could be measured through enhanced disability data in EMIS, such as: access to assistive devices/technologies, presence of teacher aides, access to specialist staff, relevant teacher qualifications and professional development, and accessibility of buildings and school transport.
Ministry of Education questions regarding disability information
In considering an improved approach to disaggregation of the Fiji EMIS, it is valuable to understand the range of questions that the Fiji Ministry of Education, AQEP and other stakeholders are grappling with as they strive to make substantial shifts to an inclusive education system. Some of the questions are: How do we calculate loading for the school grants as the schools enrol children with disabilities? How do we determine staffing needs? How should the individual needs of the child be assessed? How do we verify whether school figures on disability are correct? Which level of functional difficulty should be interpreted and counted as a ‘child with disability’? How do we know which schools should be prioritised for Braille or sign language specialists? How do we plan for additional supports that are outside of the education system (eg. rehabilitation needs)? How do we measure change against the national policy? How much will implementation of the national policy cost? How do we measure change related to the out-of-school children with disability? How do we link with other information systems, in particular the Ministry of Health and Ministry of Social Welfare?
Given the bureaucratic and political will to move towards disability-inclusive education in Fiji, and the desire for improvement in a range of related information processes, there is an opportunity for establishing a comprehensive information system that meets the reporting and planning needs at national and district levels, as well as assessment and planning needs at an individual student level. Some of the processes or forms that need development, testing and linking include: screening and identification processes, student profile forms, Individualised Education Program (IEP) processes and forms, and district education office inspection/verification processes.
Linkages between information systems and the role of a unique identifier
A multi-sectoral stakeholder meeting was held in May 2014 to discuss issues related to identification of children with disabilities, disaggregation of the FEMIS, and to map the current and potential strengths and weaknesses of various information systems in relation to children with disabilities. This situation analysis is important as the earliest stage of identification of children with or at risk of disabilities is likely to occur within health services, including community-based maternal child health services, and there is a partnership required between health and education in relation to any diagnosis and service-provision following potential education-based screening processes. In addition to government services, Fiji relies heavily on non-government organisations for screening and services related to vision, hearing, and mobility, including access to assistive devices and technologies. A role that the national DPO fulfils includes managing the eligibility and verification process and database for people with disabilities to access free transport vouchers through the Ministry of Social Welfare. This includes collecting copies of relevant medical records. The databases resulting from the work of NGOs and DPOs are important to be aware of in relation to the potential for the Ministry of Education to verify information and potentially link with.
Information currently available through the health system does not provide enough information to identify children with disabilities. The Child Health Card, with developmental milestones, only goes to 24 months and is an emergent system; the School Health Team does not record whether children have disabilities; and the National Health Number (NHN) is only provided to people upon hospitalisation. However the Ministry of Health is working to link the NHN to the Births and Deaths register, or to ensure that every child is given an NHN at birth. If the NHN is strengthened and covers the whole population, this would be potentially a strong system to link to. However the Ministry of health advised that this process may take years, and that in fact they currently access information from the FEMIS to provide basic denominators for their work. It is clear that, whilst linking the education and health information systems for disability data is a vital piece of work, it is a long-term process and the Ministry of Education must move ahead with establishing processes that meet its own information needs in the short and medium term.
Interestingly, due to the natural disasters occurring in the Pacific related to climate change, some DPOs in the region are being funded to develop detailed databases of people with disabilities and their functional limitations, health conditions and assistive devices so they can be identified and supported in emergencies. A unique identifier/number which links people with disabilities across the various databases has the potential to minimise both duplication of data-gathering efforts and the risk of people being missed by these important sector-based initiatives. A strong consensus of the multi-sectoral stakeholder meeting was that a unique identifier/number for children was very important, in particular in children with disabilities, and that the linking of the various information systems would have mutual advantages and be an important aim.
Approaches to disability disaggregation of EMIS in the Pacific
Several countries in the Pacific disaggregate their EMIS by disability, using a variety of approaches. The Cook Islands records the total number of boys and girls who are receiving official teacher aide support from the MoE, and the level of needs – low, moderate and high. Tonga uses individual Student Profile Forms, matched to a Student National ID Number, and the Health section asks: Disability: Yes or No, and Illness: Yes or No, with prompts for comments on how attendance is affected. Vanuatu records total number of boys and girls with impairments in each year level, categorised into a range of impairments. Uniquely in the region, the Vanuatu school census also asks schools to record numbers of children in the community who are not able to attend school due to disability or impairment. It is not clear what instructions or training is provided to teachers or head teachers in any of the countries in order to complete these school census forms.
In relation to the indicators mentioned earlier in the paper (enrolment, transition, completion, ‘dropout’ rates and achievement), based on the Pacific EMIS examples above, disability-disaggregated data is generally only provided on the number of children enrolled. Analysis could be done by comparing census forms over time to provide an estimate of transition, completion and dropout. Indicators that require information on achievement (learning outcomes) of children with disabilities is unattainable through current school census mechanisms. The validity of the data is difficult to determine in terms of how decisions are made which children are categorised as having disabilities, except in the Cook Islands where the level of needs is classified by the MoE Inclusive Education Adviser directly.
This approach of direct assessment by the Cook Islands MoE is not possible in a large country like Fiji where the population is vastly greater and the distances, travel costs and limited specialist human resources in the MoE require a different system for identification of children with disabilities. Given the variety of questions facing the Fiji MoE discussed above, a research process is being planned to provide a basis for an improved disability-disaggregation of FEMIS.
Research to inform disability-disaggregation of the FEMIS
The research, which is planned to commence in early 2015, aims to strengthen the capacity of Fiji’s Ministry of Education to implement and monitor its inclusive education policy. The objectives of the research are: (1) to develop feasible methods for the Fiji Ministry of Education to: (a) identify children with disability in schools and communities; (b) assess learning support needs of children with disability in schools and communities; (c) disaggregate the FEMIS by disability status and learning support needs; and (d) disaggregate FEMIS by environmental factors related to disability-inclusive education (such as accessible built environment and staff qualifications related to inclusive/special education), and (2) to test the appropriateness and validity of the methods outlined in Objective 1.
The methodology is under development, however initial thoughts for methods to identify children with disability and assess learning support needs include using the UNICEF/Washington Group on Disability Statistics Module on Child Functioning and Disability as part of a tool which would be completed by a small local team, including the teacher, village nurse, community-based rehabilitation worker and the primary caregiver. This screening process would identify children to receive formal assessment by a multi-disciplinary team.
The technical issues related to running FEMIS reports that cross-match teacher and LANA databases with student and school information will be undertaken in collaboration with the FEMIS database programmers. The feasibility of disability data within FEMIS related to environmental factors will be tested by setting up a practice module within FEMIS and recruiting a sample of schools to trial the module and reports.
Data challenges for disability-inclusive education in Fiji
There are a number of challenges in establishing a system which supports Fiji’s Ministry of Education to implement and monitor its inclusive education policy.
Teachers and head teachers in Fiji are transferred relatively frequently and so knowledge of the child with disability (in terms of health condition, functional limitations and participation) may be stronger in other people. Whilst the role of teachers in picking up children at risk of disability is unquestionable, building a system that is focused solely on teacher-based screening may not be the best approach in Fiji.
There is a common practice in Fiji of labelling children as “non-readers” and interpreting this to be a disability, despite evidence of multiple factors unrelated to neurodevelopmental function that affect children’s reading, eg. lack of access to books, under nutrition, etc. Appropriate and feasible identification of specific learning disabilities such as dyslexia has been identified by the MoE as a priority in the context of large numbers of children being labelled ‘non-readers’ and the issue of some ‘non-readers’ being encouraged to drop out of regular schools and enrol in special schools. Accurate identification is required so that appropriate responses are implemented for children who do have specific learning disabilities, as well as appropriate and different responses for children who may need social protection and community and family engagement. Acknowledging the challenge of accurately identifying specific learning disabilities, AQEP is funding a separate research study.
The FEMIS is inherently limited by its focus on children who are enrolled in schools. Many children with disabilities are not enrolled in school and disability data through FEMIS will need to be compared with national data to measure the proportion of children with disabilities accessing schools out of the total of children with disabilities. It is hoped that by integrating the UNICEF/Washington Group Module into the identification tool to be used at the school level, this may inform and link to national efforts in childhood disability measurement.
Linked to this is the issue of children with disabilities who access non-formal education, including education at home. With access to information technology outpacing efforts to make transport accessible, the potential for children with disabilities to access high quality education from home is enormous. The advantages and disadvantages of home-schooling can be debated, however it is a growing option and therefore the data related to children accessing education outside of the formal setting should be considered.
A distinct challenge is finding the balance between developing a system that is feasible and valid to disaggregate FEMIS for the primary purpose of data, and one which provides detailed enough information to inform individual eligibility for services or financial aid, as well as accurate assessment required to provide appropriate individualised supports.
Aside from disability-related data, there are distinct structural challenges that the MoE and schools face in relation to FEMIS more broadly. Access to electricity is limited for many schools, so an online system relies on generators. However, solar systems have become more affordable and many schools are purchasing these with current government grants. Internet access is also limited in some places, which affects data entry. There is the back-up option of collecting disability data in paper formats during district inspections, so this will be considered in developing the system.
Another significant challenge relates to the fact that as disability-inclusive education is expanding, the relevant MoE officer for special/inclusive education is completely over-stretched and would require support to undertake the analysis of the disability-disaggregated data within FEMIS. If schools are undertaking a substantial effort to gather disability data, and they are not receiving information back which demonstrates that the data is being analysed and valued, then the completion rate will drop for disability sections of FEMIS, as will potentially the validity of the data if teachers and head teachers do not value it and are rushing their responses.
Summary of recommendations for disability-disaggregation of EMIS’
Whilst the research in 2015 will provide more relevant information for making recommendations about disability-disaggregation of EMIS’, the reflections from this paper may be useful:
Where possible, processes that determine if a child is counted in the EMIS should directly link to program support eligibility criteria in policies.
Where disability data within EMIS’ link to financial incentives for school grants, verification processes need to be established.
As ministries of education take up disability-inclusive education, education officers can become extremely stretched; external support for analysis and use of EMIS disability data may be important in the short term so schools perceive that data entry effort is useful and valued. Where education-sector programs exist, building support for disability data analysis into the workplan may be a useful mechanism.
Regular multi-sectoral discussions, including relevant ministries, DPOs, non-government service providers and the Bureau of Statistics regarding various disability data initiatives are useful in aligning opportunities and minimizing duplication.
It is valuable to investigate options for use of a unique identifier/number which works across databases.
Determination of disability for EMIS may be more valid if decisions are made by a small group of people who know the child well rather than simply on one teacher’s opinion.
Where processes and forms are used for assessing the function, learning support needs and progress of children with disability, these should be the basis of data that is entered in the EMIS (eg. Student Profile Form and Individual Education Program).
Special attention needs to be given to how disability-disaggregated data can be achieved in relation to learning outcomes/achievements.
Where possible, approaches to determining disability within the EMIS are compatible with national household approaches to estimating numbers of children with disabilities; thereby enabling a denominator for inclusive education targets.
Consideration should be given to data for out-of-school children with disabilities, both those accessing non-formal education (including online home schooling), and those who are not accessing education at all.
Support for data systems as a whole, such as adequate electricity or human capacity for data analysis, must be addressed to enable disability data.
Central Intelligence Agency. (2014). The World Factbook: Fiji. Retrieved 28 June, 2014, from https://www.cia.gov/library/publications/the-world-factbook/geos/fj.html
Fiji Ministry of Education, N. H., Culture & Arts and Youth and Sports. (2011). Policy in Effective Implementation of Inclusive Education in Fiji. Suva, Fiji: Retrieved from http://www.education.gov.fj/images/Legislation/Policies/POLICY%20IN%20EFFECTIVE%20IMPLEMENTATION%20OF%20INCLUSIVE%20EDUCATION%20IN%20FIJI.pdf.
Mitra, S. (2013). A data revolution for disability-inclusive development. The Lancet. doi: http://dx.doi.org/10.1016/S2214-109X(13)70016-0
Pacific Islands Forum Secretariat. (2009). Pacific Education Development Framework (PEDF) 2009-2015. Suva, Fiji: Pacific Islands Forum Secretariat.
Pacific Islands Forum Secretariat. (2012). Forum Education Ministers' Meeting. Port Vila, Vanuatu, 14-15 May 2012. Session 4 - Update on the implementation of the Pacific Education Development Framework - country and regional level (Vol. PIFS(12) FEdMM.03). Suva, Fiji: Pacific Islands Forum Secretariat.
Robson, C., & Evans, P. (2005). Educating Children with Disabilities in Developing Countries: The Role of Data Sets.
Task Force on the Global Partnership for Education (GPE). (2013). Background Note for the Global Partnership on Children with Disabilities.
UNESCAP. (2012). Incheon Strategy to “Make the Right Real” for Persons with Disabilities in Asia and the Pacific. Bangkok: United Nations Publication.
UNESCO Regional Bureau of Education for Latin America and the Caribbean. (2011). Regional Education System on Students with Disabilities. SIRIED. Methodological proposal. Santiago: UNESCO Regional Bureau of Education for Latin America and the Caribbean.
United Nations Committee on the Rights of Persons with Disabilities. (2009). Guidelines on treaty-specific document to be submitted by states parties under article 35, paragraph 1, of the Convention on the Rights of Persons with Disabilities Geneva: United Nations.
Sprunt, B. (2014). Efforts to improve disability disaggregation of the Fiji Education Management Information System. Working Paper. Nossal Institute for Global Health. The University of Melbourne. Melbourne.