We focused on the knowledge construct as an example of how to utilize the initial findings from the faculty questionnaire in planning staff development and measuring efficacy of intervention at a later time. The knowledge construct consisted of: (a) knowledge about legal mandates pertaining to disabilities in higher education; (b) knowledge regarding instructional and examination accommodations; (c) knowledge about the office of disability services, and 4) general knowledge about disabilities. In the following, results are reported for the four knowledge items and the overall knowledge composite variable for Northern Illinois University.
Knowledge about disabilities. When faculty were asked about their level of knowledge regarding disabilities in general, almost one fourth (25%) in Year One reported that they had no knowledge at all (NAA) in contrast to only 1% in Year Three. Moreover, about half of the respondents in Year One reported that they had very limited knowledge (1 or 2 on a six-point scale) in contrast to 12% in Year Three. The remaining one fourth in Year One in contrast to one third in Year Three had a great deal of knowledge (5 or 6 on a six-point scale). The mean score for the Likert scale responses for this item increased from M = 1.78 ( SD = 1.45) to M = 3.94 (SD = 1.34), confirming the significant increase in knowledge about disabilities with a large effect size, t(240) = 12.83, p < .001, d = 1.53 (see Table 2).
Knowledge about legal mandates. A bimodal distribution was reported in Year One with regard to knowledge about the legal mandates germane to higher education (ADA and Section 504 of the Rehabilitation Act). Twenty-one percent of the faculty indicated that they had no knowledge at all (NAA) regarding ADA and Section 504, and more than half (58%) rated themselves as having very limited knowledge (1 or 2 on a six-point scale). In contrast, 19% indicated they were very knowledgeable (5 or 6) about the legal mandates. The Likert scale mean was 2.33 (SD = 1.11), almost identical to the mean for the item regarding knowledge about disability services office. In Year Three, only 7% responded that they had no knowledge at all about the legal mandates, 16% rated their knowledge as very low, and 35% rated themselves as very knowledgeable. When the means for this Likert scale item were compared, there was a significant increase in Year Three (M = 3.90, SD = 1.44) as compared to Year One (M = 2.33, SD = 1.11) which was a large effect size, t(186) = 8.9, p < .001, d = 1.24 (see Table 2).
Knowledge about accommodations. In spite of limited knowledge about disabilities and the law in Year One, many faculty members were fairly knowledgeable about providing accommodations. Only 2% reported no knowledge at all (NAA) in providing accommodations, 19% reported very limited knowledge, and 31% reported they had a great deal of knowledge in providing accommodations. When the means for this Likert scale item were compared, a significant increase was found in Year Three (M = 3.99, SD = 1.28) compared to Year One (M = 2.22, SD = 1.03) which was a large effect size, t(319) = 13.23, p < .001, d = 1.59 (see Table 2).
Awareness of the need for accommodations generated proactively by faculty was described as “I teach an intro course, and this semester is the first time I have ever had students with hearing trouble. I use videos and films. This is now a big problem as the videos are not closed caption(ed).” The statement also implies an ongoing need for assistance to provide an accommodation. As a result of awareness of this problem, the university instituted a policy regarding purchase of videos/DVDs only if they were closed-captioned and the purchase of software to provide closed captioning in all smart classrooms.
Knowledge about the disability services (DS) office. Knowledge about accommodations did not seem to be acquired as a result of direct contact with the office of DS, because 9% of the faculty reported that they had no knowledge at all about the DS office, 37% had limited knowledge about the DS office, and about one fourth were very knowledgeable about this office. After Year Three, only 1% reported no knowledge at all, and those who had limited knowledge went down to 15%. At the same time, 40% had a great deal of knowledge about the DS office in Year Three. Not surprising, there was a statistically significant increase in the means on this Likert scale item, from M = 2.29 (SD = 1.06) to M = 3.95 (SD = 1. 38), t (184) = 11.34, p < .001, d = 1.39, indicating a large effect size (see Table 2). A wish for increased knowledge and a deeper level of knowledge was expressed by one faculty when this person said “I always wish I had firsthand knowledge of this NIU office.”
Knowledge – combined construct. The faculty knowledge construct that combined the previous four items naturally also showed a statistically significant increase from Year One to Year Three, t(175) = 15.3, p < .001, d = 1.88, indicating a large effect size. The mean in Year One was M = 1.96 (SD = 0.97), which increased to M = 3.90 (SD = 1.16) in Year Three (see Table 2).
Faculty Knowledge Mean Comparisons Between Years One and Three
Note. All means were significantly different between years one and three at p < .001.
Topics of Interest
We asked faculty to indicate their level of interest in acquiring more information about eight topics related to disabilities as well as how they preferred acquiring information. Congruent with the lack of faculty knowledge regarding the disability services office, faculty identified this as the topic of greatest interest (M = 4.90, SD= 1.36), followed by a need for more information regarding policies and procedures relevant to students with disabilities (M = 4.70, SD = 1.33), and test accommodations (M = 4.70, SD = 1.41). General information about disabilities and legal mandates were rated next highest, M = 4.55 (SD = 1.44) and M = 4.43 (SD = 1.39), even though the faculty also assessed their level of knowledge about these two topics also very low. These findings indicate that the faculty had a greater need for practical knowledge about accommodations and policies than for background information regarding disabilities and legal mandates. Qualitatively, faculty expressed a need for information about universal design, how to help students become self-advocates, how faculty can be fair, and how the university can afford the cost of accommodations.
When given a choice of eight alternative methods for acquiring information, tied for first place was a desire for expert advice and information to be available 24/7 online (M = 4.55, SD = 1.38) and one-on-one consultation (M = 4.57, SD = 1.50; see Table 3). This was reinforced with comments like “It is most helpful to be able to call someone for specific, up to date guidance about reasonable accommodations.” Faculty also identified workshops and on-site seminars and speakers next in order of preference. Least desirable were distance learning courses, teleconferences, and credit or non-credit short courses or graduate courses. The latter two alternatives were included so as to be all-encompassing and appropriate for use with faculty, staff, and administrators at all types of institutions, not just faculty in a large, public, doctoral-degree granting institution.
The reliability and validity of the scores from the self-reported assessment provided on the questionnaires were evaluated by examining internal consistency (knowledge and topics of interest) reliability and by external confirmative evidence corroborated by examining syllabi and analyzing identified paragraphs. We learned from student report that one of the most powerful methods for faculty to send a welcoming message to students with disabilities is to include a paragraph in their syllabi regarding their desire to meet with them if they need accommodations. This paragraph has been interpreted by students to indicate that faculty members know about disabilities and suggests that they are willing to make accommodations.
Two strategies were used to determine the prevalence of this practice and whether or not the paragraph wording and location sent a positive message. The first strategy was to include a question in the faculty questionnaire asking how frequently faculty included a welcoming paragraph in their syllabi. The second strategy involved actually reading syllabi and searching for paragraphs. When located, the paragraphs were analyzed for wording and placement of paragraphs in syllabi.
In Year One, more than half of the faculty (55%) reported that they had never included such a paragraph in their syllabi. However, one third reported that they did so very often (M = 2.72, SD = 2.24). In Year Three, 29% reported that they had never included such a paragraph in their syllabi and 57% did so very often, M = 5.12 (SD = 1.41), indicating a significant increase in this practice in Year Three. A noteworthy finding is that no faculty in Year One reported including such a statement in their syllabi “Very Often,” whereas in Year Three 80% of faculty reported including the statement “Very Often” (Likert scores of 5 or 6).
Analyzing paragraphs. Several strategies were used in order to corroborate questionnaire findings from the syllabi themselves. In Year One we drew a randomized stratified sample of 304 undergraduate and graduate courses listed in the fall and spring registration bulletins. Forty-seven (17%) of the 304 syllabi included a paragraph about accommodations, confirming that few faculty included a welcoming paragraph in syllabi. When the 47 paragraphs were analyzed, the following five exemplary characteristics and/or content were identified: (a) correct reference was made to the fact that the university abides by Section 504 of the Rehabilitation Act; (b) appropriate terminology was used such as person-first wording, (e.g. “students with a learning disability” rather than “learning disabled students”); (c) faculty invited students with disabilities to contact them early in the semester if they needed accommodations; (d) students who needed accommodations and had not yet registered with the DS office, were encouraged to do so; and (e) the paragraph was placed in a neutral location rather than embedded within a list of rules regarding class preparation or decorum. (A sample paragraph appears in Figure 2.)
Topics of Interest
Disability services office
Policies and procedures
General information about disabilities
Note. The higher the mean, the greater the interest.
Figure 2. Recommended paragraph to include in syllabus.
This institution abides by Section 504 of the Rehabilitation Act of 1973, which mandates reasonable accommodations be provided for students with documented disabilities. If you have a disability and may require some type of instructional and/or examination accommodation, please contact me early in the semester so that I can provide or facilitate in providing accommodations you may need. If you have not already done so, you will need to register with the office of disability services, the designated office on campus to provide services for students with disabilities. The office is located at __________ (address and telephone number). I look forward to talking with you soon to learn how I may be helpful in enhancing your academic success in this course.
In the final semester of the Project all available online syllabi were scanned to determine if changes in the practice of inclusion of welcoming paragraphs had occurred. Of the 144 course syllabi located, 56/144 (39%) included a paragraph, which represents an increase of 22%. Of these 56 paragraphs, 40 (71%) included paragraphs that were identical or similar to the recommended paragraph and located in a neutral location representing a significant increase as compared to previous years.
The twofold purpose of this study was to (a) evaluate faculty knowledge, attitudes, practices, and topics of interests, regarding students with disabilities and (b) assess the effectiveness of the faculty questionnaire for evaluating campus climate for students with disabilities. The change in faculty knowledge was striking from Year One to Year Three. The change in knowledge was a large effect and encompassed all four aspects of knowledge about disabilities: disabilities in general, legal mandates, accommodations, and the disability services office. Although the effect cannot be directly attributed to the interventions, it is unlikely that such a large effect would occur in two years without the interventions.
It was clear that campus climate for students with disabilities had improved by Year Three with regard to faculty knowledge. It was also noted where areas that faculty felt that they needed the most information were about practical knowledge regarding accommodations and policies, not background information regarding disabilities and legal mandates. With respect to the second purpose of the study, the findings indicated that (a) the scores from the revised faculty questionnaire were reliable, and (b) scores from the faculty questionnaire were valid, as corroborated from inspection of faculty syllabi.
Results from the use of the faculty questionnaire in Year One confirmed in many respects the student-identified barriers of limited faculty knowledge regarding disabilities in general (M=1.78) and the legal mandates requiring accommodations for students with documented disabilities (M=1.69). Moreover, few faculty were aware of the DS office (M=2.29) and its role in providing test accommodations. These findings were confirmed by the students who commented that faculty do not know very much about disabilities, as one student wrote: “…understanding that my disability is really a disability and that I (am) not trying to take advantage of it/make things up.”
In addition, the faculty members themselves corroborated their lack of knowledge when they identified these two topics as the ones of greatest interest. Faculty identified in Year One, from highest to lowest rating, their desire to learn more about the DS office (M=4.90) and the need for more information about test accommodations (M=4.70), followed by the need for more information about legal issues (M=4.55). The knowledge items combined with the interest in specific topics and comments from the faculty questionnaire enabled us to determine that faculty wanted to acquire more practical knowledge than background knowledge regarding disabilities in general or the legal mandates. For example, one faculty expressed, “I am really hazy about what to do with the physically disabled in lab situations.” Similarly, another faculty acknowledged, “I would like to know how much of what I say is being transmitted, and what I need to do to make it easier (for the students) to comprehend. I also would like to know how to design my course to make it more available to blind students, since I depend much on overhead outlines and PowerPoint™.”
Despite lack of general information about disabilities and legal mandates, faculty members were very knowledgeable about making accommodations and were, in general willing to make accommodations. Nonetheless, they expressed interest in learning more about providing test accommodations (M = 4.70). Those who attended workshops especially valued the aspect of the workshops that provided videotaped interviews with students with disabilities as described by a faculty member who said “student voices have always been the most powerful in faculty workshops…particularly when they shared with faculty what worked well in class and what did not work.”
The most meaningful information for planning staff development activities was gathered from close examination of the Likert scale item responses, verbal responses, including Not At All (NAA) and comments, and the Likert scale means and standard deviation for individual items. We strived to achieve greater understanding by balancing self-reported responses to items, for example, knowledge regarding disabilities and self-reported perception of need for information. Such information from Years One and Three for faculty was especially useful in planning staff development activities, determining efficacy of staff development activities, and summative evaluation of the Three-Year Project.
Given eight options for faculty development opportunities, there was a two-way tie between online web-based expertise available 24/7 and one-on-one consultation. Faculty expressed their preference for web-based expertise possibly because it provided targeted information instantaneously, when they wanted it and needed it. In response to the desire for an infusion of information to be available online, we developed the Enhancing Success Web site with many useful links to websites within the university, slide presentations, recommended readings linked to full text, as well as online links to other informative national Web sites. Also identified were workshops, seminars, and speakers.
Other methods for infusion of information that contributed to the significant increase in knowledge included e-learning graduate courses on Disabilities and Higher Education. These courses drew graduate students from throughout Illinois, from various institutions, and in a variety of roles including administrators working in admissions, registration and records, advising, career planning, housing and dining, and graduate school. The Project also donated media and print materials to institutional libraries and sponsored a series of presentations given by a motivational speaker with a disability. In addition, workshops regarding students with learning disabilities, visual impairments, and hearing impairments were offered in which a variety of speakers, including DS and information technology staff, made presentations and responded to questions. During the workshops, DVDs allowed the participants to view video-taped interviews from students with disabilities (Project PACE, 2003). The key principle in staff development that emerged was the need to provide input using a variety of methods and a variety of timeframes given the limited time faculty have to attend workshops (Scott & Gregg, 2000).
Last, some of the highest-ranking administrators at the departmental, college, and university level encouraged faculty to adopt the practice of inclusion of a welcoming paragraph in syllabi. At the end of Fall 2003 and Spring 2004 semesters, the associate vice provost responded favorably to our request to email all faculty members at the end of each semester reminding them to include the recommended paragraph in future syllabi. In addition, two reminders appeared at the end of each semester encouraging faculty to incorporate the recommended paragraph in syllabi. One article appeared in the faculty/staff weekly newsletter and a second, from the student perspective, in the student newspaper.
Although these methods resulted in a significant increase in the inclusion of the welcoming paragraph in syllabi, many faculty still did not adopt this practice. In an effort to further understand the factors that influence campus climate and how staff development has to be adapted to individual institutions, Vogel, Leyser, Burgstahler, Sligar, and Zecker (2006) compared faculty knowledge, awareness, practices, and attitude on three contrasting institutions. One of their dramatic findings was that almost all faculty members in the community college setting included a welcoming paragraph in their syllabi. Upon inquiry, it was discovered that in this specific institution, disability workshop attendance was not only mandatory but also included in annual evaluations and considered favorably in determination of salary increment. Moreover, at the workshop the welcoming paragraph was presented and its inclusion in syllabi mandated.
An interesting contrast between various departments at NIU was also noted. Upon close examination, it was determined that in some departments the chair mandated and enforced the inclusion of a welcoming paragraph. Top-down leadership is a significant factor in the practice of inclusion of the paragraph. Furthermore, we learned through the fine-grain analysis of wording and placement that the paragraph alone does not make a specific classroom a welcoming environment. The paragraph has to be properly worded and placed. Further, the correct placement of a properly worded paragraph on its own is not enough to make a classroom a safe environment in which to disclose one’s disability. Several factors in combination make a classroom safe or welcoming, and it is this composite that makes a campus a safe environment in which to disclose one’s disability and an environment that is facilitative of academic success.
The four instruments, referred to Assessment of Campus Climate to Enhance Student Success (ACCESS), are thought to provide 360-degree feedback from all constituents who contribute to and are affected by the campus climate. All respondents contribute from their own perspective. The richness of the data allows for targeted staff development, alternative methods of providing information based on the target audiences’ preference, triangulation, and cross-validation. Most importantly, the students with disabilities speak in their own voice since they are the ones directly impacted by the campus climate. Utilization of ACCESS empowers students with disabilities to contribute to changing their environment and thereby to enhance their chances of success.
Some of the unexpected benefits of having administered the ACCESS questionnaires and gathering corroborative evidence during the three-year project were to increase awareness of the institution’s web presence and the need for the enforcement of Section 508, to provide training and assistance for faculty, administrators, and computer lab technicians regarding assistive technology, accessible Web sites, universal design in instruction and assessment, online accessible learning experiences, and accessible institution marketing campaigns. Given the recent tragic events on college campuses, there is an even greater need for courses similar to the e-learning course to meet the informational needs of undergraduate and graduate students without disabilities. In addition, a ripple effect went beyond NIU when the Illinois Board of Higher Education instituted a statewide initiative regarding institutional web presence assessment and enhancement of accessibility to be in compliance with federal and state legislation.
Every effort was made to encourage a robust response rate. In administration of the questionnaires the highest response rates achieved were 62% among administrators and 42% among faculty members, rates that we attributed to the support of the president of the university and university-wide planning. More often, however, the response rate was around 24%. Efforts have to be made to garner institution-wide support surrounding the administration of the questionnaires. Response rate will be enhanced if dissemination of the questionnaires is supported by the highest administrative officials, the DS office, cross-functional teams, shared governance involvement, effective public relations, broad-based campus enlightenment, and all disability and diversity advocates. Moreover, they have to be viewed not as evaluation instruments, but as a method to make the respondents more comfortable and successful in fulfilling their responsibilities and meeting the needs of students with disabilities. They should also be viewed as assisting the institution in fulfilling its mandates to increase diversity, enhance retention, and enroll more under-represented students. Rather than adding to their work load, the administration and faculty need to see the questionnaires as tools to give them the information they want, now they want it, and when they need it.
Partnership with AHEAD
The four questionnaires are licensed to AHEAD which will make online versions available on a cost-recovery basis to those who want to administer one or more of them. They will be accompanied by a manual that includes suggestions to make them as effective as possible. In addition, a CD-ROM will provide supplementary information such as sample letters and technical reports.
This opportunity is offered as part of a service/research partnership between users and AHEAD. At the first level, AHEAD will provide users with a service package that will include hosting the online questionnaire, data collection, and provision of raw data and code book for ease of data analysis. Partners will contribute their data to the AHEAD database, which eventually will enable benchmarking so that institutions will be able to compare their findings with grouped data from similar institutions and also monitor changes over time.
At the second service level, AHEAD will provide basic data analysis, (quantitative and qualitative technical reports). A third level of service will integrate the quantitative and qualitative findings and describe implications and make recommendations customized to the institution. At the fourth level, consultants will be available through AHEAD to help institutions implement the recommendations that emerge from the questionnaires.
As a result of this service/research partnership, it is hoped that we can move forward to make college campuses truly welcoming and enhance not only admission, but also graduation from higher education and beyond for all individuals with disabilities.