POSTERS: ECONOMIC & COST-UTILITY ANALYSIS
EVALUATING THE ECONOMIC COSTS AND SHORT-TERM PRACTICAL BENEFITS OF TWO IMPLEMENTATION STRATEGIES FOR AN ALCOHOL USE PREVENTION PROGRAM FOR ELEMENTARY SCHOOL CHILDREN. Raamses Rider1, Mary Lou Bell2, Tara Kelley-Baker1, Maria Lyakhovich1, 1Pacific Institute for Research and Evaluation, Calverton, MD United States; 2The Bell Group, Austin, TX United States
This study compares two strategies for implementing an alcohol use prevention program for school children. The Protecting You/Protecting Me (PY/PM) program, developed by Mothers Against Drunk Driving, is a classroom-based alcohol use prevention and vehicle safety program for elementary students with the ultimate goal of preventing the injury and death of youth from underage alcohol use and riding in vehicles with impaired drivers. The PY/PM educational curriculum has been implemented in two ways: using elementary school teachers to teach the curriculum, and using high-school students enrolled in a Peer Helper program to teach the curriculum. This study compares the economic costs and the short-term practical benefits of these two implementation strategies. Specifically, we calculate and compare the total costs and specific costs for each strategy, the total number of students reached with each strategy, the cost per student in each strategy, the effectiveness of each strategy on a range of outcome measures using effect size as a common metric, and the ratio of the difference in costs between the two strategies relative to the difference in effectiveness, among other analyses. Prior research has shown both of the strategies for implementing PY/PM to be effective, but a direct comparison of the two has been lacking until now. Conclusions and implications are presented as well as limitations of the study vis-à-vis the Standards of Evidence established by the Society for Prevention Research.
POSTERS: EFFECTIVENESS TRIALS
AN EVALUATION OF THE NORTH CAROLINA MEDIA LITERACY EDUCATION PROJECT (MLEP). Janis Kupersmidt1, Laura Feagans1, Michael Eisen2, Richard Hicks1, 1Innovation Research and Training, Inc., Durham, NC United States; 2North Carolina Department of Health and Human Services, Raleigh, NC United States
The current study presents results of an evaluation of the North Carolina Media Literacy Education Project (MLEP), a media literacy curriculum delivered by college students to middle school children to change their cognitions associated with alcohol and tobacco use. The MLEP is based on past research that has found teaching children critical viewing skills surrounding pro-drug media messages can make them less vulnerable to media persuasion. Consistent with the National Youth Anti-Drug Media Campaign (2001), the specific goals of the MLEP were to: (1) increase student recognition that alcohol and tobacco advertisements are biased messages designed to persuade viewers to use tobacco and alcohol products, (2) increase students´ ability to identify specific persuasive strategies used by alcohol and tobacco advertisers, (3) decrease students´ positive and increase negative attitudes and beliefs about tobacco and alcohol use, and (4) decrease students´ current use and future intention to use alcohol and tobacco.
Health Education classes in two racially diverse Southeastern middle schools were assigned to curriculum delivery and wait list control conditions. Students receiving active parental consent completed pre- and post-test questionnaires. College students serving as media literacy educators completed training prior to curriculum delivery and completed fidelity measures after each of the six lessons. 146 students participated in the intervention group while 239 students participated in a wait list control group.
The Media Literacy Education Program successfully achieved change in each of the main goals of the program. MANCOVAs, controlling for baseline differences between intervention and control groups in gender, race, grade, and previous substance use revealed that this short intervention produced significant change in the desired direction for most objectives. Intervention students (1) increased in media skepticism (this effect varied by the students´ past alcohol usage levels) (2) demonstrated a greater ability to recognize strategies used by advertisers to sell tobacco and alcohol (3) increased in their likelihood to report that teen smoking and drinking are wrong (while the control group reported finding teen smoking and drinking to be more acceptable at post-test) and (4) decreased in intentions to use alcohol and tobacco (while those in the control group increased in intentions to use alcohol and tobacco). This effect was more pronounced for boys than girls. Students also indicated they liked learning about media literacy. Consequently, media literacy education seems to be a valuable substance use prevention intervention.
REAL-WORLD IMPLEMENTATION: EFFECTS OF PROGRAM STRUCTURE AND PROVIDER CHARACTERISTICS. Katie Maucione1, Laura Hill1, 1Washington State University, Pullman, WA United States
The field of prevention has relied mainly on research demonstrating program efficacy to evaluate program success. Recently, large-scale effectiveness trials (e.g. the bi-state PROSPER project) have also documented factors related to successful program outcomes, dissemination, and sustainability in real-world settings. However, most science-based programming examined in these studies is both initiated and supervised by researchers. To date, we know little about the realities of provider-driven dissemination and implementation of evidence-based programs. Such knowledge is important, since real-world programming is most often initiated and conducted not by researchers but by providers, whose goals, priorities, and knowledge base may differ significantly from those of researchers. Ultimately, program effectiveness will depend on the extent to which those goals and capabilities shape program delivery outside of a research context. The purpose of the current investigation was to explore the real-world, provider-driven implementation of a science-based program. Our questions included 1) whether there were systematic deviations in the structure of the program-as-delivered from that of the program-as-intended; 2) why such deviations occurred; and 3) whether program structural characteristics and provider characteristics were related to short-term program outcomes.Efficacy of the Strengthening Families Program for Parents & Youth 10-14 (SFP) has been extensively documented in longitudinal trials. SFP has been widely disseminated in Washington State, sponsored by school districts, state agencies, and the Extension system of Washington State University. We tracked statewide dissemination and collected implementation and outcome information from 30 programs delivered by 75 providers and representing 568 attendees. We also conducted 60 in-depth provider interviews to investigate 1) reasons for, and 2) provider correlates, of deviations from program content and structure.Results indicated that many facets of implementation were substantially different from the program as intended and as researched in efficacy trials, and that these differences were related to proximal program outcomes. For example, although the program is intended for families with youth aged 10-14, actual youth ages ranged from 8-22, and the number of families per program ranged from 3-15 rather than 8-10 as documented in research trials. Program size, composition, and other structural variables were related to provider satisfaction and program coverage as well as to outcomes. We present these outcome-related findings and qualitative data documenting reasons for systematic program changes. Findings have implications for training, technical assistance, and real-world effectiveness trials.
THE IMPACTS OF A COMPREHENSIVE PRESCHOOL PROGRAM ON EMOTION SKILL AND SOCIAL COMPETENCE. Heather K Warren1, Brittany Rhoades1, Celene Domitrovich1, 1Pennsylvania State University, University Park, PA United States
Building young children´s social-emotional skill and fostering positive adult and peer relationships buffers the effects of social risk and provides important opportunities for language and cognitive development (Bierman, Greenberg & CPPRG, 1996). Increased interest in the development of universal prevention programs focused on the promotion of preschool social competence has led one urban, disadvantaged school district in Pennsylvania to develop an initiative to provide all eligible three and four year olds and their families with comprehensive educational, health, and social services. The initiative includes an enriched classroom environment which focuses upon child-directed learning and literacy. In addition, it utilizes the research-based preschool Promoting Alternative THinking Strategies curriculum (Domitrovich et al, 2000) to address social-emotional skill development. There is strong empirical evidence demonstrating the PATHS curriculum improves protective factors and reduces risk across a wide variety of elementary school-age children (Greenberg & Kusché, 1998). The current study contributes to the literature validating the program for preschool populations by presenting findings related to social-emotional skill promotion from a quasi-experimental evaluation of the program delivered to a primarily African-American (33%), Hispanic (10%) or multi-racial sample.
Four and five year old participants in the 2002-2003 program (N = 66 and N = 78) will be compared to district students not exposed to the program (N = 104 and N = 114), matched on school, gender, age and SES. Previous outcome analyses of the program in its first year indicated that the least competent four year olds in the program showed improvements in their social skills and a reduction in problem behaviors according to their teachers. Preliminary analyses of the emotion skill targets of preschool PATHS program suggest that, when compared to children in the district not exposed to the program, exposure to the program at three years of age significantly improved children´s conceptual understanding of labels for other children´s emotions (F = 7.29, p < .05). Their exposure to the program at four years of age significantly improved children´s expressive and receptive identification of basic emotions (F = 4.62, p < .05 and F = 25.808, p < .001, respectively), and the accuracy of their emotion attributions (F = 9.86, p < .05), and marginally improved their knowledge of conventional emotion situations (F = 2.71, p = .10). Forthcoming analyses will examine program impacts on children´s anger bias (Shultz & Izard, 2001) and the relations between the emotion targets and social competence outcomes. Child and parent moderaters of student success in the program will also be examined.
EFFECTIVENESS OF TRANSACTIONAL ANALYSIS (TA) PROGRAM FOR SCHOOL MALADAPTEES. Sung-Kyu Lee1, 1Washington University in St. Louis, St. Louis, MO United States
Purpose: The problems of school maladaptees are affected by several factors, including school environments, family environments, and individual factors. A Transactional Analysis (TA) program has been demonstrated to approach individual factors based on life position, which is an attitude of self and others around self. The purpose of this study is to examine the effectiveness of the TA program for school maladaptees in improving their life position and school adaptation ability.
Methods: The subjects of this study are 20 female school maladaptees, ages 14-15, referred by teachers in a middle school in Seoul, Korea. Baseline characteristics were matched between two groups: experimental group (n=10) and control group (n=10). The TA program was conducted with the experimental group, one time per week, for two hours, for eight weeks, while the control group did not have the program. The Life Position test (I´m OK-You´re OK, I´m OK-You´re not OK, I´m not OK-You´re OK, and I´m not OK-You´re not OK), and the School Adaptation test (relationship with teachers, relationship with peers, aptitude in class, and keeping school rules), were used to measure the differences in their life position and school adaptation ability. The Wilcoxon Matched-Pairs Signed-Ranks test was conducted both pre- and post- intervention of each group.
Results: The results indicated that students who participated in the TA program showed considerably positive changes in their life position and school adaptation ability. In terms of life position, students in the experimental group had significant changes in increasing I´m OK-You're OK (Z=-2.680, p=.007) and decreasing I'm not OK-You're not OK (Z=-2.539, p=.011). However, there were no significant differences in I´m OK-You´re not OK and I´m not OK-You´re OK between the two periods. In addition, students in the experimental group had significant improvement in their school adaptation ability (Z=-2.814, p=.005). Meanwhile, students in the control group did not have significant changes in either life position or school adaptation ability.
Implications for practice: Findings from this study indicate that the TA program is a useful intervention for school maladaptees in improving their life position and school adaptation ability. Finally, this study suggests that the TA program is useful for individual or group approaches such as school counseling for school maladaptees. In addition, it can be utilized in diverse settings such as youth counseling centers, shelters, and other youth service agencies, particularly for school maladaptees.
PICTURE SUCCESS-INNOVATIVE PREVENTION PROGRAM DEVELOPED IN MISSISSIPPI. Betty Streett1, Perry Moore1, 1Region I Mental Health, Clarksdale, MS United States
Picture Success is a multi-strategy innovative prevention program, focusing on enhancing protective factors, reducing risk factors, and possibly relieving depression in high-risk, alternative school adolescents. Through the use of photography and interview, Picture Success involves the youth with successful persons in their community, from the mayor to the driver of the trash truck, from the superintendent of schools to the lunchroom worker, allowing the young people to change their picture of success from rock and NBA stars and drug dealers to local people who serve their families and their communities in positive ways, making success achievable for them. The photos and interviews are displayed on a web site developed by the youth, and are displayed in area schools and libraries, as well as the Chamber of Commerce. Multi-strategies include; Information Dissemination, Prevention Education, and Alternative Activities. Risk factors addressed are; low self esteem, alienation, rebelliousness, favorable attitudes toward problem behavior, lack of school commitment, self defeating perceptions regarding substance abuse, and belief in substacne abuse myths. Protective factors addressed are; resilient temperamant, positive social orientation, bonding, healthy beliefs by increasing harm perception, and positive standards for behavior. Evaluation through University of Mississippi Department of Psychology and the University of Mississippi Social Science Research Lab use multiple evaluation instruments. They are; standardized student survey (compliant with CSAP Core Measures), Baruth Protective Facors Inventory, Perceived Harm Scale, and the Reynolds Adolescent Depression Scale.
ALL STARS SR.: HIGH SCHOOL UNIVERSAL PREVENTION. David Wyrick1, Muhsin Orsini2, Rita O'Sullivan2, Denise Halfors3, Christina Hardy4, Bill Hansen1, 1Tanglewood Research, Greensboro, NC United States; 2University of North Carolina at Chapel Hill, Chapel Hill, NC United States; 3Pacific Institute for Research and Evaluation, Chapel Hill, NC United States; 4Tanglewood Research, Inc., Greensboro, NC United States
Existing high school drug abuse prevention programs primarily target youths who already have defined drug use problems. Effective research-based prevention programs that target a universal audience are not yet widely available. This project examined All Stars, Sr., a program to improve mediating processes important to late onset prevention and self-initiated cessation. Mediators targeted include: lifestyle incongruence, normative beliefs about the prevalence and acceptability of drugs, commitment to avoid high-risk behaviors, resistance skills, beliefs about social and psychological consequences, goal setting, decision making, and stress management skills. The program is designed to augment (rather than replace) existing high school health textbooks. A randomized field trial was conducted, involving high schools in three states (North Carolina, Nebraska, and Missouri). There were 13 treatment and 11 control schools. Students were measured at three points in time over a two-year period to assess changes in mediators and the prevalence and frequency of substance use. At pretest, 3,129 students completed surveys. At the immediate posttest, there were 1,995 valid surveys. At the one year follow-up, we surveyed 1,711 students. There was no differential attrition by condition. At pretest, there was pretest equivalence for all measures. By the initial posttest, there were significant reductions among treated students in lifetime and past month alcohol use, risky alcohol use, problem behavior, and alcohol-related problem use. At the one year follow-up, effects of the program continued for past month alcohol use, problem behavior and problem use. There were no effects on other substances (tobacco, marijuana, inhalants, or illicit drugs). The program altered targeted mediating variables. At follow-up, compared to control students, students in the treatment group had improved goal setting, decision making, school attachment, stress management, commitments, lifestyle incongruence, perceived pressure to engage in risky behaviors, and resistance skills.
DO TRAINING DOSAGE AND EXPOSURE MAKE A DIFFERENCE IN COMMUNITY COALITION PROGRESS IN ADOPTING AND IMPLEMENTING EVIDENCE-BASED PREVENTION? THE STEP DIFFUSION TRIAL. Maykami Mc Clure1, Peter Bunce2, Anna De Jesus1, Jennifer Corniea1, Mary Ann Pentz1, 1University of Southern California, Alhambra, CA United States; 2One Great River, Inc., Shreveport, LA United States
An increasing question in taking evidence-based programs to scale is whether there is a critical dosage of prevention training and a critical proportion of consumers that are necessary to achieve diffusion effects. As part of a large multi-state, multi-city prevention trial, STEP, the present study evaluated the relationship of training dosage (% of 6 teleconference training sessions received) and exposure (% of targeted trainees who actually participated in training) to community coalition progress in achieving 12 outcomes as reported by community leaders (3 individual community leader, 8 community coalition, and 1 community). STEP is a large randomized prevention trial, with 24 cities in 5 states randomized to one of three conditions (technical assistance + teleconference prevention training, training alone, or prevention as usual control). A total of 928 community leaders, teachers, and parent representatives participated in training. Training consisted of 3 sessions aimed at community leaders (risk and protective factors assessment, community organization, media advocacy), 2 at teachers (media literacy, and drug resistance skills programs for students), and 1 at parents (family management). Proc.mixed regression analyses were conducted on community leader survey data from 227 community leaders with matching id´s and complete data from baseline to two year follow-up. Analyses were conducted at the individual level, controlling for city. Predictors were overall dosage and exposure, based on a combination of research staff ratings and site facilitator archival reports. Covariates included assigned experimental group (technical assistance + training, training alone, standard “prevention as usual”). Results showed that after controlling for effect of the entire intervention (experimental group or treatment effect), % of targeted trainees had no additional effect, and dosage had only marginal effects on one individual outcome (intentions to participate in the coalition in the future), one community outcome (readiness), and one organizational outcome (empowerment). Findings suggest that overall involvement in prevention, rather than particular dosage or numbers of trainees reached in prevention training, may be a better indicator of community prevention outcomes.
THE IMPACT OF A COMPREHENSIVE PRESCHOOL PROGRAM TARGETING SCHOOL READINESS: EXAMINING THE AGE OF PROGRAM EXPOSURE. Celene Domitrovich1, Nicole Freeman1, Linda Jacobson1, 1Pennsylvania State University, University Park, PA United States
One of the ways to counteract the effects of poverty for young children is to ensure that they receive a high quality early childhood educational experience. Programs that focus on emergent literacy and social-emotional skills have the potential to protect children against negative outcomes such as mental health problems and school failure. One urban school district in Pennsylvania city has developed a comprehensive full-day preschool program to provide all eligible 3 and 4 year old children with the services they need to achieve academic and social success. The program uses the Head Start model but enhances the curriculum by adding a social emotional curriculum and research-based literacy activities. The program is being evaluated using a quasi-experimental design with a matched comparison group. Child assessments were conducted in the fall of each program year. Teacher ratings were gathered at the beginning and end of each program year. Kindergarten outcomes for participants exposed to one year of the program have already been presented. The purpose of this study is to present findings regarding the same child outcomes (direct assessments and teacher report of behavior) for a younger population (i.e., outcomes at age 4) and to compare these to the outcomes for participants who were exposed to the program when they were one year older. The sample used for participant verses non-participant comparisons was the group of children new to the program at age 4. The significance of within group change scores (pre to post) on standardized measures was tested using paired t-tests and compared to findings from the national Head Start evaluation (FACES study). Findings indicated significant increases from pre to post in children´s vocabulary (PPVT-R; Dunn & Dunn, 1981), emergent literacy, math and writing skills (Woodcock & Johnson, 1989) and these increases were larger than those found in the older participant group. Between group differences (participants vs. non-participants) were analyzed using ANOVA and found on the majority of child measures. Participants had higher vocabulary scores (F (1,139) = 8.27, p < .01), emergent reading abilities (F(1,139) = 9.50 p <.01), early numeracy skills (F(1, 139) = 6.6 p < .05). Between group differences favoring the intervention group were also found on teacher ratings of child problems behaviors. Findings will be discussed in term of their implications for the timing of early intervention efforts and the importance of participant retention.