Evaluation of an infused alcohol and drug prevention programme

Abstract

Purpose

Alcohol, tobacco, marijuana and other drugs use typically increases in prevalence and frequency during middle and late adolescence. School health instruction often focusses on providing facts and rarely provides tools for addressing the psychosocial risk factors needed to prevent substance use. The purpose of this paper is to report about the effectiveness of a prevention programme delivered in US high school health classes. The intervention augments typical instruction by providing teachers with activities that can be infused in their daily teaching.

Design/methodology/approach

In total, 26 schools were randomly assigned to receive the intervention or serve as controls. Pupils were pretested near the beginning of the school year, posttest near the end of the school year and administered a final test near the beginning of the following school year. Teachers in treatment schools were provided with activities designed to target psychosocial variables known to mediate substance use onset and self-initiated cessation. These include normative beliefs, intentionality, lifestyle incongruence, beliefs about consequences of use, peer pressure resistance skills, decision-making skills, goal setting skills and stress management skills.

Findings

Hierarchical modelling analytic strategies revealed the intervention to have definable positive impacts on alcohol and cigarette use. Moreover, the intervention had strongest effects on alcohol and cigarette use among pupils who were identified at pretest as being lower-than-average risk.

Originality/value

This research provides support for providing teachers with a strategy for preventing alcohol, tobacco and other drugs that can be used in a flexible manner to augment the instruction they are already mandated to provide.

Citation

Orsini, M.M., Wyrick, D.L., Hansen, W.B., O’Sullivan, R.G., Hallfors, D., Steckler, A.B. and Ridenour, T.A. (2019), "Evaluation of an infused alcohol and drug prevention programme", Health Education, Vol. 119 No. 3, pp. 230-243. https://doi.org/10.1108/HE-07-2018-0035

Most drug prevention programmes that have been developed and evaluated in school settings are delivered in the elementary or middle school grades (Das et al., 2016; Faggiano et al., 2014; Tobler and Stratton, 1997). There are comparatively few experimental studies of universally targeted high school-delivered primary prevention and non-clinical approaches to early cessation. In their review of school-based programmes, Onrust et al. (2016) identified nine universal smoking prevention programmes, six alcohol prevention programmes and six drug use prevention programmes that targeted 10th through 12th graders. These include evaluations of the High School Anti-Smoking Project (Johnson et al., 1986), Class Action (Perry et al., 2002), Project SUCCESS (Morehouse and Tobler, 2000), Positive Action (Flay and Allred, 2003; Flay et al., 2001) and Project SPORT (Werch et al., 2005).

Botvin created and tested a high school version of Life Skills Training (Botvin et al., 2015; Resnicow et al., 2008). The programme consisted of ten sessions that were based generally on the middle school Life Skills Training programmes. Sessions focussed on teaching generic personal and social skills and included activities that focussed on teaching about the consequences of use, correcting erroneous normative beliefs and developing skills for resisting peer and media pressure to use substances. The evaluation included measures of the frequency of smoking, drinking, drunkenness and marijuana use. An evaluation of this universal high school prevention programme demonstrated reductions in daily alcohol use, drunkenness, cigarette smoking and marijuana use.

Most recently, Vallentin-Holbech et al. (2018) examined a social norm universal intervention aimed at 13- to 17-year-old pupils in Denmark and found positive effects. Overall, the research on high-school-based alcohol and drug prevention is promising, with several of the studies cited presenting positive outcomes.

Several programmes available for high school age pupils are designed to target select populations of at-risk pupils. These include Project Towards No Drug Abuse (TND) (Rohrbach, Gunning, Sun and Sussman, 2010; Sussman et al., 2002) and Reconnecting Youth (Eggert et al., 1990; Eggert, Thompson, Herting and Nicholas, 1994; Eggert, Thompson, Herting, Nicholas and Dicker, 1994). These at-risk-student-focussed programmes have been implemented with mixed success (Hallfors et al., 2006; Shamblen and Ringwalt, 2008; Sun et al., 2006).

In addition to its implementation as an indicated programme tested for its effectiveness in continuation high schools (Dent et al., 1998; Lisha et al., 2012; Sun et al., 2006; Sussman et al., 2003, 2012), TND has also been implemented as a universal intervention evaluated with a regular high school sample (Dent et al., 2001; Little et al., 2013; Rohrbach et al., 2007; Rohrbach, Gunning, Sun and Sussman, 2010; Rohrbach, Sun and Sussman, 2010). The initial universal version of TND consisted on nine classroom sessions delivered over a three-week period, with a focus on communication and listening skills, avoiding stereotypes and correcting erroneous normative beliefs, consequences of drug abuse, goal setting and lifestyle incongruence, stress and coping, increasing self-control, decision making and building a personal non-abuse commitment. In the first trial involving regular high school evaluation projects, reductions in alcohol and hard drug use (cocaine, hallucinogens, stimulants, etc.) were observed, but no effects on cigarette or marijuana use (Dent et al., 2001; Sussman et al., 1998, 2002). In the second regular high school trial, the programme was expanded to include additional sessions that targeted cigarette and marijuana use. With this expanded version, researchers observed effects for alcohol, cigarettes, marijuana and hard drugs (Sun et al., 2006; Sussman et al., 2003). In the final trial involving regular high schools, a national dissemination study was conducted and showed programme effects for marijuana and hard drug use, but only for baseline non-users (Rohrbach, Sun and Sussman, 2010).

There is a significant need for effective universal prevention for high school pupils that can be widely disseminated. As evidenced by Monitoring the Future, an annual survey that samples approximately 50,000 US 8th, 10th and 12th graders (Miech et al., 2017), it is during middle adolescence that the greatest increases in initial use and overall frequency of use of substances occur. For example, the proportion of 10th and 12th pupils who report alcohol use during the past 30 days increases markedly (19.9–33.2 per cent, respectively) as does the proportion who report past month cigarette smoking (4.9–10.5 per cent, respectively) and marijuana use (14.0–22.5 per cent, respectively). Deterring the onset of consumption, reducing the frequency of consumption and reducing problematic situational use remain a critical concern of society.

In high schools, drug education is typically included as part of a semester-long health class. As a preliminary step in the current research project, we reviewed a variety of high school health textbooks. We found that textbooks and course presentations focus exclusively on what drugs are and what health consequences ensue from ingesting them (Hansen and McNeal, 1999; Tolan and Lounsbury, 1982). In contrast, our strategy was to provide teachers with easy-to-implement activities that focus on targeted mediators known to be associated with behaviour and attitude change. The intervention we examine here, All Stars Senior, did not aim to replace existing textbooks but was promoted as a strategy for augmenting instruction with highly engaging activities that could be taught as part of regular classroom instruction. This provided a way of infusing psychosocial strategies as supplements to regular class work. This strategy involved providing teachers with a menu of interactive activities that might increase pupils’ engagement and effectiveness of health education (Hansen et al., 2019).

The programme content to be included in the intervention was based on research findings that suggested psychosocial variables that are worthy of targeting for change (Hansen and McNeal, 1996), variables that predicted late onset of (McNeal and Hansen, 1999) and self-initiated cessation from (Hansen and McNeal, 2001) the use of alcohol, tobacco and marijuana. From this research, variables selected for intervention included manifest commitment to avoid drug use, peer pressure resistance skills, normative beliefs (perceiving drug use to not be common or acceptable to the peer group), beliefs about negative social consequences from using substances, lifestyle incongruence (perceiving drug use to interfere with life goals), decision-making and impulse control skills and goal setting skills.

The variables targeted for change in All Stars Senior are like those targeted in many school-based drug-prevention programmes. For example, lifestyle incongruence, commitment and normative beliefs are targeted in the middle school All Stars Core (Harrington et al., 2001). Decision-making skills, goal setting skills and peer pressure resistance skills training are targeted in All Stars Plus (Hansen and Dusenbury, 2004). Many of these mediators are also targeted in widely disseminated middle school programmes, such as Project Alert (Ellickson et al., 1993), Life Skills Training (Botvin and Diaz, 1995; Botvin et al., 2015), Keepin’ It REAL (Hecht et al., 2003), Taking Charge of Your Life (Sloboda et al., 2009) and TND (Dent et al., 2001; Sussman et al., 2003). These topics can also be found in risk-group-focussed high school interventions such as Reconnecting Youth (Eggert, Thompson, Herting and Nicholas, 1994).

Classroom observations have revealed that the central strategy utilised by classroom teachers for preventing substance use among high school pupils is to provide facts about drugs and list potential consequences of use (Hansen and McNeal, 1999). Improvement in pupil knowledge about drugs and consequences of use, however, has not been shown to significantly change pupil attitudes and related behaviour (Connell et al., 1985). In contrast, most research-based prevention programmes draw from multiple theories and use a variety of approaches (Hansen et al., 2006). Thus, while variables were selected because of observed statistical relationships, it is noteworthy that these mediators represent a diverse set of theories and assumptions about individual behaviour such as Social Learning Theory (e.g. peer pressure resistance) (Bandura and Walters, 1977), the Theory of Reasoned Action (e.g. intentions and normative beliefs) (Ajzen and Fishbein, 1988) and the Health Belief Model (e.g. beliefs about consequences) (Rosenstock, 1990).

The purpose of this paper is to report the effects of All Stars Senior on substance use, problem behaviour and substance use associated with problem behaviours. This project was completed between 2001 and 2004 (Orsini et al., 2012).

Method

Participants

Schools in the states of Nebraska, North Carolina and Missouri participated in this study. Information was sent to high school principals and district administrators informing them about the project and requesting a response if there was interest. Site visits were made to each school that responded. In total, 32 schools indicated interest in the project. Of these, 26 (80 per cent) agreed to participate. An approximately equal number of schools came from urban/suburban and rural settings.

Schools that agreed to participate were randomly assigned to the treatment or control conditions after stratifying within state for urbanicity, school size, ethnicity and per cent of pupils receiving free or reduced lunch. Each stratifying variable was standardised with a mean of 0 and standard deviation of 1. The differences between standardised scores of each pair were calculated. Nearest neighbours were paired and randomly assigned to condition.

In all, 13 schools each were assigned to the treatment condition (North Carolina=8, Nebraska=3 and Missouri=2) and the control condition (North Carolina=8, Nebraska=3 and Missouri=2). After assignment to conditions, two control schools had a change in principals and never participated in any data collection and were dropped from the study, leaving 24 schools. Health teachers (n=27) from treatment schools participated by delivering All Stars Senior to pupils.

All pupils enrolled in high school health were invited to participate in the evaluation (n=3,922). Financial incentives were provided to schools and teachers throughout the study based on level of adherence to the study protocol, and classrooms received a free pizza/healthy snack party at the conclusion of the study.

Parents were provided the opportunity to withdraw their child from the study and pupils could withdraw using an opt-out informed consent procedure that had been approved by the federal funding agency and the responsible Institutional Review Boards (Human Subjects Protection Committee). Additionally, the funding agency provided a “certificate of confidentiality” to ensure that data collected would be protected from any attempt to reveal the identities of participants. The parents of 121 pupils (3 per cent of the population) did not provide consent for their teen to participate and 344 pupils (9 per cent of the population) did not provide personal assent to participate. The remaining 3,467 (88.4 per cent of the population) were invited to complete the pretest survey. A total of 3,129 (90.2 per cent of those pupils eligible to participate in the pretest survey) completed pretest surveys at the beginning of the school year. Of those who completed pretests, 1,992 (61.8 per cent of the pretest sample) completed immediate posttest surveys at the end of the same school year after the programme had been completed. At the beginning of the following school year, 1,709 (54.6 per cent of the pretest sample) participated in one-year follow-up surveys. Analyses were based on pupils who completed the pretest and at least one of the follow-up surveys (2,415; 77.2 per cent of pretest sample).

The final sample included 1,547 pupils from schools assigned to the treatment condition and 868 control group pupils. The pupil sample was 52.3 per cent female, 67.4 per cent Caucasian, 19.0 per cent African-American, 5.6 per cent Hispanic, 2.4 per cent Asian and among the remainder, 3.9 per cent classified themselves as being more than one race. About half of the pupils (49.7 per cent) were 14 years old with another large proportion (29.2 per cent) reporting they were 15 years old, 7.4 per cent were 16 years old, 7.6 per cent were 13 years old and 6.1 per cent either 17 or 18 years old. Gender (females were 55.0 per cent treatment and 56.7 per cent control; χ²=0.44, ns), race (whites were 69.9 per cent treatment and 71.1 per cent control; blacks were 11.7 per cent treatment and 14.5 per cent control; χ²=1.51, ns) and age distributions (t=0.81, ns) did not differ between conditions. Most pupils in the treatment condition (65.6 per cent) and control condition (88.6 per cent) were in 10th grade at the time of the pretest.

Intervention

The All Stars Senior curriculum manuals provided 24 activities related to ATOD, with an additional 18 activities related to personal health and 21 activities related to nutrition. These were designed to be interactive and engaging, using small group activities, games, discussions and worksheets that asked pupils to apply knowledge to real-world tasks. For each topic, targeted mediating variables included normative beliefs, commitment to avoid risky behaviour or participate in healthy behaviour, perceptions that risky behaviours would interfere with desired lifestyles and healthy behaviours would contribute to desired lifestyles, and the development of goal setting, decision-making, stress management and peer pressure resistance skills. Table I presents the number of activities available to teachers in each of the three manuals.

Measures

Pupil data were collected using a 134-item survey. Behavioural outcome measures included self-reports of lifetime and past 30-day alcohol consumption (beer, wine, wine coolers and liquor), risky alcohol use (drunkenness, riding in a car with someone who had been drinking and driving after drinking), tobacco use (cigarettes, cigars and chewing tobacco), use of marijuana and hashish, inhalants, hallucinogens, MDMA (ecstasy), PCP, cocaine, crack, methamphetamine, heroin, amphetamines, barbiturates, benzodiazepines, narcotics, flunitrazepam (Rohypnol) and steroids, and the use of needles to inject drugs. Indices were calculated for alcohol use, risky alcohol use, tobacco use, marijuana use, inhalant use and illicit drug use. For each index, calculated values ranged from “0” to “more than 51 occurrences.”

Pupils also reported whether they experienced any of a variety of problems since the start of the school year including being sent out of a classroom for bad behaviour, being suspended, skipping school or class, destroying something on purpose, having been in a fight, having been in trouble with parents, having been in trouble with police and having been involved in a serious accident. Response options included 0 (did not happen), 1 (happened once) and 2 (happened more than once). Pupils’ responses were summed across items to create a problem behaviour index. Pupils were also asked if these problems had occurred on days they had been drinking alcohol, smoking marijuana or using illegal drugs. Similar ratings (0–2) were applied and pupils’ responses were summed across items to create a problem use index.

Procedure

Treatment teachers received 6 h of face-to-face training. A prevention specialist explained prevention theory and health education research that relate to the curriculum, introduced the manual and described activities. Teachers were asked to implement three required activities and three additional activities during each week of instruction in their high school health class.

Pupils in treatment and control groups completed pretest surveys at the beginning of the school year, posttest surveys at the end of the same school year and the one-year follow-up survey at the beginning of the next school year. Schools were encouraged to administer each of the pupil surveys within a two-week window of receiving them via postal mail. Surveys were identified with unique imprinted bar code identifiers, so that pupil responses could be matched across administrations.

Analysis plan

Hierarchical modelling (Collins, 2006) was used to test programme effects on trajectories of substance use and problem behaviour outcomes with Mplus 5.1 (Muthén and Muthén, 2008). Analyses consisted of three-level hierarchical models, the most levels that can be analysed with this software. Teacher effects were statistically controlled by analysing them at the highest level (using the CLUSTER option). Intervention effects (differences in trajectories between control and treatment pupils) were analysed as condition × time interactions at the second level, and at the lowest level were within-individual growth, demographics (age, gender and race), and interactions between condition × high-risk status or demographics. None of the condition × demographics interactions were statistically significant (p>0.05). The best fitting model for each outcome (intercept, linear slope and quadratic slope) (Muthén and Muthén, 2010) was determined using likelihood ratio χ² and sample-size adjusted BIC to accommodate changes in sample size between analyses due to missing data (Hedeker and Gibbons, 2006; Schafer and Graham, 2002).

High-risk status was based on cluster analysis that found a two-group solution using pretest recent behaviours (alcohol consumption, risky alcohol use, tobacco use, marijuana use, inhalant use and illicit drug use), problem behaviour and problem use as clustering variables. In total, 159 (11 per cent) treatment and 97 (12 per cent) control pupils were classified as high-risk using this method.

Two types of hierarchical models were conducted, one for continuous outcomes, the other for outcomes with large percentages of zeros. Continuous outcomes were lifetime and 30-day alcohol use, lifetime risky alcohol use, lifetime and 30-day tobacco use, lifetime marijuana use, and 30-day problem behaviour and problem use. These models were estimated using linear hierarchical modelling and maximum likelihood with robust standard errors (Hedeker and Gibbons, 2006). To model the effect of time, fixed factor loadings for trajectories equalled the number of months into the study that data were collected (pretest=0 months).

For outcomes with large percentages of zeros, zero-inflated Poisson (ZIP) models (Olsen and Schafer, 2001) were used and estimated with maximum likelihood with robust standard errors. ZIP models are also referred to as two-part models because the logistic regression part models the difference between a zero vs non-zero outcome, whereas the linear hierarchical part models growth among the non-zero outcomes. ZIP models tested efficacies for outcomes of 30-day risky alcohol use (76.6 per cent=0), 30-day illegal drug use (87.2 per cent=0), lifetime illegal drug use (78.8 per cent=0), 30-day inhalant use (96.8 per cent=0), lifetime inhalant use (92.7 per cent=0) and 30-day marijuana use (87.7 per cent=0). In addition to the second-level and lowest-level (condition × high-risk status) tests for efficacy (described earlier), ZIP models tested for intervention effects over time for the binary outcome of zero vs non-zero outcomes.

Human subjects protection

This project was reviewed and approved by two institutional IRBs associated with Tanglewood Research and the University of North Carolina at Chapel Hill. In addition, each school district reviewed and approved parent informed consent letters and pupil assent procedures prior to agreeing to participate in the research project.

Results

Exposure

Teachers in the treatment group delivered between 6 and 65 All Stars Senior activities (M=13.6, SD=13.4) to pupils. In all, 4 teachers taught 9 or fewer activities, 13 teachers taught between 10 and 19 activities, and 8 teachers taught 20 or more activities.

Alcohol and drug prevalence

Table II presents descriptive statistics across pretest, posttest and follow-up surveys for all pupils. The means represent the average number of times used for the measure. For example, at pretest, pupils in the treatment group reported an average consumption of 5.10 alcoholic beverages during their lifetime, whereas pupils in the control group reported an average consumption of 6.08 alcoholic beverages during their lifetime. For almost all measures, pupils in both the treatment and control groups reported an increase of use from the pretest to follow-up.

Retention and attrition

There was differential attrition between conditions (χ²=78.81, p<0.001) for those who completed any posttest. Fewer control pupils (69.1 per cent) than treatment pupils (82.6 per cent) completed at least one posttest survey. Retention rates for control and treatment pupils, respectively, were 59.7 and 66.3 per cent at the immediate posttest (χ²=14.51, p<0.001), and 54.5 and 54.7 per cent at the one-year follow-up survey (χ²=0.01, ns). Among those who completed the initial posttest, there was no difference between groups in the proportion who then completed the follow-up survey (χ²=0.13, ns). These results suggest that data are at least missing at random and possibly missing completely at random (Schafer and Graham, 2002). In analyses comparing treatment and control pupils who only completed pretests, only 1 variable out of 14 outcomes, pretest 30-day tobacco use, showed statistically significant difference (t=2.68, df=615, p=0.007). Treatment pupils who completed only the pretest on average consumed more tobacco (M=2.94) than did similar control pupils (M=1.94). Overall, these results suggested that attrition was not systematic.

Efficacy

Table II presents efficacy estimates from linear hierarchical models, controlling for age, gender, minority race and risk cluster. Each of the hierarchical linear models estimated a reduced increase in lifetime alcohol consumption, lifetime risky alcohol use, 30-day alcohol consumption and 30-day tobacco use associated with receiving All Stars Senior. Statistically significant interaction intervention effects with risk group support the idea that greater intervention effects were seen for the high-risk group.

Cohen’s d (Cohen, 1988) effect size estimates (Table III) were small for outcomes in the low-risk group. Programme efficacies in the high-risk group were small for lifetime alcohol use and lifetime risky alcohol use; however, effect sizes for the high-risk group were medium to large for 30-day alcohol and tobacco use. Comparing treated and control pupil’s outcomes for the high-risk sample did not yield meaningful effects for lifetime alcohol use, lifetime problem alcohol use, 30-day alcohol use or 30-day tobacco use. Programme effects were modest for low-risk pupils’ 30-day alcohol use at the end of year posttest and largest for this groups’ 30-day alcohol use at the follow-up survey. Effects of the programme were pronounced for low-risk 30-day tobacco use. This was true at both the end of year posttest as well as the follow-up survey conducted in the following school year, which was a reversal from the observed pretest difference between conditions. Overall, the programme had effects for low-risk pupils; less so for high-risk pupils.

Efficacy estimates from ZIP hierarchical models, controlling for age, gender, minority race and risk cluster were unstable and did not permit adequate testing of alternative models for the following outcomes: past month risky alcohol use, lifetime illicit drug use, past month illicit drug use, lifetime inhalant use, past month inhalant use, past month marijuana use and problem use.

Discussion

Interpretation of findings

All Stars Senior provides a universal approach to alcohol and drug prevention and health promotion that allows the intervention to be infused into standard high school health education courses. Although health education textbooks are plentiful, our review of these revealed that psychosocial interventions for drug prevention, personal health and nutrition were lacking. All Stars Senior provided teachers with easy-to-use activities that encouraged increased attention to appropriate psychosocial variables that prior research has demonstrated to be functional mediators of preventive outcomes. Based on what is currently available in the drug prevention literature, this research embodies the first attempt at providing an infused programme. Even though this project involved a randomised control trial, it must nevertheless be considered to be a pilot project. As such, results observed can only be thought of as an initial attempt at infusion.

Prior to our efforts to develop activities, we conducted an analysis of commonly adopted high school health education textbooks. We then developed age-appropriate activities. We then cross-referenced the activities we had developed with chapters in these textbooks. We hoped this would provide teachers with a ready resource they could use to select All Stars Senior activities.

Teachers were required to teach three activities. Over 80 per cent of teachers taught many more than the required number. This suggests that the approach of providing teachers with engaging activities that were naturally linked with the content they otherwise already need to cover is entirely feasible. It further suggests that providing such a resource does not place an additional burden. That teachers taught more than was required supports the idea that they generally did not perceive implementing activities to require extra effort on their parts.

All Stars Senior’s primary effects were observed for reducing increases in alcoholic beverages consumption and the extent to which pupils began using tobacco. There were no effects of the programme on substances used less often by pupils such as marijuana, inhalants and illicit drugs. This is not surprising since the primary emphasis of All Stars Senior activities was on preventing alcohol and tobacco use.

Unlike programmes that specifically target high-risk pupils (e.g. TND; Rohrbach, Gunning, Sun and Sussman, 2010; Rohrbach, Sun and Sussman, 2010; Sussman et al., 2002), All Stars Senior targets a universal audience. The programme was most effective for pupils who had relatively little experience with alcohol and tobacco. Among low-risk pupils, the programme appeared to suppress the level of drinking. There was a deterrence of drinking among those pupils identified at who had little experience with alcohol. The programme is intended to be a universal intervention, and that is where primary benefits were observed.

In accordance with recent findings (Onrust et al., 2016) that found that universal programmes that targeted high school pupils were effective when they address problem solving and decision-making skills, refusal skills and social norms. They conclude that, overall, the social influences approach is especially effective for preventing alcohol use. In addition, they conclude that programmes also addressed future planning, akin to our approach that emphasised developing perceived lifestyle incongruence with alcohol and other drug use. Our findings corroborate these findings as these were among the topics included in All Stars Senior.

Implications for practice

All Stars Senior was designed to allow health teachers freedom to choose from a menu of activities. This adaptive approach for achieving the goal of infusing prevention strategies into regular classroom instruction is unique and may enable wider dissemination of programmes. From an evaluation perspective, however, this freedom reduces the ability of evaluators to understand how teachers achieved results. Teachers were required to deliver only three specific activities. Beyond that, what they chose to deliver varied greatly. Therefore, what constitutes core components of the intervention cannot be specified. Attributing cause in this case is problematic. For this evaluation, it is not possible to define how results were achieved or which activities must be taught for future implementations to replicate effects. Given general concerns in the field of prevention about fidelity of implementation (Durlak and DuPre, 2008; Dusenbury et al., 2003; Hansen et al., 2019), the quality with which All Stars Senior must be delivered to achieve effects is not yet known. Should the programme become disseminated, future research ought to pay special attention to this issue.

Infusion of drug prevention into standard health education courses has great appeal in that it allows teachers great flexibility. This strategy also fits within the social ecology of how health education is delivered. Additionally, health textbooks are typically fact based and do not provide teachers activities that address psychosocial processes known to be related to desired prevention outcomes. Prior research (Hansen and McNeal, 1999) demonstrates that teachers have little natural ability to create lessons on their own that address these issues. Our approach allows teachers to select activities from a menu, which increases the potential for those who use standard health education textbooks to deliver activities that engage pupils that would otherwise not exist.

Many evidence-based prevention programmes include activities that are similar to those provided in All Stars Senior. The difference between such programmes and All Stars Senior is that these other programmes require activities to be sequenced and completed in toto. One might assume that if other programmes’ activities were linked to textbook content then a similar approach might be feasible and adoption rates could be increased. However, as this idea has not been tried, we cannot draw conclusions about whether this would work or not.

Limitations

There are several methodological issues that must be noted that limit the validity of this research. Differential attrition has the potential to limit internal validity and, in the case where attrition in the control group is greater than the treatment group, may mask programme effectiveness. Prior research has demonstrated that pupils who drop out of studies tend to be more likely to be substance users (Hansen et al., 1985), a threat to external validity. In this evaluation, early dropouts – those pupils from both the treatment and control conditions who completed the pretest and no other survey – used tobacco more heavily than those who were retained. Late dropouts – those pupils from both the treatment and control conditions who completed the immediate posttest but not the one-year follow-up survey – used more alcohol and tobacco than those who were retained. Among pupils who only completed the pretest and immediate posttest surveys, control pupils were more likely than treatment pupils to have engaged in alcohol and tobacco use. This differential attrition leads to one possible caveat to conclusions about the observed effectiveness of the programme: It is possible that a truer estimate of the rates of these two behaviours for the control group at the final posttest might be higher than was observed. If this were the case, the effect size estimates of All Stars Senior on alcohol consumption and tobacco use may have under-estimated long-term effectiveness. Nonetheless, differential rates of attrition reduce confidence in interpreting data.

Table I

Availability of activities in each of three health education manuals

Targeted construct	ATOD	Personal health	Nutrition
Creating positive norms	4	4	3
Lifestyle incongruence	5	1	4
Resistance skills	2	1	3
Decision making	4	2	3
Beliefs about consequences	4	3	3
Goal setting	2	7	4
Stress management	3	0	1
Total activities	24	18	21

Table II

Descriptive statistics for behavioural variable measures

	Pretest (n=2,415)		Posttest (n=1,992)		Follow-up (n=1,709)
	M (SD)		M (SD)		M (SD)
Measure	Treatment	Control	Treatment	Control	Treatment	Control
Lifetime alcohol consumption	5.10 (10.09)	6.08 (11.16)	9.66 (12.93)	11.32 (13.83)	7.14 (12.17)	8.40 (13.27)
Past month alcohol consumption	1.04 (2.78)	1.23 (3.27)	1.34 (3.03)	1.88 (3.78)	1.40 (3.19)	1.93 (3.84)
Lifetime risky alcohol use	3.81 (8.87)	5.14 (11.19)	4.32 (10.06)	5.77 (11.84)	4.80 (10.80)	5.04 (10.41)
Past month risky alcohol use	0.30 (0.72)	0.37 (0.88)	0.42 (0.92)	0.50 (0.99)	0.40 (0.88)	0.45 (0.91)
Problem behaviour	2.79 (0.35)	2.80 (0.34)	2.72 (0.40)	2.68 (0.42)	2.82 (0.33)	2.77 (0.39)
Problem use	2.90 (0.36)	2.89 (0.38)	2.88 (0.38)	2.83 (0.45)	2.91 (0.33)	2.86 (0.41)
Lifetime tobacco use	4.75 (9.95)	5.64 (11.41)	5.29 (11.44)	6.34 (12.45)	5.37 (11.13)	6.33 (11.98)
Past month tobacco use	1.27 (3.37)	1.50 (3.95)	1.61 (4.10)	1.93 (4.58)	1.57 (3.86)	1.95 (4.41)
Lifetime marijuana use	2.31 (7.26)	2.55 (8.30)	2.82 (8.51)	3.04 (8.80)	2.69 (8.20)	3.14 (8.39)
Past month marijuana use	0.59 (2.42)	0.67 (2.75)	0.98 (3.32)	0.98 (3.19)	0.86 (3.17)	1.01 (3.14)
Lifetime inhalant use	0.67 (4.60)	0.82 (5.37)	0.96 (5.97)	0.89 (5.30)	1.02 (6.12)	1.14 (6.40)
Past month inhalant use	0.17 (1.50)	0.29 (2.20)	0.47 (2.60)	0.44 (2.52)	0.36 (2.30)	0.51 (2.70)
Lifetime illicit drug use	0.59 (3.48)	0.60 (3.01)	0.71 (3.53)	0.88 (3.67)	1.06 (4.82)	0.80 (3.76)
Past month illicit drug use	0.15 (1.13)	0.22 (1.37)	0.40 (2.00)	0.54 (2.32)	0.39 (1.89)	0.44 (2.07)

Table III

Efficacy estimates among risk groups

	Time 1 Means (SD)	Time 1 Cohen’s d	Time 2 Means (SD)	Time 2 Cohen’s d	Time 3 Means (SD)	Time 3 Cohen’s d
Lifetime alcohol use
High-risk controls	24.2 (17.2)	0.12	24.4 (15.8)	0.12	21.5 (17.4)	0.08
High-risk All Stars	23.5 (16.3)		21.4 (15.5)		16.5 (17.1)
Low-risk controls	3.8 (7.6)	0.04	9.8 (12.7)	0.19	7.0 (11.9)	0.29
Low-risk All Stars	3.0 (6.3)		8.4 (11.9)		6.1 (11.0)
Lifetime problem alcohol use
High-risk controls	13.4 (18.0)	0.15	11.8 (16.4)	−0.15	9.0 (14.7)	−0.06
High-risk All Stars	12.7 (15.9)		12.0 (15.8)		11.9 (16.7)
Low-risk controls	3.9 (9.1)	0.04	5.1 (11.1)	0.01	4.7 (9.9)	0.18
Low-risk All Stars	2.7 (6.8)		3.5 (8.8)		4.1 (9.7)
30-day alcohol use
High-risk controls	7.3 (6.4)	0.08	4.9 (5.3)	0.16	5.3 (5.6)	0.13
High-risk All Stars	6.2 (5.8)		3.7 (4.5)		2.7 (4.1)
Low-risk controls	0.5 (1.5)	0.18	1.6 (3.4)	0.24	1.6 (3.4)	0.54
Low-risk All Stars	0.4 (1.1)		1.1 (2.7)		1.2 (3.0)
30-day tobacco use
High-risk controls	11.1 (4.8)	0.11	8.8 (6.5)	0.06	7.2 (6.4)	0.06
High-risk All Stars	9.8 (4.5)		6.8 (6.4)		5.4 (5.8)
Low-risk controls	0.2 (0.9)	−0.28	1.2 (3.6)	0.31	1.4 (3.7)	0.30
Low-risk All Stars	0.3 (0.9)		1.0 (3.3)		1.2 (3.4)

Notes: For comparison between two means, Cohen’s (1988) d equals the number of standard deviations by which they are separated. For interpretive purposes, Cohen described d=0.2 as small, d=0.5 as medium and d=0.8 or greater as large. Tabulated results statistically control for age, gender and race