Evaluation of Environmental Classroom Option Program
(ECO)
Rob Lillis
Evalumetrics Research
2002
Summary
An evaluation of the Marcus Whitman ECO Program found that upon return to the
mainstream school in 8th grade, ECO students academically perform at least as well as
students who were in the mainstream school throughout grades six through eight.
In terms of Risk and Protective Factors, compared to the control group, ECO students
were significantly more likely to enjoy being in school and to find their courses to be
interesting. ECO students were also significantly more likely to report their parents expect
then to call if they are late and significantly more likely to report clear family rules
regarding alcohol and drug use.
ECO students were significantly more likely to report that teachers ask them to work on
special projects and that they have lots of chances to be part of class discussions and
activities. ECO students were also significantly more likely to report that teacher notice
when they do a good job and that teachers praise them for hard work. ECO students
scored significantly higher on the protective factor scale, “Rewards for pro-social behavior
in school.”
ECO students were significantly less likely to report switching from activity to activity and
more likely to say they try to work hard at school.
In regards to the environment, ECO students were significantly more likely to report they
know a lot about nature, significantly more likely to want to work to protect the environment
and significantly more likely to report they could catch a fish without help from an adult.
Significantly more ECO students gave correct answers to questions about the environment
including the definition of a watershed, identifying threats to streams and lakes and
recognizing the implications of sensitive bugs in water.
ECO students were more likely to report missing one or more days of school in the four
weeks prior to the survey.
Introduction. The Marcus Whitman ECO program provides an alternative educational
program for 6th and 7th grade students. Students are selected at random from a list of
students who volunteer and whose parents have given permission to participate in the
program that provides all required academic programs in an alternative school setting.
The setting for ECO is currently the Cummings Nature Center in the town of Naples. The
program has dedicated classrooms and facilities for numerous environmental activities. In
addition to classroom learning, ECO students tend to livestock, tap maple trees and
prepare syrup, build canoes and complete numerous horticultural, woodworking and craft
projects.
The intent of the ECO program is to instill knowledge about and an appreciation and
concern for nature and the physical environment while teaching all normal and required
academic subjects.
This evaluation addresses three specific questions:
I. Does the ECO program affect academic performance when students return to
the mainstream in 8th grade?
II. Does ECO affect selected Risk and Protective Factors that have been
demonstrated to predict substance abuse, violence and other behavior problems?
III. Does ECO affect students’ knowledge about and appreciation and concern for
the physical environment.
Academic Performance. Students attend ECO in 6th grade, 7th grade or both depending
on selection. The first evaluation question addresses academic performance in 8th grade
for former ECO students compared to all other 8th graders who did not attend ECO.
Specifically, it addresses the question of whether extended participation in an alternative
school setting has a deleterious effect on ability to perform upon returning to the normal or
mainstream setting. Analysis includes a comparison of grades in Math, English, Science
and Social Studies in 5th grade, i.e. prior to ECO and in 8th grade, i.e. after returning to
the mainstream. Tables 1a through 1d show comparisons for ECO students (n=21) who
are 8th graders during the current (2000-2001) school year compared to all other
students who are 8th graders this year and who were students in Marcus Whitman when
they were in 5th grade.
Table 1a shows that ECO students had slightly (non-significant) lower final grades in Math
in 5th grade compared to non-ECO students. Upon returning to the mainstream in 8th
grade Math scores in the first marking period of 2000 were non-significantly lower and at
the midterm were non-significantly higher. Thus, participation in ECO had no apparent
negative effect on Math scores.
Table 1a.
Marcus Whitman Schools
Comparison of Eco Students to All Others in Same Class
2000-2001
Math Math Math
5th Grade Final* 8th Grade 1st Quart 8th Grade Midterm
Student Group Mean Grade Mean Grade Mean Grade
ECO (n=21) 2.71 79.62 85.30
NON-ECO (n=71) 3.03 84.82 85.28
Ratio 0.90 0.94 1.00
diff prob ns ns ns
*(4 point scale)
Table 1b shows that, compared to non-ECO students, Eco students had non-significantly
higher grades in English in 5th grade. ECO students scored slightly lower in the first
marking period and slightly higher at the midterm. Neither difference was statistically
significant
Table 1b.
Marcus Whitman Schools
Comparison of Eco Students to All Others in Same Class
2000-2001
English English English
5th Grade Final* 8th Grade 1st Quarter 8th Midterm
Student Group Mean Grade Mean Grade Mean Grade
ECO (n=21) 3.05 83.52 81.75
NON-ECO (n=71) 2.97 84.29 81.23
Ratio 1.02 0.99 1.01
diff prob ns ns ns
*(4 point scale)
Table 1c shows that ECO students had 5th grade Science grades slightly lower than other
students but first quarter and midterm Science grades were higher. None of these
differences were statistically significant.
Table 1d is consistent with the others showing no significant difference in Social Studies
scores either in 5th grade or upon returning to the mainstream in 8th grade.
Risk and Protective Factors. Current substance abuse, violence and other problem
prevention programs in Ontario County and other communities have been developed
using the Risk and Protective Framework. The model, which is described in attachment I,
addresses Community, School, Family and Individual factors that have been demonstrated
to influence problem behaviors. Risk Factors are those characteristics that predict
increased likelihood of substance abuse, violence or other problem behaviors. Protective
Factors are characteristics that predict a decreased likelihood of problems.
ECO students during the 99-2000 school year as well as a random sample of non-ECO
6th and 7th graders completed an abbreviated version of the Risk and Protective Factor
(see attachment II) Survey developed at the University of Washington by J. David Hawkins,
Richard Catalano, and Janet Miller. Analysis included a comparison of ECO and non-ECO
students responses to individual items and scores on scales consisting of multiple items.
Table 2a presents results of comparison of the school risk factor scale “Lack of
Commitment to School” as well as the individual items that constitute the scale. ECO
students had non-significantly lower scores on this scale, however, ECO students were
significantly more likely to “Enjoy being in school”(66.6% vs 45.0%). ECO students were
more likely to report missing one or more days of school in the four weeks prior to the
survey and were significantly more to report missing one or more days for “other
reasons.” A possible explanation for this is the distance traveled to the Cummings site
makes it more likely that students do not return to school if they have an appointment
during the school day.
Table 2b shows the results for the risk factor scale “Favorable attitudes toward antisocial
behavior.” ECO students had non-significantly lower scores on the scale as well as all
individual items except favorable attitude toward “Pick a fight with someone.”
Table 2c represents the results for the family risk factor “Lack of Supervision.” ECO
students reported a non-significantly lower score on the risk factor. ECO students were
significantly more likely to report that their parents want them to call if they are going to be
late (97.9% vs 85.2%) and to report that their family has clear rules about alcohol and
drug use (100% vs 92.6%).
Table 2d shows results for the school protective factor, “Opportunities for Prosocial
Involvement in School.” ECO students reported non-significantly higher scores on the
scale and were significantly more likely to report that teachers ask them to work on special
projects (70.9% vs 50.7%) and that they have lots of chances to be part of class
discussions and activities (100% vs 82.7%).
Table 2e presents results for the school protective factor, “Rewards for Prosocial Behavior
in School.” ECO students scored significantly higher on the scale (mean=12.2 vs 10.9). In
addition, ECO students were more likely to report their teachers notice when they do a
good job (87.5% vs 70.4%) and that teachers praise them when they work hard in school
(79.2% vs 51.9%).
Table 3 shows results for individual items related to school and other characteristics that
are not part of risk or protective factor scales. ECO students were significantly less likely
to report they switch from activity to activity (34% vs 53%) and significantly more likely to
report they try hard to do good work at school (100% vs 91.3%).
Knowledge about and appreciation and concern for the physical environment. Table 3
also shows results for items related to the environment. ECO students significantly more
likely to say the know a lot about nature and the environment (91.7% vs 69.6%),
significantly more likely to say they want to work to protect the environment (95.9% vs
70.4%) and significantly more likely say they could catch a fish without help from an adult
(100% vs 71,6%).
ECO students also scored significantly better on environmental knowledge questions
including the definition of a watershed (68.8% vs 19.8%), identifying threats to streams
and lakes (81.3% vs 45.7%) and knowing that sensitive bugs are indicative of clean water
(91.7% vs 44.4%).