Executive Summary
Major
Findings
|
|
TIMSS
1999, a successor to the acclaimed 1995
Third International Mathematics and Science Study (TIMSS), focused
on the mathematics and science achievement of eighth-grade students.
Thirty-eight countries including the United States participated in TIMSS
1999 (also known as TIMSS-Repeat or TIMSS-R).(1)
Even more significantly for the United States, however, TIMSS 1999 included
a voluntary Benchmarking Study. Twenty-seven jurisdictions from all
across the nation, including 13 states and 14 districts or consortia
(see below), participated in the Benchmarking Study.
TIMSS 1999 Benchmarking Participants
States |
Districts and Consortia |
Connecticut
Idaho
Illinois
Indiana
Maryland
Massachusetts
Michigan
Missouri
North Carolina
Oregon
Pennsylvania
South Carolina
Texas
|
Academy School District #20,
Colorado Springs, CO
Chicago Public Schools, IL
Delaware Science Coalition, DE
First in the World Consortium, IL
Fremont/Lincoln/Westside Public Schools, NE
Guilford County, NC
Jersey City Public Schools, NJ
Miami-Dade County Public Schools, FL
Michigan Invitational Group, MI
Montgomery County, MD
Naperville School District #203, IL
Project SMART Consortium, OH
Rochester City School District, NY
Southwest Pennsylvania Math and Science Collaborative, PA |
|
Each jurisdiction had its own reasons for taking part in the TIMSS
1999 Benchmarking Study. In general, participation provided an unprecedented
opportunity for jurisdictions to assess the comparative international
standing of their students' achievement and to evaluate their mathematics
and science programs in an international context. Participants
were also able to compare their achievement with that of the United
States as a whole,(2) and in the cases where they
both participated, school districts could compare with the performance
of their states.
Each participating entity invested valuable resources in this effort,
primarily for data collection and team building, but also for staff
development to facilitate use of the TIMSS 1999 results as an effective
tool for school improvement. Despite each participant's deep commitment
to educational improvement by virtue of its participation in such a
venture, it took courage and initiative to join such a high profile enterprise
as the TIMSS 1999 Benchmarking Study. Whether students' achievement
fell at the top, middle, or bottom of the range of results for countries
internationally, each participant will be asked to explain the results
to its parents and communities.
This report provides a preliminary overview of the results for the
Benchmarking Study in science. The real work will take place as each
participating entity begins to examine its curriculum, teaching force,
instructional approaches, and school environment in an international
context. As those working on school improvement know full well, there
is no silver bullet or single factor that is the answer
to higher achievement in science or any other school subject. Making
strides in raising student achievement requires tireless diligence,
as policy makers, administrators, teachers, and communities work to
make improvements in a number of important areas related to educational
quality.
Unlike in many countries around the world where educational decision
making is highly centralized, in the United States the opportunities
to learn science derive from an educational system that operates through
states and districts, allocating opportunities through schools and then
through classrooms. Improving students' opportunities to learn requires
examining every step of the educational system, including the curriculum,
teacher quality, availability and appropriateness of resources, student
motivation, instructional effectiveness, parental support, and school
safety.
Particularly since A Nation at Risk (3) was
issued eighteen years ago, many states and school districts have been
working on the arduous task of improving education in their jurisdictions.
During the past decade, content-driven systemic
school reform has emerged as a promising model for school improvement.(4)
That is, curriculum frameworks establishing what students should know
and be able to do provide a coherent direction for improving the quality
of instruction. Teacher preparation, instructional materials, and other
aspects of the system are then aligned to reflect the content of the
frameworks in an integrated way to reinforce and sustain high-quality
teaching and learning in schools and classrooms.
There has been concerted effort across the nation
at the state and local levels in writing and revising academic standards
in various academic subjects. In science, most states are in the process
of implementing new content or curriculum standards or revising existing
ones.(5) All but four states
now have standards in science.(6) Twenty-nine
states also have some type of criterion-referenced science assessment
aligned to state standards.(7) Much of this effort
has been based on work done at the national level over the past decade
to develop standards aimed at increasing the science literacy of all
students. The two most prominent documents are the
American Association for the Advancement of Science (AAAS) Benchmarks
for Science Literacy and the National Research Council's National
Science Education Standards (NSES), both of which define standards
for the teaching and learning of science that many state and local educational
systems have used to fashion their own curricula.(8)
Despite considerable energy devoted to educational
improvement, achievement in science has shown only modest gains since
1982.(9) The TIMSS results show little change in
eighth-grade science achievement between 1995 and 1999. In 1999, the
U.S. eighth graders performed significantly above the TIMSS international
average in science, but about in the middle of the achievement distribution
of the 38 participating countries (above 18 countries, similar to 5,
and below 14). In TIMSS 1999, the world class performance levels in
science were set essentially by four Asian countries and a central European
one. Chinese Taipei, Singapore, Hungary, Japan, and the Republic of
Korea had the highest average performance. The Netherlands, Australia,
the Czech Republic, and England also performed very well (see Exhibits
1.1 and 1.2
in Chapter 1).
Major Findings from the TIMSS 1999 Benchmarking Study
|
Average performance in science for the 13 Benchmarking states
was generally clustered in the upper half of the international
distribution of results for the 38 countries. All but three of
the Benchmarking states performed significantly above the international
average.
|
|
The top-performing Benchmarking participants the Naperville
School District and the First in the World Consortium (both in
Illinois), the Michigan Invitational Group, and the Academy School
District (in Colorado) all had average achievement comparable
to the world class performance of Chinese Taipei and Singapore.
However, the Benchmarking Study underscores the extreme importance
of looking beyond the averages to the range of performance found
across the nation, as performance across the participating school
districts and consortia reflected nearly the full range of achievement
internationally. In contrast to the top performers, urban districts
with high percentages of students from low-income families
the Rochester City School District, the Chicago Public Schools,
the Jersey City Public Schools, and the Miami-Dade County Public
Schools performed more similarly to lower-performing countries
such as Jordan, Iran, Indonesia, Turkey, and Tunisia, but significantly
higher than the lowest-scoring countries.
|
|
The TIMSS 1999 Benchmarking Study provides evidence that some
schools in the U.S. are among the best in the world, but that
a world-class education is not available to all children across
the nation. The TIMSS index of home educational resources (based
on books in the home, availability of study aids, and parents'
education level) shows that students with more home resources
have higher science achievement. Furthermore, the Benchmarking
jurisdictions with the greatest percentages of students with high
levels of home resources were among the top-performing jurisdictions,
and those with the lowest achievement were four urban districts
that also had the lowest percentages of students with high levels
of home resources. These and other TIMSS 1999 Benchmarking results
support research indicating that students in urban districts with
a high proportion of low-income families and minorities often
attend schools with fewer resources than in non-urban districts,
including less experienced teachers, fewer appropriate instructional
materials, more emphasis on lower-level content, less access to
gifted and talented programs, higher absenteeism, more inadequate
buildings, and more discipline problems.
|
|
It is disappointing that in science at the eighth grade, the
TIMSS 1999 Benchmarking Study shows relatively unequal average
achievement for girls and boys in many of the Benchmarking jurisdictions,
and in the United States overall. Boys had significantly higher
average science achievement than girls in 10 of the 13 Benchmarking
states, with Massachusetts, South Carolina, and Texas the exceptions.
Gender differences were less prevalent among the Benchmarking
districts and consortia, with significant differences in just four
jurisdictions: the First in the World Consortium, Guilford County,
Naperville, and the Southwest Pennsylvania Math and Science Collaborative.
This follows the national and international pattern where the
United States was one of 16 countries in 1999 where boys significantly
outperformed girls.
|
|
Of the six science content areas assessed by TIMSS, U.S. eighth
graders performed higher than the international average in earth
science, life science, chemistry, environmental and resource issues,
and scientific inquiry and the nature of science, but only at the
international average in physics. In life science and in scientific
inquiry and the nature of science, the two areas in which the
United States performed best, some of the lowest-performing Benchmarking
participants had more success than in the other content areas.
It will be important, however, for each participant to determine
its specific relative strengths and weaknesses in science achievement.
|
|
Although many countries teach eighth-grade science as separate
subjects (namely, earth science, biology, physics, and chemistry),
most jurisdictions in the United States teach science as a single
general or integrated subject. It naturally follows, then, that
teachers in the U.S. overall and in the majority of the Benchmarking
entities reported a relatively heavy emphasis given to general/integrated
science among the science content areas. In the U.S., teachers
of 41 percent of the students reported that general science was
emphasized most in their classes, compared with 28 percent for
earth science, 21 percent for physical science (chemistry/physics),
five percent for biology, three percent for chemistry, and two
percent for physics. Although results for many of the Benchmarking
jurisdictions were similar to the national profile, the content
area emphasis differed substantially from jurisdiction to jurisdiction.
For example, teachers in Idaho, the Academy School District, Jersey
City, and Rochester reported emphasizing physical science for
half or more of their students, while those in North Carolina,
Texas, the Delaware Science Coalition, the Fremont/Lincoln/Westside
Public Schools, and Guilford County did so for less than 10 percent.
|
|
Research shows that higher achievement in
science is associated with teachers having a bachelor's and/or
master's degree in science.(10) According
to their teachers, however, U.S. eighth-grade students were less
likely than those in other countries to be taught science by teachers
with a major area of study in science, and more likely to be taught
by teachers with a major in general education. In
the U.S., 47 percent of students were taught science by a teacher
whose major area of study was biology, 13 percent physics, 21
percent chemistry, 43 percent science education, 14 percent mathematics
or mathematics education, 56 percent general education, and 45
percent some other area.(11) Among Benchmarking
participants, in almost every jurisdiction the majority of students
were in science classes in which the teacher's major area was
science education or general education. Teachers with a major
in physics or chemistry were rare; only in the Academy School
District, Naperville, and Project smart were more than 30 percent
of students taught by such teachers.
|
|
In general, teachers in many Benchmarking
entities and in the United States overall expressed much less
confidence in their preparation to teach eighth-grade science
than mathematics. In the U.S. as a whole, 87 percent of the students
had teachers who felt very well prepared to teach
across a range of general mathematics topics covered by TIMSS,(12)
compared with 27 percent for science. This figure for science
ranged from 56 percent in the Academy School District to 14 percent
in the Delaware Science Coalition across the Benchmarking entities,
with half of them exceeding the national average. Teachers in
a number of the lower-scoring jurisdictions reported relatively
high levels of confidence in their preparation, possibly because
they are teaching a science curriculum that is not very demanding.
|
|
Since entering teachers make up a relatively small percentage
of the teaching force, improving teacher quality depends on providing
opportunities for professional development. Science teachers in
the United States reported a relatively heavy focus on curriculum,
pedagogy, and content knowledge in their professional development
activities. Although the national pattern held in many jurisdictions,
there was variation across the Benchmarking participants. For
example, the percentage of students whose teachers reported an
emphasis on content knowledge ranged from 24 percent in the Delaware
Science Coalition to 59 percent in Miami-Dade.
|
|
The choices teachers make determine, to a large extent, what
students learn. An important aspect of teaching science is the
emphasis placed on scientific investigation. The TIMSS 1999 results
show that higher science achievement is related to the emphasis
that teachers place on experiments or practical investigations.
In the United States as a whole, 31 percent of the students were
in science classes with a high degree of emphasis on scientific
investigation, compared with 38 percent internationally for countries
with general/integrated science. There was great variation among
the Benchmarking participants, from 79 percent in Naperville,
more than in any TIMSS 1999 country, to 17 percent in the Delaware
Science Coalition. Eighteen of the Benchmarking entities were
above the U.S. average. In addition to Naperville, more than 50
percent of students were in such classes in Maryland, the First
in the World Consortium, the Academy School District, Connecticut,
and the Fremont/Lincoln/Westside Public Schools.
|
|
In general, the TIMSS 1999 data reveal that the focus in most
science classes was on teacher-centered activities. In the United
States overall, 69 percent of students reported that their teacher
shows them how to do science problems almost always or pretty
often, while only 59 percent reported that they work on science
projects this frequently. According to U.S. science teachers,
class time is spent as follows: 19 percent on lecture style teacher
presentation; 23 percent on teacher-guided or independent student
practice; 17 percent on students conducting experiments; eight
percent on teachers demonstrating experiments; nine percent on
re-teaching and clarification; nine percent on tests and quizzes,
eight percent on homework review; six percent on administrative
tasks; and three percent on other activities. The results for
the Benchmarking participants generally resembled the national
profile.
|
|
The TIMSS 1999 data indicate that the instructional time for
learning science, beyond being spent largely on teacher-centered
activities, becomes further eroded by non-instructional tasks.
In Japan and Korea, more than half the students were in classes
that never had interruptions for announcements or administrative
tasks. Among the Benchmarking participants, the results ranged
from 30 percent of the eighth graders in such classes in Naperville
to only seven percent in the Academy School District. Also, 57
percent of the U.S. students reported that they began their science
homework during class almost always or pretty often, compared
with the international average of 41 percent. In most Benchmarking
jurisdictions, the results followed the national pattern, although
the percentage varied from 41 to 74 percent.
|
|
The Benchmarking Study shows that students
in schools that are well-resourced have higher science achievement.
Among the Benchmarking participants, three-fourths or more of
the students in the Academy School District, the First in the
World Consortium, and Naperville were in schools where the capacity
to provide science instruction was largely unaffected by shortages
or inadequacies in instructional materials, supplies, buildings,
space, laboratory equipment and materials, computers and computer
software, calculators, library materials and audio-visual resources.
These high percentages exceeded those of all the TIMSS 1999 countries,
with the highest percentages (43 to 60 percent) reported by Belgium
(Flemish),(13) Singapore, and the Czech Republic.
|
|
Discipline that maintains a safe and orderly
atmosphere conducive to learning is very important to school quality,
and research indicates that urban schools have conditions less
conducive to learning than non-urban schools.(14)
For example, urban schools report more crime against students
and teachers at school and that physical conflict among students
is a serious or moderate problem. Among the Benchmarking participants
there was considerable variation in principals' reports about
the seriousness of a variety of potential discipline problems.
In several of the urban districts, however, 10 percent or more
of the students were in schools where absenteeism, classroom disturbances,
and physical injury to students were felt to be serious problems.
Also in several of these districts, 20 percent or more of the
students were in schools where intimidation or verbal abuse among
students was a serious problem.
|
Among the 27 participants in the TIMSS 1999 Benchmarking Study, there
was particularly extreme variation in science achievement among the
school districts and consortia, but less among the states. Several districts
in relatively wealthy communities had comparatively high achievement
in science, while others in urban areas with high percentages of students
from low-income families had relatively low achievement, compared with
the TIMSS 1999 results internationally. Regardless of its performance,
however, each state, district, and consortium now has a better idea
of the challenges ahead and access to a rich array of data about various
facets of its educational system. The TIMSS 1999 data provide an excellent
basis for examining how best to move from developing a curriculum framework
or standards in science to meeting the extraordinary challenge of actually
implementing the standards in schools and classrooms often characterized
by considerable cultural, social, and experiential diversity.
Footnotes
1 |
IEAs International Study Center
at Boston College reported the international results for TIMSS 1999
as well as trends between 1995 and 1999 in two companion volumes
the TIMSS 1999 International Mathematics Report and
the TIMSS 1999 International Science Report. Performance
in the United States relative to that of other nations was reported
by the U.S. National Center for Education Statistics in Pursuing
Excellence: Comparisons of International Eighth-Grade Mathematics
and Science Achievement from a U.S. Perspective, 1995 and 1999.
(See the Introduction for full citations.) |
2 |
For the most part, the U.S. TIMSS national
sample was separate from the students assessed in each of the Benchmarking
jurisdictions. Each Benchmarking participant had its own sample
to provide comparisons to each of the TIMSS 1999 countries including
the United States. Collectively, the Benchmarking participants are
not representative of the United States even though the effort was
substantial
in scope. |
3 |
A Nation at Risk: The Imperative
for Education Reform (1983), Washington, DC: National Commission
on Excellence in Education. |
4 |
O'Day, J.A. and Smith, M.S. (1993),
Systemic Reform and Educational Opportunity in S.H.
Fuhrman (ed.), Designing Coherent Education Policy: Improving
the System, San Francisco, CA: Jossey-Bass, Inc. |
5 |
Glidden, H. (1999), Making Standards
Matter 1999, Washington, DC: American Federation of Teachers. |
6 |
Key State Education Policies on K-12
Education: 2000 (2000), Washington, DC: Council of Chief State
School Officers. |
7 |
Orlofsky, G.F. and Olson, L. (2001),
The State of the States in Quality Counts 2001, A
Better Balance: Standards, Tests, and the Tools to Succeed,
Education Week, 20(17). |
8 |
Smith, T.A., Martin, M.O., Mullis, I.V.S.,
and Kelly, D.L. (2000), Profiles of Student Achievement in Science
at the TIMSS International Benchmarks: U.S. Performance and Standards
in an International Context, Chestnut Hill, MA: Boston College. |
9 |
Campbell, J.R., Hombo, C.M., and Mazzeo,
J. (2000), NAEP 1999 Trends in Academic Progress: Three Decades
of Student Performance, NCES 2000-469, Washington, DC: National
Center for Education Statistics. |
10 |
Goldhaber, D.D. and Brewer, D.J. (1997),
Evaluating the Effect of Teacher Degree Level on Educational
Performance in W. Fowler (ed.), Developments in School Finance,
1996, NCES 97-535, Washington DC: National Center for Education
Statistics; Darling-Hammond, L. (2000), Teacher Quality and Student
Achievement: A Review of State Policy Evidence, Education Policy
Analysis Archives, 8(1). |
11 |
Because teachers can have dual majors,
or different majors at the undergraduate and graduate level, percentages
do not add to 100. |
12 |
Mullis, I.V.S., Martin, M.O., Gonzalez,
E.J., OConnor, K.M., Chrostowski, S.J., Gregory, K.D., Garden,
R.A., and Smith, T.A. (2001), Mathematics Benchmarking Report,
TIMSS 1999 Eighth Grade: Achievement for U.S. States and
Districts in an International Context, Chestnut Hill, MA: Boston
College. |
13 |
Belgium has two separate educational
systems, Flemish and French. The Flemish system participated in
TIMSS 1999. |
14 |
Mayer, D.P., Mullens, J.E., and Moore,
M.T. (2000), Monitoring School Quality: An Indicators Report,
NCES 2001-030, Washington, DC: National Center for Education Statistics;
Kaufman, P., Chen, X., Choy, S.P., Ruddy, S.A., Miller, A.K., Fleury,
J.K., Chandler, K.A., Rand, M.R., Klaus, P., and Planty, M.G. (2000),
Indicators of School Crime and Safety, 2000, NCES 2001-017/NCJ-184176,
Washington, DC: U.S. Departments of Education and Justice. |
|