Science Benchmarking Report TIMSS 1999–Eighth Grade

Executive Summary
Major Findings

 

 

 

Executive Summary


TIMSS 1999, a successor to the acclaimed 1995 Third International Mathematics and Science Study (TIMSS), focused on the mathematics and science achievement of eighth-grade students. Thirty-eight countries including the United States participated in TIMSS 1999 (also known as TIMSS-Repeat or TIMSS-R).(1) Even more significantly for the United States, however, TIMSS 1999 included a voluntary Benchmarking Study. Twenty-seven jurisdictions from all across the nation, including 13 states and 14 districts or consortia (see below), participated in the Benchmarking Study.

TIMSS 1999 Benchmarking Participants

States Districts and Consortia
Connecticut
Idaho
Illinois
Indiana
Maryland
Massachusetts
Michigan
Missouri
North Carolina
Oregon
Pennsylvania
South Carolina
Texas
Academy School District #20, Colorado Springs, CO
Chicago Public Schools, IL
Delaware Science Coalition, DE
First in the World Consortium, IL
Fremont/Lincoln/Westside Public Schools, NE
Guilford County, NC
Jersey City Public Schools, NJ
Miami-Dade County Public Schools, FL
Michigan Invitational Group, MI
Montgomery County, MD
Naperville School District #203, IL
Project SMART Consortium, OH
Rochester City School District, NY
Southwest Pennsylvania Math and Science Collaborative, PA

Each jurisdiction had its own reasons for taking part in the TIMSS 1999 Benchmarking Study. In general, participation provided an unprecedented opportunity for jurisdictions to assess the comparative international standing of their students' achievement and to evaluate their mathematics and science programs in an international context. Participants were also able to compare their achievement with that of the United States as a whole,(2) and in the cases where they both participated, school districts could compare with the performance of their states.

Each participating entity invested valuable resources in this effort, primarily for data collection and team building, but also for staff development to facilitate use of the TIMSS 1999 results as an effective tool for school improvement. Despite each participant's deep commitment to educational improvement by virtue of its participation in such a venture, it took courage and initiative to join such a high profile enterprise as the TIMSS 1999 Benchmarking Study. Whether students' achievement fell at the top, middle, or bottom of the range of results for countries internationally, each participant will be asked to explain the results to its parents and communities.

This report provides a preliminary overview of the results for the Benchmarking Study in science. The real work will take place as each participating entity begins to examine its curriculum, teaching force, instructional approaches, and school environment in an international context. As those working on school improvement know full well, there is no “silver bullet” or single factor that is the answer to higher achievement in science or any other school subject. Making strides in raising student achievement requires tireless diligence, as policy makers, administrators, teachers, and communities work to make improvements in a number of important areas related to educational quality.

Unlike in many countries around the world where educational decision making is highly centralized, in the United States the opportunities to learn science derive from an educational system that operates through states and districts, allocating opportunities through schools and then through classrooms. Improving students' opportunities to learn requires examining every step of the educational system, including the curriculum, teacher quality, availability and appropriateness of resources, student motivation, instructional effectiveness, parental support, and school safety.

Particularly since A Nation at Risk (3) was issued eighteen years ago, many states and school districts have been working on the arduous task of improving education in their jurisdictions. During the past decade, content-driven systemic school reform has emerged as a promising model for school improvement.(4) That is, curriculum frameworks establishing what students should know and be able to do provide a coherent direction for improving the quality of instruction. Teacher preparation, instructional materials, and other aspects of the system are then aligned to reflect the content of the frameworks in an integrated way to reinforce and sustain high-quality teaching and learning in schools and classrooms.

There has been concerted effort across the nation at the state and local levels in writing and revising academic standards in various academic subjects. In science, most states are in the process of implementing new content or curriculum standards or revising existing ones.(5) All but four states now have standards in science.(6) Twenty-nine states also have some type of criterion-referenced science assessment aligned to state standards.(7) Much of this effort has been based on work done at the national level over the past decade to develop standards aimed at increasing the science literacy of all students. The two most prominent documents are the American Association for the Advancement of Science (AAAS) Benchmarks for Science Literacy and the National Research Council's National Science Education Standards (NSES), both of which define standards for the teaching and learning of science that many state and local educational systems have used to fashion their own curricula.(8)

Despite considerable energy devoted to educational improvement, achievement in science has shown only modest gains since 1982.(9) The TIMSS results show little change in eighth-grade science achievement between 1995 and 1999. In 1999, the U.S. eighth graders performed significantly above the TIMSS international average in science, but about in the middle of the achievement distribution of the 38 participating countries (above 18 countries, similar to 5, and below 14). In TIMSS 1999, the world class performance levels in science were set essentially by four Asian countries and a central European one. Chinese Taipei, Singapore, Hungary, Japan, and the Republic of Korea had the highest average performance. The Netherlands, Australia, the Czech Republic, and England also performed very well (see Exhibits 1.1 and 1.2 in Chapter 1).

Major Findings from the TIMSS 1999 Benchmarking Study

Average performance in science for the 13 Benchmarking states was generally clustered in the upper half of the international distribution of results for the 38 countries. All but three of the Benchmarking states performed significantly above the international average.

The top-performing Benchmarking participants – the Naperville School District and the First in the World Consortium (both in Illinois), the Michigan Invitational Group, and the Academy School District (in Colorado) – all had average achievement comparable to the world class performance of Chinese Taipei and Singapore. However, the Benchmarking Study underscores the extreme importance of looking beyond the averages to the range of performance found across the nation, as performance across the participating school districts and consortia reflected nearly the full range of achievement internationally. In contrast to the top performers, urban districts with high percentages of students from low-income families – the Rochester City School District, the Chicago Public Schools, the Jersey City Public Schools, and the Miami-Dade County Public Schools – performed more similarly to lower-performing countries such as Jordan, Iran, Indonesia, Turkey, and Tunisia, but significantly higher than the lowest-scoring countries.

The TIMSS 1999 Benchmarking Study provides evidence that some schools in the U.S. are among the best in the world, but that a world-class education is not available to all children across the nation. The TIMSS index of home educational resources (based on books in the home, availability of study aids, and parents' education level) shows that students with more home resources have higher science achievement. Furthermore, the Benchmarking jurisdictions with the greatest percentages of students with high levels of home resources were among the top-performing jurisdictions, and those with the lowest achievement were four urban districts that also had the lowest percentages of students with high levels of home resources. These and other TIMSS 1999 Benchmarking results support research indicating that students in urban districts with a high proportion of low-income families and minorities often attend schools with fewer resources than in non-urban districts, including less experienced teachers, fewer appropriate instructional materials, more emphasis on lower-level content, less access to gifted and talented programs, higher absenteeism, more inadequate buildings, and more discipline problems.

It is disappointing that in science at the eighth grade, the TIMSS 1999 Benchmarking Study shows relatively unequal average achievement for girls and boys in many of the Benchmarking jurisdictions, and in the United States overall. Boys had significantly higher average science achievement than girls in 10 of the 13 Benchmarking states, with Massachusetts, South Carolina, and Texas the exceptions. Gender differences were less prevalent among the Benchmarking districts and consortia, with significant differences in just four jurisdictions: the First in the World Consortium, Guilford County, Naperville, and the Southwest Pennsylvania Math and Science Collaborative. This follows the national and international pattern where the United States was one of 16 countries in 1999 where boys significantly outperformed girls.

Of the six science content areas assessed by TIMSS, U.S. eighth graders performed higher than the international average in earth science, life science, chemistry, environmental and resource issues, and scientific inquiry and the nature of science, but only at the international average in physics. In life science and in scientific inquiry and the nature of science, the two areas in which the United States performed best, some of the lowest-performing Benchmarking participants had more success than in the other content areas. It will be important, however, for each participant to determine its specific relative strengths and weaknesses in science achievement.

Although many countries teach eighth-grade science as separate subjects (namely, earth science, biology, physics, and chemistry), most jurisdictions in the United States teach science as a single general or integrated subject. It naturally follows, then, that teachers in the U.S. overall and in the majority of the Benchmarking entities reported a relatively heavy emphasis given to general/integrated science among the science content areas. In the U.S., teachers of 41 percent of the students reported that general science was emphasized most in their classes, compared with 28 percent for earth science, 21 percent for physical science (chemistry/physics), five percent for biology, three percent for chemistry, and two percent for physics. Although results for many of the Benchmarking jurisdictions were similar to the national profile, the content area emphasis differed substantially from jurisdiction to jurisdiction. For example, teachers in Idaho, the Academy School District, Jersey City, and Rochester reported emphasizing physical science for half or more of their students, while those in North Carolina, Texas, the Delaware Science Coalition, the Fremont/Lincoln/Westside Public Schools, and Guilford County did so for less than 10 percent.

Research shows that higher achievement in science is associated with teachers having a bachelor's and/or master's degree in science.(10) According to their teachers, however, U.S. eighth-grade students were less likely than those in other countries to be taught science by teachers with a major area of study in science, and more likely to be taught by teachers with a major in general education. In the U.S., 47 percent of students were taught science by a teacher whose major area of study was biology, 13 percent physics, 21 percent chemistry, 43 percent science education, 14 percent mathematics or mathematics education, 56 percent general education, and 45 percent some other area.(11) Among Benchmarking participants, in almost every jurisdiction the majority of students were in science classes in which the teacher's major area was science education or general education. Teachers with a major in physics or chemistry were rare; only in the Academy School District, Naperville, and Project smart were more than 30 percent of students taught by such teachers.

In general, teachers in many Benchmarking entities and in the United States overall expressed much less confidence in their preparation to teach eighth-grade science than mathematics. In the U.S. as a whole, 87 percent of the students had teachers who felt “very well prepared” to teach across a range of general mathematics topics covered by TIMSS,(12) compared with 27 percent for science. This figure for science ranged from 56 percent in the Academy School District to 14 percent in the Delaware Science Coalition across the Benchmarking entities, with half of them exceeding the national average. Teachers in a number of the lower-scoring jurisdictions reported relatively high levels of confidence in their preparation, possibly because they are teaching a science curriculum that is not very demanding.

Since entering teachers make up a relatively small percentage of the teaching force, improving teacher quality depends on providing opportunities for professional development. Science teachers in the United States reported a relatively heavy focus on curriculum, pedagogy, and content knowledge in their professional development activities. Although the national pattern held in many jurisdictions, there was variation across the Benchmarking participants. For example, the percentage of students whose teachers reported an emphasis on content knowledge ranged from 24 percent in the Delaware Science Coalition to 59 percent in Miami-Dade.

The choices teachers make determine, to a large extent, what students learn. An important aspect of teaching science is the emphasis placed on scientific investigation. The TIMSS 1999 results show that higher science achievement is related to the emphasis that teachers place on experiments or practical investigations. In the United States as a whole, 31 percent of the students were in science classes with a high degree of emphasis on scientific investigation, compared with 38 percent internationally for countries with general/integrated science. There was great variation among the Benchmarking participants, from 79 percent in Naperville, more than in any TIMSS 1999 country, to 17 percent in the Delaware Science Coalition. Eighteen of the Benchmarking entities were above the U.S. average. In addition to Naperville, more than 50 percent of students were in such classes in Maryland, the First in the World Consortium, the Academy School District, Connecticut, and the Fremont/Lincoln/Westside Public Schools.

In general, the TIMSS 1999 data reveal that the focus in most science classes was on teacher-centered activities. In the United States overall, 69 percent of students reported that their teacher shows them how to do science problems almost always or pretty often, while only 59 percent reported that they work on science projects this frequently. According to U.S. science teachers, class time is spent as follows: 19 percent on lecture style teacher presentation; 23 percent on teacher-guided or independent student practice; 17 percent on students conducting experiments; eight percent on teachers demonstrating experiments; nine percent on re-teaching and clarification; nine percent on tests and quizzes, eight percent on homework review; six percent on administrative tasks; and three percent on other activities. The results for the Benchmarking participants generally resembled the national profile.

The TIMSS 1999 data indicate that the instructional time for learning science, beyond being spent largely on teacher-centered activities, becomes further eroded by non-instructional tasks. In Japan and Korea, more than half the students were in classes that never had interruptions for announcements or administrative tasks. Among the Benchmarking participants, the results ranged from 30 percent of the eighth graders in such classes in Naperville to only seven percent in the Academy School District. Also, 57 percent of the U.S. students reported that they began their science homework during class almost always or pretty often, compared with the international average of 41 percent. In most Benchmarking jurisdictions, the results followed the national pattern, although the percentage varied from 41 to 74 percent.

The Benchmarking Study shows that students in schools that are well-resourced have higher science achievement. Among the Benchmarking participants, three-fourths or more of the students in the Academy School District, the First in the World Consortium, and Naperville were in schools where the capacity to provide science instruction was largely unaffected by shortages or inadequacies in instructional materials, supplies, buildings, space, laboratory equipment and materials, computers and computer software, calculators, library materials and audio-visual resources. These high percentages exceeded those of all the TIMSS 1999 countries, with the highest percentages (43 to 60 percent) reported by Belgium (Flemish),(13) Singapore, and the Czech Republic.

Discipline that maintains a safe and orderly atmosphere conducive to learning is very important to school quality, and research indicates that urban schools have conditions less conducive to learning than non-urban schools.(14) For example, urban schools report more crime against students and teachers at school and that physical conflict among students is a serious or moderate problem. Among the Benchmarking participants there was considerable variation in principals' reports about the seriousness of a variety of potential discipline problems. In several of the urban districts, however, 10 percent or more of the students were in schools where absenteeism, classroom disturbances, and physical injury to students were felt to be serious problems. Also in several of these districts, 20 percent or more of the students were in schools where intimidation or verbal abuse among students was a serious problem.

Among the 27 participants in the TIMSS 1999 Benchmarking Study, there was particularly extreme variation in science achievement among the school districts and consortia, but less among the states. Several districts in relatively wealthy communities had comparatively high achievement in science, while others in urban areas with high percentages of students from low-income families had relatively low achievement, compared with the TIMSS 1999 results internationally. Regardless of its performance, however, each state, district, and consortium now has a better idea of the challenges ahead and access to a rich array of data about various facets of its educational system. The TIMSS 1999 data provide an excellent basis for examining how best to move from developing a curriculum framework or standards in science to meeting the extraordinary challenge of actually implementing the standards in schools and classrooms often characterized by considerable cultural, social, and experiential diversity.

next chapter >


Footnotes

1 IEA’s International Study Center at Boston College reported the international results for TIMSS 1999 as well as trends between 1995 and 1999 in two companion volumes – the TIMSS 1999 International Mathematics Report and the TIMSS 1999 International Science Report. Performance in the United States relative to that of other nations was reported by the U.S. National Center for Education Statistics in Pursuing Excellence: Comparisons of International Eighth-Grade Mathematics and Science Achievement from a U.S. Perspective, 1995 and 1999. (See the Introduction for full citations.)
2 For the most part, the U.S. TIMSS national sample was separate from the students assessed in each of the Benchmarking jurisdictions. Each Benchmarking participant had its own sample to provide comparisons to each of the TIMSS 1999 countries including the United States. Collectively, the Benchmarking participants are not representative of the United States even though the effort was substantial
in scope.
3 A Nation at Risk: The Imperative for Education Reform (1983), Washington, DC: National Commission on Excellence in Education.
4 O'Day, J.A. and Smith, M.S. (1993), “Systemic Reform and Educational Opportunity” in S.H. Fuhrman (ed.), Designing Coherent Education Policy: Improving the System, San Francisco, CA: Jossey-Bass, Inc.
5 Glidden, H. (1999), Making Standards Matter 1999, Washington, DC: American Federation of Teachers.
6 Key State Education Policies on K-12 Education: 2000 (2000), Washington, DC: Council of Chief State School Officers.
7 Orlofsky, G.F. and Olson, L. (2001), “The State of the States” in Quality Counts 2001, A Better Balance: Standards, Tests, and the Tools to Succeed, Education Week, 20(17).
8 Smith, T.A., Martin, M.O., Mullis, I.V.S., and Kelly, D.L. (2000), Profiles of Student Achievement in Science at the TIMSS International Benchmarks: U.S. Performance and Standards in an International Context, Chestnut Hill, MA: Boston College.
9 Campbell, J.R., Hombo, C.M., and Mazzeo, J. (2000), NAEP 1999 Trends in Academic Progress: Three Decades of Student Performance, NCES 2000-469, Washington, DC: National Center for Education Statistics.
10 Goldhaber, D.D. and Brewer, D.J. (1997), “Evaluating the Effect of Teacher Degree Level on Educational Performance” in W. Fowler (ed.), Developments in School Finance, 1996, NCES 97-535, Washington DC: National Center for Education Statistics; Darling-Hammond, L. (2000), Teacher Quality and Student Achievement: A Review of State Policy Evidence, Education Policy Analysis Archives, 8(1).
11 Because teachers can have dual majors, or different majors at the undergraduate and graduate level, percentages do not add to 100.
12 Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., O’Connor, K.M., Chrostowski, S.J., Gregory, K.D., Garden, R.A., and Smith, T.A. (2001), Mathematics Benchmarking Report, TIMSS 1999 – Eighth Grade: Achievement for U.S. States and Districts in an International Context, Chestnut Hill, MA: Boston College.
13 Belgium has two separate educational systems, Flemish and French. The Flemish system participated in TIMSS 1999.
14 Mayer, D.P., Mullens, J.E., and Moore, M.T. (2000), Monitoring School Quality: An Indicators Report, NCES 2001-030, Washington, DC: National Center for Education Statistics; Kaufman, P., Chen, X., Choy, S.P., Ruddy, S.A., Miller, A.K., Fleury, J.K., Chandler, K.A., Rand, M.R., Klaus, P., and Planty, M.G. (2000), Indicators of School Crime and Safety, 2000, NCES 2001-017/NCJ-184176, Washington, DC: U.S. Departments of Education and Justice.

Click here to return to the ISC homepage

TIMSS 1999 is a project of the International Study Center
Boston College, Lynch School of Education