Download TIMSS 2019 Assessment Design (pdf)
TIMSS 2019 Student Booklet Design
A major consequence of TIMSS’ ambitious reporting goals is that many more questions are required for the assessment than can be answered by any one student in the amount of testing time available. Accordingly, TIMSS uses a matrix sampling approach that involves packaging the entire assessment pool of mathematics and science items at each grade level into a set of 14 student achievement booklets, with each student completing just one booklet. Each item appears in two booklets, providing a mechanism for linking together the student responses from the various booklets when data from all booklets are taken together. Booklets are distributed among students in participating classrooms according to assignments predetermined by the TIMSS within-school sampling software, so that the student samples completing each booklet in each country are approximately equivalent in terms of student ability.
After the assessment has been administered and the data collected and processed, TIMSS uses item response theory scaling methods to assemble a comprehensive picture of the achievement of the entire student population of a country from the combined responses of individual students to the booklets that they are assigned.1 This approach reduces to manageable proportions what otherwise would be an impossible student burden, albeit at the cost of some complexity in booklet assembly, data collection, and data analysis.
To facilitate the process of creating the student achievement booklets, TIMSS groups the assessment items into a series of item blocks, with approximately 10 to 14 items in each block at the fourth grade and 12 to 18 at the eighth grade. As far as possible, within each block the distribution of items across content and cognitive domains matches the distribution across the item pool overall, as described in Chapters 1 and 2. Similar to the TIMSS 2015 assessment, TIMSS 2019 has a total of 28 blocks at each grade, 14 consisting of mathematics items and 14 consisting of science items. Student booklets are assembled from various combinations of these item blocks.
Following the 2015 assessment, eight of the 14 mathematics blocks and eight of the 14 science blocks at each grade were secured for use in 2019 as a basis for measuring trends. The remaining 12 blocks (six mathematics and six science) were available with permission from IEA for use in publications, research, and teaching, and had to be replaced by newly developed items for the TIMSS 2019 assessment. Accordingly, the 28 blocks in the TIMSS 2019 assessment comprise 16 blocks of trend items (eight mathematics and eight science) and 12 blocks of new items to be used for the first time in 2019.
As shown in Exhibit 1, the TIMSS 2019 mathematics blocks are labeled MP01/ME01 through MP14/ME14 and the science blocks SP01/SE01 through SP14/SE14.2 Mathematics and science blocks ending in odd numbers (01, 03, 05, etc.) contain the trend items from the 2015 assessment, as do blocks ending in 06. The blocks ending in even numbers (except 06) contain the items developed for use for the first time in TIMSS 2019.
Exhibit 4.1: TIMSS 2019 Item Blocks—Fourth and Eighth Grades
| Mathematics Blocks | Source of Items | Science Blocks | Source of Items | |
| MP01/ME01 | Trend Block M13 from TIMSS 2015 | SP01/SE01 | Trend Block S13 from TIMSS 2015 | |
| MP02/ME02 | New items for TIMSS 2019 | SP02/SEO2 | New items for TIMSS 2019 | |
| MP03/ME03 | Trend Block M08 from TIMSS 2015 | SP03/SE03 | Trend Block S08 from TIMSS 2015 | |
| MP04/ME04 | New items for TIMSS 2019 | SP04/SE04 | New items for TIMSS 2019 | |
| MP05/ME05 | Trend Block M09 from TIMSS 2015 | SP05/SE05 | Trend Block S09 from TIMSS 2015 | |
| MP06/ME06 | Trend Block M10 from TIMSS 2015 | SP06/SE06 | Trend Block S10 from TIMSS 2015 | |
| MP07/ME07 | Trend Block M11 from TIMSS 2015 | SP07/SE07 | Trend Block S11 from TIMSS 2015 | |
| MP08/ME08 | New items for TIMSS 2019 | SP08/SE08 | New items for TIMSS 2019 | |
| MP09/ME09 | Trend Block M04 from TIMSS 2015 | SP09/SE09 | Trend Block S04 from TIMSS 2015 | |
| MP10/ME10 | New items for TIMSS 2019 | SP10/SE10 | New items for TIMSS 2019 | |
| MP11/ME11 | Trend Block M12 from TIMSS 2015 | SP11/SE11 | Trend Block S12 from TIMSS 2015 | |
| MP12/ME12 | New items for TIMSS 2019 | SP12/SE12 | New items for TIMSS 2019 | |
| MP13/ME13 | Trend Block M14 from TIMSS 2015 | SP13/SE13 | Trend Block S14 from TIMSS 2015 | |
| MP14/ME14 | New items for TIMSS 2019 | SP14/SE14 | New items for TIMSS 2019 |
Fourth grade students are expected to spend, on average, 18 minutes on each item block, and eighth grade students, 22½ minutes. Consequently, the 28 blocks of fourth grade items are estimated to contain almost 8½ hours of testing time and the eighth grade blocks about 10½ hours. In previous TIMSS cycles, National Research Coordinators from participating countries agreed that the testing time for any one student should not be increased from previous assessments. Thus, as in the past, the assessment time for each student booklet must fit into 72 minutes for the fourth grade and 90 minutes for the eighth grade. An additional 30 minutes for a student questionnaire also was planned at each grade level.
In choosing how to distribute assessment blocks across student achievement booklets, the major goal was to maximize coverage of the framework while ensuring that every student responded to sufficient items to provide reliable measurement of trends in both mathematics and science. A further goal was to ensure that achievement in the mathematics and science content and cognitive domains could be measured reliably. To enable linking among booklets while keeping the number of booklets to a minimum, each block appears in two booklets. TIMSS has used the same booklet design since 2007.
The TIMSS 2019 booklet design shows how the 28 assessment blocks are distributed across 14 student achievement booklets (see Exhibit 4.2). The fourth and eighth grade booklet designs are identical, although the fourth grade blocks contain 18 minutes of assessment items and the eighth grade blocks 22½ minutes. Each student booklet consists of four blocks of items: two blocks of mathematics items, and two of science items. In half of the booklets, the two mathematics blocks come first, and then the two science blocks, and in the other half the order is reversed. Additionally, in most booklets two of the blocks contain trend items from TIMSS 2015 and two contain items newly developed for TIMSS 2019. For example, as may be seen from Exhibit 4.2, students assigned the computer-based Booklet 1 complete two blocks of Mathematics items, ME01 and ME02, and two blocks of science items, SE01 and SE02. The items in blocks ME01 and SE01 are trend items from TIMSS 2015, while those in ME02 and SE02 are items new for TIMSS 2019. Similarly, students assigned the computer-based Booklet 2 complete two science blocks, SE02 and SE03, followed by two mathematics blocks, ME02 and ME03. SE02 and ME02 contain the new items and SE03 and ME03 the trend items.
Countries participating in TIMSS aim for a sample of at least 4,000 students to ensure that there are enough respondents for each item. The 14 student booklets are distributed among the students in each sampled class according to a predetermined order, so that approximately equal proportions of students respond to each booklet.
Exhibit 4.2: TIMSS 2019 Student Achievement Booklet Design—Fourth and Eighth Grades
| Student Achievement Booklet | Assessment Blocks | |||
| Part 1 | Part 2 | |||
| Booklet 1 | MP01/ME01 | MP02/ME02 | SP01/SE01 | SP02/SE02 |
| Booklet 2 | SP02/SE02 | SP03/SE02 | MP02/ME02 | MP03/ME03 |
| Booklet 3 | MP03/ME03 | MP04/ME04 | SP03/SE03 | SP04/SE04 |
| Booklet 4 | SP04/SE04 | SP05/SE05 | MP04/ME04 | MP05/ME05 |
| Booklet 5 | MP05/ME05 | MP06/ME06 | SP05/SE05 | SP06/SE06 |
| Booklet 6 | SP06/SE07 | SP07/SE07 | MP06/ME06 | MP07/ME07 |
| Booklet 7 | MP07/ME07 | MP08/ME08 | SP07/SE07 | SP08/SE08 |
| Booklet 8 | SP08/SE08 | SP09/SE09 | MP08/ME08 | MP09/ME09 |
| Booklet 9 | MP09/ME09 | MP10/ME10 | SP09/SE09 | SP10/SE10 |
| Booklet 10 | SP10/SE10 | SP11/SE11 | MP10/ME10 | MP11/ME11 |
| Booklet 11 | MP11/ME11 | MP12/ME12 | SP11/SE11 | SP12/SE12 |
| Booklet 12 | SP12/SE12 | SP13/SE13 | MP12/ME12 | MP13/ME13 |
| Booklet 13 | MP13/ME13 | MP14/ME14 | SP13/SE13 | SP14/SE14 |
| Booklet 14 | SP14/SE14 | SP01/SE14 | MP14/ME14 | MP01/ME01 |
2 Each item block exists in both modes of administration, the paper-based version with the letter “P” in its label and the computer-based version with the letter “E” in its label.