TIMSS 2019 Countries Develop Mathematics and Science Items in Hamburg
At the TIMSS 2019 2nd National Research Coordinators Meeting in Hamburg, Germany, representatives of participating countries worked together to develop mathematics and science items for the assessment.
Most item writers huddled in clusters of two or three, working with intense focus. One murmured to himself, deep in thought, while carving out geometrical shapes in the air. Some shared their thoughts on the process of item writing, and what makes a good item.
“It would have a real world applicability, a realistic context,” said Terry O'Brien of Ireland. “The child would be familiar with the scenario.”
Most item writers underscored the importance of brief, easily understood wording that still is probing. Some suggested the addition of graphics for visual interest, or starting with a clearly defined framework.
Mark Chetty of South Africa said the TIMSS framework is a very good one for a basis on which to construct an item. He added that a good item writer has content knowledge and assessment knowledge. “Often it’s one or the other, and to get the package is a difficult mix,” he said.
“You need experience in the teaching field, to know where to calculate,” added his colleague, Mariette Visser. Teachers assess students on an ongoing basis, and are familiar with how they think, she said.
While several item writers said their focus was on the cognitive domains of knowing, applying, and reasoning, June Chau of Singapore said she was thinking of items to fit into the eTIMSS framework, and visuals to accompany them. “I was thinking of how to build in the e-features,” she explained. “The drop-down menus, the sorting, and so on.”
Some countries including Singapore, the United Arab Emirates, Norway, and South Africa came prepared with items already written, and had a head start.
Item writers often professed a preference for mathematics or science. Some favored writing Grade 4 items because they previously taught at that level, or found the concepts and language more straightforward. Others preferred to create Grade 8 items because they could relate to the higher level of knowledge at that grade.
“I think the Grade 4 science items are the hardest ones to write because the language needs to be relatively simple for students to read,” said Robyn Caygill of New Zealand. “Those students tend to be lower in science [competency], and our knowledge is way above the level of the students. So bringing ourselves to that level can be quite difficult.”
Other item writers echoed some of those thoughts. “Science items are more complicated because of the length of the situation you have to establish,” said Marc Colmant of France, saying science items require more wording. “It’s more difficult to find a short situation in science.”
TIMSS Executive Directors Michael Martin and Ina Mullis, IEA Executive Director Dirk Hastedt, and IEA Hamburg Director Heiko Sibberns welcomed participants to the TIMSS National Research Coordinators Meeting in Hamburg, Germany
Ina Mullis, TIMSS & PIRLS International Study Center Executive Director, presented an overview of activities for TIMSS 2019.
Michael Martin, TIMSS & PIRLS International Study Center Executive Director, delivered a progress report for TIMSS 2019.
Dirk Hastedt opened the meeting by welcoming item writers from all the TIMSS 2019 countries to Hamburg.
From left: Robyn Caygill of New Zealand, Gia Khatisashivili of Georgia, Aizatul Adzwa Md Basri of Malaysia, Aigul Baigulova of Kazakhstan, and Qasim Alshannag of Abu Dhabi prepared science items together.
Jenny Lindbom of Sweden and June Chau of Singapore teamed up to brainstorm mathematics items.
Terry O’Brien of Ireland and Marc Colmant of France collaborated on mathematics items.
Pierre Foy, Director of Sampling, Psychometrics, and Data Analysis at the TIMSS & PIRLS International Study Center, discussed survey operations for the TIMSS 2019 field test.
David Ebbs of IEA Amsterdam presented an update on translation and verification of the TIMSS 2019 field test instruments.
Mark Cockle of IEA Hamburg delivered a presentation on data management.
Marcin Chrzanowski and Dominik Marszal of Poland (left), and Juan Antonio Carchano Lopez of Spain developed science items.
Mariette Visser, Mark Chetty, and Vijay Reddy, all of South Africa, and Samuel Sollerman and Maria Axelsson of Sweden were assigned to mathematics items.
Jamie Scott of Ontario, Canada, and Katrin Schulz-Heidorf of Germany paired up to produce science items.
Clockwise from left: Nada Abu Baker Husain Ruban, Mai Khalaf, Moza Rashid Ghufli, Wedad Meh Al Shehhi, Khadeijah S.M. Hamdan Alhmoud, Shiekha Mohammed Al Seriady, Badreya Ali Abdullah Alshehhi, and Nayel Itijateen, all of Abu Dhabi or the United Arab Emirates, worked together on mathematics items.
Balázs Szalay of Hungary and Ahmad Issa Altweissi of Jordan developed science items together.
Hilal Humaid Ahmed Al-Qassabi and Haitham Zahran Sa’ud Al-Forqani, both of Oman, wrote science items.
Ketty Savioli of Italy, Bettina Toferer of Austria, and Louis Scerri of Malta teamed up on mathematics item writing.
Fumi Ginshima (right) consulted with Milena Taneva of IEA Hamburg on data for Japan.
Duygu Savaşcı of IEA Hamburg assisted Joel Rapp with data for Israel.
Sue Thomson (right) consulted Sylvie LaRoche of Statistics Canada on Australia’s data.
Duygu Savaşcı and Umut Atasever of IEA Hamburg provided consultation to Rezana Vrapi, Endri Qejvanai, and Jeton Stojku of Albania.
Milena Taneva assisted Hawraa Al-Qattan of Kuwait with her data.
Sofia Vlami and Nikos Papadouris of Cyprus consulted with Duygu Savaşcı and Umut Atasever of IEA Hamburg regarding their data.
TIMSS & PIRLS International Study Center thanks our our item writers, who contributed close to 1,000 draft mathematics and science items for the TIMSS 2019 field test by week’s end.
