‘What’s an avocado?’: Localising NAPLAN questions lifts scores

A pioneering study has found adjusting NAPLAN questions to make them culturally relevant to regional and Aboriginal students could close the Indigenous reading gap by 50 per cent and reduce the urban-rural gap by a third.

More than 1100 students from 20 schools in Dubbo in central western NSW were involved in a randomised control trial in which half the test papers were changed to replace unfamiliar references with local ones, such as a reading prompt that referred to the Parkes “dish” rather than a lighthouse.

Dubbo College Delroy Campus year 8 students Jordan Butcherine, 14 (left), Zana Dillon, 14 (centre) and Khynan Roworth, 14 (right), sat the culturally contextual NAPLAN test.Credit:Kate Geraghty

Researchers from the University of NSW’s economics of education knowledge hub said the improvement of students with the adjusted test suggested familiar content could increase reading scores for year 5 Indigenous children by nearly 25 NAPLAN points, lifting them above the national minimum standard.

NSW Education Minister Sarah Mitchell described the findings as “incredibly powerful for our Indigenous and culturally diverse students”.

In a study that also involved the NSW Department of Education and the Aboriginal Education Consultative Group, half of year 6 and 8 students were given a standard NAPLAN paper. The other half were given the same paper with the questions changed to include objects and images familiar to local students.

A prompt about looking after a guinea pig was changed to one about looking after a dog, a story about getting lost was situated in the Wellington Caves. In the numeracy test, half an avocado was replaced by half a chicken and a treasure map was substituted by one using Aboriginal tools.

There was little difference in the numeracy results between the groups. But in reading, there was a significant impact; the researchers found the “average treatment effect” for those sitting papers with culturally contextual questions could lead to an improvement of more than 12 points on the NAPLAN scale, compared with those who did not.

Indigenous students could improve by nearly 25 points, pushing them from band 4 to 5, which is above the minimum standard. “The different in performance on contextualised reading tests represents 33 per cent of the rural-urban gap and 50 per cent of the Indigenous-non-Indigenous gap,” the researchers said.

Former NSW education minister Professor Adrian Piccoli was one of those involved. Teachers had long believed students struggled with unfamiliar questions, but the impact had never been measured.

“People knew it made a difference, but it had never been tested, literally, like this,” he said.

Another researcher, economics Professor Richard Holden, who has been involved in education-related randomised controlled trials in the United States, said the magnitude of the difference in results suggested culturally appropriate classroom materials, such as textbooks and handouts, could also have a big impact on learning outcomes.

Dubbo College Delroy Campus year 8 students Jordan Butcherine, Khynan Roworth, 14, and Zana Dillon, 14, who all sat the NAPLAN test that is more appropriate for Indigenous students.Credit:Kate Geraghty

While the results left a question mark over whether the Indigenous education gap might not be as big as previously thought, “it’s still a pretty big gap – it might be reduced by half, but it’s still a big gap,” he said. “This is a step towards seeing how we can bridge the rest of the gap”.

Professor Holden urged authorities such as the Australian Curriculum, Assessment and Reporting Authority (ACARA), which runs NAPLAN, to use more generic images and ideas in its test questions. “I think just paying extra attention to those sorts of things might be one way forward,” he said.

Warren Richardson, a Ngunnawal man, was involved in contextualising the questions on behalf of the Aboriginal Education Consultative Group. “I remember never being able to see myself within the questions, that’s what was driving me and my team to get these questions right,” he said.

“For me, if you don’t understand the question, you’re not going to be able to answer it. What’s an avocado? There were two parts to this; understanding the question and seeing themselves within the question.

“We’ve worked a lot in camps – STEM and language and culture camps. These give kids access to STEM through a cultural lens. We could see the difference [in the students’ understanding], but we never had evidence. This is pure evidence.”

A spokeswoman for ACARA said NAPLAN materials were reviewed by panels from all states and territories. “[They] include, wherever possible, Indigenous representatives and great care is taken during item development to ensure cultural contexts do not impact the students’ opportunities to display their knowledge,” she said.

The Morning Edition newsletter is our guide to the day’s most important and interesting stories, analysis and insights. Sign up here.

Most Viewed in National

From our partners

Source: Read Full Article