Shawnee Mission student performance on state tests, ACT has fallen slightly during pandemic

COVID-19 Johnson County schools

Shawnee Mission students' Kansas Assessment and ACT scores decreased slightly from the last pre-pandemic school year, 2018-19, to the 2020-21 school year, according to new data. Above, Shawnee Mission South students take a test. File photo.

Shawnee Mission School District officials, families, teachers and students have said for months how challenging the COVID-19 pandemic has been.

Now, the district has a clearer picture of just how much the past two years have impacted students in the classroom.

Newly compiled assessment data shows that since the onset of the COVID-19 pandemic nearly two years ago, SMSD student performance has declined somewhat on assessments used by both the state and more broadly.

Dan Gruman, director of assessment and research, presented an update of student assessment to the school board earlier this month.

The presentation detailed Shawnee Mission students’ score data from annual state tests, as well as ACT scores taken during the 2020-21 school year.

Prior to presenting the data itself, Gruman explained to the board how COVID-19 impacted results.

“Sometimes we jump right into the data, and this time around I want to make sure we all have a clear picture of, one, where we’ve been in the last two years and the context and, two, making sure we have a common understanding of what these scores predict,” Gruman said.

Key data points

Students receive a score for annual state-mandated tests that fall between a range of 220 and 380.

Those numeric scores are then translated into performance levels, ranging from 1 to 4.

The performance level language used by the state has recently been adjusted.

Currently, the state says students on Level 1 show “a limited ability to understand and use the skills and knowledge needed for post-secondary readiness.”

Students at Level 2 show “basic” ability. Students at Level 3 show “effective” ability, and students on Level 4 four “show excellent” to use college-ready skills and knowledge.

Below is a bar graph, generated by the Post using Shawnee Mission data from Gruman’s presentation, that looks at SMSD English state assessment data for all students in 2021 compared to 2019:

Loader Loading...
EAD Logo Taking too long?

Reload Reload document
| Open Open in new tab


Here is a bar graph, generated by the Post using Gruman’s data, that looks at SMSD mathematics state assessment data for all students in 2021 compared to 2019:

Loader Loading...
EAD Logo Taking too long?

Reload Reload document
| Open Open in new tab


In both subjects, the percentage of SMSD students finishing in Level 4 — the highest level — declined by roughly 2%.

At the same time, the percentage of students finishing in Level 1 increased in both subjects, but the jump was more substantial in math. In 2021, the percentage of students finishing in Level 1 in English went up 3% and up by 5.8% in math.

The percentage of students in Level 2 for both English and math stayed relatively the same, while the percentage of students in Level 3 in both subjects also declined slightly.

Meanwhile, the average ACT composite score decreased from 22.4 for the class of 2020 to 21.5 for the class of 2021, according to the presentation.

That mirrored state and national trends and also continued a downshift in average ACT scores the district had marked starting before the pandemic.

Shawnee Mission students’ composite ACT scores have been declining for the last four years.

Graphic via SMSD Board of Education documents.

Still, Shawnee Mission’s average ACT composite score is above the national average of 20.3 and the state average of 19.9, according to the district.

Additionally, the 21.5 average also meets the criterion laid out by all six state universities in Kansas that calls for a score of 21 or better. All except the University of Kansas will accept students who either meet a certain grade point average or get a 21 or above on the ACT.

No Shawnee Mission student — or any student in Kansas — took a state assessment test in 2020 due to COVID-19. In 2021, 1,154 Shawnee Mission students missed the state assessment test, Gruman said. Above, a Shawnee Mission South student takes a test. File photo.

COVID-19 impact

The pandemic brought on a number of challenges when it came to annual testing, Gruman said.

For starters, the pandemic canceled the administration of state assessments in the 2019-2020 school year, which is why there is no data for that year.

Additionally, many students still missed out on testing in the 20-21 school year.

There were 46 students who missed taking state assessments districtwide during the 2018-19 school year, the last full year before the pandemic.

In contrast, during the 20-21 school year, there were 1,154 students who missed the assessments, Gruman said.

That was due, in part, to the high number of students learning remotely that school year. The number of students who missed that year’s assessments include students who may have simply refused or opted out of taking tests, which still needed to be administered in person.

Schools had to find time to reach out to the families of remote-learning students — who made up about 27% of the district’s student population that year — and schedule a time and place for students to take state tests in person because the tests could not be administered remotely, Gruman said.

Additionally, there were fewer ACT testing opportunities for students in the class of 2021 — which resulted in many students only taking the ACT one time because of the pandemic.

Test scores a ‘modest predictor’ of success

While test scores matter, there are other predictors of college readiness for students like GPA and socioeconomic status, Gruman said.

For instance, Gruman argued that there is no evidence to suggest that students landing at Levels 2 or 3 on state assessments in math or English are not for college classes.

Heather Ousley, Shawnee Mission at-large board member, said Gruman’s assessment presentation was “well-timed” amid discussions about Shawnee Mission students’ success. File photo.

“At best, I would say test scores, they do serve as a modest predictor of success in the absence of other data, but a single test score does have some challenges and can be problematic by itself,” Gruman said.

Students’ college and career readiness was an issue invoked during November’s school board contests, in which some challengers argued declines in student performance showed the district was not preparing enough students for post-secondary life.

Mary Sinclair, SM East area board member, said this month that student test data during the pandemic has sometimes been miscast.

Sinclair said her takeaway from Gruman’s report is that college prep coursework, GPA and socioeconomic status are more effective predictors than test scores.

Additionally, Gruman said nearly 70% of all students in the class of 2019 enrolled in college, and 90% persisted in college from their first to second year — no matter their performance on state tests.

Heather Ousley, at-large board member, said the report is “well-timed” with the legislative session and “the current discussions about the success of our students.”

“To have roughly 70 to 80% of our students performing well entering college and persisting in college I think is really important information for people to have, because there is a narrative that that is not the case,” Ousley said.