Key metric plummets after JoCo health department changes COVID-19 dashboard — here’s why

Those who keep close tabs on the Johnson County Department of Health and Environment’s COVID-19 dashboard likely noticed a significant change this week.

On Monday, JCDHE rolled out several adjustments to how it calculates and displays COVID-19 data for the county. One of the most notable differences involves Johnson County’s percent positivity rate, or the rate of new tests that come back positive.

The metric is a key indicator used by public health officials across the U.S. to assess how much community spread is potentially occurring in their area. In Johnson County, percent positivity is also one of the main data points public schools have been using to make decisions about when and how to bring students back for in-person learning.

The change Monday by JCDHE resulted in a significantly lower percent positivity figure. Before the change Monday, JCDHE displayed Johnson County’s percent positivity at 11.3%. After the change, it fell to 6.6%.

This is the second time in less than a week that JCDHE has made major changes to how it communicates COVID-19 data to the public. Last week, county health officials adjusted their gating criteria, relaxing standards for allowing secondary students to have some in-person learning.

These changes come as county health leaders and local public schools officials continue to face intense pressure from some parents to reopen schools more fully amid the pandemic. Parents and students have protested at school board meetings across Johnson County, waving signs and chanting messages like, “Let them play!” and “Let us learn!”

The shift could have major ramifications for schools as they continue to monitor county health trends and more students return for in-person learning in coming weeks. We’ll get to that, but first: what’s behind the data change? And how should Johnson Countians interpret these new numbers?

Three methods to calculate percent positivity

JCDHE spokesperson Barbara Mitchell wrote in an email to the Post that county health officials opted to change their methodology for calculating percent positivity to reflect a “truer picture of what is going on in our community.”

That’s not unprecedented. Public health officials from Texas to Minnesota have made adjustments to how they calculate percent positivity to reflect changing conditions of the pandemic in their locales.

In an explainer sheet sent to the Post, JCDHE said there are three “main ways” that percent positivity can be calculated, all involving different numbers by which to divide the total number of new positive cases over a given time frame, typically 14 days.

  • Method 1: New positive cases/ “unique tests” —This is the method JCDHE had been using until Monday, which had been yielding higher percentages.
  • Method 2: New positive cases/individuals tested — This is the method JCDHE is now using and mirrors the method used by the CDC.
  • Method 3: New positives/total tests — This is the method currently used by the Kansas Department of Health and Environment.

This graphic from JCDHE shows the long-term trend line of 14-day percent positive rates for all three methods, with the green line demonstrating JCDHE’s new method. Method 1, the method JCDHE had been using up until Monday, consistently yielded the highest overall number. But JCDHE officials point out the trend lines themselves all appear similar. 

How each method yields different rates

This might be best illustrated through a fictional example.

A Johnson County resident, let’s call her Jo, has gotten tested multiple times during the pandemic. She got tested initially in April, during the the early days of the spread of the disease and tested negative. She got tested again in September, as a precautionary measure before getting together with her grandkids for the first time in months, and again, she tested negative.

The way Jo’s multiple negative tests will be counted affect the ultimate percent positivity rate.

Using Method 1 (New positive cases/”unique tests”)

This method only counts Jo’s initial negative test in April and not the later one in September. “If that first test is negative and they go back for additional tests, those subsequent tests would not be counted,” the JCDHE explainer says.

This would lower the denominator (the total number of “unique tests”) for 14-day positivity trends in September, thereby increasing the final percent positivity rate. Furthermore, let’s say Jo got tested in September not because she wanted to see her grandkids but because she was feeling sick, and she tested positive. Her positive test in September, then, would be counted in the numerator of the 14-day calculation at that time, but she would still only be counted once in the denominator, as a “unique test.”

JCHDE officials acknowledge Method 1 does not take into account the increasing number of Johnson County residents getting retested as testing has become more widespread.

“At the beginning of the pandemic, [using Method 1] wasn’t an issue because testing was limited and few people were taking more than one COVID-19 test,” the JCDHE explainer says.

But critics — including some parents who have been pushing local school districts to reopen more fully — have argued JCDHE’s percent positivity rates using Method 1 were artificially high.

Using Method 2 (New positive cases/individuals tested)

The method JCDHE is now using counts all individuals tested within a 14-day window, regardless of whether they tested negative previously.

The new method being used by the Johnson County Department of Health and Environment counts all individuals tested within a 14-day window, regardless of whether they tested negative previously. File photo.

So, Jo tested negative in April and got counted towards the county’s overall positivity rate then. Under Method 1, she would not have been counted again when she went back for a new test in September. But under Method 2, she does get counted, thereby increasing the denominator used for the county’s percent positivity calculation for that 14-day period in September.

That, extrapolated over thousands of tests, is what leads to a lower percent positivity rate overall.

There is a key nuance in Method 2, however. Individuals who test negative multiple times in the same 14-day window will only be counted once towards the final percent positivity rate. That’s because there are some individuals — front-line health care workers and professional athletes, for instance — who get tested nearly daily.

Including all their negative tests could lead to what some public health officials feel is an artificially low positivity rate.

Using Method 3 (New positive cases/total tests)

But that is what the Kansas Department of Health and Environment does. They divide new positive cases by all tests conducted in a certain time frame, regardless of whether the tests may come from individuals being frequently retested.

That’s made KDHE’s percent positivity figure for Johnson County consistently lower than the figures put out by JCDHE. Even with JCDHE’s new methodology, Johnson County’s percent positivity as calculated by the state remains lower.

As of Monday, KDHE said Johnson County’s percent positivity was 5.4%.

It would be as if Jo got tested twice in consecutive weeks in September and tested negative both times. Using the state’s method, both of those tests would have been included in a final calculation for a 14-day percent positivity rate, increasing the denominator and leading to a lower overall figure.

What this means

The county’s percent positivity rates have been a critical part of the debate of how to approach in-person learning. At a recent Shawnee Mission school board meeting, a group of parents submitted a detailed letter arguing for an expansion of in-person learning options for SMSD students, and they based part of their argument on the lower percent positive rate for Johnson County put out by the state of Kansas, compared to JCDHE’s figures.

But public health officials caution that, regardless of what final number people settle on, the percent positive figures being published suggest community spread remains high in Johnson County, making activities like classroom learning potentially unsafe.

Speaking recently on the KU Health System’s daily COVID-19 briefing broadcast on Facebook, Amanda Gartner, KU’s director of quality and safety, said the “real takeaway” from the discussion over the different percent positive figures is that all the data suggest “the disease is still prevalent in our community.”

“The numbers really shouldn’t change our behaviors at his point. Once we get a stable number and it is on the decline and stays on the decline, that will give us a different message,” Gartner said. “But I think right now — regardless of whether it’s 6, 8, 10, 12% — we still have a high positivity rate and it’s still active in the community.”