What happens when data on the COVID-19 pandemic goes missing?

Experts say the information was important for individuals to make decisions about whether and when to test, quarantine or wear a mask.

An area line chart on graph paper showing a significant drop in cases over time and a large question mark where the drop occurs. There is text that reads: where did the data go?
Andjela Padejski / WBEZ
An area line chart on graph paper showing a significant drop in cases over time and a large question mark where the drop occurs. There is text that reads: where did the data go?
Andjela Padejski / WBEZ

What happens when data on the COVID-19 pandemic goes missing?

Experts say the information was important for individuals to make decisions about whether and when to test, quarantine or wear a mask.

WBEZ brings you fact-based news and information. Sign up for our newsletters to stay up to date on the stories that matter.

In August, Colette Matejcak sat down with her two boys — a second grader and a fifth grader at a private school in the western suburbs — to discuss how they were going to navigate COVID-19 this school year.

Even as a family nurse practitioner, Matejcak said she wasn’t sure what direction the pandemic was heading and what data to monitor in order to help protect her family. She had come to rely upon updates from her children’s school when officials there confirmed a COVID-19 case. Sometimes, she’d get multiple notifications a week. But so far this school year, she hasn’t heard anything.

“I’d love to think that nobody’s gotten COVID since the school year started, but I also don’t know that that’s true,” Matejcak said. “Information has dropped off pretty precipitously.”

Matejcak’s hunch about the missing data was right. Earlier this year, the Illinois Department of Public Health (IDPH) stopped publishing data on COVID-19 outbreaks by school and potential COVID-19 exposures. In addition, the Office of the Cook County Medical Examiner no longer publishes detailed information about most COVID-19-related deaths.

These were datasets that parents, residents, advocates and journalists relied on to track the pandemic. Their disappearance has made it harder for the public to understand COVID-19’s risk and its impact in their communities and schools.

Experts worry having fewer community-level datasets on COVID-19 means individuals now have even less information to make informed decisions about whether and when to test, quarantine or wear a mask.

“It’s very alarming,” said Dr. Enbal Shacham, a behavioral science and public health researcher at Saint Louis University, on the lack of public data.

“Data helps us inform how to engage in preventative behaviors,” she said. “If you don’t have the information to make those decisions, I don’t understand how we can continue making individual-based decisions for ourselves or our children.”

This scaling back of data on the pandemic’s presence in our communities has also come at a time when another fall and winter surge in COVID-19 cases could be around the corner, as colder weather moves more social activities inside and holiday travel picks up. In the fall of 2021, the appearance of a highly contagious omicron strain led to a surge in cases and hospitalizations that accelerated over the holiday season through early 2022.

Public health officials are hoping uptake of the new booster shot now will be high enough to avoid the worst effects of a surge later. But what still remains to be seen is how many people actually get the shots and whether new variants will arrive in the coming months that are able to circumvent the vaccine’s protections. So far, less than 10% of Chicagoans eligible for the updated booster have received it, and 60% of the shots have gone to white residents.

Recently, the Centers for Disease Control and Prevention (CDC) guidance has put the burden on individuals to know their own COVID-19 risk thresholds and make decisions accordingly.

In August, the CDC updated its COVID-19 guidance, lifting the requirement for individuals to quarantine if exposed to the virus, regardless of vaccination status. The CDC also dropped the “test-to-stay” guidance that required schools to administer frequent tests to students exposed to the virus in order to keep them in classrooms and out of quarantine.

Parents left in the dark

In April, IDPH stopped updating data on school outbreaks and potential exposures following changes to its contact tracing policy, according to Michael Claffey, public information officer for IDPH.

“It was no longer possible to reliably report all school outbreaks,” Claffey wrote in an email to WBEZ. “Instead, the system would highlight those schools that were working hard to identify them — but miss many others that were not making an effort.”

For suburban parents like Matejcak, the data on school outbreaks was vital because there weren’t many other places where they could find it. Information included whether the outbreak affected staff, students or both, as well as the case count.

“Earlier on in the pandemic, it was easier to find that information — either through the Illinois Department of Public Health or through the Kane County Health Department,” she said. “The data just kind of disappeared.”

In Chicago, there’s more data available for parents. Chicago Public Schools (CPS) publishes daily and weekly cases across the district, daily and weekly cases by school, and student and staff vaccination rates by school.

But even CPS data has limitations, and aren’t necessarily reflective of the entire district. Data on cases published on the CPS dashboard comes from two sources: self-reporting and weekly COVID-19 testing, which is based on an opt-in system, a districtwide testing program that requires consent from students, staff or parents for students under 18, for participation. On Oct. 11, 2022, the districtwide opt-in rate for weekly testing was just under 11%, according to the CPS COVID-19 dashboard.

Because testing is so low, parents don’t have a complete picture about new COVID-19 cases in their children’s classrooms, said Arturo Carrillo, deputy director of violence prevention, neighborhood health initiatives and clinical services at Brighton Park Neighborhood Council.

“Parents are finding out about possible exposures by way of anecdotal stories that are shared by their children once they come home from the classroom,” he said.

Missing details on COVID-19 deaths

According to IDPH, more than 35,000 people in Illinois have died from COVID-19 since the start of the pandemic. In 2020 and 2021, COVID-19 was the third-leading cause of death in Illinois, behind heart disease and cancer, according to the state’s public health department.

But it’s difficult to get a clear picture of who’s dying and where the deaths are occurring. While city, county and state agencies all provide information about COVID-19 deaths, the data is not complete and lacks key metrics.

In Cook County, the medical examiner’s office was a key source of disaggregated data that helped show patterns or disparities in deaths related to COVID-19. The medical examiner reported detailed information on people who died: where they lived, where they died, time of death, race, age, as well as the primary and secondary causes of death.

But because of a change in what deaths the medical examiner was responsible for investigating, the public no longer has access to a lot of the detailed data about most COVID-19 deaths.

In April 2022, the medical examiner’s office stopped determining the cause of COVID-19 deaths that occur in hospitals, nursing homes or hospice.

“The Medical Examiner’s Office takes jurisdiction of any case where manner of death is not natural and any other cases which require investigation or when there is no treating physician to sign the death certificate,” wrote Natalia Derevyanny, a spokesperson for the Cook County Bureau of Administration, in an email to WBEZ.

“We are pleased the data our Office collected during the pandemic was able to inform public health agencies, journalists, government officials and the public at large. We understand the importance of this data and were diligent in collecting and disseminating it. The [medical examiner’s office] can only take cases that fall under our jurisdiction and once more was known about the pathology of the virus and deaths decreased dramatically, we determined that treating physicians now had jurisdiction of these natural deaths,” Derevyanny explained.

Illinois law requires health care providers to report information on every death related to COVID-19 and include demographic details such as the patient’s age, sex and ethnicity as well as medical information about their death, but that information is not available to the public. WBEZ reporter Kristen Schorsch requested the data in 2020, but IDPH denied the request and gave only a snapshot of the number of positive COVID-19 cases such as the types of jobs the patients had and the number of confirmed cases by the type of medical setting, such as a hospital or a daycare center.

Still, the data the state collected was woefully incomplete. A few months into the pandemic, in nearly 80% of cases, hospitals or others didn’t list the types of jobs patients had, for example, making it difficult to track and prevent potential outbreaks.

The medical examiner now only designates cause of death in cases where someone died at home and did not have a treating physician that could sign a death certificate, Derevyanny wrote.

A banner at the top of the Cook County medical examiner’s COVID-19 Related Deaths dashboard points visitors to the state health department’s COVID-19 dashboard, which includes the aggregated number of COVID-19 deaths by race and age for the entire state.

A screenshot of the banner at the top of the Cook County Medical Examiner's COVID-19 Related Deaths dashboard that reads: Effective April 1, 2022, the Cook County Medical Examiner’s Office no longer takes jurisdiction over hospital, nursing home or hospice COVID-19 deaths unless there is another factor that falls within the Office’s jurisdiction. Data continues to be collected for COVID-19 deaths in Cook County on the Illinois Dept. of Public Health COVID-19 dashboard.
A screenshot of the yellow banner at the top of the Cook County Medical Examiner’s COVID-19 Related Deaths dashboard. Cook County Medical Examiner

But the Illinois Department of Public Health’s COVID-19 tracker doesn’t provide any details about each individual who has died from COVID-19 as the Cook County medical examiner once did. The state also doesn’t report the aggregate number of COVID-19 deaths by ZIP code, listing that data as “N/A” on the tracker, “because of the concern that victims could be identifiable,” wrote Claffey in an email to WBEZ.

Death data by race and ZIP code, or by age and ZIP code, is also not published for the same reasons, wrote Claffey. “I am assured that we have never posted COVID deaths by ZIP code,” he wrote.

IDPH deaths by ZIP code screenshot
A screenshot of the COVID-19 cases and deaths by ZIP code from the Illinois Department of Public Health displays no information on the number of deaths for each ZIP code. Illinois Department of Public Health
The Chicago Public Health Department does publish deaths by ZIP code on the city data portal. Based on city data, we know the 60621 and 60636 ZIP codes — which, combined, cover all of West Englewood, practically all of Englewood and portions of Chicago Lawn, Greater Grand Crossing, Gage Park and Washington Park — have had the highest rates of deaths related to COVID-19 in the city, but it’s hard to determine those rates for communities in suburban Cook County or for many places in the rest of the state.

Data on mortality by ZIP code is necessary for community advocates, such as Carrillo with the Brighton Park Neighborhood Council, who are working to address racial and geographical disparities in medical care.

“We know that there’s a severe health access issue when it comes to receiving speciality and sometimes primary care on the South and West sides of Chicago, where we’re located,” said Carrillo. “And we know that has absolutely contributed to the higher infection, and severe infection rates of COVID in our area.”

However, mortality data by ZIP code is not readily available for many areas outside of Chicago. Carrillo says it’s much-needed information. “The more informed people are, the more able they are to take precautions and the more we can mitigate the spread of COVID,” he said.

Another implication of the medical examiner’s decision to change the data reflected on the dashboard is that it may impact the community’s ability to understand the severity of COVID-19.

According to the latest data published by the Cook County medical examiner’s office, there were only 36 COVID-19 deaths from April 1, 2022, through Oct. 11, 2022. That’s just 2% of the number of such deaths over the same period in 2021, according to a WBEZ analysis of the medical examiner’s data.

This underreporting of deaths could mislead people about the true toll of the pandemic, especially when there is already a perception among some that COVID-19 is no longer a serious threat.

“There’s kind of this perception that COVID is not as dangerous because there’s this perception that COVID isn’t leading to higher levels of mortality that it once was, and that’s a concern for us in the community because we know that community spread is still happening,” said Carrillo.

What COVID-19 data do we still have?

The primary metric the CDC and many public health departments at the state and local levels publish is a “COVID-19 community level” — categories for an area’s risk ranked low, medium or high.

Three factors determine an area’s COVID-19 community level:

  • The number of new cases per 100,000 in the past seven days

  • The total number of new hospital admissions per 100,000 in the past seven days, and

  • The percentage of staffed inpatient hospital beds that are occupied by COVID-19 patients in the past seven days

The CDC launched its community-level metric in February to “help communities decide what prevention steps to take based on the latest data,” according to the CDC’s website.

The CDC publishes individual-level prevention guidelines to take at each COVID-19 community level. At all risk levels, they recommend people stay up to date on vaccination and, if they test positive, follow recommendations for isolation. At the medium level, they recommend people who are immunocompromised to wear masks; and at the high level, everyone should wear masks.

Community levels are updated by the CDC every week for each county in the nation. As of Oct. 12, Chicago and Cook County are both at a low community risk level.

“Things are continuing to look good from a COVID perspective here in Chicago,” said Dr. Allison Arwady, commissioner of the Chicago Department of Public Health, in a virtual Q&A on Oct. 11. She urged everyone over the age of 12 to get the updated COVID-10 vaccine this fall in order to keep cases, deaths, hospitalizations and ER visits as low as possible.

Community levels tell us how many people are getting really sick from COVID-19, said Landon, the infectious disease specialist.

But Landon says the problem with only looking at the number of people who are being hospitalized, and not everyone who is getting COVID-19, is that it overlooks the potential long-term risks associated with getting COVID-19 at all.

“The fact of the matter is that people are not getting very sick with COVID,” Landon said. “The problem is that there are these long-term consequences that are not well fleshed out yet.”

What data should we be watching?

With a potential surge in COVID-19 over the fall and winter months, some experts encourage the public to pay attention to a few additional data points, beyond COVID-19 community levels.

One of these metrics is the CDC’s Community Transmission, which is the higher of the number of cases per 100,000 people or the percentage of positive tests over the past seven days. Landon believes Community Transmission is a better metric to gauge the risk of contracting any degree of COVID-19, which matters when it comes to preventing long COVID, she said.

Long COVID refers to people who were previously infected with COVID-19 and continue to experience a wide range of symptoms — including difficulty breathing, sleep problems and brain fog — sometimes for months or years. About one in every five people who have survived COVID-19 experienced persistent complications in the following year, according to a May 2022 CDC study. The risk was higher for people 65 years and older.

Community Transmission rates are categorized into High, Substantial, Moderate and Low levels. As of Oct. 12, almost 50% of counties across the U.S. were at a high transmission rate, meaning there are 100 or more cases per 100,000 people in the last seven days.

Two maps from the CDC of the US by county with a mostly green map of community levels on the left and a mostly red map of transmission rates on the right.
Map of CDC COVID-19 community levels on the left, and a map of CDC transmission rates on the right. Each map paints a different portrait of risk. Amy Qin / WBEZ

“I really look at [transmission rates] as our potential opportunity to prevent long-term consequences,” Landon said. “It’s important to think about your risk of acute COVID and your risk of long COVID — or other organ damage from COVID — separately.”

Although the CDC’s transmission rates are likely an undercount — because most people do not report the results of at-home tests — and almost certainly lag behind what’s actually happening on the ground, that information provides useful trend data, said Landon.

When cases are going up, it’s likely the risk of getting infected is already higher than what the numbers say, and when cases are going down, it’s likely the risk is lower than what the reported numbers show due to the lags in reporting, Landon said.

“The trending of things is way more important than the raw numbers,” she said.

Despite the changes we’ve seen to local COVID data reporting and availability, the state of public data is “far, far better” than what it used to be in the early stages of the pandemic, according to Dr. Jerry Krishnan, associate vice chancellor for population health at the University of Illinois Chicago. Krishnan, who is also a professor of medicine and public health, uses both the CDC and IDPH websites to monitor acute COVID cases.

“If you’re trying to figure out what’s going on in your community, you have to look at the best data that you have in your community,” Landon said. “You want to look at everything that they have.”

Amy Qin is WBEZ’s data reporter. Follow her @amyqin12. Charmaine Runes is WBEZ’s data/visuals reporter. Follow her @maerunes.