You can’t catch what you don’t see. New Yorkers will remember how this saying defined the early weeks of the city’s COVID-19 pandemic, as the boroughs and the nation struggled to develop a test for the virus. Delayed or absent detection meant people showed up to hospitals with their symptoms too far advanced, raging and beyond recuperation.

Laboratory testing gradually built itself up and evolved to cope with case surges, but it is now entering a new evolutionary phase with omicron. The highly transmissible variant has propelled the use of at-home COVID-19 tests, especially when lines for in-person tests were hours long or difficult to book around the holiday season. The U.S. Postal Service will now deliver kits to your door — if you can bypass the kinks in its ordering website. Supplies have improved online and at retail stores after deep shortages. And New York City schools are handing out millions of kits to exposed students and staff as well as symptomatic people who don’t want to stand in lines during this frigid winter.

But city officials are not tracking the results of the at-home tests, unlike other counties in New York and other major cities such as Washington D.C. and Austin, Texas. As Gothamist reported last week, this rise of the home kit is contributing to the sharpest drop in conventional testing that New York City has experienced during the pandemic.

As community transmission cools off, testing numbers tend to fall, but never quite as fast as current data show. Some health experts worry this decline is part of a larger national pattern with coronavirus testing, one that could make it harder to spot new variants.

“The biggest worry that I have is that we're going to miss changes in the epidemiology of this virus because we're losing that surveillance,” Dr. Jennifer Nuzzo, a senior scholar at the Johns Hopkins Center for Health Security and the lead epidemiologist for the Johns Hopkins COVID-19 Testing Insights Initiative, told Gothamist.

A line stretches over two avenues long, as New Yorkers wait to receive free home test kits in Harlemn, December 24th, 2021.

A line stretches over two avenues long, as New Yorkers wait to receive free home test kits in Harlemn, December 24th, 2021.

A line stretches over two avenues long, as New Yorkers wait to receive free home test kits in Harlemn, December 24th, 2021.
Debra L Rothenberg/Shutterstock

City health department and Test and Trace corps officials said they don't include at-home test results because it could muddy the city's official record keeping of COVID cases. They said at-home results couldn't be independently verified, and they believe that enough people are still heading to clinics and labs. (They’ve also recently launched a choose-your-own-adventure guide for people who are exposed and at-home testers.)

“Our current level of surveillance data accurately reflect transmission trends around the city,” Michael Lanza, a spokesperson for the city’s Department of Health and Mental Hygiene, said in an emailed statement. “The City is doing a high level of PCR and antigen testing, which allows us to gain an accurate understanding of the trajectory of the pandemic.”

Yet this winter, other jurisdictions such as Tompkins County in upstate New York potentially caught up to an extra 25% of cases by allowing their residents to report at-home test results. Translate these patterns to the five boroughs, and New York City could have spotted an additional 250,000 infections on top of the 1 million cases that have been recorded since it began handing out free home tests on December 16th. The same week, omicron became the dominant variant in the city.

We asked city officials to further explain their stance against tracking at-home results.

People won’t report when they’re negative — or would they?

The most common concern cited by city health officials has centered upon the reliability and quality of at-home testing.

Their statements expressed worries that people wouldn’t report the results because there is no requirement to do so. That could throw off the accounting of the city’s positivity rate — or what portion of tests come back with a positive result. But by definition, officials can only calculate this rate if they know how many people are taking tests overall and how many are negative, which is easy to do via clinics and labs.

But people in places recording at-home tests, such as Washington D.C., have actually been reporting their negative results, too, which health officials in the nation’s capital described as a welcome surprise.

“We want individuals to report that data to us, just so we have an understanding of who is testing positive and who is testing negative,” said Patrick Ashley, the senior deputy director for the health emergency preparedness and response administration at DC Health. The District has registered about 9,000 positives and 60,000 negatives since it began allowing people to self-report home results on October 15th.

DC Health launched this tracking program after it noticed a desire among the community to adopt rapid antigen testing. The swabs became the cornerstone to tracking cases in schools, where students and parents must upload documents attesting that they’re not fibbing about the result. But the general public can report their positive or negative results without sharing photos of the test, though they have to submit their name and contact information.

Because it is an honor system, DC Health keeps a separate spreadsheet with self-reported results, and these numbers don’t factor into its official case counting or positivity records. Ashley said he and his colleagues are fully aware of the data quality issues that come with this self-reporting.

“Sometimes, we'll see people that report multiple positives,” he said. “They tested positive, and a couple of days later, they may take the test again and still report that they're positive.”

But data quality isn’t the goal here. Ashley said self-reporting offers better visibility into how the coronavirus is afflicting the community. The District committed heavily to the process, buying more than 6 million rapid antigen tests in late December to hand out for free — or about eight kits for every resident.

By contrast, New York City has issued 8.1 million kits, of which 6.6. million went to schools. That’s fewer than one kit per resident.

“We've always really wanted individuals to know their status, and we haven't shied away from making that data available,” Ashley said.

Are outbreaks missed without at-home test results?

At a recent lecture for the National Academy of Medicine, Nuzzo from Johns Hopkins predicted omicron cases would decline quickly in the U.S. — but not as fast as what has been seen in other countries.

That’s because multiple studies have shown that the coronavirus tends to spread in social clusters, rather than steadily moving through a population. So as businesses reopen and states like New Jersey drop mask mandates in schools, the virus may find new opportunities to jump between pockets of people — extending the duration of the current wave.

Thanks to vaccination and the widespread abundance of immunity, most people will be spared severe consequences. But a slow burn of cases, hospitalizations and deaths could simmer and thrive in at-risk communities, causing tens of thousands of deaths.

Read More: Why Coney Island and Brighton Beach were hit so hard by omicron

While Nuzzo supports at-home tests, she said now is a precarious time to develop blindspots in official case reporting.

“We want to understand who [these new cases] are and if the cases are occurring in a different demographic group than what we would typically expect,” she said.

Such reasoning explains why Tompkins County opted to start recording at-home test results in late December.

“It's just something that we're doing to be able to have an idea of what's going on with home tests and our community,” said Frank Kruppa, Tompkins County public health director and mental health commissioner. “Most importantly, when someone reports a positive test result to us, they get information back on what they should do for their own health around isolation.”

The county has recorded 1,550 positive self-tests since December 24th, while conventional testing documented 5,767 cases. Stated differently, the at-home program detected up to an extra 26% of cases.

New York City officials said it had kept track of some home test results, namely 30,000 positives from the 6.6 million kits that it had handed out to public school students and school staff since January 3rd.

The city admitted that these 30,000 cases had not been included in its COVID counts for schools nor in how it calculates in-school positivity. In fact, officials only know about the at-home outcomes because a positive or negative result dictates when a child or staff can return in person. And similar to Tompkins County, these at-home positives could have added an extra 21% of cases to the tally for New York City schools, which has accumulated 142,000 infections since classes resumed on January 3rd.

But if the city is open to registering some results, why not open it up to the general public?

Omicron changed the value of regular testing

Or maybe it doesn’t matter? The decline in testing numbers could be a result of “COVID fatigue,” or the general sense of Americans “learning to live with the virus.”

Though laboratory numbers are falling, New York City’s clinics are still averaging about 71,000 PCR tests and 15,000 antigen tests per day — bringing viral surveillance back to its pre-omicron levels. The city said this level of testing is still adequate and that it is also considering other metrics.

“In addition to cases, we also focus on other measures beyond case rates, like hospitalizations, deaths, and other measures of disease severity, in order to gauge the overall situation,” said Lanza from the city health department.

Yet if the official testing continues to slide, keeping track of the coronavirus could end up mimicking how officials track flu every year.

“We don't do this much testing for flu,” Nuzzo said. “We don't even count nationally adult flu cases.”

Instead, the Centers for Disease Control and Prevention focuses on flu hospitalizations from a fairly small selection of places — more than 70 counties across 14 states places — and then extrapolate a quick estimate of how many flu cases and deaths likely happened. The U.S. and other countries have been using this process to faithfully measure flu burden for a few decades.

As we're entering this new phase in the pandemic, we are going to have to possibly rethink how we conduct surveillance.

Jennifer Nuzzo, Johns Hopkins Center for Health Security

Repeating this habit for COVID-19 would essentially mean the country has collectively given up on trying to monitor the disease in real-time, which would give the coronavirus more reign to spread.

“Partly that is because we don't try to interrupt individual chains of transmission of flu like we are trying to do for COVID,” Nuzzo said. That’s less of an issue if a high abundance of people keep taking boosters or updated versions of the vaccine to thwart future variants.

The American experience with seasonal influenza teaches us that only about 50% of adults and about 60% of children take their flu vaccines annually. As a partial consequence, flu has averaged about 35,000 fatalities per year since 2010.

COVID-19’s fatality rate is still about five to 10 times higher than the flu’s. An endemic of this coronavirus will not be harmless.

“As we're entering this new phase in the pandemic, we are going to have to possibly rethink how we conduct surveillance,” Nuzzo said.