One of the most talked about issues around Covid 19 is testing. Everyone seems to want a test. However, little is said about the details of the tests, such as kinds of tests, good/bad tests and what do you do with the results. The answers to these questions will help answer the questions about how important the tests are to reopening of businesses of keeping a business open.
A reliable, fast diagnostic needs to be established and available to manage the reopening of businesses. The ability to identify infections before they become clusters, or clusters before they become outbreaks will be significant in how successful we will be in reopening businesses.
The tests. There are two kinds of tests being talked about in the news and the reporting does not always make it clear which test they are talking about. In most cases, they are talking about a DIAGNOSTIC test. A diagnostic test will determine whether or not the Covid 19 virus is present at the time of the test. The other kind of test is generally termed a serology (blood) test. The serology tests determine the presence of antibodies that are created if the patient had been been previously infected with Covid 19.
Diagnostic Testing. Most diagnostic tests use a method abbreviated RT-PCR (Reverse Transcription-Polymerase Chain Reaction) that will detect the presence of the DNA from the virus. There are several manufacturers of these tests around and THEY ARE NOT THE SAME in reliability. The difference in testing methods and reliability should be remembered when analyzing data from different sources.
In most cases, the testing being discussed are the results of diagnostic testing. That is, diagnostic methods are used when the daily number of confirmed cases is provided. For instance, diagnostic tests were used to report the 1,391,316 confirmed cases of Covid 19 in the US (5/12/20).
A diagnostic test will determine if a virus is present at the time of testing. It can not tell if the person was infected and then recovered nor is it predictive of if the patient will get the disease tomorrow or any time in the future. If you are showing severe symptoms, you should go get medical attention whether or not you have been tested. If you are feeling ill in any way, you should stay-at-home regardless of whether or not you are tested. If you test negative, there are also uncertainties. You may have been infected just before the test and the virus has not spread enough to be detected, or you could get infected tomorrow. If you have no symptoms but are positive in a diagnostic test, you are among the 80% of infected people who have no or minor symptoms.
Serology – Antibody Testing
Serology tests are blood-based tests that can be used to identify whether people have been exposed to Covid 19 by looking for specific antibodies. The mechanisms for antibodies was topic of an earlier blog regarding vaccines. Covid 19 Vaccine. Where are we in the process? How will they work? What’s taking so long? The presence of antibodies would indicate that the patient had been infected with Covid 19. As mentioned above, because the vast majority of people who are infected have no or minor symptoms, antibody testing is the only way to determine how many people are actually infected. Earlier blogs have indicated that the actual number of people infected as determined from antibody testing is up to 20x higher than the number of infections detected with diagnostic testing.New York: Nearly 3 million infections – not 276,000
It is important to know:
- It is not known if the presence of antibodies makes the patient immune from further infection.
- If the patient does become immune to Covid 19, it is not known how long the immunity will last.
- The immediate benefit from antibody testing will be to determine how many people were infected.
Specificity and Sensitivity. These are KEY factors in the reliability of tests but are seldom discussed in the news. Specificity is a measure of how reliable the test is. In other words, if the test indicates you have the disease, do you really have the disease? Or is the test somewhat unreliable because it can provide a positive result when you don’t have the disease. This is termed a false positive result. Specificity is a measure of how many false positives a test will give. A test with a specificity of 80% means that only 80% of those who tested positive actually are positive. 20% show an incorrect positive result.
Sensitivity is a measure of how reliable the test is, if the test says that you are negative. It is a measure of the false negatives. A test with a sensitivity of 90% means that 90% of the people who test negative, truly do not have the disease but 10% of the negatives are really positives and have the disease. Some of the reports from South Korea suggests their test had a sensitivity of 80-85%.
Each test should be evaluated for both it’s specificity and sensitivity. It is possible for a test to have a high specificity but a low sensitivity.
These seem like details, but a 5% false negative means that if 1,000,000 people test negative, there are really 50,000 people who are actually infected. Labs, doctors and patients should be very aware of the specificity and sensitivity of the tests they are administering. There are over 20 different tests being conducted around the world, each with it’s own, sometime unknown specificity and sensitivity. This makes comparative data very difficult.
As the choice of what test to run is determined locally (state, county, hospital), differences in specificity and sensitivity between tests likely exist.
There is an old joke in testing labs about what level of service a client can request. For each test, you can choose how fast you get the results back, how accurate the results are and the cost of the test. Unfortunately, you can only choose two of the three choices. This is true for Covid 19 testing as well.
Test Approval Process.
It would normally take more than a year or more to get a diagnostic test to get FDA approval. This is because of the number of patients and time in clinical trials that are required to demonstrate sensitivity and specificity. Under the FDA Emergency Use Authorization guidelines, manufacturers only need to test 30 laboratory samples and demonstrate 95% specificity (positives) and 100% sensitivity (negatives). That means that the test must detect 95% of the samples that contain the virus. A 100% sensitivity means that there can be no false negatives. I am unaware of any test whose accuracy has been evaluated by an independent organization (eg not the company who manufacturers the test).
For diagnostic tests, it is important to know that the virus used in these laboratory tests are not from patients but from lab sources. It is also not clear that all labs are using the same concentration of virus in all of their tests. Last, the tests are conducted under laboratory conditions without issues of sample collection and other factors present when testing patients. There is no requirement that any clinical data from patients be part of the Emergency Use Authorization approval process. This does not mean that the tests are unreliable, it just means the tests have not been as fully evaluated as they would have in an non-emergency environment and we don’t know the number of false positives and negatives we are getting (other than the data supplied by the manufacturer).
There are also factors not related to the actual diagnostic test that can effect the test results.
- When the patient was infected. If the patient was infected very recently, there may not be enough of the virus to detect.
- Where the virus is. Most tests are from nasal swabs. However, as the disease progresses, the virus moves into the lungs, so the viral load in the nasal passages goes down.
- Incorrect sample collection. If the nasal swab is too superficial (not deep enough) then the virus may not be detected.
- Rapid testing after collection. The tests usually must be run within 8 hours, unless it is refrigerated in which case it much be tested within 72 hours. Testing outside the windon decreases the chance of detection.
More attention must be paid to determine the specificity and sensitivity of all tests. Too many false negatives will release infected people into the population and too many false positives can overwork or overload the health care system. The situation is made more difficult with the increasing number of new tests being introduced in the US and around the world.
The same specificity and sensitivity issues apply to the Serology-antibody tests. Again, there are over 20 different tests of this type being run around the world. Like the diagnostic tests, they are being approved for use under emergency use authorization so specificity and sensitivity values are being supplied the manufacturer who makes the tests under laboratory conditions. This does not make the tests unreliability and I am not criticizing the manufacturers, but independent evaluation under normal clinical evaluation guidelines should be done.
It is a difficult time as development and distribution speed is now prioritized over normal safety and effectiveness procedures. This undoubtedly has allowed the use of some tests that would not have been approved under more normal conditions.
There are two examples of what can happen if specificity and sensitivity are not well established. In May, the United Kingdom reported that it had purchased 2 million antibody test kits from China for $20 million (₤16 million). However, the purchase was apparently made before independent UK analysis was done. When the kits were received, independent tests showed both too many false positive and too many false negatives. The 2 million kits are useless. https://www.bloomberg.com/news/articles/2020-04-07/new-test-hopes-dashed-as-u-k-finds-antibody-kits-don-t-deliver
The city of Laredo, Texas also bought 20,000 antibody test kits from China for $500,000. These kits were not approved by the FDA in any way. The city took a risk to secure antibody testing. Once received, the tests, like the ones purchase in the UK, turned out to be unreliable and useless. The kits were, nonetheless, seized by the FDA.
It is not my intention to criticize the incredible speed at which these highly complex tests are being developed. I have not discussed the actual technology, but it is quite incredible. We usually just see a box or a device without knowing the complex test that is going on inside. The balance of speed of development and reliability is a point the world is trying to find. The purpose of this blog is to discuss the factors involved in testing that have not been often discussed and to provide some background to the daily news regarding testing.
