This is a continuation of my ‘A Beginner’s Guide to False Positives’ from two days ago. In that post I described how false positives can swamp true positives when the prevalence of the disease or condition in question is low. This was the situation in Britain with Covid for quite a few months. However, more recently the percentage of tests that are positive has increased, so I thought I should take a look at the situation in regards to the latest figures.
(I should point out that, as we have recently seen, Public Health England, which handles this data, has proven itself to be less than competent, so we should take this data with a large handful of salt.)
Here’s a quick and dirty graph (click to enlarge) that I did tracking the recent changes in percentage of tests that are positive (data from https://coronavirus.data.gov.uk/). There’s various issues with this data (eg. the inclusion of antibody tests, although the numbers for that are small) so, even forgetting our more general suspicions about any data that comes from PHE, it’s not going to be very accurate, but it will illustrate the general picture.
Take 30 September, for example, the day the percentage was highest (5.63% positive). On that day we had 184,624 tests done, of which 10,387 were positive. If the false positive rate is about 0.9%, as Matt Hancock intimated – in fact, let’s say 0.8%, as some people have claimed – then we should expect 1477 false positives. As we had 10,387 positives, then we must have had 8909 true positives. It appears, then, that we are back to a situation where the true positives swamp the false positives, rather than the opposite situation that we had over summer, in which case the relevance of false positives is diminished. This is also indicated by the fact that the percentage of positive tests on that day, 5.63% is far higher than the false positive rate (0.9%).
However, things aren’t quite that simple. Over summer, the percentage of tests that were positive apparently went down to 0.4%. Logic tells us that that’s the false positive ceiling. It can’t be any higher than that. Or rather, it can’t be any higher at that time. It could be higher at other times, though, unless the false positive rate cannot change. Can it change? Is it fixed? It’s fixed if all the things it depends upon never change, or never deviate very much from an approximate level. But we know that one of the ways a false positive can be produced is if it gets a very similar bit of RNA. So it’s possible that false positives could increase over autumn and winter as other viruses circulate. So it’s possible that some of the increase we are seeing in the rate of positive tests is due to the rate of false positives increasing.
Possible, but how likely? Looking into this, the scientists who work in this area are very insistent that the PCR tests are tailored very specifically for SARS-CoV-2, and they make sure that they can’t be triggered by other coronaviruses, or flu, or any other virus, except in fluke circumstances. I’m going to accept that, at least provisionally.
I don’t mean that I’m entirely convinced. Years ago I used to be friends with some really, really top molecular biologists and they used to be pretty scathing about all the things that go wrong in their field, especially when you’ve got clueless Research Assistants doing the grunt work. How things are supposed to work in theory is one thing, how they work in practice is another thing altogether. You’ve also got SAGE papers that won’t commit to any more than 5% as a false positive ceiling, and you’ve got this recent Lancet paper saying “The current rate of operational false-positive swab tests in the UK is unknown; preliminary estimates show it could be somewhere between 0·8% and 4·0%.”
So maybe there are more false positives than the government lets on, and maybe not; that’s not something that can be decided on just by finding a few internet links. (Feel free to knock yourself out talking about what you’ve found in comments, although I’m not going to take seriously any Off-Guardian-type nonsense about Koch’s Postulates.)
On the other hand, we’ve got this graph from Christopher Bowyer, which indicates that the rate of hospital admissions per 1000 positive test has not gone down since the summer (after decreasing markedly over summer), If a lot, or all, of the recent rise in the positive test rate is due to colds causing false positives then this rate should have decreased since summer. But it hasn’t.
There’s also the issue of false negatives. The false negative rate is sometimes estimated to be higher than the false positive rate, which blunts the false positive-style critique.
Besides, the crux of my lockdown scepticism does not depend on claiming that Covid is now completely gone, which I doubt is true anyway. I think it’s very unlikely that the recent increase in positive tests is not being driven at least partly by an increase in SARS-CoV-2. So to some extent I think the focus on false positives is barking up the wrong tree. The case against lockdown applies even if (as seems likely) Covid is increasing again.
But there are some other serious issues with PCR tests that I want to look at (eg. the cycle numbers, dead fragments, and the large numbers of asymptomatic positives), which I’ll do in a further post.
13 thoughts on “More on false positives”
My concern on the false positives issue is not the reliability of the test itself but that the test is being used to justify actions that demand a higher level of justification than one test. No doctor would start you on chemotherapy based on a blood test.
That the number of tests is a justification for government action also raises the problem that any statistic on which the government can look either good or bad will be misrepresented by that government.
And actually, I believe the test will, in time, be shown to be far from reliable; not in and of itself but due to human errors in handling, processing etc. And for another reason; any scheme set up, managed and run by government or its agencies will be cocked up.
How about the positive test rate figures are almost totally bogus by a factor of 3 – retests are not included in the denominator but a positive result in a re test is factored in the top line.
Dr Clare Craig FRCPath
@ClareCraigPath
· Oct 5
If you are testing for fairies at the bottom of the garden, does it matter how many are false positives or whether none are?
**although I’m not going to take seriously any Off-Guardian-type nonsense about Koch’s Postulates.)**
Who cares what the medium is? Off-Guardian sources are no different from those of the German corporate fraud lawyer who spells out in forensic detail how and why the PCR technique, whose inventor insisted it was of no diagnostic worth: “With PCR you can find almost anything in anybody” – was fraudulently reconstituted as a “test” by Christian Drosten and WHO, He’s bringing criminal charges based on “PCR tests” being fraudulent and criminal. Dr Yeadon also calls the “tests” ‘criminal’. Surely if the “PCR test” itself is invalid, “false positives” is otiose? I quoted from Off-Guardian earlier on the other thread but the source is Dr Stoian Alexov, President of the Bulgarian Pathology: ‘No One is Dying from Corona Virus’:
https://off-guardian.org/2020/07/02/no-one-has-died-from-the-coronavirus-president-of-the-bulgarian-pathology-association/#comment-203305
“maybe there are more false positives than the government lets on, and maybe not”: thank God we have a government so competent that it knows the false positive rate.
Without getting into the whole Type I and Type II statistical errors discussion, we know that no test regime is perfect. There definitely are some False Positives. The peculiarity of the current situation is the lack of official interest in what that number might be.
We also know that the number of tests run per day has been increasing exponentially. Think about a factory where production demand has more than doubled. People getting burned out working overtime and weekends; second & third shifts added with under-trained workers; old equipment pressed into service; time for cleaning, maintenance, calibration being sorely pressed. It would be very unlikely for the False Positive rate to remain a constant under these circumstances — let alone that it would be as good as the rate achievable under ideal laboratory conditions.
Since we don’t have good data on False Positives, let’s look at what we do have. Based on the data Hector showed, testing 100,000 English people (ie about a half-day’s load currently) produces 3,000 – 5,000 positive test results. However, only about 150 – 250 of those people with positive tests are admitted to hospital. What about the other 2,850 – 4,750 people with positive tests?
Perhaps those people with positive tests who do not end up in hospital were victims of False Positives. Perhaps there simply was nothing wrong with them — no illness, despite have a positive test; let’s call them Healthy Positives, to distinguish them from the Genuinely Sick Positives. The tests that really matter are the Genuinely Sick Positives, and they appear to be only about 0.5% of Total Positives, based on hospital admissions. Whether the other 99.5% of positive tests are False Positives or Healthy Positives is … irrelevant.
Simple question, why hasn’t an EQA been done for the PCR test on this coronavirus? We know the range of EQAs for this PCR tests on other virus gives FPRs of 0.8-4%. Your analysis would be completely different if the EQA showed it at 4% not 0.8%. The only constraint is the ONS numbers over the summer. But the ONS tests are not Pillar 2 tests. The ONS FPR can be much lower than Pillar 2 because of process.
Having said all this its the hospitalisation and death numbers that are somewhat more significant. ‘Somewhat’ because both of them can be easily manipulated as well. For instance hospitalisation numbers can include one night stays from ER patients, and deaths can include muslims who have to be buried within 24 hrs.
In other words the government and public health authorities can produce whatever numbers they want in support of their aims.
Antibody tests as meaningless as RT-PCR tests – they are a blunt instrument and not present if T-cells defeated virus quickly
Also
See also:
https://www.google.com/search?q=anamnestic+response
Hancock et al ignore this
Not disputing real cases rising, that was always inevitable when lockdown relaxed – lockdown again is simply kicking can down road and causing more non-C19 deaths. The low prevalence and low deaths suggests a lot of FPs distorting actual numbers
Hancock says “Err, don’t know,… <1% err, em, …. yes, yes it's 0.8%"
Rabb says "It's 7%"
So we have 0.8% to 4% (Gov), 1% to 4% (WHO) and 0.8% to 7% (Gov)
No matter what Hancock says (he's repeatedly shown ignorance of facts) I would say 2% to 3% FP most plausible
More on PCR Test
@Hector
PCR test is an indicator, not a diagnosis or proof. You need to address this, not only FPs
Thumb distal hyperextensibility is an indicator of whole body hyperextensibility/hypermobility. I have Thumb (& Big toe) distal hyperextensibility, inherited from father, but no other hyperextensibility or hypermobility
@James
Yes. Using PCR test as a diagnosis / proof is medical negligence, yet it is used as prima facie to fine or imprison people
@Gavin
Exactly
@JimW
Spot on
Belgium stopped using “case” numbers for policy and switched to Hospital admissions and deaths being the important number. Sensible
Just to pick on the gold standard thing, which is important: most PCR tests are validated against other PCR tests, a sort of echo chamber validation – see here:
https://dr-no.co.uk/2020/09/10/the-pillars-of-testing/
The problem is if you use a bent ruler to validate another bent ruler, you may believe both are OK when they are not.
The other thing is PCR tests numbers will be dwarfed by rapid antigen tests if/when Moonshite takes off, and they are even worse (lower sensitivity and specificity) than bent PCR rulers.
Comments are closed.