Canada Markets closed
  • S&P/TSX

    21,357.56
    +64.56 (+0.30%)
     
  • S&P 500

    4,662.85
    +3.82 (+0.08%)
     
  • DOW

    35,911.81
    -201.79 (-0.56%)
     
  • CAD/USD

    0.7968
    -0.0026 (-0.3211%)
     
  • CRUDE OIL

    84.50
    +0.68 (+0.81%)
     
  • BTC-CAD

    54,217.83
    -402.71 (-0.74%)
     
  • CMC Crypto 200

    1,037.76
    +12.03 (+1.17%)
     
  • GOLD FUTURES

    1,817.30
    +0.80 (+0.04%)
     
  • RUSSELL 2000

    2,162.46
    +3.02 (+0.14%)
     
  • 10-Yr Bond

    1.7720
    +0.0610 (+3.57%)
     
  • NASDAQ

    14,893.75
    +86.95 (+0.59%)
     
  • VOLATILITY

    19.19
    -1.12 (-5.51%)
     
  • FTSE

    7,542.95
    -20.90 (-0.28%)
     
  • NIKKEI 225

    28,124.28
    -364.82 (-1.28%)
     
  • CAD/EUR

    0.6980
    +0.0008 (+0.11%)
     

With Covid studies, the quality of the evidence matters

·2 min read
<span>Photograph: Bloomberg/Getty</span>
Photograph: Bloomberg/Getty

Putting many low-quality studies together cannot provide reliable answers about masks and ivermectin

In The Adventure of the Copper Beeches, Sherlock Holmes says: “Data! Data! Data! I can’t make bricks without clay.” Recent claims of massive benefits from wearing masks and using ivermectin against Covid-19 depended on mainly low-quality clay.

Meta-analysis is a technique for pooling the results from many studies, but it cannot make silk purses out of sows’ ears. A recent British Medical Journal review looked at six, fairly porcine, studies concerning mask-wearing and estimated an impressive 53% reduction in risk. But the single randomised controlled trial estimated the smallest effect: a reduction of about 18% (-23% to 46%) in Sars-CoV-2 infections. The “heaviest” studies, an analysis of US states and a survey of about 8,000 Chinese adults in early 2020, observed rather than experimented and its editorial highlights the risks of confounding variables influencing both wearing masks and infections and the impossibility of disentangling the effects of measures fluctuating simultaneously. Indeed, this review found an identical 53% reduction from handwashing.

After removing studies with a high risk of bias, the estimated effect dissipated

Physical models and laboratory testing suggest masks would be expected to have some effect, and to be fair, it is challenging to imagine rigorous randomised trials of the effects of mask-wearing or any other behaviour. But these are considered both feasible and essential for evaluating medical treatments and so in principle it should be easier to evaluate ivermectin, a cheap, anti-parasitic drug, heavily promoted as an overlooked treatment for Covid-19. One meta-analysis originally found ivermectin reduced Covid-19 mortality by about 56%, a huge effect. But then a preprint server withdrew the largest study and there have been further concerns over major errors and fraud in some ivermectin analyses. After removing studies with a high risk of bias, the estimated effect dissipated: the estimated decreased mortality was about 10% but with a wide uncertainty interval (- 42% to 43%), so no firm conclusions can be drawn.

Huge effort has been expended on scrutinising ivermectin research, even leading to the intriguing observation the drug appears to work in countries with a high prevalence of roundworm, whereas what is needed are the kind of good-quality studies that finally laid the claims about hydroxychloroquine to rest. Fortunately, these are now under way for ivermectin.

• David Spiegelhalter is chair of the Winton Centre for Risk and Evidence Communication at Cambridge. Anthony Masters is statistical ambassador for the Royal Statistical Society

Our goal is to create a safe and engaging place for users to connect over interests and passions. In order to improve our community experience, we are temporarily suspending article commenting