[FRIAM] Could this possibly be true?

[Roger Critchlow]

Pieter -

The initial safety and efficacy report was published in the New England
Journal of Medicine at the end of 2020,
https://www.nejm.org/doi/full/10.1056/nejmoa2034577, it has smoother
language and inline graphics.  It also has fewer deaths in the treatment
group than in the control group, but it is only reporting the first two
months of the study.

The numbers of deaths reported in the "Adverse Reactions" section of these
reports will eventually track the expected death rate of the population in
the trial, and apparently they do, since there is no comment to indicate
otherwise.   Every clinical trial that tests the safety of a treatment is
expected to agree with the baseline mortality statistics for the population
in the trial.

If you see 14 and 15 deaths out of 22000 participants and your immediate
response is that 15 is bigger than 14, then you should probably stop
torturing yourself with statistical data.  You're making and agonizing over
distinctions that the data can never support.  The number of deaths in a
population over a period of time has an average value and a variance which
are found by looking at large populations over long periods of time.  In
any particular population and period of time there are a lot trajectories
that the death count can take that will be consistent with the long term
average even as they wander above and below the average.

I append a simple simulation in julia that you can think about.

-- rec --

# from https://www.cdc.gov/nchs/fastats/deaths.htm
death_rate = 869.7              # raw deaths per 100000 per year

# simulate the action of a 'death rate' on a population of 'sample'
individuals for 'days' of time.
# convert the raw death rate to the death_rate_per_individual_per_day, ie
death_rate/100000/365.25,
# allocate an array of size sample*days, size coerced to an integer value,
# fill the array with uniform random numbers.
# if an array value is less than the death rate per person per day, score 1
death.
# this overcounts because individuals can be scored as dying more than
once, YODO!

simulate(death_rate, sample, days) =
    sum(rand(Int(sample*days)) .< death_rate/100000/365.25)

# accumulate an ensemble of death rate simulation results.
# run 'trials' simulations of 'death_rate' for 'sample' individuals for
'days' time.
# accumulate an array with the number of deaths in each simulation
accumulate(death_rate, sample, days, trials) =
    [simulate(death_rate, sample, days) for i in 1:trials]

# check the model: run the simulation with death_rate for 100000
individuals and 365.25 days,
# the result averaged over multiple simulations should tend to the original
death_rate.
# we report the mean and standard error of the accumulated death counts
julia> mean_and_std(accumulate(death_rate, 100000, 365.25, 50))
(868.34, 31.64188002361066)

# That's in the ball park
# Now what are the expected deaths per 22000 over 180 days
julia> mean_and_std(accumulate(death_rate, 22000, 180, 50))
(94.3, 10.272312697891614)

# that's nowhere close to the 14 and 15 found in the report.
# Probably the trial population was chosen to be young and healthy,
# so they have a lower death rate than the general population.
# let's use 14.5 deaths per 22000 per 180 days as an estimated trial
population death rate
# but convert the value to per_100000_per_year.
julia> est_death_rate = 14.5/22000*100000/180*365.25
133.74053030303028

# check the model:
julia> mean_and_std(accumulate(est_death_rate, 22000, 180, 50))
(14.96, 3.6419326558007294)

# in the ball park again.

# So the point of this simulation isn't the exact result, it's the pairs of
results that this process can generate
# let's stack up two sets of simulations, call the top one 'treatment' and
the bottom one 'control'
# treatment and control are being generated by the exact same model,
# but their mutual relation is bouncing all over the place.
# That treatment>control or vice versa is just luck of the draw

julia> [accumulate(est_death_rate, 22000, 180, 20),
accumulate(est_death_rate, 22000, 180, 20) ]
2-element Vector{Vector{Int64}}:
 [12, 12, 13, 11, 22, 13, 14, 16, 13, 14, 21, 17, 13, 14, 19, 11, 20, 11,
9, 19]
 [11, 14, 15, 17, 11, 19, 17, 12, 16, 14, 18, 16, 11, 16, 12, 16, 10, 14,
17, 13]


On Wed, Sep 15, 2021 at 2:25 AM Pieter Steenekamp <
pieters at randcontrols.co.za> wrote:

> In the Phizer report "Six Month Safety and Efficacy of the BNT162b2 mRNA
> COVID-19 Vaccine" (
> https://www.medrxiv.org/content/10.1101/2021.07.28.21261159v1.full.pdf) ,
> I picked up the following:
>
> "During the blinded, controlled period, 15 BNT162b2 and 14 placebo
> recipients died"
>
> Does this mean the Phizer vaccine did not result in fewer total deaths in
> the vaccinated group compared to the placebo unvaccinated group?
>
> I sort of can't believe this, I obviously miss something.
>
> But of course, there are clear benefits in that the reported vaccine
> efficacy was 91.3%
> - .... . -..-. . -. -.. -..-. .. ... -..-. .... . .-. .
> FRIAM Applied Complexity Group listserv
> Zoom Fridays 9:30a-12p Mtn GMT-6  bit.ly/virtualfriam
> un/subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
> FRIAM-COMIC http://friam-comic.blogspot.com/
> archives: http://friam.471366.n2.nabble.com/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20210915/74b13ea3/attachment-0001.html>