Report-9 Review plus Conclusions & Recommendations on Scientific Advice

Introduction


In the UK during February and March 2020 the government was supplied with scientific advice generated by a group called the Scientific Advisory Group for Emergencies (SAGE) with the government’s Chief Scientific Officer (CSO) plus Chief Medical Office (CMO) being the interface between the science and politicians.

We know that events overtook both the scientists and government resulting in the government imposing a nationwide lockdown on the 23rd March.  It was reported that a significant element in the government’s decision to lockdown was a report by Imperial College, which predicted hundreds of thousands of fatalities plus overwhelming the NHS’ capacity.  This report was published on the 16th March 2020, entitled: “Report 9: Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand” (Report-9).  The lead author was Professor Neil Ferguson, a member of SAGE but based at Imperial College in London.  The work was (part?) funded from the UK Medical Research Council under a concordat with the UK Department for International Development, plus the NIHR Health Protection Research Unit in Modelling Methodology and Community Jameel.

Imperial College made Report-9 available via their website1 together with a brief summary of the work.  There are a number of other reports available from the Imperial website at the time of writing.

At first glance, Report-9 looks like a relatively innocuous, small piece of work.  However, in the context of a new highly infectious virus spreading rapidly across the world, looking at what kinds of non-medical strategies could be used to reduce both fatalities and NHS demand, isn’t trivial at all.  Particularly as this work appears to have been the only study of its kind in the UK, at that time.

This article is predominantly about the contents of Report-9.  In early May 2020, my review of Report-9 was supplied in hardcopy to Professor Ferguson, Dr Sabine van Elsland and the President of Imperial College, Professor Alice Gast.  No acknowledgement or response was received back.  The cover letter accompanying the detailed review can found in Appendix-1 and the detailed review itself, in Appendix-2.

This article also draws conclusions about the way that science is undertaken and fed into the government.  Recommendations are made to ensure the science, the government’s own scientific advisors plus politicians themselves provide respectively; robust quality assured results, clear substantiated conclusions and recommendations which policymakers can understand, challenge and be confident in their use.

Brief Summary of Report-9

Report-9 is a semi-technical report which looks at how the COVID-19 infection could spread through the UK based on data obtained from China.  It presents predicted fatalities and hospitalisations over time, firstly with no interventions and secondly, with a selection of interventions or combinations thereof.  It also looked at the impact of switching on and off interventions (triggering) on the resulting deaths plus demand on medical services.

The range of interventions modelled were:

1.    Persons with symptoms isolate at home.
2.    Persons with symptoms quarantine together with their immediate family at home.
3.    Social distancing of all people over 70 years old.
4.    Social distancing of everyone.
5.    School and University closure.

In terms of results the graph below shows, for the Do Nothing Case, the predicted fatalities per day per 100,000 people during April through August 2020, highlighting the estimated total number of fatalities in the UK at 510,000. With the peak fatality rate being at the end of May (Figure-1).

 

Figure-1 Graph of Modelled Do Nothing Case Fatalities per Day per 100,000 People Over Time

Let’s imagine you are a politician and you are told that the most recent modelling predicts around half a million deaths over the next five months or so.

Figure-2 shows the estimates of fatalities as a result of a range of interventions.

 

Figure-2 Graph of Modelled Do Nothing Case plus Other Interventions’ Critical Care Bed Requirements per 100,000 People over Time

Figure-2 shows that interventions could reduce the requirement for beds but none of the individual interventions bring the bed requirement anywhere near the bed capacity at the time of this work.

Figure-3 shows the modelled bed occupation profiles for combined interventions assuming these are in force for five months.

 

Figure-3 Graph of Modelled Do Nothing Case plus Combinations of Interventions’ Critical Care Bed Requirements per 100,000 People over Time

Figure-3 shows that selected combinations of interventions can reduce the bed requirements below the then current capacity but, as soon as the interventions are removed, the model predicts a sharp peak within a month.

Back to our politician, who now considers the Do Nothing Case is probably valid, but not a viable strategy plus, that none of the individually modelled interventions are predicted to keep patient numbers within the current NHS capacity.  But, based on Figure-3, a combination of interventions (“restrictions”) could avoid overwhelming current NHS capacity.  This largely explains the end-result: Total lockdown and rapid expansion of NHS capacity.

Unfortunately Report-9 failed to identify an intervention which could have significantly reduced the need for hospitalisation and therefore the fatality rate.

Simple logic, together with the actual data, shows that older people once infected are more likely to need hospital care and are at higher risk of dying.  Therefore if you ensure the high risk people are properly protected from infection then the requirement for additional capacity is much reduced, as well as the fatality rate.

Simple.

No modelling was required to reach this conclusion.

The report goes on to look at the potential impacts of switching on or off the various interventions, concluding that this triggering did not have a significant impact on fatalities or the demand on NHS capacity.

The report appears somewhat confused about the difference between Mitigation and Suppression strategies but errs towards Suppression with caveats about the length of time required, plus likelihood of infections picking up after the relaxation of restrictions.  The authors illustrate a model of Adaptive Suppression whereby relaxations are introduced and then reapplied if the infection rate / hospital capacity criteria become threatened.

On the 23rd of March the government implemented a full Suppression strategy, which then evolved into a form of Adaptive Suppression.

Interestingly, but unfortunately too late to gain sufficient traction, in early May 2020, van Bunnik et al published a paper entitled “Segmentation and Shielding of the Most Vulnerable Members of the Population as Elements of an Exit Strategy from COVID-19 Lockdown”2.  As the title indicates, this work looked at isolating the most vulnerable from infection plus ensuring medical and care staff didn’t introduce infection, to the vulnerable.  This approach was pitched as a mechanism for exiting Lockdown; however, it should have been the most effective strategy from the very beginning.

Conclusions

1.    Report-9, although probably considered to be a relatively small piece of work by its authors; was, in conjunction with evidence of exponential growth in infections, almost certainly the catalyst for the UK government’s decision to implement a full lockdown in late March 2020.

2.    However, the actual conclusions and recommendations that SAGE did provide to the government at this particular time are, unfortunately not known.

3.    The scope of the work was, in fact, too narrow in that the most logical and significant strategy of preventing the vulnerable from getting infected, was not evaluated.  In other words, the intervention information from this report is not actually robust enough to be a foundation for government policymaking.

4.    At subsequent government committee reviews, when asked about how reports such as this were quality assured, Professor Ferguson indicated this was undertaken by other members of SAGE.

5.    Having reviewed the published report it is clear that it hasn’t had a proper review; however, you can judge for yourself by looking at my detailed review below.

6.    It would appear that both peer assist and peer review standards are lacking, which is a significant failing in an organisation contracted to provide scientific advice to the government.

7.    This raises the question as to whether this report and by extension SAGE, were fit for purpose?

8.    At a subsequent government committee review the CSO reiterated that the role of SAGE was solely to provide scientific advice to the government.  This doesn’t include economic assessments of scientific advice.  He reiterated that the government then develops policies based on said scientific advice.

9.    This approach clearly failed on a number of levels, possibly originating from the results of Report-9, which were likely provided to the government in early March.

10.    It appears reasonable to conclude that in February and March 2020 the scientific assessment of the progression of COVID-19 was inaccurate leading to delayed and drastic policy decisions ultimately feeding into unnecessarily high fatality rates, plus significant economic damage.

Recommendations

1.    If the existing scientific advice structure is maintained, all work undertaken by scientific advisors to the government must, before being allowed go forward to policymakers, be:

a.    Independently peer reviewed then,
b.    Made available publically in order to solicit wider feedback.
c.    Integrated with economic and commercial information.
d.    Formal quality assurance should be a contractual requirement between the scientific advisors and the government.
e.    Must make clear whether results are deterministic or stochastic, plus include an uncertainty analysis.

2.    In the event that time is insufficient to implement a proper independent review, the CSO or CMO should:

a.    Take full responsibility for the adoption of any scientific advice that has not been properly quality assured and
b.    Ensure policymakers are aware of the lack of quality assurance in said advice.
c.    Make the material publically available at the earliest opportunity for wider feedback.

3.    The CSO or CMO must formally summarise, at minimum, in the form of conclusions and recommendations, plus publish as quickly as possible, the scientific advice provided to the government and,

a.    Ideally combine the scientific advice with economic or commercial information to illustrate the validity, or otherwise, of the advice.
b.    Failing this, ensure the government understands that economic evaluations have not been undertaken and that these must be conducted in order to identify the most effective policy option(s).

4.    Government departments receiving scientific advice should:

a.    Ensure proper quality assurance has been implemented and if unclear, clarify with the CSO or CMO.
b.    Understand whether the scientific advice being provided is deterministic or stochastic and, if the former, what the probability of the deterministic scenario represents.

5.    Accepting the premise that the system for scientific advice was not fit for purpose, replace it with an alternative one which provides consistently high quality, timely, integrated economic validated scientifically based conclusions and recommendations.

6.    Review other countries’ systems such as Germany, Korea, Singapore and Sweden.

References

As of the 26th September 2020, with increasing regional restrictions being switched on and off in the UK, John Bishop sums up the government's attempts to "control the virus" in this brilliant short video.



Appendix-1

 

May 11th, 2020

 

Dear Professor Ferguson,

Report-9 Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand. 16 March 2020

Having read Report-9 and emailed you on the 17 April seeking some clarifications regarding the Interventions modelled, I have to express a number of concerns over this Report.

1.    Most significantly, the range of Interventions modelled is incomplete, which makes comparison of the results, also incomplete.  Hence the actual value of the results is impacted, potentially to being misleading, particularly for policy-makers.

The data available at the time of your work clearly shows that people over the age of 60 are at increasing risk of death and it doesn’t take a great step to identify the most-likely co-morbidities that COVID-19 attacks, irrespective of age.

Report-9 failed to identify the most significant Intervention: Identify & Protect the Vulnerable. Somewhat belatedly, van Bunnik et al(1) now appear to have addressed some of this. If this key Intervention had been fully incorporated in your work in February/March, the country would almost certainly be in a much better situation, than it currently is.

2.    Aside from the hugely significant issue above, I am concerned about the quality of Report-9, particularly as it was an input to crucial Government decision-making.

Was the work and Report peer reviewed and signed-off before being issued; if so by whom and when?

I have seen technical writing skills deteriorate since my Masters at the Royal School of Mines in the late Eighties. Attached is a print-out of my review of Report-9, which highlights poor writing but more fundamentally, poorly thought out technical work.

    2.1    Irrespective of who the end-user of a technical report is, but absolutely crucially if the report is for the Government; it must address the following:

    2.1.1    State whether the report has been (preferably independently) reviewed and include who did this; or make crystal clear the report has not been reviewed.

    2.2    Start with prioritised Conclusions in simple language, so that even the most uneducated reader can understand these. Based on the contents of Report-9, I would have written the following prioritised conclusions, which policy-makers would readily understand and almost certainly use in the Government’s forward programme, plus in messaging:

    2.2.1    Based on the input data and our modelling, this indicates the vast majority of the population, 64.5 million people, out of a total population of 65 million people, should survive infection with COVID-19.

    2.2.2    In terms of deaths, the data indicates the risk of fatality increases as follows:

    a. Age 50 to 59: Infection Fatality Rate (IFR)=0.6%
    b. Age 60 to 69: IFR = 2.2%
    c. Age 70 to 79: IFR = 5.1%
    d. Age 80+: IFR = 9.3%
    e. IFR means the percentage of infected people estimated to die. So, 9.3% of infected people over 80 are predicted to die.
    f. The IFR for people younger than 50 ranges from 0.15% down to 0.002% for under 9’s, indicating the virus is not a threat to healthy people below 50.
    g. For the entire UK population the overall IFR is currently estimated to be 0.9%.
    h. If you assumed the entire UK population of 65 million people became infected (which is not a realistic assumption), then the IFR translates into 585,000 fatalities.
    i. Include the range of IFR’s estimated in mild to severe flu seasons and make a comparison to the IFR for COVID-19.

3.    The data and model indicate that people over 50 and particularly anyone with medical susceptibilities must be protected from infection until a vaccine becomes available. Protecting the vulnerable is likely to be the most significant and effective strategy.

4.    Healthy people under 50 are predicted to cope adequately with infection and can maintain a near normal life whilst adopting basic infection control procedures, such as enhanced hygiene.

5.    Then summarise the results of the Intervention scenarios.

6.    Follow with prioritised Recommendations, for example:

    6.1    Modelling indicates 99.1% of the population should cope adequately with the infection, so Interventions need to be urgently focussed on protecting the 0.9% deemed to be vulnerable.

    6.2    Summarise the recommended Intervention: Protecting the Vulnerable.

    6.3    Summarise any Interventions which are recommended to cope with the infection passing through the 99.1% of the population.

7.    Next, the report should have an Executive Summary covering the Terms of Reference (including the objectives), the input data, data processing, assumptions and results, all written in plain English so politicians and the public can understand it.

8.    Next is the Introduction which can be written in technical language as can the rest of the report.

9.    The quality assurance, peer assist, peer review and sign-off procedures, for such reports, has to be rigorous even when time-scales are very tight.

You can see in my review notes attached some key comments, some of which I’ll explain below:

No scientific work that is produced for a government should be divorced from Economics. Report-9 makes it clear Suppression is a long term strategy, but then washes its hands of any economic elements, when it’s obvious, even to the layman, that Suppression itself, is not a viable approach. Yet Report-9 then goes on to recommend Suppression, together with Adaptive Suppression, in order to keep hospital capacity available as infections rise after a slight lifting in restrictions. The timeframe for these is not realistic for the UK; hence these strategies are not appropriate and therefore should not be recommended. To reiterate, the best strategy is to focus on the minority of vulnerable people; not lockdown 65 million people unnecessarily.

We can see, since late March, that Report-9’s “thinking” has almost certainly informed Government policy and unfortunately it continues to do so as the Government’s approach to Lockdown Phase-2, is clearly Adaptive Suppression as briefly summarised in Report-9.

How has the Government failed to understand the straight-forward science about the actual severity of COVID-19? Is SAGE simply not fit for purpose? The discussion at the Health and Social Care Committee session, on the 5 May 2020, suggests unfortunately, this is probably the case.

In the ideal world, testing, trace and contact is the recommended approach. The UK was nowhere near the ideal world, yet it is trying to play catch-up on the wrong elements of the science. Is this another failure in policy as well as advice from the scientists? What is the real world value of monitoring the infection as it passes through 99.1% of the non-vulnerable population? The time, resources & costs are long, vast and high; making the ultimate value, low. Academically, testing, trace and contact are logical processes, even when it’s too little, too late. But context is crucial to scientific advice. Academic science must be put into the correct context, incorporating economics, then properly reviewed, before any recommendations are made to policy-makers.

The Government appears to have latched on to the R0 parameter as the foundation of their policy. How has SAGE “informed” the Government about infection rates? The science clearly shows that for the vast majority of the population, the infection is not fatal, consequently suppressing it broadly, in these people, is actually counter-productive. In reality, the non-vulnerable population, plus the country as a whole, actually benefit from the infection passing through, with certain minor controls to avoid a sharp peak, which could potentially overburden the NHS. There’s no indication the Government actually understands the real science, which has to be interpreted as a failure of the scientific advice. Sadly, Report-9 is an element in incomplete, poorly focussed and poorly reported science. Not good.

There have been consistent criticisms of the apparently pessimistic predictions from your models. This is highlighted by Sweden who dismissed your conclusions and results, in Report-9 and actually used the correct science to let the infection pass through the population with minimal economic impact (which is what the UK and other countries should have done).

It’s imperative that all future models provide a full distribution of inputs and outputs so that the range of uncertainty is described.

Academic science is quite commonly somewhat divorced from the world at large. I experienced this at a College event entitled “Mobilising Business, Acting on Climate Conference”, where the College’s staff were some way behind the actual situation “out in the wild”. Academia can be leading edge within academia but tends to be trailing edge when it comes to the outside world. This doesn’t appear to be widely appreciated within Academia, or by policy-makers.

I am saddened, being an Imperial College Alumni, that Report-9 reflects poorly on the College and that it appears to have had a disproportionate impact on policy when, in its current form, it shouldn’t have passed a proper review process.  And the Government appears to be basing policy on misplaced, academic, incomplete science; potentially yours?

The College urgently needs to revise its internal review and sign-off procedures to prevent such a serious issue occurring in the future.  The implications of Report-9 are going to last perhaps a decade as the Government continues down the dead end of Adaptive Suppression having been led there by incomplete, inadequately thought-out science.  The Government are on solid ground when they say they are using the science.  Unfortunately the science they’re using missed the crucial Intervention and has actually guided the Government down the wrong path!

I’m going to break my own rules now by articulating a key recommendation, right at the end of this missive.  I strongly recommend that you use your new status, outside SAGE, to publically support the van Bunnik(1) work and tirelessly promote the Protect the Vulnerable strategy in order to persuade the Government out of the current, non-viable Adaptive Suppression strategy.  Eventually the country would likely thank you for doing this.

Lastly, attached is a brief summary of where the UK could be today if the simple, science-based Protect the Vulnerable recommendation had been successfully recommended by SAGE.  Interestingly, SPI-B in their 3rd March Return to SAGE, include “Isolation of vulnerable groups” as a SAGE intervention, but this appears to go no further, leading to the ongoing national tragedy.

Yours sincerely,

Jeremy Daines

 

cc: Professor Alice Gast, Dr Sabine L. van Elsland & Mr Jeremy Hunt  MP

 

Reference

1. Segmentation and shielding of the most vulnerable members of the population as elements of an exit strategy from COVID-19 lockdown. van Bunnik et al. 5th May 2020.  
www.ed.ac.uk/covid-19-response/latest-news/two-tier-approach-could-begin-lockdown-end

ATTACHMENT-2

Imagine the situation where the UK government had started the lockdown in March using the following fully scientifically supported approach:

1. The data we have so far (i.e. Chinese data used in Report-9) estimates that overall 0.9% of the UK population, which, in the current (Imperial College) model, amounts to approximately 500,000 people in the UK; are at risk of dying, if infected. For these people, the sole remedy is a vaccine.

2. Consequently, analysis of the data predicts that 99.1% of the UK population, or approximately 64.5 million people, are not predicted to die.

3. The NHS cannot cope with 500,000 sick people; no country’s health service could adequately cope with this number of infected people.

4. Consequently, the most effective plan is to rapidly suppress the virus from infecting everyone by implementing a country-wide lockdown, so that the Government can do the following:

a. Notify then rapidly protect the vulnerable from getting infected.

b. The current data indicates that people over 50 and very importantly people over 80 must isolate themselves immediately in order to minimise the potential of getting infected.

c. Details of how to do this can be found here: (website URL with details of isolation recommendations, including for care homes)

d. Likewise anyone, of any age, with medical issues listed on (website URL) are also vulnerable and urgently need to be isolated.

e. If you have parents or grandparents who could be vulnerable please telephone them immediately to ensure they know what to do in order to implement isolation. Do not visit them without ensuring you have implemented the necessary infection control processes.


5. In parallel with the urgent need to protect the vulnerable, the Government will re-purpose the NHS to deal with infected patients that do need specialist care.

6. In the first instance, if you think you have direct Coronavirus symptoms, most specifically a fever, a new dry cough and possibly a loss of smell, call NHS Direct (111) and follow their advice closely & immediately. This is the flu season, so many people will have colds & coughs that are not COVID-19, so don’t panic if you think you may be infected; the vast majority of people will cope with the virus. Do not go to hospital or your GP surgery and avoid all physical contact with people, where possible.

7. The lockdown restrictions will not prevent any ongoing medical or emergency care that you already have, or are deemed to need. You should use the NHS in the same way during the lockdown as you did before the lockdown, albeit minimising the risk of getting infected. If you are deemed vulnerable (or you are in any doubt about your pre-existing treatment) call NHS Direct (111) for information on how to attend your treatment appointments.

8. Once the vulnerable have been properly protected from infection and the NHS fully re-purposed, the lockdown can be released because the current data indicates the vast majority of the population will cope with the infection and the systems will be in-place to prevent the most vulnerable from getting the virus.

9. At this time, it’s not possible to say exactly how long it will take to protect the vulnerable and re-purpose the NHS; however, the government will report regularly on progress, which will then allow a more accurate prediction of when lockdown can end. It is likely to be several weeks, or potentially months, depending on how quickly these two objectives can be achieved.

Stay Home, Protect the Vulnerable & Save Lives

Based on what has taken place since the end of March, it’s highly likely that the Government could have ensured the vulnerable were protected and NHS capacity enhanced to the level where the lockdown could have been removed at the end of April or early May.  Sadly the massively detrimental lockdown we are currently experiencing cannot be switched-off because the wrong strategy has been adopted.

Unfortunately, this is what SAGE and therefore the government both appear to have missed?

Appendix-2

 



March 2021
Here's an interesting read on behind the scenes in the UK government during 2020.

Comments

  1. I subsequently found out from a podcast with an interview with Ivor Cummins, that some seriously qualified scientists wrote to Imperial College, about Report-9. They didn't get any response. So, they organised an even higher powered letter which got a response....acknowledging receipt of their letter. This group accused Imperial College, according to Ivor Cummins, of over-estimating covid fatalities by a factor of 10.

    There have been other indications that Imperial College's Report-9 has been largely binned by most qualified scientists but sadly, the damage has been done and continues to be repeated in the form of Adaptive Suppression.

    ReplyDelete

Post a Comment