That such methodology may be 'peer reviewed' is not in question, but its content may be lacking, weak, assumed or 'tortured' to fit the model. We quote from our contributer's view on just this type of prediction, and its subsequent conclusions which have been described by some as 'coherent argument' offered by the ISG to support their unequivocal conclusions.
To quote F J Bourne:
'Scientific findings indicate that the rising incidence of disease can be reversed and geographical spread contained, by the rigid application of cattle-based control measures alone.' (ref. his letter to Miliband).
‘The ISG also conclude that rigidly applied control measures targeted at cattle can reverse the rising incidence of disease, and halt its geographical spread……….Having shown that the main approach to cattle TB control should be rigorously targeted to cattle, we hope that the overwhelming scientific evidence we have provided to support this view……..’(ref. the ISG press release of 18th June 2007.)
‘Our modelling work indicates that implementation of cattle control measures outlined in this report are, in the absence of badger culling, likely to reverse the increasing trend in cattle disease incidence that has been a feature in GB for decades. It is also possible that more effective cattle controls will lead to a decline of the disease in badgers, although the timescale for this is likely to be slow.’ (ref. Chairman’s overview, section 12.)
‘ Analysis of a simple mathematical model suggests that rigorously enforced movement testing would halt the epidemic and indeed produce some steady decline in incidence…..’(ref. Recommendations and Conclusions. Section 31.)
The comment continues, emphasising that the above quotes were read before reading the main body of the report. The writer admits surprise at these conclusions, but asserts the report was read with an open mind, the reader ready to be reluctantly convinced. What the reader was not expecting was such a lack of scientific rigour in the arguments. As a full analysis is not possible here, the writer concentrates on the mathematical modelling that seems to be the basis of the ISG’s ‘overwhelming scientific evidence’ as follows:
‘
7.24 The effect of changes cannot be assessed directly from available data but simple mathematical models, combined with the large amount of data now assembled, do allow some very tentative predictions. The infection rate concerns all sources of infection for cattle, local infection for example across farm boundaries, infection from animals bought, in particular but not only, from high incidence areas, and infection from wildlife, especially badgers. All these are important but their relative importance, and that of cattle-to-badger transmission, cannot be estimated directly’ .
"So far, so good. The predictions are tentative, indeed, very tentative, and the relative importance of the most important data is unknown. So what scientific or even logical explanation can there be for the next sentence?"
’In the following calculations we assume all three sources to be roughly equally important.’
"If there is available evidence to show that that assumption is a reasonable one what is it? We know that Bourne does not believe there to be such a thing as a closed herd".
‘People claim to have a closed herd, but they simply do not exist’ (Bovine TB seminar for MPs September 14th 2004).
"There is overwhelming evidence that closed herds do exist, so there is one piece of evidence that he is not taking into account at all". (This site was set up by four owners of such herds, precisely for that reason - ed)
"From section 7.19 there is some discussion of the mathematical model , and towards the end of that section the report says":
7.29 These conclusions are subject to substantial uncertainty and should be taken as broad guidance only.
The comment continues: "That is the conclusion I would expect a reputable scientist to come to. There are too many uncertainties and unknowns, and the model is too simple. Substantial uncertainty is not overwhelming scientific evidence".
Kitching et al say in their paper "The Use and Abuse of Mathematical models": …..Rev.sci.tech.Off.int.Epiz.2006.25910, 000-000)
It is not necessary to be mathematically literate to appreciate that no model will produce the right output, when fed the wrong input. In the future, care should be taken to ensure that lessons are learned – a bad model is like a bad x-ray because it invariably results in erroneous conclusions and a wrong course of action.’
(Incidentally, I would very much recommend reading the whole article. There is much more that is very relevant.)
The writer concludes: "So how can such vague inputs into the ISG model come to such a concrete conclusion as this":
‘Analysis of a simple mathematical model suggests that rigorously enforced movement testing would halt the epidemic and indeed produce some steady decline in incidence…..’(ref. Recommendations and Conclusions. Section 31)
The writer also notes that Christl Donnelly, vice-chairman of the ISG, worked on the mathematical model for FMD, which was the basis of the paper published by Kitching et al and entitled "The Use and Abuse of Mathematical Models".
3 comments:
There was a computer model in New Zealand that showed that vector control (in this case the possum rather than badger) was the only factor that made a significant difference to the number of infected herds (the OIE measure of disease incidence). Changes to cattle testing factors (including test sensitivity or frequency) made relatively small differences without removing the primary source of infection. Inputting historical data into this model produced very realistic results that mirrored actual results.
Also by applying vector control - albiet using methods not suitable for badgers in the UK - the North Island of New Zealand is now TB free (less than 0.2% herds infected). The South Island is well on the way to TB freedom.
Much scientific information was gathered during the application of vector control methods that were considered best practice at the time - with input from science, policy and field staff from both disease control and vector control.
Thanks for that. Glad the input into the computer for NZ's problem was less 'tentative' or 'assumed' or vague than the ISG's efforts.
And well done you for attaining OIE TB free status at less that 0.2 per cent of herds affected.
We have almost 7 per cent of herds under restriction, a figure that is rising, a point not unnoticed by our trading partners and customers.
By a happy coincidence your two pieces posted on 27th June “The use and abuse of mathematical models” and “From the Outside Looking in” complement each other in a way which is not immediately apparent.
The first piece, among a lot of good material, points out the dubious assumption by the ISG that three possible sources of infection for cattle should be given equal weight. The three possibilities are:
“local infection for example across farm boundaries, infection from animals bought, in particular but not only, from high incidence areas, and infection from wildlife, especially badgers.”
Very properly you raised the question: "If there is available evidence to show that that assumption is a reasonable one what is it?
In the second piece, Mr O’Donnell points to a paper which, everyone knows, proves that cattle movements are the major, if not the only significant means by which bovine tuberculosis is spread. But what everyone knows simply is not true.
The flyer on the cover of the edition of “Nature” which in which the letter from Gilbert et al was published read: “Bovine Tuberculosis – Cattle Movements Spread Disease.” But that was not what Gilbert et al said.
O’Donnell was not quite right in saying: “In the model predicted by Gilbert et al., 2004, where cattle movement is the primary motivator for BTB spread it also implies cattle to cattle spread.”
I don’t blame him for misreading the paper; it is chloroform in print and some key information is hidden away in the supplementary information which is concealed in the online version of Nature.
Gilbert et. al. chose a series of “predictors” for bovine tuberculosis and tested them for statistical significance. The only one which came out as significant was cattle movements. I searched the letter and supplementary information for any suggestion that Gilbert et. al. had understood a misleading anomaly of MAFF/DEFRA recording categories and found none. For many years, it has been almost mandatory within SVS/Animal Health to record the “origin” of a TB “herd breakdown” or “confirmed new incident” as “purchased animal” where the only reactor is a purchased animal. In my experience, almost invariably when this was so, the purchased animal was, and remained, the only reactor in the “breakdown.” In other words, although an animal infected with tuberculosis was introduced into the herd, there was no transmission of disease to other animals in the herd.
In the absence of any evidence that Gilbert et. al. allowed for this fact their work only serves to highlight what has been known for many years and is almost self-evident: that if you infect an animal on one farm and move it to another it will fail a test just as well after the journey as before!
Gilbert et. al. did consider badgers as a possible “predictor” but chose to use old (1965-90) and patchy data about sightings of any badgers and to ignore, completely, available data about findings of infected badgers.
What did deserve to be trumpeted on the cover of Nature was that they also considered proximity to other infected farms as a “predictor” with negative results.
So, while telling us nothing about the significance of cattle movements and badgers for the risk of infection they cast serious doubt on the possibility of acquiring infection from the farm next door.
Yet the ISG invite us to assume that the three possibilities have equal significance.
Kitching et. al. are right indeed: no matter how good your mathematical model, feed that kind of data into it and you will certainly come up with wrong answers. No wonder the spoligotypes tell a different story.
Post a Comment