Thursday, August 16, 2007

"I've gone off the ISG people"

We are grateful to the Small Farms Association for a copy of his critique of the RBCT Badger Dispersal Trial, written for them by Dr. John Gallagher, which we publish below.

TB and those culling trials – the ISG final report.

After spending the most depressing day for many years listening to the full findings of the so called Independent Scientific Group (ISG) on the culling trials and TB problem in cattle, just how badly the trials had been carried out started to dawn on me. I had been working in the Putford / Holsworthy area at the time and talking with farmers and seeing what they were reporting made me realise that the trapping was going badly. But to listen to the slick ISG presentation describing these trials as the “only robust scientific data on which policy can be formulated” you would have thought they had been conducted in an impeccable manner. We were told culling badgers did not work and whilst it could result in modest reductions in numbers of cattle herd outbreaks in the culled areas, in the surrounding areas a rise in outbreaks was found as a result of infected badgers dispersing. This they claimed was an inevitable effect and an insuperable problem of culling. But what we were being told didn’t add up. You were left wondering just how robust was this science?

The ISG’s report is some 280 pages long and most of the data they have used on which to base their conclusions derives from the Randomised Badger Culling Trials. But there were huge problems with these trials. DEFRA Wildlife staff were tasked with trying to catch the badgers in cage traps yet trapping was continued for only an average of 8 days per annum, of the 15,666 traps set, 8,981 (57%) were tampered with and 1,827 (12%) were stolen and trapping was carried out when least likely to be effective during November to January in 16 of the total of 51 culls. Indeed one of the DEFRA trapping managers was so concerned about the poor trapping efficiency and procedural interference by the ISG that he submitted a written submission to that effect to the EFRA Select Comittee, 2006. All these problems as well as landowner refusal to participate added up to a disastrously low culling rate confirmed by DEFRA of from 20% to 60%. This means from 80% to 40% of infected badgers were dispersed to spread their infection making this more a study in dispersal of TB rather than a culling trial to control it. However, in a vigorous defence the ISG assert they removed from 32% to 77% !

But in all other trials in this country (Thornbury, Avon; Steeeple Leaze,Dorset and Hartland, Devon) and in the Republic of Ireland (East Offaly and the Four Counties) culling success has been over 80%. In the trials using gassing in the mid 70’s, 100% removal was achieved resulting in complete cessation of TB cases over 10 years before other infected badgers moved in to start the problem again (Thornbury) and 7 years in the other (Steeple Leaze). At the latter the farming group switched to arable after that time.

Culling efficiency is everything when dealing with a wildlife reservoir host such as the badger which is organised into hierarchical social groups. Due to the confined air space in the sett, mutual grooming, communal sleeping and the gregarious nature of badgers, once there is a diseased badger in the sett all inmates will become infected. Most infections remain in a dormant phase with maybe only one or two developing progressive disease straight away. But the dormant cases may break down and develop disease as a result of stress caused by malnutrition, intercurrent diseases or social disruption. This is how poor culling approaches spread disease. So culling must always be aimed at complete removal of all the social group or sett occupants.
When the trials were set up there were to be three areas each of 100 sq.Km. One was to be a no action area, acting as a control, whilst reactive culling was to be carried out the second as and when outbreaks occurred. In the third all the badgers were to be culled to see whether that halted outbreaks in the cattle, the proactive cull area. But in this last area the ISG decided not to cull out the badgers but rather to reduce the population and keep it “suppressed” throughout the trial. Surely this was a certain formula for disrupting and dispersing social groups? On studying the trial data it was apparent that 5 of the first culls in these 10 areas were carried out in the Winter (4 in Dec/Jan and 1 in Nov). It is well known that Winter is the least successful time of the year for trapping thus giving worrying doubts as to what the ISG were attempting to do ?

Consciously choosing such a course of action ensured this first cull removed minimal numbers of badgers and maximised social disruption and subsequent dispersal. At site B, Putford, N Devon the second cull was also in the Winter exacerbating the disruption. Overall 16 of the 51 culls were conducted at this time. Were they actually trying to turn this into a study of the effects of social disruption and dispersal in the spread of TB ?

We have long known that poor culling rates can spread infection and the initial cull must always attempt to maximise the removal of as many sett inmates as possible to avert this problem.

The last large cull, using the same cage trapping method as used in the RBCT, was at Hartland, N Devon, in 1984 and resulted in a fall in confirmed herd outbreaks of TB in cattle from 15% of herds to 4% in 1985. Thereafter annual incidence declined and held at around 1%. In excess of 80% to almost 90% of badgers were removed which required protracted trapping efforts in some of the area, sometimes for about three months. No evidence of significant dispersal of infected badgers was found here.

However, waiting for badgers to emerge at dusk and chance catching them using traps or shooting them makes the likelihood of complete removal of social groups poor as well as logistically hugely more difficult. And the one unequivocal finding from the ISG is that if culling is not done completely disruption of badger groups and their dispersal will result.

Since badgers live in underground tunnel systems the obvious approach is to try and dispatch them whilst still asleep underground, during the working day, by a toxic gas. Carbon monoxide mixed with dioxide from petrol exhaust fumes would seem the obvious choice but DEFRA have only recently started to test this approach and the trials and licensing of the method seem almost a year away. Whether, in the light of the ISG’s report they will continue with this work is debateable. Cyanide had been used previously but whilst very effective there were concerns about humaneness and obviously other connotations to its use. But it is only the gassing approach that has resulted in a certain and complete dispatch of all the sett inmates.

The culling method is crucial in eliminating infected groups and stopping spread to other badger groups and eventually cattle and the several other species that may pick infection. To date this includes deer, alpacas, lamas, sheep, pigs, cats, ferrets and a dog. Infection of man is an accident waiting to happen.
In the MAFF annual report of 1995 the Chief Veterinary Officer stated that 90% of outbreaks were considered due to infected badgers and this was also affirmed by MAFF’s Senior TB Epidemiologist. Indeed, in the two gassing trial areas the complete cessation of TB in cattle following removal of the badgers indicated that they were the sole source of infection. Thus there and throughout the areas where TB infection is endemic in badgers cattle have been acting as sentinels of active disease in the badger But the ISG say they have been unable to quantify the role of badgers in cattle outbreaks although they do admit they can be a source of infection for cattle. However, they consider that cattle to cattle transmission is greatly more important !

Test, test and test again and then use the blood test as well. This is the only way forward according to the ISG. They consider there is a huge reservoir of undiagnosed infectious cattle out there and more “rigorous testing” will more effectively cull out all infected cattle and new herd infections will reduce, so they tell us. But I see no sense in killing cattle this way as they are the innocent sentinels of active disease in badgers. The TB test may not be perfect but repeat herd testing and slaughter of reactors has eradicated TB from all other Member States in Europe other than the UK and Ireland. And in Ireland a combined policy of control in cattle and culling infected badgers has resulted in a 42% fall in reactors since 2002. We are the only country with a deteriorating problem. And from their recommendations based on their dubious work, the ISG would appear to be quite content with that. But the Minister has yet to decide what to do.

So £45 million later with no end to the carnage of our cattle and the human and economic costs involved, there is still no intent to face the reality of this serious disease which is becoming even more entrenched in our badger population.

I must admit I’ve gone off the ISG people."

This very easy-to-read piece is substantially what the Irish scientists have said, but in rather more scientific language which we covered in our posting, here
But, as Dr. Gallagher points out, the ISG have offered a seductive morsel to ministers in the shape of a drop in cattle Tb if they adopt more cattle based measures. We have attempted to point out that this may be the triumph of hope over experience , but this view is 'bourne' out by the £2.8 million Pathman project which failed to find a single mucosal cattle sample testing postive for onwards transmission of bTb - even from reactors with lung lesions.


another anon said...

Thank you for repeating your example of skewed statistics. The good old "57% of traps were interfered with". This is a rather shocking misuse of statistics as I've pointed out previously on this blog.
Think about this logically - if you were to use some sort of measure to assess how much impact interference had on the quality of trapping, how could the absolute number of traps inferred with possibly provide what you want? One trap that is destroyed can be replaced by another one and how many times was that trap successfully used before it was destroyed? The ONLY measure which makes sense is number of trap NIGHTS - i.e. one trap night is one trapping opportunity, so one that is interferred with was (potentially) a trapped badger not trapped. What has happened to individual traps is meaningless, only a measure of trapping OPPORTUNITIES can possibly tell us what we need to know. And that figure is 8.1% (reference ISG final report), rather different to 57%.

Matthew said...

Another anon.
As before, we really cannot get to grips with your logic. However in this comment you do explain:

"One trap that is destroyed can be replaced by another one and how many times was that trap successfully used before it was destroyed? The ONLY measure which makes sense is number of trap NIGHTS - i.e. one trap night is one trapping opportunity, so one that is interferred with was (potentially) a trapped badger not trapped"

The PQ which revealed the extent of trapping inefficiency in the RBCT was couched in just those terms and was explained thus:

"Management records indicate that - over 116 culling operations, across 19 trial areas between December 1998 and 10th October 2003, during which 15,666 traps were sited - there were 8,981 individual occasions where a trap was interered with, and 1,827 individual occasions when a trap was removed".
[ 8 dec 2003: col 218W. 141971]

Now that seems to us to be 15,666 individual opportunities (nights) to trap a badger, of which 57 per cent failed due to that individual trap opportunity being 'interfered with' and a further 12 per cent of the primed individual traps set on one night went AWOL.

That is, one trap set at dusk on a particular night, and its efficacy recorded the following morning. 15,666 times.

For sure the 8,981 traps 'interfered with', would be used again, but the trial log showed how many times they were used unsuccessfully. And gave a figure of 57 percent of those individual trapping opportunities - or 'nights' up to October 2003.

Paul Caruana, a WLU manager working on the trials said in his submission to the EFRAcom:

" Krebs had too many anomolies and weaknesses in the strategy for it to be successful. It took us four years to steer away from trapping setts that had been interfered with by Animal Rights Activists, to be able to trap badgers anywhere, in order to eliminate them."

The figures that Dr. Gallagher quoted from the PQ's (and supported by Mr. Caruana's observations as an operator of the trial) are from that first 4 years of culling, and possibly would have been diluted in the proactive areas during the final cull. But not of course the Reactive areas, which had been culled themselves in October 2003.

Later validation of the trial's trapping efficiency put the figure at between 32 - 77 per cent (Smith and Cheeseman 2007). Do you consider these figures 'skewed' as well? We would have thought that the 'individual traps sets v. badgers caught' figure revealed by PQs sat squarely in the middle of that final analysis.
And furthermore their basis seems to agree with your analysis of 'trapping opportunities or nights'- if not its answer.

The bottom line is that here at the coal face, in a RBCT Reactive area, 8 nights very inadequate trapping in 2000 was followed by a gap of three years until the next slightly more effective 8 night hit-and-run foray. This left a highly infected population of badgers stirred up and shaken. They were not removed. They were not culled. They were not caught at all due to inadequate trapping techniques, hence the PQ to discover just how inadequate. And we think in terms such as you are advocating - that is trapping opportunities (or nights).

RBCT trapping efficacy was rubbish, and no amount of verbal gymnastics in the RBCT final report can alter that.

another anon said...

what an incredibly verbose response. the figure you want is 8.1%. PQs are often made up on the spur of the moment in response to being put on the spot. whatever gripes you may have with the RBCT, the figures in the final report are based on data - up to the end of the trial and not made up on the spur of the moment. quoting other figures is misrepresenting statistics.

i also notice that you've made no mention of the paper published recently by the ISG in the Journal of Zoology which examines trapping efficiency in the RBCT. The actual data are in there and you can analyse them to your hearts content - or you could believe made up numbers.

Science is about basing results on data. i believe "rubbish" is not a very scientific word.

Matthew said...

Another Anon.
I'm not sure the people who patiently answered 538 PQs would appreciate your description of their 'made up answers, on the spur of the moment'.

I'll cut and paste their answer to trapping opportunity and thus efficiency again for clarification;

"Management records indicate that - over 116 culling operations, across 19 trial areas between December 1998 and 10th October 2003, during which 15,666 traps were sited - there were 8,981 individual occasions where a trap was interered with, and 1,827 individual occasions when a trap was removed".
[ 8 dec 2003: col 218W. 141971]

Whether you consider our detailed response 'verbose' or not is a matter of opinion. But 8,981 non effective trapping opportunities logged out of 15,666 set traps plus of course the 1,827 which went walkabout, is not 8.1 per cent. Or not when I last went to school.

Perhaps you should ask John Bourne for his comments, instead of throwing gratuitous insults at the people who gathered the information which we and many others have used. The Irish critique which we published last week deftly points out several points of weakness.

And if as participating farmers, we found its original message 'misleading' its methodology wanting and its conclusions based on the former, wildly unrealistic, then we reserve the right to say so.

It was rubbish.

Jo said...

Another anon (or is it the same one?), is right to refer us to the ISG report. The following taken from it and from their oral evidence. He says “the figures in the final report are based on data - up to the end of the trial and not made up on the spur of the moment.” I’m having problems with the data as presented.

With reference to disrupted traps, anon should read more closely. The measure referred to in section 2.56 is not trapping nights but setts. “. Occasionally, interference and capture of nontarget species together meant that NO TRAPS WERE AVAILABLE TO BADGERS AT A PARTICULAR SETT, …. On an average trap night, 6.1 % of trapped setts in proactive areas, and 3.4% of those in reactive areas, were affected in this way.” Table 2.7 then refers to the number of trap nights disrupted, which is another matter. These figures of 6.1% and 3.4% only refer to setts where no badgers were captured at all.

Section 2.55 says “The numbers of traps placed at each sett exceeded the total number of badgers that experienced field staff expected to capture there…”, and we know that on the proactive cull there were 298.5 traps deployed per night on each of 51 operations. What they don’t tell us is the number of traps per sett. I am sure there is a reader of this blog who can tell me if that figure should be 10, 12, or 15. Either way the figures don’t stack up. Surely there wouldn’t be as few as 2 traps per sett? Perhaps another anon could tell me?

Section 2.56 “AFTER ACCOUNTING FOR SUCH FACTORS, ”How have they accounted for them? Are they included in the final figures or are they discounted? We need to know .”on the first night of culling operations badgers were found in 20.1% of traps in proactive areas and in 30.2% of traps in reactive areas (Woodroffe et al., in press). Capture rates declined rapidly after the first night, averaging 6.1 % in proactive areas (Table 2.7), and 8.8% in reactive areas over the whole trapping period (Table 2.8).”

Looking at Table 2.7, the percentage of badgers caught, (6.1%) is less than the number of trap nights disrupted (7.9%).

There were 160,893 trap nights, 8,891 badgers caught (plus 1,619 of other species), 277 badgers released, and 12,775 other interferences in Table 2.7. It seems that the percentage of interference went down after 2003. But then the number of badgers caught declined too. There is no mention at all of traps stolen. And we still don’t know if the figures include those setts which have been totally disrupted.
The total capture rate does strike me as extremely low. 20.1% on a first capture night, and then declining to an average of 6.1% . Bourne, in his evidence taken before the environment, food and rural affairs committee, 18th June 2007 says “…systematic culling carried out by a professional team over a five-year period where badger removal was about 70-73 per cent, which suggests a trapping efficiency of 80 per cent,” . How do these correlate? I would really like to know.

I came across another strange use of statistics. Look at Table 4.3 on page 66.
“Statistical analyses (log-linear regressions adjusting for triplet and log-transformed land area available for surveying) revealed no significant differences between proactive and survey-only areas in sett (p=0.31) or latrine (p=0.39) densities. “
It’s true, the mean numbers are very similar. Active sets per km2 were 3.62, 3.32 and 3.26 in Proactive, Reactive, and Survey-only areas.
But in some triplets the densities vary wildly between Proactive and survey-only areas. In A it is 1.45 and 2.18, in B it is 3.82 and 1.01, and so on, with only 3 of the 10 being remotely similar.
I would be interested to hear comments on that.
It is also interesting that in table 4.13 the figures are different again. (An average of 6.34 setts or 0.67 main setts in the proactive A triplet for example). I would have thought that where measurements are given for sett densities in a scientific paper it would be useful if the same system of measurements was used throughout.

Jim said...

The fact is that a large number of traps were interfered with - 1827 were actually removed. Now, if one wanted to disrupt the trials, presumably the simplest way to do this would be to un-set a trap or possibly hit it hard with a handy lump of wood. Why bother to cart it away (and risk being caught with the evidence) unless it already had an occupant? Perhaps someone on the badger side of the debate could answer this. Otherwise anyone who moved a trap-cum-badger (and I believe there is evidence that this was indeed done) risks the charge of gross irresponsibility in helping to spread bTB (to other badgers as well as cattle).

It seems to me that taking away infected badgers and releasing them elsewhere is just as likely, if not more likely, than cattle movements to explain why "stray" spoligotypes have appeared in unexpected places. Infected badgers which were moved to a new location no doubt took their spoligotype of origin with them - while the latest research from the VLA (summarised previously on this site) shows that cattle-to-cattle transmission must be on a much smaller scale than supposed.

Jo said...

Looking at the figures in the ISG report again, I'm wondering about the 298.5 traps used on each operation. Surely just under 300 traps to use on an area of 100 sq klm (admittedly, not all that area was accessible - MAFF couldn't trace the owners of 13% of the land!) is not enough.

Matthew said...

Anon 9.42
A senior ex.Defra epidemiologist has commented on your extraordinary claim of written PQs being "made up on the spur of the moment".
His reply may help your grasp of the situation.

He said:
"It is virtually a hanging offence for a Civil Servant to mislead a Minister by providing incorrect or misleading information for the preparation of a reply to a written PQ. It is just not done to put one's job and pension at risk by so doing. Details of trapping efficiency, cage tampering etc were given in a written PQ."

This as opposed to a throwaway comment spoken by a minister who may or may not have full grasp of its context.

And he goes on to refer to the credibility issue of the RBCT input data, highlighted in the Irish paper which we posted last week. This as Jo is discovering while investigating trap density, would obviously influence the output of the mathematical modelling into which it was compressed.

At the risk of being verbose again, I will explain in simple terms what I do not see from the RBCT final report, which according to Professor Sir John Krebs were methodology constraints which he thought were vital to the success of his proposals.

1. Number of trapping nights, (there were 8 only at each widely spaced visit)and follow up;
2. Length of time between trapping opportunities on an individual farm. (RBCT target was 10 months(?), but reality 2/3 years, if at all. Krebs was much more specific)
3. Prevention of recolonisation of infected setts.( None)
4. Change of boundaries to RBCT areas. Some outside at the beginning were scooped up half way through, conversely other areas abandoned. (Data would apply to only half the time scale of RBCT for each parcel of land thus affected)
4. Cattle testing data calculated within RBCT time scale; i.e not taking into account test latency + time of test. On annual testing 14 months could elapse before tested cattle showed any benefit, or conversely where a problem was, after RBCT attempted culls.

As we saw this charade unfold, written PQs were asked to establish the actual basis on which the RBCT would report, which they very effectively did.
Matt 3

George said...

I believe that any answers to PQs are as accurate as they can be. Any inaccuracy or inconsistency discovered later would be a considerable cause for concern, and would certainly be a serious problem for the writer.