More on that 74 percent.

We touched the surface of the most recent headline grabber from Defra in the posting below, but having had a trawl through the project again, we think we have sorted where the 74% comes from.
How accurate is its reporting, is quite another matter. But it was not only the Badger Trust and media which ran with it, Defra issued the original high profile press release on November 8th, stating that :

A key finding of the field study, conducted over four years in a naturally infected population of more than 800 wild badgers in Gloucestershire, was that vaccination resulted in a 74 per cent reduction in the proportion of wild badgers testing positive to the antibody blood test for TB in badgers.

As we said in the posting below, if that was true, when do we start?

But as usual, the devil is in the detail. This project was not ever about efficacy of BCG. It was to study the health and welfare of badgers vaccinated with BCG. Thus no postmortems were done to support the blood assays, assertions or conclusions. And the project morphed into an 'estimated efficacy' on blood assays and cultures alone.

Blood assays are notorious for giving less than optimum results. Results are measured by 'sensitivity' or the ability to diagnose disease or antibodies associated with that particular disease. The lower the sensitivity, the less confidence can be placed in it.

For example, the opinion of the ISG contained in their Final Report

on the old 'Brock' test:
1.7 [] ... A live test for badgers had been developed and subject to trial from 1994-96, but its sensitivity was much poorer than had been hoped, successfully detecting only about 40% of infected badgers (Clifton-Hadley et al., 1995-a,Woodroffe et al., 1999)

Thus the 'sensitivity' of this Brock blood test test, (which our PQ answers also put at 'very low' and other research at around 47%) was described as 'much poorer than was hoped" by the ISG and hovers around the 47% mark, at which point the live test trial was abandoned. Bookmark that sensitivity rating and the ISG comment on it.

So back to this paper on the Health and Welfare of badgers vaccinated with BCG.

As we pointed out, the original 844 badgers were taken back to secure laboratory facilities and screened using GammaIFN / Elisa and Statpak Rapid blood assays. Out of the headline 844 badgers trapped, 265 took part in the annual research. Does that mean that at the first screening test, and before vaccination, almost 70% had TB antibodies? But we digress. Efficacy of BCG is the subject.

Much if not all, depends on the sensitivity of the test applied, and as we pointed out above, the old 'Brock' test was roundly condemned for a low sensitivity. So what of the blood assays used to estimate BCG efficacy here? The project cites work done on assays which was supported by postmortems, by Chambers et al, in 2008:

4.7 Immunology Results
Two immunological tests were applied in this study: the IFN EIA and the Brock TB Stat-Pak Test (Chembio Diagnostic Systems, Inc.).
The first test [GammaIFN/EIA] measures the production of IFN following stimulation of whole heparinised blood with bovine and avian tuberculins and has an estimated sensitivity of 80.9% and an estimated specificity of 93.6% (Dalley et al., 2008).

So the gamma IFN / Elisa test has a sensitivity of 80.9% and vaccinated badgers in this project had a protection level of - what?
19% according to the results.

And the second blood assay :

The Brock TB Stat-Pak is a lateral flow assay to test for the presence of antibodies in serum to M. bovis antigen MPB83. It has an estimated sensitivity of 49.2% and an estimated specificity of 93.1% based on a study of 1464 badgers naturally infected with M. bovis as determined by culture (Chambers et al., 2008). Sensitivity of the Stat-Pak varies according to disease severity, such that sensitivity was found to be 34.4% in infected badgers with no visible lesions at post mortem, 66.1% in infected badgers with visible lesions at post mortem, 41.7% in infected badgers that excrete M. bovis; rising to 78.1% in so-called "Super-Excretor‟ badgers (Chambers et al., 2008).

From that we see that Chambers postmortemed his badgers to qualify sensitivity, and found the StatPak test variable, with a combined sensitivity of 49.2% , including super excreters and 'badgers which excrete m.bovis'.

So what criteria have the authors of this project used?

"Here an “all test” exclusion criterion was used such that any badger positive by any of the three tests at T1 or at the time of first capture/vaccination (T2 onwards) was excluded from the analysis. This analysis addresses more directly the prophylactic effect of BCG vaccination since the effect of vaccination is measured in badgers considered to be free of TB by virtue of negative results in all three tests. Whilst this does not rule out infection completely, it is the best measure of TB status in the live animal. As the combination of all three tests would not be 100% sensitive, some badgers regarded as TB-free by this criterion would actually harbour infection."

The authors observe that should badgers actually be already infected with TB when vaccinated, "This would have the effect of reducing the measure of vaccine efficacy."

Quite. So, having excluded from the modeling any badger pre screened as positive at the time of vaccination, the results for gammaIFN / Elisa were?

Against this background, the incidence of IFNγ EIA positivity was reduced by vaccination from 35.0% cases (95% confidence interval: 23.0%, 49.3%) to 28.5% cases (95% confidence interval: 20.8%, 37.7%) but it was not significant at the 5% level. The analysis presented currently for the IFNγ EIA test alone provided no conclusive (P < 0.05) evidence that BCG vaccination was able to prevent infection with M. bovis, although the trend was in that direction.

And the modelled result was the '19 percent' reduction contained in the phrase 'Between 19 and 74%'. Gamma has a sensitivity described in this paper of 80.9%

And the StatPak results about which the headlines are so enthusiastic?

In contrast, vaccination was found to have a significant effect on reducing the incidence of positivity for both Stat-Pak or Stat-Pak and culture combined. The incidence of Stat-Pak positivity was reduced by vaccination from 17.1% cases (95% confidence interval: 10.8%, 25.9%) to 4.4% cases (95% confidence interval: 2.4%, 8.2%), which was significant statistically (P < 0.001). [that is the 74% - ed]
The incidence of Stat-Pak and culture combined positivity was reduced by vaccination from 21.7% cases (95% confidence interval: 13.5%, 32.9%) to 8.3% cases (95% confidence interval: 4.9%, 13.9%), which was also significant statistically (P = 0.008). As the likelihood of a positive Stat-Pak result or excretion of M. bovis increases with disease progression/severity (Chambers et al., 2008; Gallagher & Clifton-Hadley 2000) this study provides evidence consistent with the progression/severity of TB being significantly reduced in BCG vaccinated badgers after they become infected.

It does? With a sensitivity very similar to Brock test, abandoned and severly trashed by the ISG? A sensitivity averaging 49.2 per cent, and with no postmortems to confirm?