BMJ 2001;322:184-185 ( 27 January )

Editorials
How policy informs the evidence
"Evidence based" thinking can lead to debased policy making

Education and debate p 222

Who would not want health policy to be based on evidence? "Evidence based
medicine" and "evidence based policy" have such reassuring and self
evidently desirable qualities that it may seem contrary to question their
legitimacy in relation to reducing health inequalities. However, these
terms are now so familiar that it is easy to forget the important question
about what sort of data provide appropriate evidence for particular types
of decisions. The sort of evidence gathered on the benefits of
interventions aimed at individuals may not help in guiding policies
directed towards reducing health inequalities.

In this week's BMJ readers have the opportunity to assess part of the
process leading to the recommendations of the Independent Inquiry into
Health Inequalities (the Acheson inquiry),1 established in 1997 to help the
government formulate policy to reduce health inequalities. The inquiry
established an evaluation group to report on the quality of the evidence it
used to reach its conclusions and support its recommendations.2 This group
critiqued submissions to the inquiry, and a list of its own remedies for
health inequalitiestheir "10 steps to health equality"was released before
the Acheson inquiry had itself reported (see box on bmj.com).3

The evaluation group appears to have applied evidence based principles to
its consideration of ways to reduce inequalities in health. Essentially it
wanted evidence from controlled intervention studies, and its main
evaluation consisted of checking each recommendation against three earlier
reviews (two conducted within an explicit evidence based framework) and the
Cochrane Library.

The task of the Acheson inquiry was to make recommendations that would
reduce inequalities in health, not merely have a positive overall health
benefit. For most of the evaluation group's suggested interventions there
are no high quality controlled studies showing that they would reduce
health inequalitiesfor example, the evidence that fluoridation of drinking
water would reduce inequalities in dental health is scanty.4 Indeed, some
of these interventions could increase inequalities. Smoking cessation may
be more successful in advantaged groups. Drugs education in schools may
have less impact on those most at risk, because they are more likely to be
truants and thus less exposed to it.

On the general question of what sort of evidence is useful to set policy in
the public health domain, it is helpful to think back to earlier eras. In
the first half of the 19th century there were no "evaluation groups" to
point out the lack of evidence from controlled intervention studies showing
the health benefits of, for example, stopping children under 9 from working
in cotton mills, fencing off dangerous machinery, or reducing the number of
hours children could work to only 10 a day. With an evaluation group,
implementation of the Factory Acts could have been resisted. The factory
owners were certainly keen on "evidence": the claim that working class
children aged 5-10 had lower death rates than middle class children was
used to suggest that factory labour was good for the under 10s.5

Clearly the situation is now different, but health inequalities are still
large and have increased over the past two decades.6 Premature death rates
are over three and a half times higher in Glasgow Shettleston than in
Wokingham,6 and a remarkable three quarters of premature deaths in Glasgow
Shettleston would not occur if it had the mortality rates of Wokingham. It
is no surprise that in Glasgow Shettleston child poverty rates are over six
times, and unemployment rates over five times, higher than in Wokingham.
Clearly the need is for substantial reductions in socioeconomic inequality,
which can follow only from the concerted implementation of policies of
progressive taxation and substantial income redistribution.

The evaluation group states that randomised trials of income support have
been carried out and could, in principle, have examined health outcomes.7
However, the effects of income redistribution would not be to give a few
people a little more money while they remain living in a highly unequal
society, but to change the nature of the society. Health is influenced by
micro and macro social environments,8 and societies with high levels of
income inequality are characterised by a wide range of social-structural
attributes that have a detrimental impact on health.9

As Schwartz and Carpenter have pointed out, inappropriately focusing on
individual level determinants of health while ignoring more important
macrolevel determinants is tantamount to obtaining the right answer to the
wrong question.10 Consider the situation of examining risk factors for
unemployment. Conventional individual-level studies would probably find
that low education, not dressing smartly for interviews, being short, being
over 50, or being a member of a minority ethnic group predict being
unemployed. Indeed these "risk factors" would probably explain a high
percentage of the variance in unemployment. A controlled study finding that
counselling on how to dress and behave at job interviews increases success
in getting a job could be added to the Cochrane Library. The same risk
factors may explain a high percentage of the intra-individual variance in
unemployment, both when unemployment is 1% and when it is 14%.

The big difference for the populationand thus for the individual risk of
unemploymentis, however, the 14-fold difference in overall levels of
unemployment at times when different fiscal policies are being implemented.
High variance apparently "explained" by individual-level risk indicators
(or markers manipulable in a discrete way within populations) does not mean
that they are important determinants of the population level of any
outcome.11 These are, however, precisely the factors that evidence based
research focuses on. Despite occasional rhetorical interest in wider
determinants of health, evidence based assessments are largely restricted
to individualised interventions. The Cochrane Library is unlikely ever to
contain systematic reviews or trials of the effects of redistributive
national fiscal policies, or of economic investment leading to reductions
in unemployment, on health.

The insidious nature of this mismatch between evidence and policy is
highlighted by the fact that the evaluation group is, as one would expect
of such informed commentators, aware of the problem, while implicitly
ignoring it. One of the evaluation group stated when launching the "10
steps to health equity", "Our recommendations are quite medical because
those are the sort that tend to have evidence behind them."3 Health
differentials between social groups, or between poor and rich countries,
are not primarily generated by medical causes and require solutions at a
different level.

One source of the scientific innovation that was institutionalised within
the Cochrane Collaboration was a powerful critique of a complacent and
uncritical form of health care delivery.12 The establishment of the
evidence based medicine movement is a remarkable achievement with an
unquestionably favourable influence on the probability that individuals
will receive health care that benefits them and be protected from
interventions that harm them. It would be ironic, and inconsistent with
Cochrane's radical instincts, if the inappropriate applications of those
ideas were to provide a complacent barrier to implementing those measures
necessary to redress health inequalities.

George Davey Smith, professor of clinical epidemiology.
Shah Ebrahim, professor of epidemiology of ageing.
Stephen Frankel, professor of epidemiology and public health.

Department of Social Medicine, University of Bristol, Bristol BS8 2PR



Footnotes

 A box listing the evaluation group's remedies to health inequalities
appears on the BMJ's website


-------------------------------------------------------------------------------

1.  Independent inquiry into inequalities in health. London: Stationery
Office, 1998.
2.  Macintyre S, Chalmers I, Horton R, Smith R. Using evidence to inform
health policy: case study. BMJ 2001; 322: 222-225[Free Full Text].
3.  Laurance J. Experts' 10 steps to health equality. Independent 1998;12
Nov:14.
4.  NHS Centre for Reviews and Dissemination. A systematic review of public
water fluoridation. York: University of York, 2000.
5.  Bennett A. A working life: child labour through the nineteenth century.
2nd ed. Launceston: Waterfront Publications, 1995.
6.  Shaw M, Dorling D, Gordon D, Davey Smith G. The widening gap: health
inequalities and policy in Britain. Bristol: Policy Press, 1999.
7.  Connor J, Rodgers A, Priest P. Randomised studies of income
supplementation: a lost opportunity to assess health outcomes. J Epidemiol
Community Health 1999; 53: 725-730[Abstract].
8.  Diez-Roux AV. Bringing context back into epidemiology: variables and
fallacies in multilevel analysis. Am J Public Health 1998; 88:
216-222[Abstract].
9.  Lynch J, Davey Smith G, Kaplan G, House J. Income inequality and
mortality: importance to health of individual income, psychosocial
environment, or material conditions. BMJ 2000; 320: 1200-1204[Free Full
Text].
10.  Schwartz S, Carpenter KM. The right answer for the wrong question:
consequences of type III error for public health research. Am J Public
Health 1999; 89: 1175-1180[Abstract].
11.  Rose G. Sick individuals and sick populations. Int J Epidemiol 1985;
14: 32-38[Abstract].
12.  Cochrane AL. Effectiveness and efficiency. London: Nuffield Provincial
Hospitals Trust, 1972.

-------------------
Problems/Questions? Send it to Listserv owner: [log in to unmask]


To unsubscribe, send the following message in the text section -- NOT the subject header --  to [log in to unmask]
SIGNOFF SDOH

DO NOT SEND IT BY HITTING THE REPLY BUTTON. THIS SENDS THE MESSAGE TO THE ENTIRE LISTSERV AND STILL DOES NOT REMOVE YOU.

To subscribe to the SDOH list, send the following message to [log in to unmask] in the text section, NOT in the subject header.
SUBSCRIBE SDOH yourfirstname yourlastname

To post a message to all 1000+ subscribers, send it to [log in to unmask]
Include in the Subject, its content, and location and date, if relevant.

For a list of SDOH members, send a request to [log in to unmask]

To receive messages only once a day, send the following message to [log in to unmask]
SET SDOH DIGEST

To view the SDOH archives, go to: https://listserv.yorku.ca/archives/sdoh.html