Interesting article in the Annals of Internal Medicine that further
undermines CD4 counts and viral load as surrogate markers of disease
progression and antiviral drug efficacy.
——————–
Fleming TR, DeMets, DL. Surrogate end points in clinical trials: Are we
being
Misled? Annals of Internal Medicine, 1996 Oct 1, 125:605-613.
Abstract: Phase 3 clinical trials, which evaluate the effect that new
interventions have on the clinical outcomes of particular relevance to
the
patient (such as death, loss of vision, or other major symptomatic
event),
often require many participants to be followed for a long time. There
has
recently been great interest in using surrogate end points, such as
tumor
shrinkage or changes in cholesterol level, blood pressure, CD4 cell
count or
other laboratory measures, to reduce the cost and duration of clinical
trials. In theory, for a surrogate end point to be an effective
substitute
for the clinical outcome, effects of the intervention on the surrogate
must
reliably predict the overall effect on the clinical outcome. In
practice,
this requirement frequently fails. Among several explanations for this
failure is the possibility that the disease process could affect the
clinical
outcome through the surrogate, with the intervention’s effect on these
pathways differing from its effect on the surrogate. Even more likely,
the
intervention might also affect the clinical outcome by unintended,
unanticipated, and unrecognized mechanisms of action that operate
independently of the disease process. We use examples from several
disease
areas to illustrate how surrogate end points have been misleading about
the
actual effects that treatments have on the health of patients. Surrogate
end
points can be useful in phase 2 screening trials for identifying whether
a
new intervention is biologically active and for guiding decisions about
whether the intervention is promising enough to justify a large
definitive
trial with clinically meaningful outcomes. In definitive phase 3 trials,
except for rare circumstances in which the validity of the surrogate end
point has already been rigorously established, the primary end point
should
be the true clinical outcome.
Page 610: HIV Infection and AIDS. The use of surrogate end points has
probably been more intensely discussed in the design and analysis of
clinical
trials of HIV infection and AIDS than in any other area. In a review of
AIDS
trials, Fleming summarized results from the largest trials that
evaluated
effects of nucleoside analogues on surrogate end points and clinical
outcomes. The summary of results from a 1993 state-of-the-art conference
shows that the effect of treatment on the most popular surrogate, the
CD4
cell count, did not accurately predict the effect of treatment on the
clinical outcomes, that is progression to AIDS or time to death.
In this review, which involved 16 major AIDS trials, the surrogate end
point
of CD4 cell count was significantly favorable in 7 of the 8 trials in
which
treatment improved the clinical outcome of progression to AIDS or death.
Unfortunately, the CD4 cell count was significantly favorable in 6 of
the 8
trials in which treatment did not improve progression to AIDS or death.
For
survival, the CD4 cell count was significantly favorable in only 2 of 4
trials in which treatment showed a significantly favorable effect on
survival
and, even worse, was significantly favorable in 6 of 7 trials in which
treatment had no effect on survival. Three additional trials, including
the
Concorde Trial showed an inverse relation between survival and improved
CD4
cell counts.
The Concorde Trials involved 1749 asymptomatic HIV-positive patients who
were
randomly assigned to receive immediate or deferred treatment (when
symptoms
occurred) with zidovudine. During a follow-up period of 3 years, the
decline
in CD4 cell counts was slowed by immediate zidovudine therapy, with an
average difference of 30 to 35 cells/mm^3 between the two treatment
groups.
In addition, patients in the group that received deferred treatment with
zidovudine more quickly achieved a 50% decline in CD4 cell counts.
However,
the clinical outcomes did not reflect these changes in the surrogate end
point. Time of progression to AIDS-related complex, AIDS or death was
essentially unaffected (175 event in the immediate zidovudine treatment
group
compared with 171 in the delayed zidovudine treatment group). For death
alone, the results actually favored the delayed zidovudine treatment
group
(95 compared with 76 deaths). The early pressures to use zidovudine
treatment
in asymptomatic persons with HIV were not supported by these longer-term
clinical events.
Page 611. Conclusions. Effects on surrogate end points often do not
predict
the true clinical effects of interventions. Although there are many
explanations for this failure, such as the existence of causal pathways
of
the disease process that are not mediated through the surrogate end
point and
that might be influenced differently by the intervention, the most
plausible
explanation is usually that the intervention has unintended mechanisms
of
action that are independent of the disease process. These unintended
mechanisms can readily cause the effect on the true clinical outcome to
be
inconsistent with what would have been expected solely on the basis of
evaluation of surrogate end points. These mechanisms are insidious
because
they are often unanticipated and unrecognized.
Unfortunately, the failure of surrogate end points to predict true
outcome is
not an isolated problem. Table 1 [see excerpt below] shows various
examples
from several disease and treatment and prevention strategies, including
many
comprehensive meta-analyses that involve scores of clinical trials.
Several
other examples of the failure of surrogate end points can be seen in
other
settings, ranging from trials of vaccines that use the presence of
neutralizing antibodies or cell-mediate immune response as the surrogate
end
point, thrombolytic agent trials that use vessel repercussion, and
cancer
screening strategies that use stage of detected disease to trials of
vitamin
supplementation for treatment of retinitis pigmentosa using decline of
electroretinograms as the surrogate end point, oxygen supplementation in
severe chronic obstructive pulmonary disease using physiologic
variables,
dental treatments using probing attachment levels, and surgery using
excision
of disease or establishment of the blood flow.
The validity of a surrogate end point has rarely been rigorously
established.
Occurrence of false-positive and false-negative results must be low,
typically in the range of 2.5% to 10%, in definitive trials evaluating
the
effects of interventions on clinical outcomes. Hence, to be a valid
replacement end point, a surrogate must provide a high level of accuracy
in
predicting the intervention’s effect on the true clinical end point.
Predictions having an accuracy of approximately 50%, such as the
accuracy
seen with the CD4 count in the HIV setting, are as uninformative as a
toss of
a coin. Methods for validation surrogate end points have been discussed
by
Lin and colleagues, Freedman and associates, and DeGruttola and
colleagues.
Statistical methods for validation usually require meta-analyses because
the
sample sizes needed are much larger than those required for the typical
phase
3 evaluation of interventions. Proper validation of surrogates also
requires
an in- depth understanding of the causal pathways of the disease process
as
well as the intervention’s intended and unintended mechanisms of action.
Such
insights are rarely achievable.
Surrogate end points should be used where they perform best–in
screening for
promising new therapies through evaluation of biological activity in
preliminary phase 2 trials. Such results in turn can guide decisions
about
whether the intervention is sufficiently promising to justify the
conduction
of large-scale and longer-term clinical trials. Although information on
surrogate end points in these definitive phase 3 trials can provide
further
valuable insight into the intervention’s mechanisms of action, the
primary
goal should be to obtain direct evidence about the intervention’s effect
on
safety measures and true clinical outcomes.
——————–
Page 607: From Table 1, titled "Speculation on Reasons for Failures of
Surrogate End Points."
For HIV infection or AIDS, with the intervention of antiretroviral
agents,
using the surrogate end points of CD4 levels and viral load, and the
clinical
endpoints of AIDS events or survival, the authors state that the likely
or
plausible reasons for failure of surrogate end points are (1) of several
causal pathways of the disease, the intervention only affects the
pathway
mediated through the surrogate; (2) the surrogate is not in the pathway
of
the intervention’s effect or is insensitive to its effect; and (3) the
intervention has mechanisms of action that are independent of the
disease
process.