Notes on the Whole System Demonstrator

Whole system demonstrator cluster RCT

The introduction says that the “monitoring of blood glucose” is an example of of useful monitoring that can be done by patients. In fact, Cochrane is pretty clear about how useful this is for people with type 2 diabetes not using insulin (the majority of people with diabetes), saying that “there is no evidence that self monitoring of blood glucose affects patient satisfaction,, general well being, or health related quality of life”. Similarly, with self monitoring of blood pressure, there is evidence that this can lead to small changes in blood pressure only, and there is no data on mortality or morbidity improvements for this intervention.

The WSD study examined patients with chronic obstructive airways disease, heart failure and diabetes. This is a wide range of conditions and the effect of telehealth on each these conditions were not analysed separately.

The randomisation was not done in the usual way but in terms of practices either assigned to the telehealth or the control group. Why?  “Initial discussions with sites indicated that individual randomisation of patients would probably not be acceptable to stakeholders. Therefore, we used a pragmatic approach to randomise general practice”. However there are large problems with this approach, as we have seen in breast screening trials for example. Cluster randomisation is beset with tendency to bias and I am not sure that the reasons given, for such a large and expensive trial, are good enough.

What was the intervention being tested? This to me is the biggest problem with this trial. Take note “Choices of telehealth devices and monitoring systems varied among the three trial sites, and there was no attempt to standardise these technologies across sites. We included a broad class of technologies, and the study was not designed or powered to examine differences between specific devices or monitoring systems.” In other words, the intervention varied hugely. This is crucial: what was the trial showing the effect of? The only constants were glucometer for diabetes (effect described above) weighing scales for heart failure (a commonly used device already, well outwith ‘telehealth’ trials) and an oximeter for COPD (which is of uncertain benefit in monitoring patients compared with usual care – this is important, because it might lead to either false reassurance or false alarms.)

Indeed, what is telehealth anyway? I don’t think that telephones or email communications between doctors and patients count as telehealth. The DoH had spent millions on the ‘black box’ monitoring devices well before the results of the WSD trial were published. These are “base units” which send data about health from the patients’ home. , either as questionnaires or clinical readings.  The outcomes analysed were for mortality and emergency admissions.

The graphs are here

How sure can we be that this apparent small benefit to intervention patients was real? The study was powered to detect a 17.5% change in admissions to hospital, however ” the magnitude of the group difference in admission proportion was relatively small (10.8%, 95% confidence interval 3.7% to 18.1%), and smaller than the size that the planned study design was able to detect (17.5%), raising questions about the clinical relevance of the results”. As for death rates “Intervention patients were significantly less likely to die within 12 months than controls. We also observed small differences in the mean number of emergency admissions per head between the intervention and control groups (crude rate 0.54 v0.68; difference 0.14). These changes were significant in the unadjusted comparisons and when we adjusted for a predictive risk score, but not when we adjusted for baseline characteristics”. In other words, the more the researchers tried to reduce the bias, the less sure the benefit appeared to be.

The study also found that there was an increased emergency admissions in the control group; “There could have been a propensity to select controls with a higher risk of short term admission and intervention patients with a lower risk.” It’s difficult to know, certainly we cannot exclude bias even with the case mix adjustments the researchers made.

So a comparable intervention wasn’t made, there are design issues in relation to the study design that increase the risk of bias, and the interventions being offered lack evidence themselves of benefit. The scale of savings is small. Neither has the WSD compared normal use of technology in practice (telephone, skype for remote access patients etc) against the expensive intervention being tested. For here is the conflict of interest. The DoH had bought these machines before the trial was completed, with publicity attached to it’s investment to “support the UK life sciences industry.” I’d personally like to see a bit more investment in the people who work as home carers, providing the unglamorous but essential help to wash, feed and assist people directly.

 

 

 

16 Responses to “Notes on the Whole System Demonstrator”

  1. WordPress › Error

    There has been a critical error on this website.

    Learn more about troubleshooting WordPress.