New publication looking at the impact of point-of-care genotyping on routine clinical practice
We collaborated with researchers at the Manchester Centre for Genomic Medicine on a paper looking at the impact of introducing point-of-care genotyping on routine clinical practice. LINK TO PUBLICATION.
Background
Aminoglycoside antibiotics are used to treat infections in neonates. Unfortunately, in babies with a rare genetic variation this treatment causes permanent loss of hearing. For those without this trait, the treatment is safe and effective.
The objective of the study was to understand the practicability of carrying out genetic tests on patients before administering treatment. The results of which will then guide physicians in their treatment choices.
Comparing treatment times
There is a trade-off that comes with introducing any additional point-of-care tests: while valuable information might be gained, critical time may be lost. One of the key study outcomes was the impact of introducing the genetic tests on patient treatment, in particular on the time to antibiotic therapy.
We used an equivalence test to compare times in the control and treatment groups. These are a simple modification of the standard null hypothesis significance test.
Typically when comparing the difference between two values, our objective is to falsify the claim that there is no difference, i.e. our null hypothesis is that the difference is zero. In this case however, we’re want to falsify the claim that the difference is greater than some pre-specified, clinically acceptable amount.
Cohen’s d
Cohen’s d is a data transformation used to present differences / effect sizes on a standardised scale, i.e. in terms of standard deviations. It is simply the difference in the means divided by the standard deviation of the pooled data. Typical thresholds of 0.3 or 0.5 are used for effect sizes considered meaningful. In our case, we used 0.5 as our ‘equivalence bound’, i.e. the difference considered meaningful is half of one standard deviation.
Two-one-sided significance tests (TOST)
The testing procedure involves testing two nulls: H01: Δ ≤ –ΔL and H02: Δ ≥ ΔU. Where ΔL, ΔU are expressed in terms of Cohen’s d. If we reject these two tests at an appropriate significance level, we can conclude the two mean times are equivalent, i.e. that there is no significant impact on the time taken to administer antibiotics.
In practice, this is simply a question of constructing an appropriate confidence interval and verifying it sits within our Cohen’s d bounds.
The mean difference between the treatment and control groups was -0.87, so the mean in the treatment group was actually fractionally less than in the control group (though the standard deviation was a little higher).
To successfully reject the nulls, we are implicitly carrying out two separate (Welch's) t-tests, at the upper and lower end of the confidence interval. To retain a Type-I error rate of 2.5% we need a 95% CI (2.5% at each end).
Looking at this plot, we can see the confidence interval falls fully within the equivalence bounds, so we reject the null that the difference is beyond these acceptable limits.
As always, thank you to the team at Manchester University NHS Foundation Trust for inviting us to be involved in another great project.