Bias in laboratory results – how we can see this in real patient data and the correlation with External Quality Assurance

In this blog we will consider just the mean potassium result for samples taken in the ED, plotted here in a statistical process chart (Figure 3).

Figure 3 Statistical process chart for mean monthly potassium taken in the ED. Mean results pre and post analyser change are shown (dotted lines). Results within 2 standard deviations of the mean prior to the analyser shift (solid line) are shown in the shaded area

We can see a shift in the ED mean potassium in March 2018. In the language of the SPC, the system went “out of control.” This correlates with the laboratory switching its analyser provider.

Correlation with EQA performance

We can look at the performance of the laboratory in the external quality assurance scheme over these periods. The following plot (Figure 4) shows the bias of different analysers in this scheme. This laboratory moved from the 13OL, which has a strong negative bias, to the 13BO analyser, which has a positive bias. It is not therefore surprising that we saw the shift in laboratory mean when the analysers were changed.

Figure 4 Bias of different analysers in NEQAS potassium returns.

The impact of the shift in laboratory performance on patients

Although potassium results essentially form a continuous distribution, clinicians tend to view these as categorical results (see blog 2 Table 1). This means that it is not that meaningful to look at distribution parameters if we are to understand the clinical impact of changes. Instead, we need to look at how this impacts on the proportion of results that fall into different result categories

Using theoretical “normal” populations to model the analyser effect

To analyse this, we applied a theoretical distribution to our data (using a technique called Kernel Density Estimation; Figure 5). We then calculated the impact of the shift in the potassium mean (due to the analyser change) on the expected numbers of patients who would fall into each category.

Figure 5 Histograms of potassium results from the ED before and after analyser change, with theoretical distributions as determined by KDE analysis.

We can use these distributions to calculate the proportion of results falling into different result categories before and after the analyser switch (Table 2).

Table 2 Proportion of ED potassium results falling into different result categories before and after analyser change in 2018. (Note the normal range is from 3.5 to 5.3 inclusive)

We can then apply these proportions to the number of patients that are tested in our ED each year (approximately 27000 tests;Table 3). In our small ED, 2 patients a day would have been diagnosed with hypokalaemia on the old analyser, but would be normal with the current analyser. 3 patients a week are diagnosed with a significantly higher potassium since the switch to the new analyser.

Table 3 Predicted change in annual number of patients in different result categories before and after analyser change.

Conclusions

We are making no value judgement about which analyser is correct. This blog merely notes the effect that laboratory performance has on patient outcome – 3% of ED potassium results are significantly altered (i.e. change category) merely by changing the analyser.

Setting aside wider considerations of normality and uncertainty, there are mechanisms to remove these biases in laboratory performance. For instance, real time inter-laboratory comparison of performance with continual adjusting of parameters can ensure that result distributions remain constant.

For the non-biochemist, it is hard to understand how different analysers can produce different results when, according to the ISO15189 standard to which they all comply, all measurands must be traceable back to an international standard. These are clinically important uncertainties that are perhaps under-recognised.