UTI diagnostic algorithms

On the impact of seasonal variation in potassium results in primary care

Although we do not know the reason for the difference between potassium result distributions in ED and primary care, we do know that there are seasonal differences in primary care results that can be removed by specimen stabilisation. We can see the impact of this by looking at the predicted number of patients in primary care who would be placed in different categories if all practices performed at the level of Black Torrington and Lynton (Table 8)

	Black Torrington / Lynton	All GP
Low	0.015	0.013
Normal	0.98	0.96
Low High	0.0058	0.022
Moderate high	0.00075	0.0017
High high	0.000068	0.00027

Table 8  Actual proportion of potassium tests in different result categories comparing the two practices that stabilise all specimens by centrifugation, and all other practices 

From this analysis, we can work out the number of patients in primary care who would be placed in different categories if all practices stabilised specimens (Table 9).

Predicted number of additional patients in primary care who would have a low potassium if specimens stabilised

Low

168

	Predicted number of additional patients per year in primary care who have a high potassium due to seasonal variation
Low high	1953
Mod high	116
High high	25

Table 9 Predicted annual number of patients who fall into different result categories in primary care due to seasonal variation compared with practices that stabilise all specimens (based on 124,000 tests per year in primary care).

Conclusions

From the data in this blog we can estimate that about 2% of primary care potassium results across the year are mis-categorised as a result of specimen stability issues. If we combine this effect with the systematic shift in results described in the previous blog, we can estimate that around 1 in 10 potassium results from primary care would be significantly different if taken in the emergency department.

The current focus of laboratory accreditation has done little or nothing to address the issues that lead to this, and as a result patients and clinicians are being falsely reassured about the accuracy and validity of results. We can say with confidence that the current standard of practice is leading to patient harm on a large scale. This should be an issue of national importance.

On the systematic difference between ED and GP potassium results

Primary care potassium results show large seasonal variation due to problems with specimen stabilisation. This may mask other underlying reasons for variation. If we plot the distribution of potassium results from ED and primary care side by side we can see what appears to be a systematic difference between the two populations (Figure 6)

Figure 6 Histogram showing potassium results obtained from ED and primary care.

In North Devon, two practices (Black Torrington and Lynton) have centrifuged all specimens on site since 2014. This has resulted in much more stable potassium results. We can see the same shift in mean potassium result in these practices that occurred when the analyser was changed in 2018. However, we also see a large systematic shift in the result between the primary care and ED populations (Figure 7).

Figure 7 Mean monthly potassium in ED compared with 2 practices that stabilise all specimens at point of care by centrifugation. Note that in this analysis we only looked at “first time tests” by removing any test that was repeated on a specific patient within 3 months

Impact of demographic differences on potassium result

One explanation for this result is that the people tested in ED are very different to those tested in primary care. The following population pyramid shows the differences in the demographic groups that are tested in ED compared with Lynton and Black Torrington (Figure 8).

Figure 8 Population pyramids showing demographic differences in people having a potassium test in the ED compared with Black Torrington / Lynton GP practices.

To examine whether these demographic differences could account for the systematic difference in potassium results, we looked at the results of first time testing (i.e. ignoring repeats within 3 months) in specific demographic groups. For simplicity, we looked at just the potassium results before the analyser change, as this gave us the largest dataset. We can see that the difference in mean potassium exists across all demographic groups (Table 4).

Table 4 Difference in mean potassium between ED and primary care in different demographic groups.

Impact of disease severity on potassium result

Patients in the ED who have a test are more likely to be acutely unwell than patients in general practice. In an attempt to determine whether the difference in potassium might be accounted for by differences in clinical reason for the test, we repeated our analysis looking at different groups that we could identify either through the clinical details on the request form, or from their testing history. The comparator group might be expected to contain patients who were more likely to be acutely unwell (Table 5). We can see no significant difference between the patient groups and their comparators, and conclude that disease severity is unlikely to be a reason for the systematic difference in potassium between the ED and primary care.

Table 5 Difference in mean potassium between patients in different clinical groups, as identified from clinical details on request form.

Location of patient group	Patient group	Comparator	No of patients in group	Number of patients in comparator	Mean K in group	Mean K in comparator	Difference
GP	Blood requested for chronic disease monitoring (asterisk in clinical details)	Not requested for chronic disease monitoring	667	3978	4.68	4.59	-0.09
GP	Patient has no repeat test within next week	Patient has repeat test in hospital within next week	162161	1381	4.46	4.51	0.05
ED	Patient has no repeat test within next week	Patient has repeat test within hospital within the next week	13996	1017	4.21	4.24	0.03
ED	Patient has no repeat test within next week	Patient has repeat test on ICU within the next week	13996	241	4.21	4.22	0.01

What does this mean for patients?

We cannot account for this systematic difference in potassium results between the ED and primary care. It does not appear to be due to demographic or disease differences. It would seem to be unlikely to be due to specimen stability as we see the effect in locations where specimens are stabilised at the point of draw and where results appear stable across the year. Both ED and primary care use the same equipment manufacturers and specimens are run on the same analysers.

Nonetheless, we can estimate the impact of this difference on the rate of abnormal results. The following table shows the proportions of patients who fall into different potassium result groups (Table 6).

Table 6 Proportion of potassium tests in different result categories comparing ED and the two practices that stabilise all specimens by centrifugation

From this analysis, we can work out the number of patients in primary care who would be placed in different categories if all practices stabilised specimens and if analyser performance was the same as that seen for ED specimens (Table 7).

Table 7 Predicted annual number of patients who fall into different result categories using actual primary care mean compared with theoretical mean derived from ED results (based on 124,000 tests per year in primary care).

Conclusions

We will talk in a subsequent blog about the impact of specimen stability on potassium results. This blog looks only at the clinical impact of a systematic shift in result depending on the location of where the blood has been taken.. We can estimate that about 1 in 12 potassium results taken in primary care would be in a different result category if the test was done in a different location. We do not understand the reasons for this but clearly it is important to study this further, and to understand whether this is a phenomenon seen in other regions.

Bias in laboratory results – how we can see this in real patient data and the correlation with External Quality Assurance

In this blog we will consider just the mean potassium result for samples taken in the ED, plotted here in a statistical process chart (Figure 3).

Figure 3 Statistical process chart for mean monthly potassium taken in the ED. Mean results pre and post analyser change are shown (dotted lines). Results within 2 standard deviations of the mean prior to the analyser shift (solid line) are shown in the shaded area

We can see a shift in the ED mean potassium in March 2018. In the language of the SPC, the system went “out of control.” This correlates with the laboratory switching its analyser provider.

Correlation with EQA performance

We can look at the performance of the laboratory in the external quality assurance scheme over these periods. The following plot (Figure 4) shows the bias of different analysers in this scheme. This laboratory moved from the 13OL, which has a strong negative bias, to the 13BO analyser, which has a positive bias. It is not therefore surprising that we saw the shift in laboratory mean when the analysers were changed.

Figure 4 Bias of different analysers in NEQAS potassium returns.

The impact of the shift in laboratory performance on patients

Although potassium results essentially form a continuous distribution, clinicians tend to view these as categorical results (see blog 2 Table 1). This means that it is not that meaningful to look at distribution parameters if we are to understand the clinical impact of changes. Instead, we need to look at how this impacts on the proportion of results that fall into different result categories

Using theoretical “normal” populations to model the analyser effect

To analyse this, we applied a theoretical distribution to our data (using a technique called Kernel Density Estimation; Figure 5). We then calculated the impact of the shift in the potassium mean (due to the analyser change) on the expected numbers of patients who would fall into each category.

Figure 5 Histograms of potassium results from the ED before and after analyser change, with theoretical distributions as determined by KDE analysis.

We can use these distributions to calculate the proportion of results falling into different result categories before and after the analyser switch (Table 2).

Table 2 Proportion of ED potassium results falling into different result categories before and after analyser change in 2018. (Note the normal range is from 3.5 to 5.3 inclusive)

We can then apply these proportions to the number of patients that are tested in our ED each year (approximately 27000 tests;Table 3). In our small ED, 2 patients a day would have been diagnosed with hypokalaemia on the old analyser, but would be normal with the current analyser. 3 patients a week are diagnosed with a significantly higher potassium since the switch to the new analyser.

Table 3 Predicted change in annual number of patients in different result categories before and after analyser change.

Conclusions

We are making no value judgement about which analyser is correct. This blog merely notes the effect that laboratory performance has on patient outcome – 3% of ED potassium results are significantly altered (i.e. change category) merely by changing the analyser.

Setting aside wider considerations of normality and uncertainty, there are mechanisms to remove these biases in laboratory performance. For instance, real time inter-laboratory comparison of performance with continual adjusting of parameters can ensure that result distributions remain constant.

For the non-biochemist, it is hard to understand how different analysers can produce different results when, according to the ISO15189 standard to which they all comply, all measurands must be traceable back to an international standard. These are clinically important uncertainties that are perhaps under-recognised.