Overdiagnosis and reference ranges ... again.

Here are our reference ranges for adult patients for a number of common analytes

	Low	High
Na	133	146
K	3.5	5.3
ALP	30	130
BIL	0	21
ALT	0	33
ALB	35	50
Prot	60	80
Visc	1.5	1.72
CRP	0	5
TSH	0.27	4.2
Hb	115	160
WBC	4	11
Plt	150	400

Looking at an adult primary population, the percentage of specimens with results outside these ranges are :

	Low	High
Na	3.2%	0.5%
K	1.2%	3.4%
ALP	0.3%	8.0%
BIL	0.0%	3.5%
ALT	0.0%	13.3%
ALB	2.0%	2.4%
Prot	1.3%	2.1%
Visc	1.7%	31.0%
CRP		34.0%
TSH	5.0%	13.0%
Hb	10.0%	8.3%
WBC	4.4%	5.6%
Plt	6.9%	4.2%

Now it is certainly true that this mixes up the pathogenic and the non-pathogenic. But it is surely also true that for most common tests that are carried out in primary care, most will be "normal". Or at least will be giving us very little diagnostic information. If a high proportion of the results are abnormal then the balance between sensitivity and specificity is almost certainly wrong. Do we really know the benefit of labelling a substantial proportion of a population abnormal? This appears to be a particular problem for tests we know are hard to interpret : ALT, ALP, TSH, inflammatory markers. With evidence that laboratory testing is helping drive an epidemic of overdiagnosis.

These are what the reference limits would be if we were to make just 5% of population "abnormal"

	Low	High
Na	133	144
K	3.7	5.3
ALP	44	189
BIL	0	23
ALT	7	70
ALB	36	49
Prot	62	80
Visc	1.51	1.99
CRP	0	85
TSH	0.08	8.89
Hb	96	161
WBC	3.69	13.1
Plt	122	444

Clinically this makes more sense. I have learned to ignore ALTs below 70. I don't worry about CRPs below 100. This is not to say that patients with an ALT below 70 don't have liver disease, or that a patient with a CRP below 85 doesn't have an inflammatory condition. This is why we still need clinical skills. It's just about thinking about when the test alone should prompt action - and this is how I think the reference range should be used.

The haematology results here are interesting. The platelet and white cell cut-offs suggested do fit better with clinical feeling. Not sure if there is evidence on haemoglobin. Because this is what we really need - what is the predictive power of a test for specific pathologies?

Creatinine shows a way forward here. The absolute clinical value of creatinine is low. But acute kidney injury flags based on relative changes feel clinically useful. 2.6% of renal function tests from primary care generate an AKI flag. This then might be a better way of thinking about normality. But unless we are very sure we know the benefit of reference ranges that medicalise large swathes of the population, I will continue to argue for wider limits.

ALT reference ranges

ALT is a frequently performed blood test, and is a marker of liver cell damage.  However, many patients with liver damage have a normal ALT. So what is a logical approach to setting cut-offs for ALT? And let's get real - we know (because we've studied it) that most clinicians see reference ranges as normal ranges, and so hence results outside reference ranges are seen as abnormal.

As a clinician, given that we know that a "normal" ALT does not rule out liver disease, what we probably want to know is "what level of ALT is clearly abnormal and needs explanation?" (with the knowledge that for a patient with risk factors for liver disease we will not be relying on liver transaminases as a screening test).

Here's the cumulative frequency graph for ALT for all specimens from primary care for men and women. I have ignored all results above 100 - which is about 10% of all results. This moves into results that are much more likely to come from patients with pathology. So this graph then represents a reasonably "normal" distribution of ALT in our population, with the caveat that obesity rates run at about 25%. 

Looking at different cut off values, we can estimate the percentage of a normal population that would be above this value.

ALT cut off = 35  leads to 13% of women and 18% of men with abnormal result

                    = 40                11% of women and 14% of men

                    = 50                  7% of women and 10% of men

                    = 60                  4% of women and  6% of men

                    = 80                  2% of women and  2% of men

What is the uncertainty of measurement? We can look at people with relatively normal ALTs (12-50) and see what the value was if was repeated on the same day. This can be plotted as a cumulative frequency graph and we can see that we can have about 90% certainty that a result will lie somewhere between +/- 20% of the initial value.

This fits well with the published data (Westgard) which suggests a CVi of 20.

There are some implications of this for clinical interpretation and quality control.

1. We probably shouldn't worry about ALTs until they get to about 80.

2. We need to make sure that our laboratory quality control, which focusses purely on technical precision and accuracy, reflects this much larger total uncertainty in the result. Again the Westgard site is helpful, suggesting we need to make sure that precision is within 9.7%; inaccuracy within 11.5%; and total error within 27%. 

We do not need to be that accurate in measuring ALT - it just does not have the clinical imperative that we might see for things such as potassium and creatinine. I want clinical results that reflect this.

As an aside, here's a graph of ALT distributions across the years, and it is easy to spot the analyser change in 2018. I do not understand why this problem cannot be sorted out by standardisation to common reference material; or just through EQA schemes. This just creates confusion and seems both overly scientific in a failure to accept clinical reality; and under-scientific in that it clearly does not relate the measurement to a stable gold standard. 

CRP - the most overused and overrated test of all time?

Most days it feels like half the calls I get as a consultant microbiologist are from junior doctors who are worried because a patient's CRP has gone up. My cure for that problem is to tell them to stop measuring it. CRP used to be a test that was used carefully. It is now used as part of routine monitoring. There are calls to use it as a rule out test for sepsis in primary care. So what is the performance of this test for detecting significant infection in the real world?

We have looked at the ability of CRP to predict bacteraemia, which is probably a reasonable proxy measure of sepsis. We looked back at all blood cultures sent to our lab between Jan 2011 and August 2017. This identified 36396 sets of cultures, of which 32703 were negative, 1210 grew a likely contaminant, 1800 grew a likely pathogen and 683 grew an organism isolated in a duplicate set. We then linked this data with concurrent laboratory requests for a CRP. The number of specimens with a CRP measured within 7 days were : 29403 (90%) for negative; 1063 (88%) for contaminant; and 1699 (94%) for an isolated pathogen.

Figure 1 shows the day of the peak CRP measured in the episode. We can see that in episodes with proven infection, there is a lag to peak CRP.

fig.1 Day of peak CRP

We then looked at how the initial CRP predicts bacteraemia. Figure 2 shows the cumulative percentage of episodes in each category at increasing CRP cut-offs (in other words, the number of episodes that would be missed for a particular cut-off CRP value). We can see that 30% of cases of significant bacteraemia have a CRP below 50 on presentation, and over 50% have a CRP below 200.

fig 2

The small difference in the curves for the three categories also suggests that initial CRP is not good at discriminating between infection and not infection. We can see this if we plot a receiver-operator curve (fig 3). In a ROC, a test with no value shows a straight line, whereas a good test has a hyperbolic distribution. The initial CRP looks very straight on the ROC. You can draw your own conclusions about the performance of the initial CRP in predicting bacteraemia.

fig 3. ROC for initial CRP in predicting bacteraemia

We then repeated the analysis for the peak CRP measured in the week after the blood culture draw. (fig 4 and 5). So it performs a bit better than the initial CRP, but not a lot.

fig 4

fig 5. fig 3. ROC for peak CRP in predicting bacteraemia

So this is my main conclusion :

It is not possible to find a cut-off value for CRP that tells you anything meaningful about the probability of bacteraemia

We can probably also conclude that there is often a delay between onset of serious infection and rise in CRP. So CRP is really bad at telling us what is going on in an acute setting.

Why are people pushing CRP then? I guess when it was carefully performed in selected cases when it was first brought in as a test, it was quite useful. We can see from our data that there is some signal there, but it's drowned out by noise.

I also suspect that in most cases a CRP is telling us absolutely nothing, but we are using the result to justify our decision making. When it suits us and it confirms our prejudice. Of course, the problems come when there is a mismatch between these two things. People cope with this dissonance in many ways. The most annoying of which is to phone the microbiologist. The most dangerous of which is to make an inaccurate diagnosis.

Antibiotic resistance is a distraction in community onset sepsis

1. Sepsis usually presents rapidly. Most patients with sepsis know that they are seriously unwell, and go straight to the ED, without presenting to primary care in the preceding days. Sometimes this involves a telephone triage by 111. Occasionally there is a prior review by a primary care doctor. This process appears to work well.

2. There are occasional cases which present more insidiously to primary care in the days preceding onset of sepsis.

These are harder to recognise and do not fall into patterns that are easy to define. Here, clinical judgement is crucial and should be valued and supported. Protocols for recognition of sepsis do not seem to translate easily into primary care, and we should be wary about adopting guidance in which secondary care based assessment protocols have been adapted according to expert opinion with no evidence base. Local practitioners felt that following rigid protocols in primary care could lead to overdiagnosis of sepsis, particularly in self-limiting viral illness. However, they also voiced concerns around medicolegal issues that may result from not following these protocols. Our cases also show that sepsis presents in ways that may be missed by secondary care protocols. Our cases suggest some features of severe infection that practitioners should be aware of when assessing patients in primary care, such as change in mental state and history of rigors. Current body temperature seems unhelpful. These observations fit well with recommendations in some scoring systems (eg. qSOFA). There may be a need for better rapid diagnostics to support clinical decisions. However, we noted that 19 of the 50 patients had a CRP below 100 on admission to the ED. We also saw that white blood cell counts were frequently normal. We did not examine the potential benefit of other inflammatory markers (eg, procalcitonin). There may be constellations of common tests that add value to the diagnostic and therapeutic processes. For instance, 12 patients had evidence of acute kidney injury on admission. This is likely to be important in determining severity and prognosis.

3. We need to recognise the importance of good quality care for chronic disease. 

eg. Diabetes, ulcers, catheters. Good management of these conditions is likely to reduce the incidence of downstream problems, including sepsis. We noted that care of these conditions is becoming fragmented, falling between different agencies (eg. doctors, community nursing, practice nursing, specialist nursing, care homes) and we often saw a lack of joined up care planning. In addition, despite the high prevalence of these problems, there is often a lack of evidence on which to base decisions (for example, prevention and management of catheter blocking). We may also need to consider more innovative models of care delivery for these patients. For instance, “leg clubs” have been set up for patients with chronic leg ulcers to encourage mutual support . Such psychosocial approaches may be more effective in persuading patients who struggle with compression to persevere with a treatment that is effective but uncomfortable.

4. There are a group of recurrent /chronic infections for which we lack good guidance on management.

Eg. UTI in men / recurrent UTI / cellulitis in ulcers / vaccine preventable pneumococcal infection in chronic lung disease. We need to develop new approaches to these. Some simple key primary care messages may help, for instance, around appropriateness of patient directed self-start antibiotics in recurrent UTI 7 and clearer guidance on when patients should be referred for investigation of underlying problems, such as urinary retention. A further problem in these cases is loss of continuity of care. When episodes of infection are relatively infrequent, we saw that each episode tended to be treated in isolation. A loss of continuity of care has been identified as a factor leading to hospital admission. We need to find ways to compensate for these changes to healthcare delivery, and, for instance, laboratories may be well placed to identify and alert carers to emerging patterns.

5. There is an ongoing need to ensure that microbiological diagnostics are aligned to clinical need.

Currently, diagnostics are significantly compromised by the poor quality of specimens that are received in the laboratory. The importance of this is illustrated by a case in our series of Group B streptococcal infection from an infected ulcer. Laboratory standard procedures target S. aureus and Groups A, C and G haemolytic streptococci from superficial wound swabs. It is hard to determine what is pathogenic in these specimens, which are often of low quality and lack clinical details, and this can lead to overtreatment of colonising flora. Improved swab methodologies (eg. debridement prior to sampling) may enable better discrimination between pathogens and colonising organisms.

6. Antibiotic resistance (in this series at least) was a minor reason for development of sepsis. 

We saw four cases of potentially avoidable admission in which the organisms were resistant to empirical (guideline) antibiotics. However, in two cases, it is extremely doubtful that antibiotics in the community could have altered the natural history, and in all cases it was hard to imagine alternative and reasonable empirical options in the absence of specific sensitivity data. We need to consider how diagnostics can support early detection of important resistance in primary care.

Laboratory accreditation as a lever for adding value to pathology services

Antibiotics and sepsis: the art and science of polarity management

With thanks to Jim Mather of the University of Stirling for introducing me to Polarity Management by Barry Johnson.

Some things in life require choices between different options. These may be complex, but once made, the world generally moves on

• should I apply for this job?
• should I choose this school for my children?
• should I start a beta-blocker to reduce blood pressure in this patient?

But there are other things which do not lend themselves to this sort of decision making. There exists on ongoing tension between polarities.

• should I spend more time doing clinical work/more time in a leadership role
• my children should be independent / we should do more as a family
• we should use antibiotics earlier in infections to stop serious consequences/we should hold back on antibiotics to prevent resistance and other problems of over treatment

These problems don't lend themselves to simple decisions. Rather they are ongoing tensions to be managed continuously. This is, unfortunately, not what tends to happen. What we see are people endlessly embroiled in attempting to solve problems at one end of the polarity or the other. How can I reduce antibiotic resistance? How can I improve the time to antibiotics in patients with suspected sepsis? And in doing so tending to lose sight of the opposite polarity. This often leads to entrenchment in negative positions, and disharmony between those with opposing world views. Alternatively, we see short term shifts towards one pole, and then lurches back the other way. Tension becomes unproductive swings in approach.

This does seem to be what is happening with the sepsis / antibiotic stewardship agendas at the moment. A few years ago, antibiotic stewardship dominated the agenda, thanks mainly to the rise of C. difficile. This has contributed to the huge successes in reducing incidence of this condition over recent years. But there has always been the potential for a backlash to this approach, with clinicians feeling constrained in their ability to prescribe what they want.

More recently, there has been a strong push to improve management of sepsis. Most hospitals will have set up teams to study sepsis. They report back that we need to give antibiotics earlier. "Yes, we think antibiotic stewardship is important, but what is really important when we think about sepsis is giving antibiotics quickly." Antibiotic stewardship is seen as a worthy thing, but not actually relevant to the patient I have got in front of me.

We can see the same tensions when we consider this problem from the perspective of the patient. On the one hand, I want to get better from whatever it is that is making me ill, be that a sore throat or life threatening sepsis. If an antibiotic makes me get better 24 hours earlier, or saves my life, that seems like a good thing. On the other hand, I am being told that using antibiotics could make future infection harder to treat, may have side effects, and may even cause problems through their effects on 'good' bacteria.

Polarity management I think has something to teach us about how we approach these problems. They can never be solved. Clinical decisions regarding antibiotics (or anything really) are rarely clear cut. Rather they exist in this tension between doing things now, and doing things a bit later (or not at all). Polarity management takes us through a few steps that are perhaps useful in managing this tension.

First we define four quadrants. On the left we have "Now", and on the right we have "Later". At the top we have positives, at the bottom we have negatives. It is usually pretty easy to fill these in. So for example, for antibiotics we might have :

Benefits of early treatment Get better quickly Prevent complications Early discharge Clinician engagement with prescribing decisions	Benefits of delayed treatment Reduce antibiotic usage Protect 'good' bacteria Reduce C difficile rates More standardisation of infection management
Harms of early treatment Overtreat More C difficile More resistance More collateral damage 'Random' approaches to antibiotic prescribing	Harms of delayed treatment Undertreat Delayed recovery More complications More serious infections Less clinician engagement

So what we often see happening in systems that have become focussed on a polarity is that people who see the opposite polarity in a particularly positive light start to identify with the negative dominant quadrant (so the 'antibiotic steward' sees the overtreatment of infection leading to antibiotic resistance). There is an increasing clamour for change. The harms of the current approach become self evident, and the system then shifts (more control on antibiotic use). Setting the cycle in train again from the other perspective (delayed treatment and clinician disengagement). Or, alternatively, it doesn't shift, and people become increasingly entrenched in their negative views of the world.

What polarity management does is really very simple, and I think has 2 main effects. First, by making the upsides and downsides of poles explicit, everyone can see that there are tensions that need to be managed. But they know that their voice is being heard. And second, there is now a route map that people can be taken on to explore the two poles. Rather than become boxed into a corner, they can be taken on a journey to the other side of the spectrum. What often works is to start off in the negative quadrant of the pole with which you most identify (eg. too many antibiotics being used). Of course, it is then very easy to think of all the upsides that counter these (moving to the upper quadrant. eg. early treatment of sepsis). Now you ask people to think about the negative things that the alternative approach would have (opposing quadrant. eg. delayed infection management). Again, this is pretty easy for them to do - this is, after all, why they like their approach. But finally you ask them to think about the final positive quadrant (eg. more standardised approaches). Because they feel safe - their voice has been heard - this is not now a dangerous place for them to be.

What happens when we do this for sepsis? I think we end up in a different place. It becomes easier to see the world from multiple perspectives, but, perhaps most importantly, we start thinking about individual patients a bit more. What are timely antibiotics in a patient with a rather undifferentiated presentation which might be due to infection? What are the most appropriate antibiotics to use in someone with a crackles on their chest and a fever, but a normal chest X-ray? And we start thinking perhaps a bit more clearly about how we can probably never resolve these into right and wrong answers, but become a bit more sophisticated in managing and enjoying the tensions inherent in complex medicine.

Antibiotic stewardship - a methodology

After a very productive meeting in primary care today talking about antibiotics, here are some new thoughts on what I think we need to do to improve antibiotic stewardship.

The learning environment is important.

1. Groups of about 15 seem good. A mix of GPs, nurse prescribers, pharmacists with access to data, and 'infection experts' seems to work well.

2. Clinical leasdership is important. This is not about being heroic. It's about having someone stand up and say "This matters, and as a group we agree this matters". It is important that this sentiment comes from those 'doing the work'. Experts need to be invited in, as part of the dialogue, not as the solution. Without this, most education is perhaps verging on the pointless. I'm sure there is a literature on this.

3. All participants need to feel 'safe' and able to contribute to any discussion in a non-judgemental way

Data and cases get you started and can build trust and common purpose

4. Practice data is a good way to start any discussion. Continuous ranking data relative to peers is probably a better way of tracking changes in prescribing habits than actual items per STAR-PU. The latter is too sensitive to seasonal and secular trends, and other external forces (eg. guidelines).

Graph showing rolling 12 month average rank of one practice vs peers

5. Individual prescribing data is a good way to continue discussion. GP prescribing systems allow this to be cut in many ways - by sex, age, type of antibiotic, time of week, phone vs face to face. This is a good way to start exploring beliefs. Be very careful about drawing any conclusions at this stage. People will get defensive if you are not careful. You just want to get interest at this stage, There are many valid reasons for variation in prescribing patterns. There are also many ways to explain away variation - you need to be careful!

6. Serious untoward events, or other 'interesting' cases, or a good way to 'reground' the discussion in 'what matters' to the patient.

Using expert opinion to guide discussion

7. The role of the 'expert' is perhaps just to facilitate discussion. Frankly, those 'doing the work' are far more aware of what the issues are.

8. The 'expert' must provide information that is salient. So for GPs these are things like : "What will this do to my workload?"; "Is it safe?"; "What will my patients think?"

9. The 'expert' can help define the unknown knowns, the known unknowns and the unknown unknowns. A lot of primary care happens in this space - and exploring it can be a rich experience for all.

So it feels this is where we got to today. The next step is to form a smaller group of nurse prescribers and GPs and do some case reviews. I think we need to focus on a specific topic. For instance, why is flucloxacillin prescribing so seasonal? Are people misdiagnosing the erythema of insect bites as cellulitis? Or are there other reasons for seasonality that we haven't thought about? Or, why are we prescribing so many antibiotics to women aged 20-39? Can we find better strategies for helping them manage without antibiotics?  Or do we feel particularly sorry for harassed young mothers, and just want to 'do something', even though we know it won't help?

I think we need to invent a structure for this peer-led case review that is helpful for driving improvement. I think this could take the same structure we are using to drive improvement in diagnostics (see other blogs). It has to avoid being process focussed, it needs to reflect the real uncertainties in this area, and it needs to be a method for promoting improvement rather than compliance. Something like :

Clean start

1. Do we think antibiotics were indicated? (Almost definitely / probably / possibly / almost definitely not). 

2. Do we think the antibiotics were appropriate (ie treating likely pathogens) - (Almost definitely / probably / possibly / almost definitely not).

3. Do we understand the 'red flags' for this condition and did the patient know these?

Clean continuation

4. Do we think diagnostics were used optimally? 

5. Do we think antibiotics were given for a reasonable duration (Almost definitely / probably / possibly / almost definitely not).

6. Do we think antibiotics should have been modified at any point? (Almost definitely / probably / possibly / almost definitely not).

Clean stop

7. Do we think the patient knows how to manage this condition in the future, if appropriate?

8. Does the patient know what they have to do to prevent recurrence?

9. Have we done everything to prevent recurrence (eg. vaccinate)?

Not sure how contrived they are, or whether there are bigger things, but structure and method for doing this feels important. We then have measures that we can use to experiment with improvement. For instance, does having better guidance for patients and their carers make any difference to any of these measures?