Urinary Tract Tnfection : the Clinical and Financial Harm of Sub-optimal Testing and How Clean In, Clean Through, Clean Out Improves Care

In a previous post, we discussed how the methodology of "Clean In, Clean Through, Clean Out" provides a mechanism with which to deliver increased effectiveness of pathology services. In this post we will show how it has been used to study urinary tract infection and has resulted in delivery of more purpose, at less cost, with less harm and more capacity.

Trends in submission of urine specimens

The rate of submission of Mid Stream Urine samples (MSUs) from primary care has showed an inexorable year on year increase (fig.1). This is a pattern seen for many diagnostic tests.

fig 1. Number of urine samples received from primary care in North Devon 2002-2009. We looked at September only as it is the most stable month for requesting (no Bank Holidays etc)

What is the cause of this increase? Is it an ageing population? Or has clinical decision making changed? We do know that testing patients in whom it is not indicated is not benign.

Some consequences of sub-optimal urine microbiology

A patient harmed by unnecessary urine culture

JM was a well 67 year old woman who attended her surgery for a routine check up for hypertension. She was asked to bring a urine sample, which was dipped for protein. The specimen showed 1+ for leucocyte esterase and the healthcare assistant submitted the specimen to the laboratory for culture. No clinical details were entered on the request form. The laboratory isolated a pure growth of more than 10^8 organisms / litre of E coli which was resistant to trimethoprim and nitrofurantoin but sensitive to co-amoxiclav and ciprofloxacin, and this was reported to the GP. The GP had never seen the patient, but presumed that she must have a urinary tract infection on the basis of this result (although there was no clinical evidence of infection). As she was allergic to penicillin, the GP prescribed ciprofloxacin. One week later, JM returned to see her GP with severe diarrhoea. A sample was sent to the laboratory and Clostridium difficile was detected.

Unnecessary antibiotics given because of unnecessary urine culture

We looked at 40 patients from primary care with a pure growth of E. coli. 30 of these patients had symptoms of infection and were already on antibiotic treatment. 10 patients had no symptoms of infection, but had had samples sent for just the reasons described above (ie. positive dipsticks on routine screening for other conditions). All these patients were prescribed antibiotics as a result of the test. This is clear harm to the patient driven by inappropriate testing. 

It is also worth considering the effect of abnormal results on the GP. The abnormal result will need to be considered when it arrives back. The choices are to ignore it or to act. Ignoring a test is not trivial, and comes with an opportunity cost - there has to be a conscious decision not to act, or there is a danger that abnormal results performed for good reasons may be erroneously ignored. This is an ever increasing problem with the inexorable rise of testing. And as we have seen the decision to act can be even more problematic - not only does it take time to 'do something', there is a very real risk that this action will lead either to direct harm to the patient (eg. antibiotic prescribing), or more insidiously perpetuate further work in the system ('failure demand') - bad for the patient and bad for the sustainability of healthcare.

We applied our definitions of points of leverage to study and act upon the problem.

Improving Clean In for urine microbiology

1. Were tests necessary?

We looked at the request details of individual specimens being sent to the laboratory. We saw a large number of requests did not include important relevant clinical information, but instead were either blank, or contained information pertaining to a urinary dipstick (fig 2). It is worth noting that urinary tract infection is generally a clinical diagnosis that the laboratory investigations can only support - so it is hard to know what "?UTI" means.

  

 fig. 2 Clinical details on request forms submitted with urine specimens from primary care 2010.

  

We asked why these tests were being sent and to do this went to individual case records in primary care. What we saw is that tests were being ordered by health care assistants who were seeing patients for chronic disease reviews and that much of the new demand appears related to the introduction of primary care quality targets for monitoring of chronic kidney disease.

Assessment of kidney damage in these patients requires an assessment of the amount of protein excreted in the urine. A quick way of doing this is to use point of care urine dipsticks.  Many dipsticks provide multiple additional tests - irrelevant in the management of chronic kidney disease, but of possible utility in other situations, such as determining likelihood of urinary tract infection in patients with vague symptoms of this condition.

And here we see the rub - the dangers of testing that is not aligned to a clinical question. These patients with chronic kidney disease who are presenting for routine 'check ups' (and so in whom we would not be worrying about infection) often give positive dipstick results for nitrites and leucocyte esterase - which are associated with infection. However, dipsticks have a high sensitivity but a poor specificity for diagnosing infection (ie. a negative test is good at ruling out infection, but a positive test does not diagnose infection). But, perhaps not wanting to ‘miss something important’, the urine sample is sent to the laboratory to look for bacterial growth - traditionally seen as the gold standard arbiter of infection.

There is now a second misunderstanding related to the gold standard nature of urine culture (interestingly perpetuated in much of the literature on urine infection). Clinicians are often taught that a growth of 10^8 organisms per litre is diagnostic of infection. The original work by Kass on this subject did not claim this - rather it showed that a pure growth of this magnitude was predictive of reproducibility of this growth in a research setting. This is most definitely not the same as a diagnosis of infection in a routine clinic setting. The incidence of asymptomatic colonisation of the urinary tract is high - and there is good evidence that this should not be treated with antibiotics, except in well defined populations (eg. pregnant women). In addition, the quality of urine specimens is often poor from routine settings, with samples frequently contaminated with perineal flora. There then may be substantial delay getting these specimens to the laboratory, and specimen stabilisation is never absolute.  The consequence of all this is that the performance of urine culture in routine practice is significantly worse than that seen in the research setting. To give some idea of the magnitude of this problem, all newly pregnant women are asked to send a urine specimen for culture (as this is a group in whom asymptomatic bacteriuria is a very high risk for causing pyelonephritis). Our laboratory requests repeat samples from all pregnant women if they have no symptoms but a pure growth of organisms. We find that this pure growth can only be reproduced in one third of patients. In other words, the specificity of a pure growth of bacteria for diagnosing UTI is very poor. As with all tests, the prior probability of the condition being present is key when trying to interpret a test result, be it the result of a point of care test (eg. dipstick) or a laboratory test (eg. culture).

Acting on “Clean In” as a point of leverage for improving the use of urine culture

It is easy to see now that acting upon the “Necessary” component of Clean In is a key point of leverage for improving the effectiveness of urine culture as part of the informed management of urinary tract infection.

To do this, we went to the source of the problem. We talked to Health Care Assistants (HCAs) about the way they work and what would help them to be better. We heard that they often work with guidelines that are partially conflicting. We heard how HCAs take great pride in what they do and are keen to ‘do the right thing’. We heard how noone had talked to them about possible harms of inappropriate testing.

To address these issues we worked with HCAs to design algorithms for the management of urine dipsticking. We came up with simple visual cues that distinguish between specimens that are for diagnosis of infection (red top tubes containing borate for bacterial stabilisation) and specimens that are for diagnosis of kidney damage (white top tubes). Our algorithms resolved or clarified areas of advice that could have been seen as in conflict.

We introduced this guideline into a pilot practice and saw an immediate and dramatic effect on urine culture submissions from 140 per month to just 7 per month. We were worried that a consequence of our advice had been that clinicians were not sending urine for culture in patients with infection. In other words, it is possible we had had a negative impact on the “Maximally Appropriate” aspect of Clean In. In an effort to improve the “Sufficient”and “Maximally appropriate”  aspects of urine submission, we promoted national guidance on the management of urinary tract infection, which includes advice on when to submit specimens for laboratory analysis. We saw specimen submissions rise again in the pilot practice to around 50% of the previous peak and the quality of information provided also improved.

With this clear success, it was easy to roll out the programme across North Devon. We promoted the dipsticking algorithm, the guidance on management of urinary tract infection and a distillation of our finding as a series of “Top Tips”.  We visited all practices and spoke to doctors and HCAs. We gave presentations at regional education days. As a consequence of this work, we have seen a sustained reduction in urine specimens submitted for culture from primary care (fig 3) and an increase in the quality of the information provided (fig 4).

fig 3. Number of urine samples received from primary care in North Devon 2002-2015

fig 4. Clinical details submitted with urine specimens from primary care

Clean Through - Using improvements from Clean In to improve Specimen processing

As a result of our work improving Clean In we saw a number of benefits in the laboratory. First, we had more time to redesign processes against purpose as we were doing less work (which had no purpose). Second, because we now knew that specimens were likely to be of clinical importance, we could easily see the value the laboratory was contributing to the management of these patients, and ask ourselves “Are we doing all we can to help?”. “Clean Through” has proved to be a useful framework.

What does it mean to process a urine specimen without error?

Laboratories have many ways of controlling processes to reduce error. Some of these are picked up as part of the accreditation process under ISO15189. How do we explain this to clinicians and patients in a way that they would find reassuring? Internal and External Quality Assurance (IQA and EQA) should be designed around answering this question. It is reassuring to know that a lab has performed well in an EQA process that is designed to test its ability to analyse accurately specimens that are relevant to a specific case. This could, in principle, form part of a pathology report. For instance, we would be in the position to add a comment for urine microbiology that states our confidence in our ability to identify correctly an organism, and to report accurately a set of antibiotic sensitivities.

EQA schemes are important ways to test and challenge performance against known standards. However, these are readily identifiable specimens and an accurate result does not necessarily imply that this would be achieved in a real life clinical context, To answer this question, we need a well designed IQA process. For urine microbiology. we think this must involve running specimens in parallel, ideally in a way that means they are blinded to all people involved in the process. If this is built into the routine working environment then we can easily spot variation.

However, microbiology is a dirty science. It is hard to get perfect specimens in a real life clinical setting, and microorganisms do not always behave in an entirely predictable way. As an example in urine microbiology, a substantial number of specimens contain mixtures of bacteria. Traditional teaching is that this represents contamination and that the samples should be rejected. However, when we look at patients who are bacteraemic from urine infections (in other words the bacteria have spilt over from the urine into the blood) we see that about 20% of patients have a concurrent urine that has mixtures of organisms in it. When we look at this mixture, we can see important clinical information that can be elucidated quickly without little work. On direct antibiotic sensitivity testing, it is obvious to the scientist reading the plates which antibiotics are likely to be good bets, and which should be avoided (fig 5).

Fig 5. Reading mixed sensitivities. This specimen from a patient with urinary sepsis contains at least 3 organisms. Some, but not all, look resistant to gentamicin, so we would use this antibiotic with caution. Most organisms look resistant to amoxicillin, so we would avoid this antibiotic. All organisms look sensitive to meropenem, and this would be a good choice if the patient had ongoing sepsis.

The use of mixed culture sensitivity in a patient with severe urinary sepsis

GB was a 82 year old man with a long term urinary catheter due to urinary retention. He was admitted to hospital with a high fever and low blood pressure. A diagnosis of urinary tract infection was made and blood cultures and a catheter urine specimen were sent to the laboratory. The patient was given amoxicillin plus gentamicin and resuscitated. The following day the urine plates had at least 3 different organisms on them, at least one of which was resistant to gentamicin. All isolates looked sensitive to ciprofloxacin. The patient was reviewed by the consultant microbiologist on the medical assessment unit. He still had a high fever and had an episode of rigors during the night. His blood pressure had stabilised but he looked unwell. The patient was switched to ciprofloxacin. Later that day his blood cultures became positive with a gram negative organism. This turned out to be a gentamicin resistant, ciprofloxacin sensitive organism. The patient made an uneventful recovery and was discharged 2 days later after a catheter change.

Error defined in a way that is relevant to patient care

The trouble for laboratories is that it is hard to standardise these processes, and under rigorous scientific conditions it would be appropriate to reject the specimen and start again. The trouble for the patient and their carers is that infection is a time limited condition - they do not have days in which to get the perfect result. What is needed is a result that is clinically helpful in a timeframe that is clinically relevant. Laboratories have a choice when things do not behave perfectly. They can reject them, or analyse them in great detail. The first of these does little to help the patient, and the second takes time and is likely to be costly. A third way is to explore ways to report mixed cultures in a way that is helpful, without overstating the certainty of the result and the potential for error. In our laboratory, we have designed new methods that allow the laboratory scientists to convey what they see on the plates to the consultant microbiologist on the preliminary report. The consultant then takes this information and adds a comment on therapeutic options based on the clinical details given by the requesting clinician. We will see the benefit of this approach in terms of reporting and timeliness below.

What does it mean to process a specimen with known variation?

Variation in diagnostics is well recognised, and yet is rarely considered when results are applied to clinical practice. Even microbiologists will see an antibiotic sensitivity result in a report and treat it as incontrovertible truth. So how confident should we be in a result? What is the variation that is intrinsic to the analytical process? What is the variation that is extrinsic to the analytical process (including, for instance, variations in expression of resistance genes or host excretion of antimicrobial factors)? Laboratories are most concerned with minimising intrinsic variation, but to a patient and their carer all uncertainty is common.

We have started to use our IQA processes to examine uncertainty in antibiotic sensitivity testing. We process test specimens in duplicate and see if we get the same result. Sometimes we see special cause errors (for example, an  E coli  reported as a Proteus) and this leads us to examine how they can be prevented (see above). But other times we see common cause, predictable variation in results. About 10% of antibiotic sensitivities can change between sensitive and resistant. This is a somewhat surprising result at first pass. But when we examine the science behind sensitivity testing we should not be too surprised. In controlled settings, antibiotic zone sizes can vary by up to 10%. When an organism can change between sensitive and resistant based on zone size differences of just 1mm, it is no surprise that there is variation. This begs the question of what this means at a clinical level. As a clinician, how do I interpret sensitivity results? This is difficult terrain, but it must start with an honest dialogue about what our results really imply.

What is ‘On time’ for urine microbiology?

The issue of timeliness depends to some extent on the question being asked, If we are asking “Does this pregnant woman have asymptomatic bacteriuria?” then there is little need to issue a result within 48 hours. If we are asking, “Does this patient with severe urinary sepsis have an organism resistant to the antibiotics I am going to give?” then the timeframe is clearly much shorter. In an ideal world, we would want a result there and then. But given that it is possible to have preliminary sensitivities from a well inoculated agar plate within a few hours, then we should ask perhaps why this is not done more frequently. A more common question in primary care is “This patient has pyelonephritis. I am going to treat them with empirical antibiotics, and then review them tomorrow. If they are not better, I want to know if it is because I have chosen an antibiotic to which the organism is resistant, and if so, what would be a better choice?” In order for this to happen, the specimen must arrive in the laboratory on the day it is taken and be plated up with direct sensitivities. It must be read, interpreted and validated the next day.

In North Devon, one of the consequences of improving Clean In was a substantial reduction in workload that meant we were able to experiment with ways of improving the number of results released the day after specimen receipt. We looked at the reasons we were not releasing results the following day using a standard panel of six antibiotics for direct sensitivity testing. We found that predictably it was because of coliforms resistant to first line antibiotics, or Pseudomonas, or Streptococci. We showed that by using 12 antibiotic discs on two plates we could get all the sensitivity and organism identification data that was required to release a report. We also showed that manual microscopy was faster than an expensive automated method for detecting pyuria, and we saved £20000 by removing this equipment (which was substantially more than the cost of the additional agar plates that were required). As described above, we also developed ways for dealing quickly and appropriately with mixed cultures.  We also changed laboratory working practices so that specimens received before 10pm were inoculated, and plates were read every day, including Sundays and Bank Holidays. The result of these changes on our ability to release next day sensitivity results is shown in fig. 6

Fig 6. Percent of urine specimens released on the day after receipt

Clean Out - Changing the way results are reported

We are at the beginning of exploring how we act upon the points of leverage identified here. We know that results are often misinterpreted by clinicians, and that this happens in many ways. We know that antibiotics are started when they are not indicated, just because organisms have been isolated; we know that antibiotics are switched on the back of resistance that is not clinically relevant; we know that mixed cultures are interpreted as contamination when the patient has clinical evidence of urinary tract infection. These are complex areas to address.

Making results more helpful

One approach to the problem is to realise that culture and sensitivity reports are often overly simplistic ways of answering the clinical question that has been posed. We have begun to add more consultant narrative to reports in a bid to aid the clinical decision (fig 7).

Fig 7.Percent of urine reports with consultant microbiologist advice

We also see education about management of infection as a key part of Clean Out. Much infection management is based on empirical clinical decision making, and clinicians have to make complex decisions about which antibiotics to give. This must take into account the local and patient-specific antibiotic resistance patterns. It should also consider the potential for harm, including that caused by broad-spectrum antibiotics. We ensure that our laboratory results are matched to this advice. In order to do this, we have produced comprehensive treatment decision aids for clinicians, and back this up with frequent and varied educational events.

Helping results to be understood

We are often told that patients ‘want’ antibiotics, but when we talk to patients we find the truth is more nuanced than this. They really want reassurance and appropriate treatments, they do not want to suffer needless side effects.  Giving patients reassurance is, again, a complex area. We work with social marketing teams in public health departments to design messages that speak to patients and help them make informed decisions, such as the Listen To Your Gut campaign. We now see how our approach of Clean In, Through, and Out complements, or even fulfils, the needs of an antibiotic stewardship programme.

Results that reflect what is normal for me, not normal for the population

MV is a 57 year old female who has suffered with recurrent urinary tract infections since undergoing an enterocystoplasty for interstitial cystitis. These infections were severe enough to prevent her from working. Despite numerous attempts to eradicate these infections (including quasi-experimental treatments such as high dose prolonged combination antibiotics; hyaluronic acid; intravesicular gentamicin) we have been unable to stop breakthrough infections, while inflicting a wide range of serious side effects, from anaphylaxis, to tendon rupture and ototoxicity. The organisms we isolate at each episode are unpredictable, with unpredictable antibiotic sensitivities, and in other patients would often be dismissed as contaminants (including, frequently, non-aeruginosa Pseudomonas and non-haemolytic streptococci). Working with the patient, we have discovered that she is aware when an infection is imminent, and can submit a specimen directly to the laboratory. We are able to plate this as soon as it arrives, and can give preliminary organism identification and sensitivity data within 6 hours. This allows early, rational antibiotic initiation. We have shown that this has reduced the duration and severity of her infections, and that her total antibiotic exposure is markedly reduced. She has returned to work and is able to play tennis again.

We know that there is a significant group of patients, like MV, with complex infections that are ill served by current services. What is ‘normal for the population’ may not be ‘normal for me’. This group of patients do not follow the accepted paradigms and require non-standard approaches. We are increasingly using the laboratory in flexible ways to support early and tailored treatment regimes. Although these are isolated cases, they serve to remind us of our purpose. We are not here to generate results, we are here to help patients and their carers make informed decisions. Sometimes this will be easy, sometimes we will be wrong, but at all times this purpose acts as a guiding principle by which we are willing to be judged.

The impact of focussing on points of leverage on system measures

As we have discussed, these changes have led to clear improvements in capacity of the service to do purposeful work. We have seen significant benefits to patient care at an individual level. Is it possible to see benefits in aggregate data? Unfortunately, the data is often poor, and open to bias. But we can see a number of things falling out of this work :

1. Cost to commissioners has reduced by approximately £200 000 per annum
2. Mortality following gram negative bacteraemia is the lowest in the region (6.1% seven day mortality; vs 7.1% and 9.0% in neighbouring hospitals).
3. Use of antibiotics in North Devon for urinary tract infection has fallen by 24%
4. Carbapenem prescribing has fallen by approximately 50%
5. Admission rates for urinary tract infection have stabilised having being rising at 10-20% per annum.
6. We have been able to absorb the loss of a biomedical scientist without replacement.
7. Consumable costs have fallen by approximately £20000 per annum

Conclusions

Understanding of purpose takes us out of the laboratory and out of the transactional context of the request for testing that the laboratory receives. It takes us to where citizens are and to what difference they want us to help them make in their lives. When we do this we see different and better problems and find different and better solutions. The points of intervention are often not where we traditionally focus in a narrow framed approach.  By identifying simple points of leverage, and by looking from the outside-in (ie from the perspective of the patient, and considering our purpose), we see our work as part of a wider system and have a mechanism for solving the problems that matter to citizens. Any other approach (eg. a focus on targets; thinking from the inside out) risks solving the wrong problems.

Acting at these points of leverage means better outcomes for the citizen, more purposeful work for those working in the system, less demand into the system and therefore more capacity to do even more good things. This creates a virtuous cycle: taking the time to understand and do what is really necessary, creates more time to understand and do what is really necessary.

In subsequent posts, we will see how this method has been applied to patients with chronic wounds, and patients with chronic diseases.

Clean In, Clean Through, Clean Out - A Methodology for Delivering Purposeful Pathology

This post summarises the work that has been happening in North Devon over the last few years to deliver more effective pathology. We will show that a focus on purpose, and attention to key points of leverage, has led to substantial improvements in the quality of care whilst making make cost savings. Our methodology has achieved this at pace and scale, and in a manner that is sustainable. We suggest that the problems facing pathology are not ones of efficiency, but rather of effectiveness.

What is the purpose of pathology? 

Our work started by talking to patients about what matters to them about their diagnostic tests. We heard

• that they want to trust those that are providing their care 
• they want to feel cared for 
• they want to know if they are normal 
• they generally want to be involved in their care. 

In essence, what this means for pathology is that our purpose is :

 "To enable citizens, and their carers, to make informed decisions about their care" 

What is the cost of sub-optimal pathology?

Diagnostic testing has been growing at about 5% per annum. There is a frequent assumption that this is linked to value, but this is not proved. As we will discuss in subsequent posts, there is considerable evidence that over-testing is now endemic in modern healthcare. There is an obvious link here to increased testing costs, which are not insubstantial. However, the greatest costs lie outside pathology. We will discuss how overtesting adds to an already stretched workload, and adds further costs from follow-on investigation. Most insidiously though, sub-optimal testing leads to considerable patient anxiety. Fran's story tells what it is like to be told as a patient you have an incidental finding of an abnormal result.

Deriving points of leverage from what matters - Clean In, Clean Through, Clean Out

Our work led us to the hypothesis that we will be delivering purpose if the following conditions of "Clean In, Clean Through, Clean Out" are met. In subsequent posts we will describe case examples (urinary tract infection, leg ulcer management, chronic disease monitoring) that provide illustration of these points of leverage. We will also consider the difference between 'inside out thinking' (in our case seeing the world from the perspective of the pathology service and the problems we have) and ’outside in’ thinking (seeing the world from the perspective of the citizen and the problems they have).

Clean In. 

Our standard definition requires tests to be 

1. Necessary

Our work showed numerous examples of tests that were not necessary to answer the clinical question being asked. As we will show this is expensive in and of itself, but that the true cost of inappropriate testing is under-recognised and lies outside the laboratory.

2. Maximally appropriate 

We also saw examples of where tests were not being done at a time when they would have most effectively answered the clinical question, leading to delayed and sub optimal decision making 

3. Informed 

Citizens often have little idea why things are being done to them in healthcare. Our work forced us to confront the implications of this, and it goes both ways. People are often surprised by the result of a test they were not expecting, and were not psychologically prepared for the implications. This goes someway to explaining why results are then not acted upon in the most rational manner. We also saw citizens who thought tests were doing something that they weren't, gaining false reassurance about their health.

4. Sufficient. 

When the above conditions are met, specimens should arrive at the point of analysis in a state and with sufficient information to allow the clinical question to be answered.

Clean through. 

Our standard definition requires specimens to be analysed :

1. Without error. 

In this context, we define error as variation that the system is attempting to avoid - or special cause variation. This should be viewed through the prism of citizen understanding, and this really means error that has a material effect on care.

2. With known variation 

This is similar in scope to “without error”, but looking at predictable variation which the system may accept (but needs to describe) - or common cause variation.

3. On time 

The results should be available within the clinically appropriate timeframe, ready to help informed decision making when it needs to happen.

Clean out. 

Our standard definition requires test results to be reported in a way that is:

1. Understood

Patients and their carers should know what the result means

2. Helpful 

The result should facilitate good decision making

3. Reflective of what is “normal for me “ 

For example, using reference ranges that reflect populations of people like me, and distinguish pathological values from outlying values.

Applying Clean In, Clean Through, Clean Out in Clinical Practice

In subsequent posts, we show that by applying these definitions to typical and predictable scenarios by looking at what happens to real citizens and why, then we are able to take simple, focussed and practical action based on what we learn. The results of doing this are predictably:

• more purpose
• more capacity
• less cost
• less harm

Antibiotics and behaviour change at the Health Foundation with BSAC

I had the pleasure of attending a meeting at the Health Foundation last week, courtesy of the BSAC. It was a great multidisciplinary event with microbiologists, junior doctors, intensivists and, perhaps most importantly, social scientists - of many different ilks. Here are some slightly unstructured thoughts (NB these are not verbatim notes and may not reflect what was actually said!)

Charis Marwick - Scottish Infection Intelligence Platform (IIP) : an attempt to get innovative ways to integrate data to support clinicians. Some interesting data that changes to surgical prophylaxis in orthopaedic surgery towards aminoglycosides led to more AKI (which was published here). New data shows that this trend has been reversed with a change in guidelines. We haven't seen this in SW England when we looked, but not sure our data is as robust.

Charles Vincent - On safer healthcare. His new book is freely available here thanks to the Health Foundation. We need to move from assurance to inquiry. Are boards and regulatory bodies up for this challenge? It's a profoundly different form of governance. But one that is more honest (what target is not gamed) and the basis of improvement. We need to think about how we build resilience. Patients themselves may be a key resource in monitoring safety and we need to think about how we do this. Is this what the CQC were getting at with their proposal to use social media as the canary in the coalmine?

Esmita Charani  - The importance of understanding culture and team dynamics. There is too much focus on guideline adherence. There is not enough focus on decision making, team working etc. For example, what do you find when you study why prescribers make the decisions they do? You see a prescribing etiquette. There is a reluctance to interfere with the decisions of others; an accepted non-compliance, with emulation of peers a far stronger determinant of behaviour than guidelines; and a clear hierarchy (juniors will defer to consultants). Any attempt to think about stewardship as a set of guidelines and audits is clearly magical thinking. It's hardwork engaging with this sort of deeply engrained behaviour, but we've got to do it.

There is also just too much intervention, and generally too much of everything. There is no time or space for thinking about what is right, or considering unintended consequences.

She finished with a lovely metaphor - all our systems are trying to build bridges across canyons of sub-optimal practice

. 

But this doesn't reflect the reality of chaotic, complex, adaptive systems that will never respond in predictable ways to simplistic interventions. We're dealing in a crowded market place, and we need to get in there and get our hands dirty.

Peter Davey suggested that perhaps we have too much focus on antibiotics...maybe we need to be focussing on higher level principles. In some situations antibiotics are just not the big medication problem. It might be warfarin, It might be insulin. By forcing people to focus on an issue that has little salience to them, maybe we reduce our credibility. Perhaps we need to be wiser in how we pick our battles.

Building on work I've been doing on primary school governance, perhaps there is a parallel in the way I have seen headteachers set a number of 'non-negotiables'. For instance, everyone must follow the marking policy. But within this framework, teachers have complete freedom to deliver purpose. By trusting and supporting people you can profoundly improve performance. There are echoes of Laloux's Reinventing Organisations here. One of the most profound books I've read in recent times.

Nick Sevdalis. Why has the WHO surgical checklist failed to live up to its promise as originally reported? After all, what other intervention has been discovered that could have the profound effects that were reported? He quoted Lucian Leape :

"The likely reason for failure is that it was not actually used"

So they studied the reasons. This was not a compliance audit - it was a proper sociological study of what actually happens at the interface of guideline and practice. A quick nod here to the concept of mindlines that I came across just after this. 

What they found, essentially, was that the checklist may have been implemented in body, but not in spirit. eg. in only a third of cases did the team actually pause. In other words, implementation was not as intended. The consequence of this is vividly portrayed in "Wrongfooted" in which a failure to follow the checklist led to the wrong foot being operated upon.

In general, what we see when these guidelines come out is a complete lack of strategy for how to implement. If you were going to be sensible, you'd focus on the bright spots and use concepts such as dissonance to promote behaviour change. Instead we drop the guideline from a great height and are surprised with the complete lack of engagement on the ground. We perhaps need to differentiate the strategic from the operational. I'm not completely sure about this - I think we need a common purpose at all levels of the organisation - perhaps the difference is how we talk about this depending on context.

So what we see with poor implementation of research or other good practice is :

• lack of fidelity (implementation was not done as described; inconvenient corners were cut)
• lack of understanding of causality. When things don't happen as we would expect we usually lack the data to know whether this is because the intervention is actually ineffective, whether is it effective but poorly applied, or whether it was effective in another setting, but not in this one.
• lack of engagement :  a vicious circle
• poor precedent : spill over effect of negativity towards other interventions

Also, people remember when you get risk mamagement wrong. 

All these things have significant implications for the way we 'do' antibiotic stewardship. We need to be rigorous, we need to get the evidence, we need to understand why things happen and we need to help people feel that they are part of the solution and have a stake in getting it right.

Implementation is a process. Attention to implementation is triggered by problems - timing is everything perhaps. We need to look for the moments of cognitive dissonance and strike when the iron's hot. Banging away at a psychological brick wall is depressing for everyone.

Finally, we need to find ways for senior management and service led teams to share a common language and purpose. Every board meeting I've been to will pay lip service to quality, but in the end only end up talking about money. Most board members do not understand value as they do not spend enough time building understanding in the workplace. We must move away from assurance by spreadsheet towards a deep understanding of the nature of demand on our services.

Detection and management of sepsis - Fabiana Lorencatte and Neil Roberts. There is too much ISLAGIATT (It seemed like a good idea at the time) in service redesign. There is no learning from success or failure. Meaningful change requires understanding of the determinants of human behaviour. This is a complex field and can seem bewildering to the non-specialist. But there are common themes underlying most behaviour change theories. They talked about TDF. Personally, I like the Switch framework. There was a lot of discussion about how we do this sort of work at pace. Quick and dirty...and wrong? This then feeds into the difficulty in knowing (at all levels of an organisation) how to to make sense of competing imperatives. How do we decide whether antiiotic stewardship is most important? Or Sepsis 6? Are these the same thing...they don't feel it. 

Carolyn Tarrant - line infections in ICU. What predicted good uptake of Matching Michigan?. 

• Passionate clinical lead who sets expecations.
• Peer education and persuasion
• Role modelling
• Active use of data
• Embedding and normalising
• Any opportunity to remind people "This is what we do"

We must avoid the temptation to focus on technical aspects of intervention. 

"Improvement and implementation is inherently social."

Discussion.

We need to move away from seeing Sepsis 6 and antibiotic stewardship as competing goals. Stewardship is too negative. Why as, as informed citizens, would we not have amoxicillin for a cold? Or meropenem for cellulitis? We need to find the salient motivators for patients and clinicians, and not see stewardship as a worthy compromise that is good for the public but bad for the patient. 

There was a lot of agreement that our common and unifying goal is optimisation of infection management.

Should we be spending so much time as clinicians worrying about intervening in individual cases? Management may often not be as we would do it, but are the marginal gains always worth the pain? Again, we come back to dissonance- is insisting on a 48 hour IV to oral switch actually always a good use of time. Perhaps better to spend the time discussing 'grey cases' with prescribers. Exploring real life problems and talking about pros and cons of different management approaches. Being seen to be working with people towards a common purpose, rather than against them in a somewhat abstracted form, must surely be a better way forward?

Mike Cooper talked about 'higher order goals'. These are things that are more likely to motivate people to change,and if we can measure them in ways that have credibility, these are powerful ways to create dissonance and promote change.

Another way to talk to prescribers is to ask them "What made you uncomfortable in the last 24 hours?" This is the basis of the 'heads up' campaign, I can't find a link for this, but it sounds a great way of engaging with what really matters to people. And this is really the essence of all behaviour change. Finding that feeling that shows people you get what matters to them, and will be there for them. They are not interested in your problems. So there it is. Easy really. 

Points of leverage in infection management optimisation?

Some work we have been doing in pathology recently shows how profoundly misguided we are to be focussing so heavily on clinical outputs over which we have no real control. These include things like cost. Or mortality rates. These aggregated measures tell us nothing about what happens to individuals, And it is these individual encounters that build up into the aggregate picture.  The other type of measure we often use to describe our services are process measures, such as turn around times. Most of us know these are pretty meaningless to patient care, but because we haven't been able to think of any good alternatives we have increasingly badged the quality of our services according to these variables,

So our only mechanism to improve (our points of leverage) are at the level of input into the care of the individual. In diagnostics, this has led us to the concept of clean in, clean through and clean out. By following this methodology we have seen dramatic improvements in care, mainly by reducing the potential for iatrogenic harm by overtesting. So we've reduced MSU submission rates by 50%, and have now done the same for liver function tests. We have started to develop new ways of thinking about laboratory error that have clinical meaning. And we are beginning to use new ways of displaying results that help patients and clincians understand what they actually mean and promote the correct actions.

So is there something here for antibiotics? If we are not going to look at resistance rates or mortality (aggregated outcome) or time to antibiotics, or guidelines adherence (process and compliance measures), what can we measure? This needs some work, but it might look something like :

Clean in to prescription

• Antibiotics are necessary (or justifiable - this will be a continuum)
• Antibiotics will cover likely (including previously and subsequently isolated) pathogens (sufficient)
• Antibiotics are maximally appropriate (without unnecessary side effects, including overly broad spectrum)
• Appropriate diagnostic tests have been taken before antibiotics (eg Urine culture- UTI; Wound swab - cellulitis with open wound; blood cultures - severe sepsis)

Clean continuation

• Antibiotics are given in a clinically appropriate time frame
• Antibiotics are given by a clinically appropriate route
• Antibiotics are reviewed as soon as results and clinical progress dictates

Clean discontinuation

• Antibiotics are stopped when clinical improvement unless clear need to continue
• Source control is achieved were necessary
• Patient understands the diagnosis and what they need to do to prevent recurrence if appropriate

I think the big difference here with things like Start Smart is the shift away from compliance based approaches to a more forgiving and learning collaborative approach. I think to do this will require infection specialists to act in a coaching role and encourage clinicians to explore their practice in a much more open manner. This will be much more fun. I'm going to start tomorrow with my very nice respiratory consultants, I'll let you know how we get on.