The last fifteen years has seen considerable effort, time and money put in to the development of initiatives to improve the situation on policing and mental health. We could talk for weeks – and I have – about what the precise problem is and I’ve written before asking “what problem are you trying to fix?” But nevertheless, undefined as the problem may still be, most people seem seized by the instinct to “do something” and we’ve cracked on regardless. So we’re now in the fourth phase of that – Right Care, Right Person – having previously seen street triage, liaison and diversion and the so-called “High Intensity” programme.
One thing in particular binds these initiatives together and it’s worth remembering: they haven’t been fully or properly evaluated, before being scaled up for roll-out.
This is quite a blunt claim, because of course it’s simply not correct to suggest that no-one, anywhere has attempted to think about what the impact of these various initiatives has been or that no-one has published papers or evaluations on these programmes. But my claim that they haven’t been “fully or properly” evaluated arises from my belief the quality of these evaluations was limited. Next, you may think this is quite a rude thing to say about researchers who are well qualified, many of them academics at good universities, so I also need to insist the observation isn’t about the quality of the researchers, either. I’ve met and discussed this specific topic with plenty of them and when I wondered aloud whether their efforts had been somewhat limited by poor data, they were the first to say so – and indeed, many of them wrote about limitations of methodology and statistics.
WHAT’S GOING ON?
So let’s go in chronological order and start with street triage. There are a number of papers, studies and evaluations available, many of them online. When the National Institute for Health and Care Excellence (NICE) did their systematic review of evidence for mental health in the criminal justice system, they searched for all available materials they could find on street triage. They then applied a basic assessment filter to each study to determine if the quality of them was sufficient to justify further examination for inclusion in their guideline document and concluded that only three studies were worthy of further examination. Anything which passes that filter is further described on a five-point scale: VERY LOW – LOW – SATISFACTORY – GOOD – VERY GOOD. One study was labelled LOW quality, the other two were VERY LOW. So NICE itself determined only three studies were worth considering, and only just worth it.
The main problem was data, from what I could tell. Many organisations who were party to the various initiatives didn’t collect baseline data across all relevant domains and variables, so when it came to analysis, there was insufficient background to allow a meaningful comparison. For example, street triage often examined the impact of the scheme on the rate of s136 usage in the area where ST was deployed. Two problems: for meaningful analysis, you’d need to then compare that with a similar area not operating triage but you also need to examine more than s136 usages because most interactions were in private dwellings where s136 cannot be used. In other words, by focussing on what happened to 136 rates, you’re evaluating less than half the initiative!
We then have Liaison and Diversion schemes (which involves placing mental health professionals in police custody to improve identification, assessment and management of those who are arrested or taken to the cells by the police). For years, there was limited evaluation of this and at one point in time, I used to attend the national meeting for L&D, where evaluation of the scheme was discussed well after it had been funded and scaled up across the country. There was a significant attempt being made by NHS England to evaluate outcomes connected to the health of those who were seen by the scheme, but nothing within that was connected to examining the justice outcomes. I always thought (and said!) that this was quite weird – if you are designing or making an intervention which impacts upon people who have been arrested for a criminal allegation whilst ill and you’re only examining health outcomes, not justice outcomes, how do you know what’s what? In other words, if health improves but prosecutions reduce or crime goes up, has L&D “worked”, or not?! And how will you know whether this has happened if all you’re doing is looking at health outcomes?
You obviously won’t – research by Professor Eddie Kane tried to get to grips with it years after it began and he concluded L&D results in longer detentions of vulnerable people (because more people are being identified as vulnerable, so more requirement for solicitors and appropriate adults which non-vulnerable adults may not need) and a higher likelihood of being charged with an offence. So what does that do to the health of the person and does this mean it “works”? – we don’t know.
SCALING THINGS UP
The ‘high-intensity’ or Serenity Integrated Mentoring scheme came next: this initiative began around 2014 as a local idea between the police and NHS on the Isle of Wight. It involved placing police officers amongst community mental health teams to focus on those of us who present repeatedly to the emergency system in an apparent vicious circle of crisis, intervention, short term respite and further crisis. The scheme subsequently won an NHS ‘accelerator’ fellowship which allowed it’s exponents to scale it up across NHS areas who became interested in the claims made about significant demand reduction for those of us who repeatedly present to the emergency system – claims of 93% reduction in demand amounting to thousands of hours of time saved. Despite the involvement of the Academic Health Sciences Network (AHSN), it was never rigorously evaluated and by the time of a service user campaign in 2022, NHS England were obliged to write to all NHS trusts ordering the scheme to stop and ordering an inquiry to establish how it managed to proliferate, despite there being little robust evidence (and despite it very obviously pushing against various principles of what good mental health care should be).
How did we get here? – I admit I’ve no good answer to this. The NHS in particular is historically far more interested in robust, evidence-based policy than the police. Indeed, the police service now talk about evidence-based policing (for which there is a society to promote evidence-based approaches), precisely because it borrowed the term from evidence-based medicine. I’ve been aware of some people’s frustration when I’ve previously raised the question about the quality of evaluations and including when I’ve written on this blog to critique poor quality research which aim to cement conclusions in our minds which simply aren’t justified by the evaluations. I remember in particular, claims that street triage was saving £1.1m in a particular police force area and when I pointed out the various evaluation problems, it seemed to annoy people. The answer to this, off course, is to do better research and for it to be part of how initiatives are planned before they are commenced.
The College of Policing, whose remit is to promote evidence-based policing and notions of “what works” uses the Maryland Scale to rate the quality of research where Level 1 research is the best and conclusions from it can be relied upon to influence other policy and where Level 5 is the lowest and any conclusions drawn should be treated with skepticism. Most street triage research is L5 or L4 at best, for example because they rely on ‘before and after studies’, without control groups, randomisation or other strategies which allow conclusions to be relied upon. It’s usually things like “We used to have X amount of s136 detentions then we did street triage and now we have Y number of s136s” OR “SIM worked with (or targeted?) six people who present repeatedly to emergency services and this year, demand is down 93% compared to last year”. Well fine, maybe street triage helped achieve that – no doubt SIM had some impact, but what if you looked at the next door area or other people found there was a reduction without that initiative? How certain can you be that ST or SIM is what brought about the change?!
UP TO DATE
Two weeks ago, at the Health and Social Care Select Committee, we saw MPs questioning Chief Constables and Chief Executives about the Right Care, Right Person initiative. During that period, it was revealed there had been internal evaluation of certain policing issues such as the reduction in the numbers of various kinds of call and how that translates in to officer-hours saved, but there was acceptance there has been little or no evaluation of the health outcomes for those who call the police and are then directed towards other agencies because of their being no obvious policing purpose within the nature of the call.
The evidence session concluded with suggestion the NHS is beginning to examine whether people are, in fact, receiving the right care from the right person as a result of the Right Care, Right Person programme but that work is not complete before the rollout. MPs seemed quite stunned this wasn’t done prior to the roll-out and I can’t say I blame them, given what we now know has happened with street triage, liaison and diversion and SIM.
If you saw it, you’ll remember how the committee session began, “To what question is this the answer?” … in other words, what problem are you trying to fix? Perhaps I should get that printed on a mug, as someone once suggested I should.
Winner of the President’s Medal,
the Royal College of Psychiatrists.
Winner of the Mind Digital Media Award
All opinions expressed are my own – they do not represent the views of any organisation.
(c) Michael Brown, 2023
I try to keep this blog up to date, but inevitably over time, amendments to the law as well as court rulings and other findings from inquests and complaints processes mean it is difficult to ensure all the articles and pages remain current. Please ensure you check all legal issues in particular and take appropriate professional advice where necessary.
Government legislation website – www.legislation.gov.uk
Mental Health Cop 