Making it work: service users and professionals as research partners

Capital is a user-led training, consultancy and research organisation based in West Sussex. When the local mental health trust decided to evaluate its assertive outreach team, Capital bid to conduct a qualitative review through one-to-one interviews with the team's clients and their carers. In this candid article the project team members describe the process, the hurdles and obstacles they encountered, how they were negotiated or overcome, and what they learned from the experience.

The Capital Project Trust was set up in 1997 as a training project for people in West Sussex who use mental health services. Today it has just under 100 members, many of whom work as volunteers delivering service user focused training or are involved in consultancy and research.

When the Chichester Assertive Outreach Team (AOT) operated by West Sussex Health and Social Care NHS Trust decided to review how it was doing, Capital was consulted about how best to proceed with the review. Anne Beales, then director of Capital, proposed that we should be the interviewers in a qualitative evaluation of the service. As Capital members would be from an outside organisation we hoped to be seen as neutral, and so encourage those we interviewed to be frank in their opinions. Our experiences as people who use services could also allow us to empathise with those we met during the review. Capital members worked closely with Mark Hay ward, clinical psychologist with the AOT, to develop a project team.

The only service user evaluation of an AOT known to us was one undertaken within the voluntary sector in East Anglia (Grayley-Wetherall & Morgan, 2001), and we agreed to use this study as a basis from which to develop our questionnaires. This process began in consultation with Roberta Grayley-Wetherall, who was refreshingly un-precious about our adapting her work. It then took several sessions of heated team discussion before agreeing the content of the questionnaires. The carers' questionnaire proved the most difficult, as we decided our study should include questions about how much their needs as carer were being met.

Jumping through hoops

But that was the relatively easy bit, compared with all the bureaucratic hoops we had to jump through! To provide indemnity insurance we initially agreed to take on honorary contracts with the trust. This necessitated Criminal Record Bureau checks and Occupational Health having to be satisfied of our fitness to undertake the work. As it turned out, by the time the CRB paperwork needed to be done, the one or two volunteers for whom this might have proved difficult had fallen by the wayside. Just filling in occupational health forms, however, proved highly stressful when some of us were obliged to answer 'yes' to almost every question relating to mental health. Before our medicals took place we sought and got assurances that these would not mean being given the third degree about our histories of mental ill health.

Then the trust's lawyers advised that their indemnity insurance would only cover us for interviews conducted on its property. We seemed to have hit a brick wall. None of us could envisage telling AOT clients that they could only be interviewed in a formal mental health setting. The only way out was for Capital itself to purchase indemnity insurance - useful, but expensive.

Safety and support

We also did our best to ensure the comfort and safety of all parties by working out group agreements and protocols outlining our agreed standard of behaviour. Either side had the option of terminating the interview should it feel too uncomfortable. We also agreed to work in pairs, allowing for a mutual debrief afterwards, as well as for someone to write and someone to talk. At the outset, we had explored our anxieties about the work, which ranged from 'arriving late' to 'being badly upset by the things they say'. A previous project had, indeed, thrown up some upsetting information, and this time we put extra safeguards in place, ensuring that interviewers had access by telephone to a member of Capital's paid staff after interviews, and the availability of an outside supervisor. Thankfully we never needed to call on him.

Getting going

By the time everything was in place our original team of interviewers had dwindled from nine to five. Mark was concerned that we might not be able to manage the work but we were able to reassure him that enthusiasm would compensate for numbers.

At last we were ready to recruit participants. We had decided it would be best if the invitation letters came from Capital, rather than the trust, to reinforce the message that the audit would be anonymous. As with the questionnaires, we took a good deal of time over the wording, hoping to make the letters as inviting and informative as possible without being overwhelming in length. We also mentioned that there would be a small payment for participating.

Response was slow. In hindsight we wondered whether sending out invitations from what was, for many, an unknown organisation was really the best approach. We will never know. Over the next couple of months we interviewed eight clients and eight carers and the team grew familiar with several local coffee shops, these being by far the most popular choice of venue among participants.

Challenges and dilemmas

As all the interview team were doing other things as well, co-ordinating availability in order to be able to send people out in pairs at times to suit interviewees could be quite a challenge. There were occasions when interviews had to be cancelled at the last minute because someone was unable to make it. One poor carer tolerated three postponements before we finally managed to meet up with her.

Another carer requested a telephone interview, fearing that a face-to-face meeting might upset the person they cared for. After some discussion we felt it was better to include anyone who wanted to have their say, rather than to refuse because they could not fit in with our predetermined methodology. For a number of reasons, telephone interviews proved difficult there were time constraints and, inevitably, the interviewer had no support to record the answers.

'It's hard to record answers and keep an interview going without it becoming stilted and awkward... I had to keep a running commentary going as I wrote. I was also trying to be as quick as possible as X was at work and anxious about the time it would take.' (Capital team member)

Another issue that arose was that a couple of the carers who came forward were paid staff from housing projects. This provoked heated debate among the review team as to whether or not it was appropriate for them to be included. The AOT's view was that it was equally important for them to know how well they were working with staff from other agencies, so we did agree to include them, even though the questionnaire was very much designed with informal, rather than paid, carers in mind.

As time went on our interview team's availability became more stretched. Two team members went off on (separate) trips to Australia, another was ill (with 'flu, not mental health problems). But we managed to keep going. Other problems were more accidental in nature. Someone's car broke down en route to an interview, and someone else knocked over a pheasant on the way to an interview. Happily the bird was only stunned.

Feedback

In general, clients and carers were a pleasure to meet, dispelling any fears we had of the 'difficult to engage' label. Most of the people were thoroughly engaging and seemed refreshingly happy with the service the AOT was delivering. As time went on this general level of satisfaction did come to worry us a bit. Were the people who volunteered to take part representative of the whole client group? Were those less satisfied with the AOT simply disengaging from the evaluation process? It was a question we could only pose and never answer.

That we were users of services ourselves was definitely an advantage when it came to talking to clients, although on occasions people only came to understand this part-way through the interview, and then information was suddenly far more forthcoming.

Sometimes we came across someone we knew, which we had feared might be difficult. In fact these were sometimes the easiest interviews to conduct. The one exclusion clause we did have was not interviewing any staff members who had treated us. This was to prevent any discomfort on either side. Some of us strongly disagreed with this ground rule: if we felt okay about doing it there should be no reason why not. However this argument did not prevail, although it probably merits wider discussion between service users and professionals.

Talking to the members of the assertive outreach team was another positive. The staff had a huge amount of enthusiasm for their work, although there were reservations too, mainly about some of the dynamics within the team. Interviewing a consultant psychiatrist was an experience not to be missed. How often do we mere services users actually get the chance to be the one asking all the questions?

Focus groups

Christmas came and went and our trickle of people coming forward to talk to us dried up. Our sample was still small, so we were left wondering how to proceed. AOT staff had tried generally encouraging people to come forward, but with little success. Holding focus groups at the AOT base came to feel like the only possible option for bringing in more people. It was not a compromise we made lightly... We were fully aware that, inevitably, focus group participants would be less anonymous, but we could still ensure that no remarks could be attributed to any particular individuals. As Anne Beales put it: Only Capital would run focus groups for people who don't like being in groups.'

This was quite possibly true but we had to try... We planned a session for clients that would be part business and part social. There was a Cordon Bleu buffet lunch (prepared by one of the team) and plenty of breaks. This time the invitation came from the AOT itself and staff both hosted the event and provided transport as necessary. We began with lunch and an icebreaker so everyone could get to know each other before starting any serious business. After this the staff withdrew.

Nine people attended, including two who had already contributed individually and one person who did not use the AOT at all but had heard there was a payment for attending and hoped to take advantage! We let him stay for lunch before he left. He then pursued us for several weeks before he was finally convinced that he was not entitled to the money, but that was the only real problem we encountered with running the group.

We split the new participants into two groups of three, each with two facilitators, and worked through the questionnaires over two sessions. It was a challenge to draw people out without leading the discussion:

'At first I thought "We're getting nowhere fast" but gradually people opened up... By the end of the day it felt really good.' (Capital team member)

It also seemed to feel good for the participants, who filled in evaluation forms saying they had enjoyed the day and would like to attend something similar again.

Following on from the success of this event we arranged a similar one for carers, but this time in the evening, so that working people could attend. Again we had a buffet and some time to socialise, but we were also conscious that this group had more constraints on their time, as some of them did not like to be away from home too long. Three of us took the four families who attended through the questionnaire. People had so much to say that it took two of us to record everything while the third person facilitated. We took turns at each role to try to prevent writer's cramp. Once more it felt a very worthwhile event, both because of the information gathered but also because it provided the opportunity for carers to meet each other and exchange ideas and experiences.

Findings

This more or less marked the end of the information gathering process. We had done all we felt we could to elicit people's views. The table above gives a sense of who participated in the review and how responsive they were to invitations to take part.

Everyone we had spoken to seemed to value the service they received. However we still could not help but wonder whether those who were less satisfied simply opted out of the evaluation process. We hope not, but it is a possibility that has to be borne in mind.

'Supportive' and 'encouraging' were the words most commonly used to describe the AOT, by both clients and carers alike. It was clear most people felt well supported:

I am very happy with the AOT. There's always someone to turn to.' (Client)

However some people did want more support and carers felt the need for more opportunities to take a break.

People appreciated the practical help they received, which included a wide range of activities, from going out shopping to dealing with head lice, as well as medication and formfilling. But sometimes this help got missed for want of time:

I need help with practical things not just my mind. Sometimes there's not enough time for that.' (Client)

Even if time was short, the quality of contact with the AOT was valued. People felt consulted and listened to:

'They asked my son what he wanted instead of telling him what he wanted. ' (Carer)

Service gaps

There were also perceived gaps in service provision. When specifically asked, few people could see improvements in their relationships with others or their use of leisure time. No one felt the support they received addressed their spiritual needs. However the positives by far outweighed the negatives and the only direct criticisms concerned shortage of time and occasional difficulties in contacting the AOT. Generally people reported having better health, keeping out of hospital and improved quality of life:

'They keep me happy. ' (Client)

Lessons learned

We were always committed to disseminating our work, as well as to ensuring that it was not just a paper exercise with no practical application. While the work was still in progress, a couple of team members helped run a workshop about it at the National AOT conference in Exeter. Another two made a poster presentation to the British Psychological Society conference. We are actively pursuing other opportunities to share this work.

Most important was to feed back to stakeholders locally and to see what lessons could be learned and applied. We held feedback events for clients and carers using a similar format to the focus groups, and attended an AOT team meeting to brief the staff on our findings. Following on from these, a stakeholder group has been set up - including staff, clients, carers and members of the research team - to plan improvements to the service, informed by our work. The first step has been to work on improving the information people are given, so they know what to expect from the service.

It was a fascinating project to be involved with and all of us hope to have opportunities to do similar things in future. We have learned a lot along the way, not least that we had to temper idealism with pragmatism. Without compromises along the way the project would have foundered. However we also have to keep challenging some of the official barriers to make it easier for groups like us to take on projects in the future.

The amount of time the project took from start to finish was far longer than anticipated, mainly for reasons beyond our control. However time spent on preparation was invaluable. Another time we would probably double our projections of how long things take. We would also charge realistically. This project was taken on as a loss leader, and time will tell if our investment pays off materially, although in terms of personal satisfaction for those of us involved, it was money very well spent.

Services often come in for more criticism than praise from the people who use them. It is reassuring to know that some services do work in ways that people really appreciate. As for us, we valued the experience of working as equal partners on this evaluation and the opportunities it provided to meet people we would not have met in other circumstances. As a group we contributed an energy, enthusiasm and persistence that demonstrates the eminent suitability of people who use services to research and evaluate them. We hope that our experience will encourage professionals to be more open to working with service users in research, and that it will inspire other service users to want to be involved. The rewards by far outweigh the hassles.

References

Grayley-Wetherall R, Morgan S (2001) Active outreach: an independent service user evaluation of a model of assertive outreach practice. London: Sainsbury Centre for Mental Health.

Mark Hayward, Clinical psychologist, West Sussex Health and Social Care NHS Trust

Clare Ockwell, Tim Bird, Howard Pearce, Sheree Parfoot and Theresa Bates are all members of Capital. Capital can be contacted at Capital Project Trust, The Lodge, Swandean, Arundel Road, Worthing BNi3 3EP e capitalprojects@btconnect.com

Copyright Pavilion Publishing (Brighton) Ltd. Aug 2005
Provided by ProQuest Information and Learning Company. All rights Reserved