Evaluation

The Shimme project is not only about designed and delivering a programme to encourage shared decision-making in relation to psychiatric medication, but also about evaluating the impact of this programme. The research component of the project will collect data to assess:

  1. The impact of the programme: does it lead to improvements in decision-making relating to medication? Is it associated with other benefits? Are there any detrimental effects?
  2. How feasible it is to implement the programme and how do those who participate experience the programme?
  3. The health economic implications of the programme: how much does it cost to deliver the programme? Is it associated with any cost savings in terms of health service use by participants?

We will assess the programme using a mixed-methods process and outcome-oriented approach. This means we will collect data in a number of ways: structured self-completion questionnaires, written feedback about the programme from participants, and semi-structured interviews. These will assess both experiences of participating in the programme (process), and changes over time following participation (outcomes).

The evaluation research component of the project is designed and conducted by members of the management team (Shula Ramon, Amy Li and Nicola Morant) with additional consultancy support from Dr David Wellsted, statistician at the Health Research Centre, University of Hertfordshire, and Professor Paul McCrone, health economist at the Institute of Psychiatry, London.

Data Collection

We will ask participants to provide data at three time points:

  1. Before the Training Programme we will ask everyone who participates in the training to complete a set of written questionnaires. This will allow us to understand what people’s views and experiences of relevant issues and topics are before they participate in the programme.
    Service UsersThe questions will ask about how decisions about medication are made between service users and practitioners, what this is based on, how service users feel about this, and their preferences for decision-making. They will also ask about relationships between service users and professionals, attitudes to medication, and use of medication and services over the last 6 months. We will use the following validated measures:

    1. Decision Conflict Scale. This is a 16 item measure of service users’ views of uncertainty surrounding their medication decisions, whether decisions are informed, value-based, likely to be implemented and whether they feel satisfied with decisions made.
    2. STAR (Scale To Assess Relationships – patient version): This is 12 item measure of therapeutic relationships between service users and mental health practitioners
      OPTIONS: A 12 item measure allowing service users’ to rate whether clinicians’ interactive styles promote shared decision-making
    3. Control Preferences Scale: A single-item rating of decision-making styles in current and preferred practice.
    4. Drug Attitudes Inventory (short version): This is a 10 item measure of a person’s views about psychiatric medication.
    5. Client Socio-demographic and Service Receipt Inventory (CSSRI): This collects basic data about participants and provides a measure of medication and service use from which costs of health and social care can be calculated.

    Completing these questionnaires should take about 75 minutes in total.

    Practitioners

    Pracitioners participating in the programme will be asked to complete the following validated measures:

    1. STAR (Scale To Assess Relationships – clinician version): This is 12 item measure of therapeutic relationships between service users and mental health practitioners
    2. Control Preferences Scale: A single-item rating of decision-making styles in current and preferred practice
    3. Basic socio-demographic and professional information.

    Completing these questionnaires should take about 15 minutes

  2. At the End of the Training Programme we will ask all participants to provide written feedback about their experiences of taking part in the programme. This will include questions about:
    • components of the programme (the group sessions, the web-site, self-study materials)
    • how the programme was delivered (the facilitators, the format of group sessions)
    • the content of the programme (which parts were useful, interesting and beneficial)
    • how it might be improved in future.

    This should take about 30 minutes

  3. One year after the Training ProgrammeWe will ask everyone who participated in the training to complete the same set of written questionnaires as they completed before the training programme. This will allow us to measure how people’s views and experiences in key areas might have changed over this period.Completing these questionnaires should take about 75 minutes in total for service users, and about 15 minutes for practitioners.In addition, some people will also be invited to participate in a semi-structured interview on the impact of the training programme over the past year. This will focus on experiences of discussions about medication between professionals and service users, the outcomes of these discussions and any associated impacts on well-being, and perceived benefits and challenges of having participated in the programme. Trained service user researchers will conduct interviews with service user participants, and the study researcher will interview professional participants.The interview will last between 30 and 45 minutes.

How will data be collected? As much as possible, we will offer a number of options for how data is collected. Participants will be able to complete self-report questionnaires in paper versions at home and post them back to us, or complete them electonically on this web-site. For those who might need additional support to do this, a researcher will available to provide support in completing the questionnaires.

Payment: We recognise that providing research data takes time in addition to the time spent participating in the programme. In recognition of this, we will pay service users £20 for completion of questionnaires before the training programme and £20 for information provided 12 months after the training programme. People who take part in interviews will be paid an additional £10.

What will happen to the data?

All the information provide will be kept strictly confidential: It will be looked at only by the research team and will have names and other identifying features removed so that individuals cannot be identified. All names will be removed from the transcribed interviews and only group findings will be published. The only exception to this is that if you give us information about intent to self harm or harm to others we will be obliged by law and good practice standards to let your care co-ordinator know about this.

Findings from the project will be written up and disseminated in the NHS and published in service user and practitioner publications. They will also be published as academic research papers. A summary of our findings can be provided to participants at the end of the project.