An article that became the guide to my research

In all scientific research it is necessary to carry out a review of the literature on the “state of the art” at that point of knowledge. In the course of my research on “scientific research in outcome evaluation in psychotherapy”, I first came across a study entitled “The effects of providing therapists with feedback on patient progress during psychotherapy: are outcomes enhanced?” (Lambert M. et al., 2001), a study that became the guide and the organiser of my own research.

What attracted me in this article? Its scientific rigour, naturally, and the challenge it awoke in me when I established connections between the results reported by Michael Lambert and my own experience in developing the ILOP for outcome evaluation in psychotherapy. It was precisely in order to support that development that I undertook this bibliographical search.

Building the ILOP

I began to develop the ILOP in July 2003 at the brief therapy centre I direct, thanks to the therapists, the clients and the specialists: the therapists on the team, under constant supervision behind the one-way mirror (cámara Gesell), who applied it; the clients who answered it; the specialists in the construction of psychometric scales; and the fundamental collaboration of the statisticians. All of this made possible the correct processing and analysis of the information provided by the data that the application of the ILOP generated. That is how, after successive transformations, the product of its constant application, the definitive instrument was obtained, with its validation and its internal consistency, in December 2004, and included in the second edition of my book (2005).

The ILOP is an inventory for evaluating the attainment of goals in psychotherapy which seeks to assess the state and the progress of the client in response to treatment. It consists of a set of self-administered items (self-motivating questions) that form the backbone of brief therapy as we conduct it in this institution. The client answers it at intake, every four sessions and at each of the maintenance interviews, before entering each of those interviews. The self-administered answer constitutes a first piece of feedback to the client on his own state. He receives a second one when the therapist puts the question to him again at the beginning of the session. Through this repetition of the question, the therapist himself receives feedback on the client’s state; and if he is working behind the one-way mirror, so does the supervision team, which widens the feedback loop. Working in this way generates a strong experience of control within the therapist-client system of the CTB, according to the comments of the therapists on the team.

Concerning the ILOP, one of the comments I received from an expert is that it is an interesting inventory, which encourages people to take responsibility for their problems and their solutions, and an admirable way of following up with the precision of someone tracking a furrow.

But will the use of the ILOP, I ask myself, improve outcomes? I am not yet in a position to demonstrate it scientifically, but Lambert and the researchers who work around him have indeed demonstrated the beneficial effect of feedback. In the study I mentioned above (Lambert et al., 2001), they conclude:

Does feedback improve outcomes?

The effects of feedback are stronger for patients identified as being at risk. When the therapist received the information that the client was improving, the therapies were shorter for an equal benefit. Feedback to the therapist made it possible to increase adherence to treatment among clients who were showing little progress. In the feedback group there was an overall reduction of 2.68% in the consumption of sessions. This corresponds to a saving of 60 hours of therapy had feedback been given to the whole sample. A modest 3% saving multiplied by thousands of patients would amount to a substantial reduction in fees. In this research the therapists had different treatment orientations, many of them subscribing to an integration of two or more theoretical systems. The most common orientations were cognitive-behavioural (42%), psychodynamic/interpersonal (19%), humanistic/existential (16%), behavioural (6%) or other (16%).

The beneficial effect of feedback is also demonstrated by Hawkins E., Lambert M. et al. (2004). In this research the patients (those who were not responding to other treatments) were randomly assigned to one of three groups: one of treatment as usual, another with feedback to the therapist, and another with feedback to the therapist and to the patient. The authors of this research reach the following conclusions:

In this study, 64% of the patients in the “feedback to therapist and patient” group achieved reliable improvement in 7.8 sessions. Two of the therapists who took part described their treatment orientation as mainly cognitive-behavioural, interpersonal and humanistic. The patients in the two treatments with feedback showed, at the point of concluding treatment, significantly greater improvement than those in treatment as usual, with no average increase in the number of sessions they attended. Providing feedback to patients and to therapists may be a viable approach for improving treatment outcomes.

Clinically significant change

For there to be “clinically significant change” (Jacobson et al., 1984), a concept mentioned in the preceding paragraph, two criteria must be met: 1) that the client should begin in the dysfunctional range and end in the functional range; 2) that the change score should fall within the magnitude of what is considered reliable. In another study, Jacobson and Truax (1991) indicate that the reliable change index has three characteristics: 1) it is a value obtained from the difference between pre-test and post-test scores, which must exceed the degree of measurement error and be greater than one standard deviation from the mean; 2) the change may be positive or negative; 3) a value below reliable change (RC) is not considered a real change. It is also through the ILOP that we seek to identify clinically significant change in the closing and the maintenance of treatments.

A new paradigm: patient-focused research

Following the connections between the ILOP and Lambert’s work, which applies the paradigm of patient-focused research, I go back to Howard K. I. et al. (1996), who introduce it. Howard sets out a “new paradigm: patient-focused research”. He states that this paradigm answers the question: is this treatment working for this patient? He considers that this paradigm goes beyond the questions of efficacy and effectiveness. In patient-focused research, Howard gives first place to monitoring an individual’s progress during treatment and to returning that information to the professional, the supervisor or the person in charge of the case. The ILOP asks continually whether this treatment is working for this patient.

I established another connection between the ILOP and a further article by Howard et al. (1986). In it they studied treatment time and patient benefit in a meta-analysis covering 2,400 patients over thirty years of research. The therapists were mental health professionals and their orientations were generally psychodynamic or interpersonal. None of the therapies were mainly behavioural or psychopharmacological.

Howard concluded that:

• from 10% to 18% of patients had shown some improvement before the first psychotherapy session;
• by the 8^th session, from 48% to 58% of patients can be expected to show measurable improvement;
• 75% of patients should have shown measurable improvement by the end of six months of psychotherapy at one session per week (26 sessions);
• no treatment is 100% effective;
• predictively, 85% of clients undergoing the same treatment would improve to a measurable extent;
• the maximum percentage of improvements would be reached in about 52 sessions, at one per week;
• 50% of the patients who improved in roughly 8 to 13 sessions of treatment had diagnoses of anxiety and depression;
• among clients with a borderline diagnosis, this level of improvement occurred later, between the 13^th and the 26^th session according to the patients’ self-assessments, and later still, between the 26^th and the 52^nd session, according to the clinicians’ assessments.

To this may be added the following observation: at the beginning of treatment, the researchers’ assessments of improvement were lower than the patients’ subjective assessments of well-being. After the 26^th session, the number of patients considered improved by the researchers was higher than the percentage of patients who thought they had improved. Will these differences of appraisal between therapist and client, as to the client’s state, deserve some consideration on the part of therapists?

The eighth session, a pivotal moment

A common denominator emerges: the 50% of patients improved after the 8^th session in Howard in 1986, and the studies by Hawkins, Lambert et al. of 2004, with the possibility of reaching reliable improvement at the 8^th interview, a pivotal moment of improvement that is observable and within reach of seriously trained or supervised therapists.

I regard this point as a great challenge for the ILOP, with which the cut-off point has already been found that defines — in agreement with the clinical impression that precedes it and with the client’s own assessment — whether the client needs to continue treatment or is in a state to move on to the maintenance phase.

We have also begun a stage that is necessary in any research bearing on an assessment instrument: the search for the correlation between the results obtained with the ILOP and those of other instruments that seek to measure outcomes. To this end, on the suggestion of an expert, we undertook a joint application of the ILOP with Lambert’s outcome questionnaire (OQ-45.2) and L. Derogatis’s symptom inventory SCL-90-R. These self-report inventories detect improvement in areas of functioning and make it possible to draw comparisons with “normal functioning”. Although they are not specific to the aims and the process of therapy, they could be moderately related to them. This comparison would make it possible to set the results of this therapeutic approach against results coming from other kinds of treatment.

To close this work, I raise my glass to continuity, and to there being ever more of us doing research on the evaluation of outcome attainment in psychotherapy.

This article was published in Perspectivas Sistémicas, no. 87, year 17, July-August 2005.

Correlation between the ILOP inventory (Szprachman, R., 2003-2005), the OQ-45.2 (Outcome Questionnaire, Michael J. Lambert et al., 1994) and the SCL-90-R, the 90-symptom questionnaire (Leonard R. Derogatis, 1977) — second part

(The first part is entitled: “Scientific research in outcome evaluation in psychotherapy”. Article published in Perspectivas Sistémicas, July-August 2005, year 17, no. 87.) (*)

Authors: Regina Szprachman; Michael J. Lambert.

The first part of the account of the ILOP’s progress (inventory for evaluating the attainment of goals in psychotherapy; Szprachman, 2003-2005) appeared in Perspectivas Sistémicas (July-August 2005, year 17, no. 87) under the title “Scientific research in outcome evaluation in psychotherapy”. I take up a few paragraphs of that first part in order to introduce the second. I said there: “The ILOP is an inventory for evaluating the attainment of goals in psychotherapy which seeks to assess the state and the progress of the client in response to treatment. Working in this way generates a strong experience of control within the therapist-client system of the CTB, according to the comments of the therapists on the team”, a team which has been applying it without interruption since its creation in 2003. I went on: “Concerning the ILOP, one of the comments I received from an expert is that the ILOP is an interesting inventory, which encourages people to take responsibility for their problems and their solutions, and an admirable way of following up with the precision of someone tracking a furrow.” Now, in conversation with my colleague and friend Claudio Des Champs, on learning who that expert was he said to me: I would have given the name of the expert. It was indeed Michael J. Lambert — whom I had so rightly described as an expert — who gave this opinion of the ILOP and who also proposed to look for the correlation between the ILOP, the OQ-45.2 (Outcome Questionnaire, Michael J. Lambert et al., 1994) and the SCL-90-R (Leonard R. Derogatis, 1977), agreeing to take part in this portion of the research.

I continue to take up the first part: “We have also begun a stage that is necessary in any research bearing on an assessment instrument: the search for the correlation between the results obtained with the ILOP and those of other instruments that seek to measure outcomes. To this end, on the suggestion of an ‘expert’, we undertook (with the CTB team, whom I must thank for their dedication) a joint application of the ILOP with the OQ-45.2 (Outcome Questionnaire, Michael J. Lambert et al., 1994) and the SCL-90-R (Leonard R. Derogatis, 1977).” In Lambert’s words: “These self-report inventories detect improvement in areas of functioning and make it possible to draw comparisons with ‘normal functioning’. Although they are not specific to the aims and the process of therapy, they could be moderately related to them. This comparison would make it possible to set the results of this therapeutic approach against results coming from other kinds of treatment.” The CTB set to work and, after six months of hard labour, I am in a position to report that the ILOP correlates significantly with the inventories mentioned. The ILOP correlates significantly, at the initial stage, at the point when the sought-after outcome is attained and during the maintenance stage, on all scales and subscales, with the OQ-45.2 (Outcome Questionnaire, Michael J. Lambert, 1994) and the SCL-90-R (Leonard R. Derogatis, 1977). This means that it correlates significantly with the three subscales of the OQ-45.2, named symptom distress, interpersonal relations and social role, as well as with the total score. It also correlates significantly with the SCL-90-R (Leonard R. Derogatis, 1977), with the global indices, the index of subjective distress and the symptom dimensions: depression, hostility, anxiety, obsessions and compulsions, psychoticism, paranoid ideation, interpersonal sensitivity, somatisation. I confess that this struck me as most remarkable: the ILOP, which explores in an interactional way the self-assessment that the subject makes of the evolution of his distress in the course of psychotherapy, correlates with instruments that assess distress fundamentally through areas of functioning.

And what contribution do these results bring to the field of psychotherapy? This: with the ILOP, on the way towards the attainment of outcomes, the client can decide (identification of the goal in terms of co-construction) where he is going in his therapy, with, in relation to the instruments to which it is compared, one difference in its favour: that of benefiting the treatment still further, by representing the outcome with quantitative and graphic support. There is also this possibility: by following the ILOP, the therapeutic system can avoid straying from the course of the treatment, since it is “like a furrow of the therapy”, as Lambert also puts it.

Well, in truth the challenge goes on. It will have to be proved — and everything always has to pass through proof! — that another kind of psychotherapy obtains results that can be correlated with those of the ILOP at the CTB.

To close this second part, I once again raise my glass to continuity, and to there being ever more of us doing research on the evaluation of outcome attainment in psychotherapy.