These notes were first presented at The Developing Group, 9 July 2011
Calibrating whether what you are doing is working
– or not.
Penny Tompkins and James Lawley
This paper explores how facilitators calibrate when “it’s working”, and when it’s not. We know there are a wide range of ways facilitators decide if their work is effective. We also know it is not easy for a facilitator to describe the in-the-moment criteria by which they navigate a session. However, we believe that even if a facilitator cannot specify how they decide what to do, they have a duty to consider the question: How do you know when what you are doing is (or is not) working?
Definitions of ‘calibrate’
Transcript annotated for calibrating
What we calibrate against
There are two strands to our interest in this topic. One is ethical 
, the other is practical.
Let’s start with the ethical aspect. Earlier this year James had a dialogue with Steve Andreas about Andy Austin’s ‘Metaphors of Movement’ process. [See the full discussion at: cleanlanguage.co.uk/articles/articles/299/]
Steve noted that:
Andy often responds in metaphor, introducing alternatives, something that clean language would consider ‘dirty,’ offering content not provided by the client.
I agree this is a fundamental difference between the two processes. And anyone using Andy's (or any other content-introducing) method needs to answer the following questions:
- How do you know what kind of content is and is not appropriate to introduce?
- How do you know when it is inappropriate to offer content?
I strongly believe that trainers of these approaches need to make explicit the calibration required by the facilitator to notice in real time, what is and what isn’t working for a client.
I recently saw Frank Farrelly work and he not only failed to be explicit about how he calibrated whether what he introduced was valuable, he deliberately sidestepped the question when asked. Given that I witnessed a client of Frank's suffer a severe negative reaction both during the session and, they reported, for several weeks after, I consider Frank's failure to answer this question unethical.
I am not suggesting what Andy is doing is as impositional as Provocative Therapy, but I am saying that when any method relies on introducing content, the facilitator should have ways of knowing when that is inappropriate for a particular client, since any method that is influential enough to get beneficial results must also have the capacity to produce harmful results.
More recently Steve Andreas was referring to someone using Nick Kemp’s ‘Provocative Coaching’ when he said “And like all such skills, it’s essential to be sensitive to the client’s response to using it; if it’s not working, try something else.”
[Both Steve Andreas and Nick Kemp have since replied, see the ongoing discussion at: realpeoplepress.com/blog/provocative-parenting]
I agree and it begs the question how would the provocative therapist/coach know that? I asked Frank Farrelly this and he just ran a provocative pattern on the question – leaving me none the wiser. And therein lies the danger. ‘Resistance’ to the therapist/coach’s approach can always be reframed as ‘the process working’ or as a signal to ‘play’ harder. That puts the client in a bind. When the client has volunteered to be a demo subject in front of a room full of people it’s really hard for them stop a session by “leaving the field” as Bateson put it. That puts them in a double bind. So how do experienced provocative therapists/coaches know when their approach is not working?
For an example of what can go disastrously wrong when a 'facilitator' ignores warning signs, see Christopher Goodwin's article, 'At the temple of James Arthur Ray', coincidently published in the Guardian
Saturday magazine on the same day as the Developing Group: guardian.co.uk/world/2011/jul/08/james-arthur-ray-sweat-lodge-arizona.Practical
In addition to the ethical question there’s a practical angle.
Each school of therapy values certain kinds of evidence that their process is working. Many years ago we gave a demonstration to a conference of Hypo-Analysts. After the demo a leading proponent of the method sidled up to us and asked: How did you get her to cry? Having the client cry wasn’t our aim, and in and of itself it meant little to us. We thought the demonstration had gone well and the client had got a lot out of it – but not because she cried. However we presumed crying was one of that Hypno-Analyst’s top criteria.
A different example took place recently. James was a guest presenter at an international certification training for coaches. He gave a demonstration of Symbolic Modelling and the audience were stunned that he stopped after 10 minutes. The interaction with the group went like this:
Why did you stop?
Because we were done.
How did you know?
Let’s ask the client [turns to client]
I was finished. I didn’t need to go on. I’d got more than I expected.
We guess that the group and James were using different methods to calibrate whether the client had got what they wanted from the session. Here is an unedited video of the demonstration:
Clean Language Demonstration-April 2011 filmed by nlpworldwide.com on Vimeo.
NOTE: An double-annotated transcript of the session can be downloaded from SyM-Lite-calibrating-transcript-v3.pdf
and is analysed below.
This response of groups to a demonstration of Symbolic Modelling is not an isolated incident. We have demonstrated
to all kinds of groups in a variety of countries and we have often noticed a pattern. During the demo we often pause to ask the group what they
are attending to. Rarely does anyone suggest something that is high on
our list. And if they do, we interpret it in different ways. Why is this? We believe
many of the differences stem from how each facilitator calibrates
whether what they are doing is working, or not.
This paper is not
about the difference between Symbolic Modelling and other approaches. We are just as interested in the way different facilitators calibrate in Symbolic Modelling. Below is a transcript of the start of a session using Clean Language. As you read it you can wonder what on earth the facilitator was calibrating for. The client started by saying: “I want to integrate diversity without dispersing myself and diversify myself without losing focus. I want to find my balance.” The session continued:
And what would you like to have happen now?
I believe I’ve just answered that.
And what kind of believe is that?
You asked me the question and I had answered it.
What happened just before believe?
I don’t know because you asked me a question to which I had answered. Your question doesn’t belong to me. [Long silence]
And when it doesn’t belong to you, that’s like what?
It just doesn’t belong to me anymore.
And when it doesn’t belong to you anymore, where is it?
The act just doesn’t belong to me anymore. It’s over there on the mat.
And what’s the relationship between over there on the mat and your outcome?
I don’t know. It’s after the act of being asked the question perhaps.
And when balance, where is perhaps?
If you say perhaps, it’s hypothetical and I can’t answer it. I know when I do feel balance it’s here [points to chest].
And is there anything else about that perhaps?
Balance is here, the act is over there [points to the mat].
And is there anything else?
I don’t think so.
And I don’t think so, and balance is here, and what is the relationship between I don’t know and balance?
I don’t know. A right reply, I guess.
And when a right reply, what kind of right reply?
Everything is adjusted.
And when a reply is adjusted, that’s like what?
I just don’t know.
And that’s ‘I don’t know’ like what?
An honest reply.
And what’s the relationship between an honest reply and balance?
Something posed in relation to a demand for an answer.
And what happens next after an honest reply?
The act is passed. None of this belongs to me anymore.
And when it doesn’t belong to you, what kind of doesn’t belong to you is that?
[Sighing and rolling their eyes] I don’t know because it doesn’t belong to me.
Penny intervened at this point. Later the facilitator asked Penny: “What do you do with such a resistant client? How can you push through their resistance and let them know their role, and that the facilitator is in charge?”
The facilitator calibrated the client as "resistant". Penny on the other hand calibrated the client as
"exasperated" which is why she intervened.
The point of the transcript is to show how using Clean Language does not immunize the facilitator from missing signs that what they are doing is not working for the client
. It also demonstrates how a facilitator can go on and on despite obvious not-working signals. We added this transcript to ensure that the debate around this subject is not about any particular modality or technique. All are subject to being misused if the facilitator is not calibrating the client's reactions, from the client's perspective
Of course it is fine for us to pose such questions, but can we answer them for ourselves?Definitions of calibrate
Over many years we have observed thousands of coaches, therapists and
other kinds of facilitators working, and we are convinced that most
facilitators pay attention to different things than we do.
Our dictionary gives three definitions for 'calibrate':
- To mark (a gauge or instrument) with a standard scale of readings.
- To correlate the readings of an instrument with those of a standard in order to check the instrument's accuracy.
- To adjust (experimental results) to take external factors into account or to allow comparison with other data.
A classic example of definition #2 is someone from the Office of Standards checking the weighing scales in a butcher’s shop. They bring along standard weights to see if the scales are working accurately. i.e. they compare the behaviour of the scales against a criterion. Note that every now and then the standard weights themselves need to be calibrated against another standard. This in turn will need to be calibrated ... and so on. At some point someone has to say: This is accurate enough for our purposes.
Apparently the origin of the word calibrate comes from the Arabic
for ‘mold,’ which itself may have been based on the Greek meaning a ‘shoemaker's last’ – the device shaped like a human foot used to fashion a shoe.
In their Encyclopedia of NLP
, Robert Dilts and Judith DeLozier say “Calibration is the name given, in NLP, to the process of learning how to read another person’s responses in an ongoing interaction”.
This involves correlating external behavioural cues to specific internal cognitive and emotional states. Pictures of Robert are used as an example of calibrating whether he is ‘satisfied’ or ‘dissatisfied’. Used in this way, calibrating an individual’s
state is more sophisticated than those popular books that tell you how to read body language based on so-called ‘universal telltale signs’.
Our use of ‘calibrating’ is based on a metaphorical extension of the second dictionary definition above. We use the term in a similar way to its use in NLP although we are less interested in matching behaviour to specific internal states, and more interested in calibrating broad categories of states so we know what to do next. For example our REPROCess model
involves calibrating when the client is experiencing a Resource, Explanation, Problem, Remedy, desired Outcome or Change, rather than any specific state within those categories. Calibrating individual states can be useful but only if you have calibrated the broad category accurately first. Of course every facilitator says they can recognise the difference between, say, Problems, Resources and desired Outcomes, and they can
, but in real time, during hurly-burly of a session, plenty of first-hand evidence suggests the majority don’t
We are particularly interested in the calibration that takes place from the very beginning of a session, before
any changes have happened. This involves calibrating broad classes of states such as ‘preparing’ or ‘getting ready’ or ‘on the way to’. In the words of Ernest Rossi the client is laying the ground for “a period of creative inner work [and] a living experiential reality [of] re-synthesizing [their] inner world”. (p. 24, Symptom Path to Enlightenment
). But how do we know the ground work will bear fruit? We can’t know for sure, but we can play the percentages.
Calibrating whether the behaviour of a client indicates that they are benefiting from the process or not, is a form of ongoing feedback
to the facilitator. While it seems like the facilitator is calibrating the client, they are also calibrating their own effectiveness. Considerations
Below are a number of aspects which we think have a bearing on calibrating what’s working. They are are more about raising questions than about giving definitive answers. For example:
What do we mean by working or not working?
How do you know when we are being effective and when we are being in
What are the degrees of ‘working’ and ‘not working’?
What is working well enough?
How do we know when we could work better/faster?
isn’t necessarily better – every natural system has a ‘too fast’
setting when the speed of change becomes counter-productive. (The same
is true of ‘too slow’.)
We can consider these questions from a number of time-frames (from longest to shortest):
- After the therapy/coaching is complete
the time the client may not be in a position to assess whether the
therapy/coaching was effective and efficient. So how long after
do you wait to enquire, and how many times do you sample? We read of a
client who changed his assessment three times in the 10 years following
therapy. It went from ‘fantastic’, to ‘I wish I never gone’, to ‘I’m
glad I did it and there would have been a better way to do it’.- In subsequent sessions- At the end of a session - During a session:
After a change
During a change
Before a change
calibration that seems to have less said about it than any of the
others is the last one. Before
a change has happened, how do we know
that what we are doing is likely
to be valuable for the client?
Also, at what scale do we calibrate?
General models, e.g. ‘self-actualised’, ‘authentic’, ‘aligned’, etc.
Common experiential categories, e.g. REPROCess
A participant undertaking an NLP Master
Practitioner modelling project wanted to know how
international-level orienteers managed to stay on course so well. He had
thought they created a sophisticated 3D mental map of the landscape.
Instead he discovered something much simpler,
they checked their map much
frequently than less-accomplished orienteers. That meant when they got
off course, which was rare, they could easily retrace their steps to the
point of departure and continue in a better direction.
the fact that navigating a client’s mental landscape is more complicated
than orientating a physical landscape – it can change without warning
and you don’t have a map – we think there is a lesson to be learned from expert orienteers. How often do we consider whether the client is
on course? How frequently do we check whether what we are doing is
Eugene Gendlin was the first person we know to shine light on this topic.
In the 1960s Gendlin engaged in research at the University of Chicago to
figure out why some therapy sessions more successful than others.
He spent a long time trying to discern consistent patterns in
therapists' behaviour but he kept drawing a blank. So he made what we
consider to be a brilliant methodological shift. He started to study
did. He discovered that clients who got the most from
therapy experienced a "felt sense" and, if given the space and time found descriptive words that resonated with the felt
sense. Usually there
followed a "felt shift", and the person would begin to
be able to move beyond their stuck-ness. Gendlin went on develop, Focusing
, a simple methodology that facilitates people to engage in these kinds of inner acts.
In outcome-orientated approaches
the client’s desired outcome will form part of the calibration, but in spite of what evidence-based therapies say, it can’t be the whole deal. Why not? Because:
What do we do when the client can’t specify a desired outcome?
Top-down or bottom-up
How do we know it’s working when the client’s outcome can’t be achieved during the session
(which is most of the time)?
A well-specified TOTE
will tell us if we have arrived at the destination, it doesn’t tell us if we’re heading in a useful direction. Edward de Bono has said “It may be necessary to go south for a while in order to journey north”, but how long do you travel south before you conclude you’re going in the wrong direction?
The specificity of the client’s
evidence for achieving their desired outcome is an important factor. For example, with a straightforward phobia success is usually very clear. When the outcome is something like ‘fulfilling my life’s purpose’, it is not obvious how anyone would know until the client is on their deathbed.
Often simple remedies are easier to calibrate than desired outcomes. It’s not difficult to know if a burst pipe has been fixed. It takes much more skill to know if a water-system is working efficiently and is well-maintained – especially if that is being measured over the life-cycle of the system (c.f. Peter Senge, et. al. A Necessary Revolution
approaches will use qualitatively different ways of calibrating; think of sailing when you can see the destination, compared to sailing blind. To avoid getting locked into top-down criteria Nassim Nicholas Taleb recommends “stochastic tinkering
There is a sweet and oft told story:
day a man was walking along the beach when he noticed a boy picking
something up and gently throwing it into the ocean. Approaching the boy,
he asked, “What are you doing?”
The youth replied, “Throwing
starfish back into the ocean. The surf is up and the tide is going
out. If I don’t throw them back, they’ll die.”
“Son,” the man
said, “don’t you realize there are miles and miles of beach and hundreds
of starfish? You can’t make a difference.”
politely, the boy bent down, picked up another starfish, and threw it
back into the surf. Then, smiling at the man, he said “I made a
difference for that one!”
There is no doubt the boy made a difference to the starfish he threw back in the ocean, but was it a valuable
difference? And valuable for whom? What natural process was he
interfering in? What were the consequences for other species and the
rest of the environment? How often has humankind, on a global or
individual level, thought it was helping only to realise later it had
made matters worse?
In addition, it seems to us that calibrating
what’s working is closely related to three topics we have
previously covered at the Developing Group: Attending to Salience
; and Scaling
If you are of a mind to undertake a self-exploration, these questions should help:
What are you calibrating?
Where is the knowing?
What qualities or characteristics does the knowing have?
How do you know when that (signal) isn’t working?
What is your metaphor for the process of calibrating?
Of course you can ask these question of yourself too much. The resulting self-doubt can render even the best facilitator ineffective. Assumptions
Given that there is so little research available on this topic, we realise we have made a large number of assumptions. Below is a list of the ones we are aware of.
Everyone is calibrating against some
If we don’t know what that criteria is, we’re doing it unconsciously. And then how do we know we’re doing a good job? Using ‘intuition’ is fine, and do you ever not follow your intuition? If so, how do you know when to act on it and when not to?
Calibrating involves two aspects:
What’s our evidence that it is / isn’t working?
What are we noticing (seeing and hearing) that is:
Explicit in the client's external behaviour?
Implicit and can be presupposed from their behaviour?
Inferred from our model of client’s model?
How do we know the evidence is or isn’t enough?
What are our internal signals for:
Evidence of existence?
Evidence of absence?
And remember, a single black swan
can turn all previous evidence on its head.
Calibrating needs to be dynamic; it has to change depending on:
- The context
- The methodology
- What is happening for the client
- Where they are in their process (before, during, or after a change)
- Their level of development (before, during, or after a stage/level shift)
- How many times they’ve been around the loop of their patterns
- The degree to which self-deception, -delusion, -denial
Calibration is a meta-assessment.
Calibrating whether it’s working or not is a level above vectoring. It asks, is the direction I’m facilitating the client to take helping them (or looking like it is going to be valuable for them)? And based on that assessment:
Do I do more or less of what I am doing?
Do I continue or change tack?
It is insufficient to only calibrate the client:
- Being in a good state
They may feel good (temporarily) but that may not be evidence that anything has changed for the better.
“Affect does not equal effect.” (as they say at the Findhorn spiritual community)
- Having an insight
“Information does not equal transformation.” (Findhorn again)
- Being congruent
Dilemmas/conflicts are sometimes as good as it gets.
- Experiencing a change
Does the change make a valued difference? If not, so what? And who decides what is ‘valued’?
- Getting what they want
If an ‘infertile’ couple does not have the child they want, was therapy a failure?
- Praising the work
Eric Berne knew the dangers of being charmed by a client’s praise. He called the TA Game: ‘Gee Professor, you’re wonderful’.
- By their referral of others to you
Great but there can be a huge time lag, and how do you know in the meantime?
It is also insufficient to only calibrate against:
- Your feelings
You might feel it’s going well or I’m lost/confused – but what’s happening for the client? That's what counts.
- The ease/flow of the session
You might think if it’s easy it’s going well – but sometimes the opposite is true.
- Having seen something similar before
This client may be the one exception.
No matter how accurate your calibration signals, there will always
be exceptions - clients who don’t fit your norms. We once had a demonstration subject who’s signal for ‘I am waiting for the next question’ was to look down in her lap. We completely mis-read the signal and thought that she was continuing to process. It wasn’t until the next day she revealed that she had been sitting waiting for us while we were sitting waiting for her!Transcript annotated for calibrating
You can download a transcript of the 10-minute video'd demonstration
mentioned above at: SyM-Lite-calibrating-transcript-v3.pdf
It has been annotated in two ways:
a. For what James was modelling in the client's information
b. For what James was calibrating in the client's responses.
An analysis of the transcript revealed that James asked 24
questions in total (excluding the Set Up and Set Down phases and the three repeating back the clients words without asking a question):
3 Like to have happen?
6 Anything else?
4 What kind of?
2 When ... What happens to ...?
4 Then what happens?
James reflected after the self-modelling:
the information and calibrating the client are two closely interlinked
processes. In this particular case I have put much less in the modelling
column than is usual. I think the client played a big part in this (a)
because it was a simple landscape with just a few symbols, (b) he gave
very concise answers, and (c) he kept close to his original frame. Much
of the time it was 'obvious' what to ask about next in order to keep the
process flowing. What we calibrate against
I think this transcript shows that the length and
complexity of a session is sometimes down to the client distracting
themselves, loosing the frame, talking for the sake of it, not being
able to select for salience, and so on. When you add in that the
facilitator can do all of these things too, it's no wonder sessions get
convoluted. And while facilitators need to learn to calibrate what is
working and what isn't – so do clients!
Below is a tentative first-pass at describing some of the factors we take into account when calibrating whether what we are doing is working or not.
In the notes for the May 2011 Developing Group
we introduced ‘Three principles of Symbolic Modelling’. The third principle states:
Calibrate whether you are encouraging conditions for change - or not.
We use the term ‘encouraging conditions’ to emphasize that we can’t make a client change, nor can a technique, and often a client is powerless to change themselves – until the conditions are right for them. Then the system re-synthesises of its own accord, often much to the surprise of the client and facilitator alike.
What are the conditions to be encouraged? And how do we know the client is experiencing them? To assist in answering these questions we have extracted the criteria being calibrated (not the client content being modelled) in the attached transcript:
Calibrations during the 6 SyM phases from attached 10-minute transcript
Set Up (Phase 1)
He [the client] considered the spatial arrangement and now seems settled
He’s actively participating in setting up the configuration
The distance and angles are ok for him
Identify desired Outcome (Phase 2)
He is getting more involved in his process
He has some access to his symbolic world
He is genuinely interested in the topic
It seems appropriate to work with this topic in this context*
His body is involved in his desired Outcome
Develop desired Outcome Landscape (Phase 3)
in touch with a rarely accessed state
he is registering each of his words
readily answers the ‘Where?’ question*
gestures are marking out his perceptual space
attending to what’s happening to him now
his inner landscape is quickly becoming psychoactive*
growing awareness of his inner spatial world
in touch with the location of the symbol
‘went deep’ to get answer
info is coming to him involuntarily
not the kind of answer I expected*
he is surprised
acquainting himself with this aspect of himself
whole body is involved
curious about his own landscape
He is making (new) connections
getting and responding to unbidden information (psychoactive)
surprise to me* and I surmise to him too
embodying enough of his desired outcome landscape*
Explore effects of desired Outcome landscape (Phase 4).
living his metaphors in present time*
[his metaphors and nonverbals suggest] maybe something is shifting
his response to the changes - he seems ok if a little bemused
huge amounts of processing is going on inside his mind and body
fully experiencing the state in the moment
maybe getting a little frustrated/helpless
Set Down (Phase 6)
the time limit*
his connection with the audience comes back
The first thing to note is that the majority of the criteria are from the perspective that Ken Wilber calls "Zone #2 singular": an interior (of an "I") looked at from the outside, i.e. a third-person modelling of a first-person reality, from their perspective.
The criteria marked with an * are from different perspectives. 
We can summerise the above as calibrating whether the client is (or is not):
1. Increasingly accessing and aware
of their inner (symbolic) world.
2. Increasingly involved and engaged
with the inner (symbolic) world.
3. Increasingly embodying
their inner (symbolic) world.
The client is at some level always embodying what’s happening. And when that becomes more overt and living-it-now it is often a prelude to a shift in the system (although embodying a memory of a trauma can be counterproductive).
The three criteria above lead to psychoactivity
- The client responding unbidden to their own inner symbolic world, especially with curiosity, wonder and surprise.
Other criteria are that the client:
4. Goes with the flow. Although not specifically mentioned in this annotation, there is
often a palpable moment when the client lets go of trying to control and
understand what is happening and "takes
the hand-brake off" as one client put it. This is not a passive 'do it to
me' but rather an active participation in the unknown.
5. Is identifying, developing and exploring the effects of
their desired outcome landscape.
6. Engages in deeper than usual processing
, jumping to a pattern level or
making new connections that lead to a different engagement with or
perspective on the material.
that they are deriving (potential) benefit from what’s happening (their nonverbals are often a kind of meta-comment too).
8. Indicates that a change or shift
- more changes (a contagion) leading to a difference that makes a valued difference
- a more resourceful response to a previously perceived problem
- an alteration to a pattern that aids the desired outcome happening
- a condition necessary for the desired outcome to occur
- the desired outcome to start happening.
- a reaction which functions to keep things the same. This indicates the client's system is responding in the here and now and a new aspect is revealing itself – and that can be worked with 'live'.
As we have hinted, for every criteria we, or anyone defines, there will be contexts when the opposite applies.Last thoughts ...
One of the
reasons some people who have lots of experience find the transition to a
clean approach challenging is because they have to set aside their
usual calibration and learn to calibrate with a new set of filters. We
can’t teach them directly how to do that, they have to acquire it for
We developed our coaching in the moment
model in part to give trainee facilitators the chance to experience
first-hand being guided by someone who has well-honed calibration
skills. The expectation is that the trainee will code the pattern of external
signs and develop a set of internal signals for when the process is
working – and when it’s not.NOTES
1 See also our comments on John Grinder’s video about ‘What’s ethical in NLP’ at: cleanlanguage.co.uk/articles/articles/244/.
2 p. 137. The section on calibration includes exercises to develop sensory acuity at: nlpuniversitypress.com/html/CaCom.html
3 See James’s blog on Desired Outcomes cleanlanguage.co.uk/articles/blogs/19/
4 This is one version of many based on the original Story by Loren Eiseley, see en.wikipedia.org/wiki/The_Star_Thrower
5 See James’ blog, 'The' Therapeutic Relationship cleanlanguage.co.uk/articles/blogs/48/
6 Since the paper was written James has been involved in discussion about a closely linked topic, modelling criteria: cleanlanguage.co.uk/articles/blogs/55/
7 Pioneered by Maturana and Varela, this is also known as biological phenomenology, since it attempts to describe the phenomenal inner world of the organism itself. For a very detailed description of Ken Wilber's "8 native perspectives" see his Excerpt D: The Look of a Feeling: The Importance of Post/Structuralism, kenwilber.com/writings/read_pdf/86.