The role of patients' meta‐preferences in the design and evaluation of decision support systems

Introduction

The arrival of new decision technologies will necessitate a profound rethinking both of the nature of the patient–doctor relationship and of the way aids and support systems designed to improve decision‐making within that relationship are designed and evaluated. The development of decision‐analysis based programmes such as CODA ¹ DARTS ^{2 , 3} PORTAL and ALCHEMIST ⁴ and WEDS ⁵ points to the growing necessity to distinguish the nature of the doctor–patient relationship from the modes of judgment and decision‐making employed within that relationship. Implicitly one‐dimensional typologies of the traditional `paternalist/shared/informed' <sup>6</sup> sort do not separate these two dimensions, nor do they provide the complexity called for by the heterogeneity of patient's `meta‐preferences' regarding each. A multidimensional matrix embodying this distinction is proposed as a framework of the minimal complexity required for the design and evaluation of the full range of decision aids and decision modes. Essentially aids should be conceived of and evaluated cell‐specifically and the search for universally satisfactory decision support systems abandoned. The decision process which emerges when patient's meta‐preferences regarding relationship model and decision mode are fulfilled may be very different from that which would constitute `shared' or `informed' decision‐making, as well as different from that which would maximize fulfilment of the patient's conventional health outcome preferences. `Shared' and `informed' are therefore best interpreted as attributes of the relationship model, with future research focusing on the more relevant, higher level, goal of better decision‐making. This is a concept which will have to embrace both meta‐ and health state preferences and patients will be genuinely empowered only when they make any needed trade‐off between fulfilment of their meta‐preferences and their health state preferences on an informed and transparent basis.

Evaluating decision aids and decision technologies

The purpose of CODA, the Clinical Guidance Program ¹ (CGP) is to provide, in real clinical time, the quantitative results of modelling a management decision, using the best available evidence‐ or more realistically a highly defensible body of evidence – in conjunction with the patient's preferences over health outcome states elicited in situ. The program is best thought of as providing a decision analysis‐based `third opinion' rather than as a decision aid or support as conventionally conceived (see below). It is accordingly very different from previous systems such as decision board <sup>7</sup> , 8, <sup>–9</sup> the probability or treatment trade‐off method <sup>10</sup> , 11, <sup>–12</sup> and various `interactive' video programmes. ¹³ , 14, ^–15 Generalising about these, Holmes‐Rovner writes

Decision supports go beyond traditional informed‐consent approaches, to engage the patient in grappling with the elements of the decision, as well as understanding the alternatives. … Some incorporate values or utilities, in a heuristic, but not prescriptive fashion. Most have decision analysis behind them to organize the presentation of the outcome data, but do not explicitly reveal the rational intent of the analysis. They deliberately leave out the `right' answer. ¹⁶

However, the fundamental aim of CODA and other CGPs is to make transparent the `rational' intent of the analysis and to offer the answer produced by that analysis. However, this `answer' is properly characterized as a defensible candidate for `best' answer, rather than for `right' answer, because the latter will rarely if ever exist. And it is only a candidate for best answer, given the assumptions, data and structural/institutional constraints built into its use. However, this qualification is true of all modes of decision‐making, including any that will be used instead of the CGP or to reject its guidance.

One vital consequence is that we are primarily concerned with the quality of its guidance and not whether that guidance is accepted and/or implemented. There are multiple reasons why guidance of the highest quality might be rejected (Dostoevsky's Notes from Underground is a more readable source than most psychological texts) and CODA makes no attempt to persuade the patient (or doctor) to follow the guidance, other than noting that the result follows from a serious analysis of the decision using the patient's preferences over outcome states. The third O'Connor criterion for the evaluation of a decision aid ¹⁷ – that it is `effective' (i.e. implemented) – is accordingly seen as inappropriate, in that it assumes either that patients are `rational' or that it is the function of the aid to make them more so.

As a practical way of implementing decision analysis in the clinical encounter the CGP is put forward rather as a mode of decision‐making– a way of arriving at clinical decisions or recommendations – not something which is seen as making a contribution to, or within, some other mode of decision‐making. The main alternative, and currently dominant, modes of decision‐making are `traditional clinical judgement' on the one hand and `evidence‐based clinical expertise' (increasingly facilitated through evidence‐based guideline statements or algorithms) on the other. ¹⁸ (Decision‐making by following a `knowledge‐based expert system' represents a third but largely hypothetical possibility.)

How, then, should an intervention such as a CGP be evaluated? Is it really an `intervention' at all? Most researchers designing evaluation instruments <sup>19 , 20</sup> assume that decision aids should be designed to help the patient make an informed choice in a decision‐making process where the mode of decision‐making used by the clinician is a given and usually assumed to be `traditional clinical judgment'. Most current research and commentary on decision aids and support systems also assumes that the key patient preferences that need eliciting (or constructing) are those impinging on choices regarding the medical management of their condition, typically phrased as the patient's preferences in relation to the knowledge and information necessary to adequately comprehend `the risks' and `the benefits' of the various options.

The new modes of decision‐making (`decision technologies') now becoming available and sure to multiply rapidly in the coming decades, call both these assumptions into question and necessitate serious re‐thinking of the methodology of evaluation. Two key issues warrant attention.

One concerns the distinction between knowledge validity and decision validity in relation to the inputs into any decision‐making process. This distinction is developed elsewhere in the context of the `condition‐specific measure' vs. `generic measure' debate in Health‐Related Quality of Life ²¹ and cannot be considered further here. The most important implication for the current paper is that the application of knowledge validity criteria in advance of decision validity criteria is a frequent accompaniment of partial or noncomparative evaluation. ^{22 , 23} In this deviation from sound practice the flaws or limitations of a particular method or approach (e.g. decision analysis), established in isolation, are regarded as conclusive without any or adequate consideration of the equivalent flaws or limitations of the alternatives (e.g. clinical judgment).

The issue on which we concentrate here concerns the way the clinical consultation is analysed and modelled. Previous analysts have constructed insightful taxonomies ^{6 , 24} but all have been implicitly or explicitly seeking a uni‐dimensional classification. The emergence of CGPs, and to a lesser extent expert‐ and/or evidence‐based clinical guidelines, makes it clear that two dimensions are needed. They derive from what can be conceptualized as the two meta‐decisions that are made before a clinical encounter proceeds. By definition, these involve deciding how to decide. Meta‐decision 1 determines which model of the physician‐patient relationship is to apply at the outset of the consultation (and, as default, throughout it). Meta‐decision 2 determines which modes of judgment and decision‐making are to be adopted within the selected model of the consultation. Currently both are made implicitly and (generally) with no explicit discussion or patient consultation. As soon as their existence is recognized it becomes apparent that patients will have different `meta‐preferences' in relation to each and should be able to make informed choices on the basis of these two sets of meta‐preferences. In the following section we propose a model/mode matrix as a framework within which patients could specify the cells, or cell clusters, within which they wish to locate the upcoming consultation. Clinicians would be entitled to (explicitly) opt out of offering consultation located in particular cells, but increasingly be expected to be competent to perform in multiple roles and modes.

The evidence for the existence of strong meta‐preferences of the first type has been around for a long time ²⁵ , 26, ^–27 and recent writings confirm it in a variety of clinical contexts. Two examples will suffice.

Interviewees were divided in their preferences for nondirective counselling: 46% would have liked to get direct advice from the counsellor, 36% said the counsellor should only provide information and not advise the couple about the right decision, and the remainder had mixed feelings. …Genetic counselling is one of the first medical areas to adopt a nondirective, patient participation approach as its norm – an approach that conflicts with client's prior experience with a directive, paternalistic medical environment. Counsellees were equally divided between those who approved and disapproved of this counselling strategy ²⁸ .

[Our] findings are consistent with other research indicating that younger, more educated patients generally prefer a more active and participatory role in medical consultations and decision‐making than do older and less educated patients. However, patients' perceptions of involvement and control over decision‐making did not differ for patients differing in age and education. …Some people (termed `blunters') generally avoid information related to their condition while others (referred to as `monitors') actively seek out this information. …Future studies …should assess patient′ preferences for their own and their physicians involvement to see if these beliefs explain why some patients are more active communicators than are others. ²⁹

In view of the consistency of this finding what is surprising is that many studies still seem frustrated by their failure to establish solid generalizations about patients (or aids). They treat the heterogeneity of patients' preferences as confounders rather than as a fundamental starting point for useful work. A number of reviews and overviews have implicitly accepted this framing. ³⁰ , 31, ^–32 The resulting frustration disappears if the evidence is interpreted in the light of meta‐preferences and in a later paper Entwistle notes that:

…individual patients and clinicians vary in their preferences for different approaches and their ability to adopt particular roles in decision‐ making. Imposing an approach with which either or both parties are uncomfortable may be inconsistent with the notions of respect for individual choice that many of the approaches aim to support. ³³

She notes that most developers of conceptual models of clinical decision involving patients have adopted a particular normative model, usually one in which the patient plays either a collaborative or an autonomous role. But full acceptance of Entwhistle's point will require abandoning two of the basic assumptions held by most of those in the patient empowerment field. One, that patients should become more involved and participatory, because it is in their interests to do so – either for the instrumental reason that it is better for their health or because they simply ought to be more autonomous and resistant to directive paternalism. (In most cases for both reasons.). Two, that becoming more informed is a necessary condition of being more involved.

Interventions to support decision‐making can be seen as affecting a wide range of processes and outcomes, such as:

…knowledge and understanding; who contributes in what way to decision‐making; what factors influence the choice made; the lengths of (and hence the costs and numbers of other available) consultations; the quality of the decision made; professional‐patient relationships (in both the short and longer term); the delivery of health‐care; health‐related behaviours; the outcomes of health‐care; people's general sense of wellbeing; and their expectations and satisfaction of patients, their families and health professionals with various aspects of health‐care and its outcomes. People will have different opinions about which of these criteria are most important and how they value particular effects. ³³

But the vital methodological issue is whose preferences are to be accorded priority in respect of any and all of such aspects. If it is accepted that it is the individual patient's preferences that count then the idea that aids and supports can be evaluated in anything but preference‐based trials, where the preferences relate to aspects of the decision‐making process and not to management options, is ruled out.

The first assumption (that patients should become more involved) is typically left latent rather than made manifest, so the fact that it clashes with the view, typically quite explicit in the same writing, that patient's preferences should be respected, is rarely noticed, let alone highlighted. But the conflict is clear and once it is exposed one needs to take a position on it. Our personal position is that patient's current meta‐preferences should be respected in the upcoming clinical consultation, and that it is the task of education, as in all other areas of personal life, to `develop' these preferences and ensure that they are based on adequate knowledge and exploration of the consequences of holding different ones. The clinical consultation is not the place for this personal preference development in relation to health‐care decision‐making and it should not be seen as exclusively or even primarily the responsibility of doctors.

The second assumption (that becoming more informed is a necessary condition of becoming more involved) seems so obviously true that it is never questioned. However, as we will see when we set up the model/mode matrix, the most `rational' and involved of all patients may be one who seeks intense value clarification and substantial decision responsibility – but wants little or no `information'.

The model‐mode matrix for clinical decision‐making

The patient is assumed to have preferences regarding the way judgements and decisions are made about their medical care as well as preferences regarding the health states that are the foci of those judgements and decisions. Specifically, they have a preferred model of the doctor–patient relationship and, conceptually independent though probably empirically correlated, a preferred mode of judgement and decision‐making within that preferred type of relationship. These two meta‐preferences are relevant to the two meta‐decisions taken before a consultation begins. At the moment these are taken implicitly, but we can envisage them increasingly being taken explicitly as the range of decision technologies expands and patients (and doctors) come to know of their availability through the internet and personal networks.

These meta‐preferences relate to the individual consultation. They may vary from case to case, not only as the presenting condition of the patient changes, but also as the sequence of consultations within an episodes progress. The degree to which the initially agreed character of a consultation can be changed during it would be up to the parties, but the existence of flexibility in this respect in no way undermines the value of the framework in locating and illuminating what is going on at any point. Neither does the `instability' of an individual's meta‐preferences through time. Such lability of preferences is well‐known to make analytical decision‐making more difficult but its existence is not a ground for ignoring or denying it.

It might seem that these meta‐preferences relate exclusively to what Elwyn and co‐workers call `the second half of the consultation'. ³⁴ They are referring to general practice, where the early rituals of the consultation can certainly take up a high proportion of the available time. But even here we suggest that a clearer identification of all management decision‐making (including decisions about diagnostic strategies as well as therapy) will bring a very high fraction of the encounter within the scope of the framework.

At this stage the precise specification and wording of the attributes and levels are still under development, as is the instrument to capture patient's meta‐preferences. However, this is the appropriate time to air the broad issue. It may or may not be necessary to point out that any suggestion that the framework is too simple needs to be made in full realization of the practical consequences of further additions. And that any suggestion that it is too complex involves denying the importance of distinguishing between the identified elements.

The multidimensional model of the relationship

The modes of judgement and decision‐making

The model‐mode matrix

Crossing model with mode creates an initial matrix of 27 cells and forms a framework within which we can explore the issues surrounding the evaluation of the clinical consultation in general and the contribution of decision aids and support systems in particular.

In fact, the 27 cells quickly reduce to 18, as Table 1 shows.

Three cells, labelled x–ia, disappear when it is recognized that higher information provision preferences cannot be fulfilled by insufficiently analytical modes. And another 6 are lost when the patient's desired level of involvement in value clarification rules out 3 as insufficiently analytical and 3 as over‐analytical (x–oa).

Of course many cell preferences (most of those where the Mode is 4 or 5) could not be accommodated at the moment, even if requested by patients from doctors willing in principle to offer such modes. It is important to remember that our intention here is to ensure that evaluation of the new technologies capable of fulfilling these preferences are properly evaluated as and when they do arrive.

There are many more than 18 possible individual sets of preferences in the framework because each of the nine valid Decision responsibility (DR) cells may, at least in principle, be coupled with any of the six valid Information provision (IP) ones and with any of the three valid Value clarification (VC) ones. The total number of permutations is therefore 162. No particular point is served at the moment by presenting a full tabulation. A few selected examples should enable the reader to assess the value of the framework for determining the suitability or unsuitability of particular aids or supports and ensuring that their evaluation is conducted properly.

The patient meta‐preference codes come in the form of a pair of numbers for each dimension. Within each pair the first number represents the preference level within the dimension concerned, the second the Mode preference for that dimension.

Conclusions

The current movement for patient empowerment largely takes for granted the virtues of `shared' decision‐making and/or `informed' decision‐making. The time has come for the focus to move from all such intermediate and partial goals to the top‐level goal of better decision‐making. Since we take it as axiomatic that any concept of `better' must have a basis in preferences our framework suggests that two fundamental sets of preferences are relevant to `better decision‐making'. One set is made up of the patient's meta‐preferences regarding the process of decision‐making, i.e. preferred relationship model and preferred judgement and decision modes. A decision will be better to the extent that it reflects these meta‐preferences. The second is the patient's preferences over the health state outcomes that are involved in the decision. Here, a decision will be better to the extent that, given these preferences, it reflects the best relevant patient‐specific evidence. If, as seems likely, there will often be conflict between maximising these distinct sets of preferences, the ethical answer is (presumably) that the competent patient is the one who must make the necessary trade‐off – subject to any constraints flowing from the publicly funded character of the health‐care system.

It has only been possible to adhere to simple, three or four way, unidimensional, models of the doctor–patient relationship because only one decision technology has been available until recently – that referred to as `clinical judgement'. But as new decision technologies, embodying increasing analytical content, become available ⁴⁴ the attempt to fit things within these simple frameworks will be increasingly invalid and unproductive. The necessity of disentangling relationship model from judgement and decision mode will become ever more starkly evident. A multiattribute model of the relationship which distinguishes the conceptually independent attributes of decisional responsibility, information provision, and value clarification, seems requisitely complex as well as mapping appropriately on to the alternative modes of judgement and decision‐making. The question of precisely which patient preferences are to be respected, and how, is made transparent and unfudgable. And the foundation for a coherent and transparent doctor–patient encounter laid.

If patient preferences are to be taken seriously they must be taken as fundamental in the same way that their clinical condition is taken as fundamental. No one would suggest trialling a drug targeted at a particular disease on a random group of patients, irrespective of their diagnosis. In relation to the design and evaluation of decision aids and modes, meta‐preferences regarding relationship model and judgement and decision‐making mode should be treated as the equivalent of the clinical condition.

Acknowledgements

I am grateful to Ian Pell for coping with all the programming difficulties associated with producing a Clinical Guidance Program that works, as well as for the insights into decision support he provided, in conjunction with other members of the project team (Aileen Clarke, Andrew Kennedy and Vanita Bhavnani). We were devastated when he died recently. Alan Williams, Richard Thomson and Angela Robinson have also made helpful comments, as have the audiences at an INSERM workshop and the inaugural meeting of the Oxford Shared Decision‐making Forum.