Friday, May 31, 2013

Improving the Accuracy of Group Judgment

We would like things to be simple. This applies to group processes as much as anything else, as reflected in my earlier post, X Ways to Y Your Meetings. However, to obtain superior results, it is sometimes necessary to use group processes that are complex. In my post on the False Wisdom of Crowds(?) I referred to a process intervention in which “small, interacting groups were able to perform significantly better than their most proficient members … when aided by an enhanced, iterative, ‘estimate-feedback-talk’ process [that involved] integration of group facilitation, decision modeling, and information technology.” In response to a recent inquiry, the specifics of this enhanced, iterative, estimate-feedback-talk process are shown below, although some of the steps, divorced from the rest of the article, will be somewhat cryptic.

Improving the Accuracy of Group Judgment: A Process Intervention Combining Group Facilitation, Social Judgment Analysis, and Information Technology
by Patricia Reagan-Cirincione
Organizational Behavior and Human Decision Processes, Vol. 58, No. 2, May 1994, pp. 246-270.

Table 1: Overview of the Process Intervention

INDIVIDUAL

1. Estimate

Specification of weights and functional relations of predictors
Holistic judgments on 25 cases

2. Feedback

Display of statistically estimated weights and functional relations

3. Estimate

Reconciliation of statistically estimated and intuitively specified weights and functional relations

GROUP

4. Feedback

Display of reconciled weights and functional relations

5. Talk

Facilitated group discussion

6. Estimate

Facilitated specification of weights and functional relations
Facilitated holistic judgments on 25 cases (each judgment followed by predicted judgment)

7. Feedback

Display of statistically estimated and intuitively specified weights and functional relations

8. Talk

Facilitated group discussion

9. Estimate

Facilitated respecification of weights and functional relations
Facilitated holistic judgments on 25 cases (each judgment preceded by predicted judgment)

10. Feedback

Display of statistically estimated and intuitively specified weights and functional relations

11. Talk

Facilitated group discussion to final consensus judgment policy