Re. Goertz et al. Extrapolating beyond the data

To the EditorI read with interest the alternate analysis offered by Goertz et al1

Goertz CM, Hurwitz E, Murphy B, Coulter I. Extrapolating beyond the data in a systematic review of spinal manipulation for nonmusculoskeletal disorders: a fall from the summit [e-pub ahead of print]. J Manipulative Physiol Ther. doi: https://doi.org/10.1016/j.jmpt.2021.02.003. Accessed April 17, 2021.

and would like to add to the comments regarding the limitations of systematic reviews, including important limitations I have observed but that were not discussed.In particular, I point out the obvious, which is that the entire systematic review process is as much a social exercise as a scientific one, and so may be subject to strong and even destructive social influences.2Collaborative irrationality, akrasia and groupthink: social disruptions of emotion regulation. My recollection is that the first invitation to the Global Summit referred to vitalism as the focus, not clinical practice, and, although later the purpose was reframed, they conflated management of nonmusculoskeletal conditions with disregard for evidence. Thus, notwithstanding whatever scientific merit the Summit claims, it appears, nonetheless, to be motivated by a desire to engineer the profession of chiropractic. This motivation in itself is not unjustified but appears to have modulated the methodology in a way that undermines the credibility of its rather far-reaching conclusions.Thus, I would like to draw attention to my observations of the social dynamics of the Summit. In the field of psychology, one will see discussion of the continuum between consensus, compliance, and coercion.3Towards a theory of collective phenomena: consensus and attitude changes in groups. In this regard, when we see a “purposive and snowball sampling” to solicit participant reviewers for the Summit, this causes us to ask what, then, was the purpose? Was it to listen to a diversity of expert opinion, or were there other considerations? Note that 42 of the 46 authors had previously coauthored articles with other members of the group, and in some cases prior coauthorships numbered in the dozens. Certainly, the chiropractic profession is a small pond to draw from; however, should one consider whether the social structure of this group likely modulated independent thought?4Culture and conformity: a meta-analysis of studies using Asch's (1952b, 1956) line judgement task.

Of equal concern, in some instances, participants were in positions of power over other participants. Did the organizers understand the pressure that the junior reviewers would be under to manifest consensus agreement with their supervisors or senior colleagues? Should it also be a concern that the authors included colleagues whose previously published opinions on the topic calls into question their ability to conduct an unbiased examination of the literature?

The dynamics of the enterprise were further contaminated by having the Summit's work take place under the examination of observers including reviewers’ employers, insurers, and professional regulators who were given the opportunity to interact with the participant-reviewers in the course of their work. In sum, these conditions prime the collective for considerable bias in the execution of its deliberations. In a work that so scrupulously examined risk of bias in the literature under review, perhaps the Summit participants ought to have turned the mirror on themselves.

Given human nature, no matter how contrived the ceremony, systematic reviews represent opinions, and consensus cannot avoid amalgamation with compliance or coercion. I believe that these concerns should also be taken into consideration with the analysis by Goertz et al. when interpreting the results and recommendations of the Summit.

References

Goertz CM, Hurwitz E, Murphy B, Coulter I. Extrapolating beyond the data in a systematic review of spinal manipulation for nonmusculoskeletal disorders: a fall from the summit [e-pub ahead of print]. J Manipulative Physiol Ther. doi: https://doi.org/10.1016/j.jmpt.2021.02.003. Accessed April 17, 2021.

Collaborative irrationality, akrasia and groupthink: social disruptions of emotion regulation.

Frontiers Psychol. 7: 8

Towards a theory of collective phenomena: consensus and attitude changes in groups.

Eur J Soc Psychol. 21: 49-74

Culture and conformity: a meta-analysis of studies using Asch's (1952b, 1956) line judgement task.

Psychol Bull. 119: 111-137Article InfoPublication History

Published online: June 15, 2021

Accepted: April 19, 2021

Received: April 19, 2021

Publication stageIn Press Corrected ProofIdentification

DOI: https://doi.org/10.1016/j.jmpt.2021.04.005

Copyright