The Guardian Australia publishes an inaccurate article about a customer satisfaction survey posing as a medical study

 On Friday, 8 September 2023, The Guardian Australia published an article by medical editor Melissa Davey, Fast access to hormone therapy in transgender adults ‘lifesaving’, study finds, which reported on a clinical trial led by the University of Melbourne. A thorough critique of the study has already been published. 

Through omission, the Guardian article misrepresents the nature of the study in an important way, stating in its second paragraph that: 

It is the world’s first study to examine the impact of access to hormone therapy on mental distress for trans people through a randomised control trial, a strong type of study crucial in medicine due to the minimal risk of confounding factors influencing the results. 

A randomised control trial is not inherently “a strong type of study”. Its strength depends on the type of controls used, the outcome measures, whether the study is blinded, and on the appropriate interpretation of results. 

Blinding refers to concealing from persons involved in a clinical research study the treatment group to which participants have been allocated. The purpose of blinding is to mitigate sources of bias which can influence the study outcome, for example, those which might arise from the participants’ knowledge of which treatment they are receiving or during the investigators’ assessment of results. 

Blinding is an important means of controlling for the influence of participants’ psychology (beliefs and expectations) on study outcomes. It does this by exploiting something known as the placebo effect. The placebo effect is the observed ability of non-active treatments (placebos) to produce variations in outcome measures. By comparing outcome measures in a placebo group and a treatment group, it is possible to estimate to what extent variation in the outcome measures in the treatment group is in fact due to the treatment rather than the participants’ psychology. 

Without blinding, it is not possible to know whether an observed improvement in symptoms is due to the active treatment or to the patient’s knowledge of treatment. For this reason, “blinding is widely viewed as a core tenet of sound clinical trial study design.” 

Crucially, the Guardian article omits to mention that the trial was open-label, i.e., not blinded. A given participant and the investigators knew whether the participant in question was receiving testosterone therapy or not. This lack of blinding is significant and ought to have been reported by The Guardian, not just because it would have alerted the reader to the absence of this very important control for bias, but also and especially because of the susceptibility to bias of the outcome measures which the study used. 

The primary outcome measure in the reported study was gender dysphoria, measured by the Gender Preoccupation and Stability Questionnaire (GPSQ, which was developed by Dr Az Hakeem and colleagues). Secondary outcome measures included assessments of depression and suicidality, also using questionnaires. The outcome measures, in other words, were all of psychological states ascertained by participant responses to questionnaires. These are susceptible to various influences including “knowledge of treatment”, i.e., the knowledge that one is being treated and how one is being treated. 

It is reasonable to expect that knowledge of treatment, especially of how one is being treated, would exercise a considerable influence on the psychological states (and therefore on their corresponding outcome measures) of the cohort of patients in question. These patients were described as “desiring” testosterone therapy, and who doesn’t feel better when they get what they desire, especially when what they desire is widely depicted as a personal affirmation, human right, and matter of life and death (“life-saving”)? 

It is therefore impossible to know to what extent the variations in the outcome measures were due to the testosterone or to the patients’ knowledge of treatment, and the beliefs and expectations this knowledge engendered. While the authors of the study acknowledge this limitation, they do not explain why blinding was not implemented to overcome it in the first instance, nor do they articulate in detail just how significant a limitation it is. The Guardian does not mention this limitation at all. 

The Guardian also fails to report on other limitations of the study which become apparent when it is appropriately contextualised. Chief amongst these is its failure to report the sex of the participants. 

We can only infer based on common assumption and the information provided—namely, that the participants were “transgender and gender-diverse individuals aged 18 to 70 years seeking initiation of testosterone therapy with no previous testosterone treatment”—that the participants were female. The study does report the “gender identity” of the participants as “binary” or “non-binary”, but these fluid categories encompass males and females. 

While the inference to female sex is probably reliable insofar as gender dysphoria is defined (implicitly and questionably) by the DSM-V as an exclusively cross-sex phenomenon, we query why—in de-identified medical literature—such a basic biological datum, commonly reported in medical studies because of its significance for the correct interpretation of findings, should have to be inferred when it can simply be reported. After all, “Biological sex assigned to you at birth” (with response options “male/female/intersex”) is included in the GPSQ. 

The study’s failure to report the sex of the participants may have enabled The Guardian’s misleading headline and accompanying article summary: “Access to immediate testosterone therapy significantly reduces gender dysphoria, depression and thoughts of suicide, research finds.” 

This was not what the research found. 

The study concludes more accurately: 

This open-label randomized clinical trial supports the use of testosterone therapy to significantly reduce gender dysphoria, depression, and suicidality in transgender and gender-diverse adults desiring commencement of testosterone therapy. These findings have critical implications for service access and delivery to ensure timely access to gender-affirming hormone therapy. 

In other words: 

• The study draws no conclusions about the speed of access to hormone therapy.  

• The study relates only to transgender and gender-diverse adults who desire testosterone therapy and, if our inferences are correct, to females who desire testosterone therapy. 

• The study does not use the term “lifesaving” at all. 

This term is used by one of the study’s investigators, Associate Professor Ada Cheung, who provided comment for the Guardian article, with the important qualification that “life-saving” is “how patients describe it”. 

The Guardian, which in theory places “Accuracy” foremost in its Code of Practice and Guidance, ought to have reported accurately the details noted above. It should also have reported that the study pertains to a cohort between the ages of 18 and 70 years. This information is relevant in the context of societal debates about and criticism of “gender-affirming care”, which relate not so much to adults as to children and youth (i.e., under-18s). 

Regarding cross-sex hormone “treatment” for children and youth, the Swedish National Board of Health and Welfare (NBWH) issued an update to its healthcare guidelines in February 2022, which among other things asserted that “the risks of hormonal treatments outweigh the benefits for most gender-dysphoric youth”, and recommended a “more cautious approach that prioritizes non-invasive interventions”. Similarly, there has been a stark shift in Denmark away from puberty blockers, hormones or surgery towards therapeutic counselling and support. 

While a clinical study cannot comment on factors which lie outside the scope of its data except to acknowledge how these may limit the significance of its findings, we think a news article which presents research findings to a wider readership not only can do so but has an obligation to do so. That obligation is not limited to accurately reporting the research, including its limitations; it extends to situating the reported findings in the wider context of research and society, and differentiating the issues involved. 

Finally, regarding the study itself, we must note that its findings may be summarised as: Adults who desire something are happier 3 months after they get it. The study's authors may be dismayed to learn that they have been beaten to this earth-shaking insight many times over by every company which runs customer satisfaction surveys. 

But there is more at stake in medical practice than whether a patient (customer?) is satisfied. Even within the field of cosmetic interventions, the practitioner, the profession, and society at large are not released from safety and ethical considerations merely because a medical intervention is what the patient (customer?) wants and because the patient feels better three months after getting it.