To the summary of update 1

Imagine you had published an excellent study conducted according to strict scientific criteria, providing solid evidence that a particular therapy can help patients suffering from a fatal disease. You have shown that the survival time of your patients can be extended considerably, far beyond what can be achieved today with conventional therapy. Not only do patients live longer, they also do so with a significantly better quality of life and feel considerably better subjectively. In short, progress could be achieved that would otherwise have to be worked hard for over decades of research.

And then someone comes along and claims that the data appears to have been manipulated. Essential study parameters were changed at a point in time when the test subjects' data was known, thereby distorting the results in a positive direction. These critics point out that the published results exhibit characteristics that arise precisely from such corrections. The result would be that, if scientific misconduct is suspected and confirmed, it could cost you your scientific reputation, and quite completely at that.

What would you do? 

You could ignore the allegations as completely absurd, because you are confident in the scientific quality of your work. After all, if serious academic criticism were to arise, you could prove at any time that your results were obtained using a scientifically sound approach. 

To put an end to the matter once and for all, you could also point to some data and/or documents that the critics, in their ignorance, probably overlooked or were unable to classify and evaluate correctly. 

Perhaps you could even clarify some of the points referred to by the critics and, if necessary, provide additional information to prevent further misunderstandings. 

Or would you try to find grounds for legal action for libel or defamation? 

Well, we actually expected something like this to happen after we wrote to the authors of the study on homeopathic adjunctive treatment for non-small cell lung cancer (NSCLC) [1] published in October 2020, on June 6, 2021. In this message, we sent the lead author and co-authors our detailed analysis of the study, which corresponds in content to the blog posts that appeared on the websites of the Information Network on Homeopathy and the Initiative for Scientific Medicine [2]. 

It would also be conceivable, although in our view rather unlikely, that a justification for the large number of exclusion criteria would be provided and an explanation given as to why other serious age-related complaints did not lead to exclusion from participation. Alternatively, an updated flow chart could be presented showing when and how many patients were excluded. Alternatively, there could be explanations for the numerous serious adaptations to the protocol, but these were apparently not deemed important enough to mention in the published study. 

But so far (end of July 2021), nothing of the sort has happened. Instead, on June 14 and 16, 2021, less than two weeks after our message, the data in the study registry entry at ClinicalTrials was revised again and a new version of the study protocol was uploaded [3]. The changes made can certainly be interpreted as an attempt to obscure all the points of our criticism. Of course, the earlier data and the earlier version of the protocol are still available in the registry, but now they are buried deeper, and you have to scroll down to the end of the entry to find the inconspicuous link titled “History of Changes.” Who would go to this trouble without a specific reason? (Addendum 3/28/2025: The clinicaltrials.gov website has since been redesigned. Some links are no longer as indicated here).

Unlike previous versions, the information in the registry now matches that in the published study, as it now includes a complete list of exclusion criteria as well as the shortened observation period for quality of life, which is the primary endpoint. Please note: Throughout the entire study period, until the end of the observation period for the last patient and until the completion of data collection, pregnancy was the only exclusion criterion and the observation period was generally set for the entire study period. All of this was only changed when the first version of the protocol was uploaded, which did not happen until two months after the end of the study—in other words, when the data was most likely already known and the first analysis results were available. 

Furthermore, a new version of the protocol has now been uploaded, this time dated February 6, 2014 [4]. However, it is obvious that this version, like the previous one, was created much later. Why else, one has to ask, was the “first version” of the protocol, supposedly dating from January 2012, uploaded to the study registry in September 2019 if this newer version had already been available at that time? 

In contrast to the earlier version, all references to a much later completion date have now been removed: References to software versions that were not released until years after the stated creation date are no longer present. The strange reference to a citation “25,” which matches the source reference in the published study but is meaningless in the protocol itself, as it does not contain a bibliography, has now also disappeared. 

Of course, this new version of the protocol also specifies the exclusion criteria in full, as well as the greatly reduced observation period for quality of life from the published study. 

What is new, however, is a section entitled “Bringing in the patient's voice,” in which the authors describe how, as part of the study, they intend to “systematically investigate the ethical, legal, socio-political, and scientific-theoretical dimensions of homeopathy using the example of non-small cell lung cancer,” for which they want to carry out an “integrated social science study” in which so-called “focus groups” of 4 to 10 participants with their relatives, friends, and caregivers will be used to capture “the interactions between individuals,  their shared and unquestioned assumptions, and the emergence of a collective meaning.” Or something like that. 

However, not a single word of this more or less meaningless but very scientific-sounding sociology jargon can be found in the published study or in the study registry. Nothing at all! Consequently, it is also futile to try to understand the significance of this project.

The only question that remains is: Why was this chapter added to the protocol? Since this “integrated social science study” was only supposed to begin after the third or fourth homeopathic treatment consultation (why, actually? Why shouldn't patients participate from the outset?), it seems obvious that this could simply be an implicit reason why the recording of quality of life should be discontinued after the third treatment consultation. 

As a result of the changes made, the information that appears when the study is called up in the ClinicalTrials study registry is now perfectly coordinated. The protocol now fully corresponds to the information in the published study and is dated to a time when the measurements were still ongoing, meaning that the results were certainly not known. Everything is in order, as a simple review of the entry would reveal.

At this point, we would like to express some criticism of the operators of this registry: The purpose of registering studies is to prevent misleading manipulation, in addition to publication bias. Without a doubt, it is a good thing that all previous data sets and all uploaded documents are preserved in this registry and remain publicly accessible. This is not the case with many national registries. However, one must have developed a certain degree of mistrust and also demonstrate some perseverance to actively comb through the history of changes to entries and verify them if necessary. If one does not go into such detail, concealing previous entries, as has apparently been attempted now, can be quite successful, and by no means only if one, as a layperson, takes only a superficial look at the entry. We therefore propose a change to the presentation on the website: if important parameters that could have a decisive influence on the result have been changed, such as exclusion criteria or observation times, this should be clearly visible in the entry and not hidden at the very end behind a “history of changes” link. 

But as far as the study is concerned, there is obviously no evidence that the study results were obtained using sound scientific methods. Nor does it seem possible to provide meaningful explanations for the reservations we have raised. 

Norbert Aust
Viktor Weisshäupl

[1] Frass M, Lechleitner P, Gründling C et al. Homeopathic Treatment as an Add-On Therapy May Improve Quality of Life and Prolong Survival in Patients with Non-Small Cell Lung Cancer: A Prospective, Randomized, Placebo-Controlled, Double-Blind, Three-Arm, Multicenter Study. The Oncologist 2020;25:e1930–e1955 https://academic.oup.com/oncolo/article-pdf/25/12/e1930/41921704/oncolo_25_12_n_a2.pdf

[2] https://netzwerk-homoeopathie.info/studienkritik-zu-lungenkrebsstudie-frass-et-al-2020/

http://www.initiative-wissenschaftliche-medizin.at/index.php?id=202

[3] https://clinicaltrials.gov/ct2/show/NCT01509612

[4] https://clinicaltrials.gov/ProvidedDocs/12/NCT01509612/Prot_SAP_001.pdf

 

To the beginning of the page

To next update 2