A response to the editorial “JFAR's role in publishing believable research findings”
David Cashley, Dundee Podiatry Clinic
13 January 2014
May I first congratulate the editorial team for the publication of an excellent journal. Whilst this journal publishes many excellent papers, the recent editorial entitled “JFAR's role inpublishing believable research findings” raises some concern. I doubt any reader would argue with the sentiment and thrust of the editorial but the assertion that “results must bebelievable” is concerning. The article states “The fundamental premise of scientific publication is the reporting of empirically verifiable facts, which means believable information.” This is fundamentally incorrect, scientifically flawed and introduces potential for publication bias. By whom must the results be believed in order to be published? This statement holds an assumption that empirically verifiable means believable and vice versa. History has shown many times that this is not the case - verifiability does not equate to believability; the two are different concepts. Einstein’s research which allowed for a static universe using the Cosmological Constant was quite believable and yet also quite wrong.(1) Conversely, Slipher’s results were quite unbelievable yet have been repeatedly verified since publication.(2) Slipher was first to demonstrate an expanding universe, yet despite this information being available, Einstein developed the Cosmological Constant in his papers to balance his equations for a static universe.(1) Einstein did this because he, and the cosmologists he consulted, did not believe the universe was expanding. He subsequently wrote "The 'cosmological constant' lamda was introduced… to account theoretically for the existence of a finite mean density in a static universe [emphasis mine]. It now appears that in the dynamical case this end can bereached without the introduction of Lamda." (3) He was not alone in his disbelief and Slipher’s unbelievable data went largely ignored until verified by Hubble two decades later.(4) Some claim Eistein felt this was his greatest error. (5) In His book “The extravagantuniverse” Kirshner relates an email conversation imploring a colleague not to publish results which “you know in your heart are not right.” The colleague, Adam Riess replied “Approach these results not with your heart or head but with your eyes. We are observers after all.”(6) In 2011 Reiss’ results won him a Nobel prize for physics. The science writer Michael Brooks says of this email conversation “…this exchange of emails shows science at its best. First, a scientist makes an observation that doesn't fit with any preconceptions of how the world should be. Colleagues then jump in, say it can't be right, and try to find the flaw. When they can't find anything wrong - even with a result of enormous significance - everyone has to accept it, and adopt a new view of how the world works, whatever their gut feeling. The heart's response is not the ultimate arbiter.”(7) This test/retest method remains robust only if initial results are published – believable or otherwise. Nobody better demonstrates this than Marshall who, in 1985 claimed peptic ulcers were caused not by stress or spicy food but by helicobacter bacteria.(8) His theory met with considerable resistance, not due to poor science but to unbelievable outcomes. Fortunately, repetition by others proved his theory and it finally gained acceptance after the work of Hentschel et al some eight years later.(9) How will future researchers retest results if they go unpublished on account of believability within current understanding? If we only publish results which seem believable then our perception will never alter from its current course. We will in consequence miss opportunities to unearth new and profound knowledge which does not currently appear to make sense. It is to the benefit of science and mankind that our forerunners, from Copernicus and Galileo to Slipher, Mitchell and many more, were courageous enough to challenge their perceptions of the world we inhabit. The editorial further states “health journals have an unambiguous responsibility to support good quality research that provides believable findings. Equally, they also have a responsibility to filter out poor quality research that provides findings that are not believable.” To equate unbelievable results to bad research in this way is a major disservice to all of the above mentioned researchers and many more besides them - and to the many to come who will happen upon unbelievable results and risk their reputation on their findings. One has only to read the life stories of Nobel laureates, or perform a cursory search of scientific history to see that such scenarios are not isolated cases. I would aver that it is not the role of the editor to decided “believability” but the role of the reader and of subsequent researchers who would endeavour to repeat the results for themselves. It is not beyond the scope of a journal to publish good science that has unbelievable outcomes with an editorial comment attached, but to not publish is, I believe, to the detriment of biomedical science, and ultimately unscientific.
References
1. Einstein, A: The foundation of the general theory of relativity. Annalen derPhysik 1916,354 (7):769-822.
2. Slipher V: Spectrographic Observations of Nebulae. Popular Astronomy 1915, 23:21-24
3. Einstein A: On the relation between the expansion and the mean density of theuniverse. Proceedings of the National Academy of Sciences, 1932,18(3):213-214
4. Hubble E: Effects of Red Shifts on the Distribution of Nebulae. Astrophysical Journal,1936,84:517
5. Gamow G: My World Line: An Informal Autobiography. Viking Press; 1970.
6. Kirshner R: The Extravagant Universe: Exploding Stars, Dark Energy, and theAccelerating Cosmos. Princeton University Press, Princeton, New Jersey; 2002
7. Brooks M. The Nobel Prize for "nutty" Physics 04/10/2011[http://www.huffingtonpost.co.uk/michael-brooks/the-nobel-prize-fornutty_b_993683.html]
8. Marshall B: The pathogenesis of non-ulcer dyspepsia. Med. J. Aust. 1985,143 (7):319
9. Hentschel E., Brandstatter G., Dragosics B., et al: Effect of Ranitidine and Amoxicillinplus Metronidazole on the Eradication of Helicobacter pylori and the Recurrence ofDuodenal Ulcer. N Engl J Med 1993,328:308-312
Competing interests
None declared
Response from the Editors regarding letter from Mr David Cashley
Karl Landorf, La Trobe University
18 January 2014
The editors of Journal of Foot and Ankle Research read with interest the correspondence from Mr David Cashley regarding our recent editorial, JFAR’s role in publishing believable research findings. We thank Mr Cashley for his eloquent and thought provoking letter.
While we welcome Mr Cashley’s comments we believe that he has taken our editorial, particularly the issue of ‘believability’, out of context for the following reasons. Firstly, we have not indicated that findings that are currently not believable cannot become believable; that is, following sound enquiry. From what we understand in his letter, it would appear that Mr Cashley has a different interpretation of what we mean by the word ‘believability’. Believability in the context of our editorial relates to findings from studies that have a minimum of confounding or bias.
Secondly, we did not indicate in our editorial that we would not publish ‘good science that has unbelievable outcomes’. Quite the contrary, the editors value good science, or good research, far more highly than whether findings come across to readers as believable or not. We would argue that if a research project has been conducted well, the findings are the findings and will be hard to argue with; that is, they will be believable.
Thirdly, there are acceptable methods to provide best evidence for research questions (which is what we argue in our editorial), and while these do change in line with knowledge, it is inexcusable in the current environment to argue against this – this is a general comment and not aimed directly at Mr Cashley’s letter.
The methods we recommend in our editorial do not prevent unbelievable findings from being further investigated, or even published in the majority of cases (most manuscripts do get published – it is often a case, rightly or wrongly, of finding a journal that will publish it). Quite the opposite, our editorial promotes that researchers tackle research questions by using robust methods, which in the long-term, will provide the best evidence for or against an issue. There is a vast body of evidence that clearly shows that poor quality research is prone to confounding and bias. Conducting research using poor methods, which leads to inaccurate or ‘unbelievable’ findings, is inexcusable in the modern environment – this is the crux of the argument in our editorial. The importance of this cannot be underestimated. As an example, if the case against smoking was left up to a plethora of uncontrolled, poor quality studies, the world at large would not have sufficient evidence that smoking increases the risk of premature death and inflicts substantial morbidity in the process.
In summary, we do not argue, as Mr Cashley suggests, that unbelievable findings cannot become believable. Our editorial was designed to introduce readers and authors to accepted methods to improve quality and therefore, improve the believability of research. Believability of research findings equates to demonstrable, observable findings independent of pre-determined belief.
Journal of Foot and Ankle Research
A response to the editorial “JFAR's role in publishing believable research findings”
13 January 2014
May I first congratulate the editorial team for the publication of an excellent journal. Whilst this journal publishes many excellent papers, the recent editorial entitled “JFAR's role inpublishing believable research findings” raises some concern. I doubt any reader would argue with the sentiment and thrust of the editorial but the assertion that “results must bebelievable” is concerning. The article states “The fundamental premise of scientific publication is the reporting of empirically verifiable facts, which means believable information.” This is fundamentally incorrect, scientifically flawed and introduces potential for publication bias. By whom must the results be believed in order to be published? This statement holds an assumption that empirically verifiable means believable and vice versa. History has shown many times that this is not the case - verifiability does not equate to believability; the two are different concepts. Einstein’s research which allowed for a static universe using the Cosmological Constant was quite believable and yet also quite wrong.(1) Conversely, Slipher’s results were quite unbelievable yet have been repeatedly verified since publication.(2) Slipher was first to demonstrate an expanding universe, yet despite this information being available, Einstein developed the Cosmological Constant in his papers to balance his equations for a static universe.(1) Einstein did this because he, and the cosmologists he consulted, did not believe the universe was expanding. He subsequently wrote "The 'cosmological constant' lamda was introduced… to account theoretically for the existence of a finite mean density in a static universe [emphasis mine]. It now appears that in the dynamical case this end can bereached without the introduction of Lamda." (3) He was not alone in his disbelief and Slipher’s unbelievable data went largely ignored until verified by Hubble two decades later.(4) Some claim Eistein felt this was his greatest error. (5) In His book “The extravagantuniverse” Kirshner relates an email conversation imploring a colleague not to publish results which “you know in your heart are not right.” The colleague, Adam Riess replied “Approach these results not with your heart or head but with your eyes. We are observers after all.”(6) In 2011 Reiss’ results won him a Nobel prize for physics. The science writer Michael Brooks says of this email conversation “…this exchange of emails shows science at its best. First, a scientist makes an observation that doesn't fit with any preconceptions of how the world should be. Colleagues then jump in, say it can't be right, and try to find the flaw. When they can't find anything wrong - even with a result of enormous significance - everyone has to accept it, and adopt a new view of how the world works, whatever their gut feeling. The heart's response is not the ultimate arbiter.”(7) This test/retest method remains robust only if initial results are published – believable or otherwise. Nobody better demonstrates this than Marshall who, in 1985 claimed peptic ulcers were caused not by stress or spicy food but by helicobacter bacteria.(8) His theory met with considerable resistance, not due to poor science but to unbelievable outcomes. Fortunately, repetition by others proved his theory and it finally gained acceptance after the work of Hentschel et al some eight years later.(9) How will future researchers retest results if they go unpublished on account of believability within current understanding? If we only publish results which seem believable then our perception will never alter from its current course. We will in consequence miss opportunities to unearth new and profound knowledge which does not currently appear to make sense. It is to the benefit of science and mankind that our forerunners, from Copernicus and Galileo to Slipher, Mitchell and many more, were courageous enough to challenge their perceptions of the world we inhabit. The editorial further states “health journals have an unambiguous responsibility to support good quality research that provides believable findings. Equally, they also have a responsibility to filter out poor quality research that provides findings that are not believable.” To equate unbelievable results to bad research in this way is a major disservice to all of the above mentioned researchers and many more besides them - and to the many to come who will happen upon unbelievable results and risk their reputation on their findings. One has only to read the life stories of Nobel laureates, or perform a cursory search of scientific history to see that such scenarios are not isolated cases. I would aver that it is not the role of the editor to decided “believability” but the role of the reader and of subsequent researchers who would endeavour to repeat the results for themselves. It is not beyond the scope of a journal to publish good science that has unbelievable outcomes with an editorial comment attached, but to not publish is, I believe, to the detriment of biomedical science, and ultimately unscientific.
References
1. Einstein, A: The foundation of the general theory of relativity. Annalen derPhysik 1916,354 (7):769-822.
2. Slipher V: Spectrographic Observations of Nebulae. Popular Astronomy 1915, 23:21-24
3. Einstein A: On the relation between the expansion and the mean density of theuniverse. Proceedings of the National Academy of Sciences, 1932,18(3):213-214
4. Hubble E: Effects of Red Shifts on the Distribution of Nebulae. Astrophysical Journal,1936,84:517
5. Gamow G: My World Line: An Informal Autobiography. Viking Press; 1970.
6. Kirshner R: The Extravagant Universe: Exploding Stars, Dark Energy, and theAccelerating Cosmos. Princeton University Press, Princeton, New Jersey; 2002
7. Brooks M. The Nobel Prize for "nutty" Physics 04/10/2011[http://www.huffingtonpost.co.uk/michael-brooks/the-nobel-prize-fornutty_b_993683.html]
8. Marshall B: The pathogenesis of non-ulcer dyspepsia. Med. J. Aust. 1985,143 (7):319
9. Hentschel E., Brandstatter G., Dragosics B., et al: Effect of Ranitidine and Amoxicillinplus Metronidazole on the Eradication of Helicobacter pylori and the Recurrence ofDuodenal Ulcer. N Engl J Med 1993,328:308-312
Competing interests
None declaredResponse from the Editors regarding letter from Mr David Cashley
18 January 2014
The editors of Journal of Foot and Ankle Research read with interest the correspondence from Mr David Cashley regarding our recent editorial, JFAR’s role in publishing believable research findings. We thank Mr Cashley for his eloquent and thought provoking letter.
While we welcome Mr Cashley’s comments we believe that he has taken our editorial, particularly the issue of ‘believability’, out of context for the following reasons. Firstly, we have not indicated that findings that are currently not believable cannot become believable; that is, following sound enquiry. From what we understand in his letter, it would appear that Mr Cashley has a different interpretation of what we mean by the word ‘believability’. Believability in the context of our editorial relates to findings from studies that have a minimum of confounding or bias.
Secondly, we did not indicate in our editorial that we would not publish ‘good science that has unbelievable outcomes’. Quite the contrary, the editors value good science, or good research, far more highly than whether findings come across to readers as believable or not. We would argue that if a research project has been conducted well, the findings are the findings and will be hard to argue with; that is, they will be believable.
Thirdly, there are acceptable methods to provide best evidence for research questions (which is what we argue in our editorial), and while these do change in line with knowledge, it is inexcusable in the current environment to argue against this – this is a general comment and not aimed directly at Mr Cashley’s letter.
The methods we recommend in our editorial do not prevent unbelievable findings from being further investigated, or even published in the majority of cases (most manuscripts do get published – it is often a case, rightly or wrongly, of finding a journal that will publish it). Quite the opposite, our editorial promotes that researchers tackle research questions by using robust methods, which in the long-term, will provide the best evidence for or against an issue. There is a vast body of evidence that clearly shows that poor quality research is prone to confounding and bias. Conducting research using poor methods, which leads to inaccurate or ‘unbelievable’ findings, is inexcusable in the modern environment – this is the crux of the argument in our editorial. The importance of this cannot be underestimated. As an example, if the case against smoking was left up to a plethora of uncontrolled, poor quality studies, the
world at large would not have sufficient evidence that smoking increases the risk of premature death and inflicts substantial morbidity in the process.
In summary, we do not argue, as Mr Cashley suggests, that unbelievable findings cannot become believable. Our editorial was designed to introduce readers and authors to accepted methods to improve quality and therefore, improve the believability of research. Believability of research findings equates to demonstrable, observable findings independent of pre-determined belief.
The Editors
Journal of Foot and Ankle Research
Competing interests
None declared