The Advancement of Science: From Paradigms to Peer Review

Paradigms and How They Shift

Understanding the role of paradigms in scientific investigation is one of the keys to approaching the revolutionary view of climate as a problem of ecosystem dynamics as opposed to one simply of excessive greenhouse gases.  The new paradigm doesn’t render the old paradigm irrelevant, but it reframes its significance and role in addressing the current climate crisis.  It exposes to open examination what was heretofore an invisible phenomenon, and avails a universe of solutions to what is, from the perspective of the greenhouse gas hypothesis, an intractable and quite possibly utterly hopeless problem.  Therefore, we will take a moment to review the paradigm process and apply it to our contending climate paradigms.

In 1962, Thomas Kuhn, a Harvard-trained physicist who became a historian and philosopher of science, published a controversial book, The Structure of Scientific Revolutions.  Prior to Kuhn, the prevailing assumptions about the way science progressed were that knowledge was gradually accumulated by generations of investigators, with occasional quantum leaps by great scientists, but in an overall smooth and continuous albeit occasionally heroic process.

Kuhn broke new ground by re-examining and reframing the process of scientific investigation.

He brought the term “paradigm” into common usage, by which he meant a body of “universally recognized scientific achievements that for a time provide model problems and solutions to a community of practitioners” (p. viii, emphasis added).  We will review Kuhn’s work briefly and apply his analysis when comparing the mainstream greenhouse gas climate paradigm and the newly evolving eco-restoration climate paradigm.

Kuhn maintained that scientific progress is episodic, characterized by long periods of “normal science,” which takes place in the context of a paradigm:

At least in the mature sciences, answers (or full substitutes for answers) to [many] questions . . . are firmly embedded in the educational initiation that prepares and licenses the student for professional practice. Because that education is both rigorous and rigid, these answers come to exert a deep hold on the scientific mind. [Kuhn 1962:5]

Normal science, the activity in which most scientists inevitably spend almost all their time, is predicated on the assumption that the scientific community knows what the world is like. Much of the success of the enterprise derives from the community’s willingness to defend that assumption, if necessary at considerable cost. Normal science, for example, often suppresses fundamental novelties because they are necessarily subversive of its basic commitments. Nevertheless, so long as those commitments retain an element of the arbitrary, the very nature of normal research ensures that novelty shall not be suppressed for very long. [Kuhn 1962:5]

Normal science is punctuated by the appearance of anomalies which cannot be explained by the paradigm’s generally accepted theories, nor tested by what the paradigm might consider reasonable hypotheses, nor resolved with current testing protocols or equipment.

When examining normal science . . . we shall want finally to describe that research as a strenuous and devoted attempt to force nature into the conceptual boxes supplied by professional education. [Kuhn 1962:5]

[W]hen [normal science repeatedly goes astray] – when, that is, the profession can no longer evade anomalies that subvert the existing tradition of scientific practice – then begin the extraordinary investigations that lead the profession at last to a new set of commitments, a new basis for the practice of science. The extraordinary episodes in which that shift of professional commitments occurs are the ones known in this essay as scientific revolutions. They are the tradition-shattering complements to the tradition-bound activity of normal science. [Kuhn 1962: 6]

and

Normal science consists in . . .  an actualization achieved by extending the knowledge of those facts that the paradigm displays as particularly revealing, by increasing the extent of the match between those facts and the paradigm’s predictions, and by further articulation of the paradigm itself.

Few people who are not actually practitioners of a mature science realize how much mop-up work of this sort a paradigm leaves to be done or quite how fascinating such work can prove in the execution. And these points need to be understood. Mopping-up operations are what engage most scientists throughout their careers. They constitute what I am here calling normal science. Closely examined, whether historically or in the contemporary laboratory, that enterprise seems an attempt to force nature into the preformed and relatively inflexible box that the paradigm supplies. No part of the aim of normal science is to call forth new sorts of phenomena; indeed those that will not fit the box are often not seen at all. Nor do scientists normally aim to invent new theories, and they are often intolerant of those invented by others. Instead, normal-scientific research is directed to the articulation of those phenomena and theories that the paradigm already supplies. [Kuhn 1962:23-24, emphasis added]

and

Paradigms gain their status because they are more successful than their competitors in solving a few problems that the group of practitioners has come to recognize as acute. To be more successful is not, however, to be either completely successful with a single problem or notably successful with any large number. The success of a paradigm . . . is at the start largely a promise of success discoverable in selected and still incomplete examples. [Kuhn 1962:23, emphasis added]

Even today, over half a century after Structures was originally published, normal science seems immune to the possibilities of paradigm shifts – such thoughts often do not occur until forced, even though the process should be reasonably well known if not entirely understood or accepted.  The prevailing opinion about paradigm shifts (if there is any opinion at all) appears to be, “It doesn’t apply to my paradigm.”

In general, a paradigm shift doesn’t only involve “objective” factors, it touches scientific practitioners at a deep emotional level as well, as any participant in or observer of academic dispute can testify:

Scientific fact and theory are not categorically separable, except perhaps within a

single tradition of normal-scientific practice. That is why the unexpected discovery is not simply factual in its import and why the scientist’s world is qualitatively transformed as well as quantitatively enriched by fundamental novelties of either fact or theory. [Kuhn 1962:7]

Therefore, the transition to a new paradigm is disruptive and challenging:

The transition from a paradigm in crisis to a new one from which a new tradition of normal science can emerge is far from a cumulative process, one achieved by an articulation or extension of the old paradigm. Rather it is a reconstruction of the field from new fundamentals, a reconstruction that changes some of the field’s most elementary theoretical generalizations as well as many of its paradigm methods and applications. During the transition period there will be a large but never complete overlap between the problems that can be solved by the old and by the new paradigm. But there will also be a decisive difference in the modes of solution. When the transition is complete, the profession will have changed its view of the field, its methods, and its goals [Kuhn 1962:84-85].

The case in point here is the comparison between old and new climate paradigms

Paradigm step	Old paradigm (greenhouse gases)	New paradigm (Eco-restoration)
Paradigm fundamentals	CO2 and equiv are greenhouse gas blankets and elevated levels cause global warming, primarily caused by burning fossil fuels	Destruction of billions of acres of land interferes with carbon and water cycles, along with oxidation of soils for over 10k years, puts gigatons of carbon into atmosphere
Weaknesses intrinsic to paradigm	Positive feedbacks underrepresented, overlooked, not calculated or estimated; biology is characterized as passive victim of climate change	Complex, interdependent systems that are difficult to model and to quantify into policy
Strengths intrinsic to paradigm	Amenable to modeling; yields numeric targets that can be translated into policy	Comprehensive of all likely drivers and their theoretical interdependencies.  Plausible upon examination of biogeologic history.
Primary investigators	Physical scientists almost exclusively from academia	Restoration ecologists and others from biological sciences; non-academic land managers
Tools	Emissions reductions via alternative energy and elimination of carbon emissions sources	Photosynthesis and regenerative land management
Costs	High	Low
Technology requirements	Extensive	Minimal
Locus of investigation	Centralized in academia – universities, scientific journals, formal test sites	Based first in local land management practice, then investigated by academia, landscape managers, local practitioners – farmers, ranchers, horticulturalists, permaculturists, indigenous cultures, etc.
Weight of evidence	Formal studies, isolated variables	Practical results, holistic assessment of land health, biodiversity, water and carbon cycling
Success criteria	Reduced emissions and atmospheric carbon burdens (target 350 ppm? lower?)	Increased biodiversity, improved water cycles, land resilience, cooling of local biospheres on a global scale, reduced floods and droughts, decline in atmospheric carbon burdens (target 280 ppm)
Duration of existence of paradigm	Roughly 200 years	Roughly 20 years with some roots going back considerably longer

The Perils of Peer Review

It may well behoove us all, including the scientists among us, to take a careful look at how science works today and how practices may improve.  For example, while peer review can be a powerful tool, it is worth keeping in mind that we’re in a world of shifting paradigms where there are libraries full of peer-reviewed papers in scientific landfills, review processes notwithstanding. Peer review may be a useful tool, but it may also be a significant obstacle to scientific progress.

Beyond the routine aging and demise of most scientific papers, however, is the problem of a system that excludes information that conflicts with or is invisible to the dominant paradigm.  That is a central issue that we face when introducing an entirely new view of climate – in the normal course of science, taking a generation or two to transition between paradigms is acceptable, even healthy; in the throes of a climate emergency, conventional peer review may be a serious obstacle to progress.  

Peer reviewers get to review their peers because they are thinking along similar lines, and are likely – perhaps even required – to reject ideas outside mainstream boundaries of thought.  Even the conventional authors of a recent USDA study had a difficult time getting their study published because its results were so unexpected [Ausmus 2014; Follett 2012].  And that doesn’t begin to touch any of the forces in the political and economic realms that impact peer-reviewed science, including what actually gets studied (and funded!) and what does not.  

Presence or absence of peer review should therefore not serve as a standard for accepting the validity of any paper or report; only the evidence is the basis for such decisions, whether it is within “acceptable” range or far beyond it.  The evidence must stand for itself, and the professionals who are readers should have the opportunity to make up their own minds.  Peer review is a standard, but not a gold standard – it is one among many, and practitioners of scientific method have an obligation to evaluate the relevance of standards.

Linkov 2006.  Whereas most tools of science have evolved over the past three hundred years, there is one that stubbornly shows its age: the scientific journal.  The author “argue[s] that the primary reason that journals have not changed is that they are ‘faith based’: we believe in them, we dare not question them.” [Linkov 2006: 596]

Linkov suggests that the journal hasn’t transformed into a new model of publication because it has never applied the scientific method to itself.

Jefferson recently presented an outstanding review of peer review and could find only 19 studies on peer review that were scientifically sound. We could find only 14 articles examining the editorial board/editorial decision making. Thus, with over 50 million articles and 300 years of the traditional journal approaches, there has been only a handful of studies questioning or testing the journal process itself. We scientists keep using the process without question, but with no data to show that it is effective. There is thus no evidence-based approach to the science of research communications. Recent studies reveal that peer review often misses major methodological problems in articles. No wonder it has not changed or improved, as there are no data questioning the process. Hypothesis testing research and randomized trials could easily and cheaply be initiated to understand the ‘grand challenges’ of research communication, but sadly they have not.

Isn’t it strange that three features that are inherent to research communication have not been looked at scientifically? There are several possible reasons for this. The most likely is that we scientists have almost complete faith in the journal process as right and unassailable. We thus take a ‘faith based’ approach to research communications. Faith is defined as a firm belief in something for which there is no proof. Many of us might view questioning of the journal process as an attack on science itself. Clearly, the scientific journal process is not a part of the scientific method. We are taught early in our training about the importance of learning to write articles (e.g. IMRaD^[23]), the power of peer review and a belief in the editorial system. We do not question the process, despite the fact that the essence of science is questioning. Questioning peer review is like questioning the Bible, Quran or Torah. One role of science is to help separate science from dogma, which we should now do with journals, and avoid a faith based approach. New approaches need to be taken – you cannot teach dogma new tricks! [Linkov 2006: 597]

It is the scientific method that is central to science, not the scientific journal. The scientific method should be central to other research communication processes, but it is not and has not been used to continuously improve how we communicate research. Because of this, we are forced into a conundrum—we cannot change the process if the process is based upon faith, not data.

Experiences of various fields, including industry, demonstrate there are other forms of quality control besides peer review that could potentially be utilized in the biomedical journals. These methodologies include 6-sigma, statistical quality control, and web based, consumer driven systems such as that employed by Amazon, eBay, and Slashdot. There are thousands of studies in business and sociology evaluating the decision making process that could be brought to bear to evaluate the decision process at the editorial level, but they have not been used. It would seem very simple to develop randomized trials to determine which system best improves the quality of publication. As Jefferson has pointed out, there are almost no data suggesting that the existing peer review systems work and none to suggest that they are better than any other system. . . .

Based upon the data, we cannot reject the hypothesis that scientific journals are faith based. [Linkov 2006: 598]

Smith 2006.  Richard Smith was editor of the British Medical Journal for thirteen years, and writes incisively and wryly about the peer-review process. He states that peer review is “the method by which grants are allocated, papers published, academics promoted, and Nobel prizes won. Yet it is hard to define. It has until recently been unstudied. And its defects are easier to identify than its attributes. Yet it shows no sign of going away.” [Smith 2006: 178]  

What is peer review?

[And] who is a peer? Somebody doing exactly the same kind of research (in which case he or she is probably a direct competitor)? Somebody in the same discipline? Somebody who is an expert on methodology? And what is review? Somebody saying “The paper looks all right to me”, which is sadly what peer review sometimes seems to be. Or somebody pouring all over the paper, asking for raw data, repeating analyses, checking all the references, and making detailed suggestions for improvement? Such a review is vanishingly rare. . . .

Robbie Fox, the great 20th century editor of the Lancet, who was no admirer of peer review, wondered whether anybody would notice if he were to swap the piles marked ‘publish’ and ‘reject’. He also joked that the Lancet had a system of throwing a pile of papers down the stairs and publishing those that reached the bottom. When I was editor of the BMJ I was challenged by two of the cleverest researchers in Britain to publish an issue of the journal comprised only of papers that had failed peer review and see if anybody noticed. I wrote back ‘How do you know I haven’t already done it?’

Smith goes on to question what peer review is for and whether it works (not very well, which is no surprise at this point).  Its drawbacks are that it’s slow and expensive; it is inconsistent, betraying the myth of being objective and reliable;  there is bias, particularly against studies with negative results; it may be abused by reviewers who are competitors; and ideas and text may be plagiarized.  Improvements may be made by standardization of the process, blinding reviewers to the identity of authors, feedback to reviewers, training reviewers and other techniques. Nonetheless, the obstacles to shifting a 300-year-old industry are daunting.  Smith concludes:

So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.

* * * * * * * *

---