Developing Research-Based Information

Wait! What? There is no research-based information on the value of Walls-of-Water around tomato plants? How can that be? They use them in the Demonstration Garden all the time. They even compared them against other tomatoes without the walls-of-water. What does it take to be “research-based” anyway?

The answer to “what does it take?” starts out simple, but there is a lot of complexity behind it. The simple answer is that the “fact” in question must be supported by an article published in a peer-reviewed scientific journal. Let’s dig a little deeper to see the more complex side.

How seriously your article is taken is influenced to some extent by the journal in which it is published. You could think of journals on a rating scale, with the best being equivalent to Nordstrom’s, and the bottom of the heap being equivalent to Target, and many others in between. You’d like to get your article published in the best journal you can. The significance of your hypothesis, the quality of your work, and the reputation of your authors will be determining factors.

To be published in a scientific journal, the article must have as its principal author a recognized, credentialed researcher in the field. Think PhD working at a respected university, or a student who is mentored by an established researcher in the field. They must work in a department related to the subject of the paper - underwater basket weaving won’t do (unless that’s what is being researched!) Before such an author would submit such an article, they should be confident the research has been well-designed, well-executed, and well-documented, or their own reputation may suffer! Much of the work may have been done by graduate students, but the lead author is putting their name on the work, and will want to be sure it is accurate. The author will spend considerable time to verify these items and guide the work, and will wish to be paid for this time. Their families like to eat.

Research is done to test hypotheses (those things we hope to call “facts”), and results must be documented in a standardized way that will be understandable by the peer reviewers and collegues. Those doing the data gathering will have to be trained in the proper methods of documentation.

The actual tests performed must accurately follow the designed protocol (a series of procedures). If they vary in any significant way from the protocols, it will invalidate the research. The protocols must account for any variables that may impact the results (temperature, water, soil chemistry, seed stock, seasonal variability, etc.)

The test conditions must cover a wide enough range of conditions to make “the fact” useful. It would not be very impressive to quote an article that could only speak to one variety of tomato, in one type of soil, with one method of watering, in one climate niche. This will almost certainly require compromise. One variety x one soil x one watering method x one climate = 1 test. 2x2x2x2 = 16 tests, and it goes up fast from there!

A reasonable protocol for our hypothetical research project would cover a range of tomato varieties, with all tomatoes of a given variety grown from the same batch of seeds, and covering a range of locations, fertilization schemes and soil conditions. It would probably require the tests be performed over multiple years.

Somebody needs to manage all this. Hire and train workers, and make sure they follow the protocols exactly. They must make sure any transitions from one worker to another is seamless. There is a good chance that the manager and the workers will want to be paid as well.

Before any of the actual testing begins, past research related to the proposed “fact” must be thoroughly reviewed.  The test design must account for previously asserted counter-facts. Failure at this point to spot an existing, relevant article that contradicts your hypothesis can turn the entire research project into a waste of time (and money). Find people skilled at searching, and pay them well.

Back at the very beginning, you must state your hypothesis. The hypothesis (the asserted “fact”) must be clearly worded using standard definitions acceptable to the related research community.

Finally, with the article written, it must be "peer reviewed". The journal considering the article for publication will assign one editor and at least two respected researchers, all with expertise in the field. Ideally the editor and reviewers will have published papers in the same area, and be aware of related research. Besides verifying that the protocols valid and justified, the data was collected accurately, and that the conclusions follow logically from the results, the editor and reviewers will also evaluate whether the results are significant given the existing research in the area. Getting your article past this review team often takes some negotiation. One reviewer may feel your “fact” is stated too broadly for the data you have accumulated. This might lead to restating your “fact” or doing further research to acquire the necessary data. Another reviewer might ask you to provide additional justification for your methods.

If all goes well, the article is published. This tells the reader that a group of informed scientists (the author(s), the editor, and the reviewers are confident that the information in the paper gives an accurate description of a phenomenon in the nature world. That, of course, may not hold true indefinitely. Other researchers will review the work, some may attempt to duplicate the research, and new researcher may shed new light on the "fact". For a while, however, you can state that your “fact” is indeed research-based.

Perhaps it is clear that developing “research-based facts” is complex and expensive. Perhaps the very first question should be, “Do you have a sizable pile of dollars to spare?”, or alternatively, “Who cares enough to put up the money to verify this “fact”?