As unfortunate as it is that I'm not part of the scientific community, I'm still a research consumer as a student. If that doesn't give me enough standing to weigh in with an opinion on this issue, then I'm not sure what would short of being a client, a member of society, or anyone on the planet that is affected by bad science. One giant point I'm trying to make here is that we're all affected by bad science, not just researchers, and not just research universities.
With that question settled (as far as I'm concerned), I think we have to consider how data is treated, particularly in this day and age. Data isn't just data, as much as this might surprise some. To offer a proof of this point, we can take two opposing theoretical stances, define our variables in relation to only one of those stances, work the same data for both theoretical scenarios and see if correlations exist.
This literally happens every day in social and psychological science, as Leslie John, Joe Simmons, Uri Simonsohn and others have suggested. And it's cheating, but one doesn't just cheat themselves. Everyone else gets to enjoy the ride too.
In order to understand why, one has to understand the process, from a top-down view of data collection. Some seem to believe it all starts with the research question. While this is an understandable error, it's still a mistake. The process starts much earlier, in the theoretical phase of investigation.
It's important to remember, a theory organizes existing data into testable questions that are falsifiable. The next step in devising a research plan is the actual hypothesis, but it's the next step after that where we really want to consider the consequences of using this data. After a hypothesis is defined, conceptualization of variables come into play, and this is where we'll find the source of the preventable mistakes made in modern science.
Defining a variable to test a hypothesis necessarily considers the accurate conception of the device(s) at work in the hypothesis. As an example we can consider the sexual acts of homo sapiens. Babies are the common result of the sexual act, but they are not the only result of the sexual act. If one wanted to know how much sex people were having, would the birth rate be a valid measure? No, it would not. The concept is different. Even though the birth rate suggests sexual activity, it does not define the breadth of it.
If, at the same time, we also collected the instance of venereal disease, age of the mother, and self-reports of extramarital sexual activity, considering that, as per the theory, babies only come from sex, but add the hypothesis that gonorrhea is produced exclusively from extramarital sex. So, the number of babies who's mothers also carried gonorrhea were produced by extramarital sex. At no time was the assumption of a positive proof mentioned of the original hypothesis which was necessary to prove the secondary hypothesis which has now been raised to the level of theory coupled with the hidden assumption of a moral philosophy that was never clearly represented, nor was the ludicrous hypothesis actually tested, but was, nevertheless, proven within statistical certainty, or enough so that we could at least shame them in public.
By the same notion, one wouldn't collect data on something as innocuous as what hair color the mothers had unless it was relevant to the presenting hypothesis only.
Finally, collecting random, irrelevant data is not merely an exercise in futility, it is a violation of the privacy of your subjects. Whether researchers are aware of it or not, they do have an affect on their research subjects. Suggesting that subjects are simply "black boxes" does not reduce professional responsibility. Schrodinger's cat is a lesson best learned before one introduces a corrosive into such a black box with a human subject inside.
Personally, I find the argument (I won't name where it came from) that the availability of finances necessitate the collection of a broad swath of data whenever possible appalling. Such sentiments are understandable, but nevertheless unethical, and lazy. The introduction of preventable error (mistakes, not error, and damn near fraud) into a system of analysis is unconscionable when subjects and clients lives depend on a researcher's methodology. Their trust, and the trust of your fellows should not be abused so lightly for so common and unworthy a cause.
Under the same reasoning it would be appropriate to run research projects outside the scrutiny of an ethics panel. In fact, that's precisely what researchers are doing when the collect data that is not relevant to their hypothesis, or use data collected under the umbrella of another project in their current work. Research review boards exist for a reason, and no researcher, no matter how trusted, righteous, or infallible, should be allowed to do as they will without any oversight. Historical lessons are painfully clear on this point. Even Goebbels had a Ph.D., was highly intelligent, and was considered a fine, upstanding member of his community.
No comments:
Post a Comment