Thursday, June 28, 2012

A Modest Proposal for Peer Review Research

With the advent of, and inexpensive nature of online sharing of information, I propose that all peer-review research should include (anonymous) raw data for each subject.  At the very least, there should be scatter plots presented for the individual data points for the main outcomes.

I frequently teach statistics, and one of the first things we discuss in that class is sort of the "first purpose" of it all.  Because before we can analyze data, first we must summarize and present the data in such a way that the "consumer" can readily glean information.  In one classic stats text -- Triola -- this part is given the acronym CVDOT.  C = Center, V = Variation, D = Distribution, O = Outliers and T = Time.  So we go through the various ways we can convey the center of a data set, it's variability, distribution, etc.  In most of the studies we discuss here, data is presented as a mean value +/- either the standard deviation or standard error (C +/- V in the acronym).  And further statistical analysis compares these means between groups for statistically significant differences.   If I have 20 subjects in a study, I can provide you with a table of all results sorted by subject number assigned randomly.  This tells you very little.  If all I do is sort the data ascending or descending, you can now readily pick out the range and "center" of the data.  Perhaps if data is of a more rounded nature, you might be able to pick out the most frequent or common values.  Outliers will jump off the page.  
Read more »

No comments:

Post a Comment