We learned in science class that we should use the scientific method to evaluate hypotheses. Yet somehow, once we enter industry we throw that early learning out the window and fall prey to several pitfalls in experimentation and data interpretation.

In the first two pitfalls that we discussed last week in part one of this series, we looked at the dangers of not running an experiment or not verifying that your supplier ran an experiment (the perils of buying snake oil).

In the second part of this series we look at how necessary it is to ensure that the experiment is valid and that our data samples are not biased.

There are numerous technical guidelines for checking the validity of an experiment or study, but one of the most fundamental is simply making sure that the experiment is representative of the real system. This pitfall is particularly insidious because in this case false or misleading conclusions are hiding behind the veil of false scientific rigor.

 …