Posts tagged Overfitting
Your Data Are Probably Biased And That's Becoming A Massive Problem Beware of black boxes - INC

Nobody sets out to be biased, but it's harder to avoid than you would think. Wikipedia lists over 100 documented biases from authority bias and confirmation bias to the Semmelweis effect, we have an enormous tendency to let things other than the facts to affect our judgments. We all, as much as we hate to admit it, are vulnerable.

Machines, even virtual ones, have biases too. They are designed, necessarily, to favor some kinds of data over others. Unfortunately, we rarely question the judgments of mathematical models and, in many cases, their biases can pervade and distort operational reality, creating unintended consequences that are hard to undo.

Yet the biggest problem with data bias is that we are mostly unaware of it, because we assume that data and analytics are objective. That's almost never the case. Our machines are, for better or worse, extensions of ourselves and inherit our subjective judgments. As data and analytics increasingly become a core component of our decision making, we need to be far more careful.

Read More