Sanity Checking

If everything was created perfect the first time, the world would already be perfect.

Accepting an answer at face value, can be detrimental to our research. Consider adding sanity checks to your tests to ensure data validity and weed out suspect responses.

The medical industry has been conducting sanity checking on results in questionnaires and academic surveys for a number of years.

The question:

“Have you ever used Derbisol?”

Would appear along with questions about alcohol, cocaine, and marijuana usage on youth-risk surveys for students. Derbisol was a fictitious drug invented to highlight a suspect paper.

Individuals who would admit to taking Derbisol could have their papers marked questionable and the data disregarded from the study; presumed to be embellishing or lying with their answers. Falsified data could then be removed from the study giving a more accurate frame of reference.

Upon reviewing data, filter results based on pre-agreed business goals for the project. If a website sold goods to the UK exclusively, data based on traffic originating in India may be disregarded in this instance as it is not a primary concern. That does not mean the data isn’t useful for other business means. Only that for this given task, the data could skew resources unnecessarily.

After conducting review sessions with users, sanity check their feedback.

Ask questions whose answers corroborate or contradict previous data given. Try not to accept everything at face value. Cross-reference responses with answers to previous questions.

Look for patterns in the data to suggest corroborative evidence. If possible plot your results on to a graph and identify common patterns or behaviours. Plotting data on a graph can also help you to easily identify subjects with suspect or inaccurate responses, whose data could be omitted from the study.