We’ve all seen some breathtaking software benchmarks out there. We’ve heard some fantastic claims. Before you get too excited, use some critical thinking - can you use this information, or is it so cooked up as to be useless?

A database company’s CTO claims updates between two data centers 6633km apart in less than a millisecond? Absolutely! Their technology is so fast, they’ve discovered a way to move TCP/IP packets faster than light.

What are some simple approaches to get a sense whether a benchmark is something solid that lets you evaluate a technology, useful for capacity planning, or a pile of poo.

This will be a concise rant about crappy benchmarks, and how to protect your mind against them.