Foot Of The Pyramid

I fight in the trenches of data collection. It's because I believe data collection is the foot of the Great Pyramid.

The better the foundation, the higher up you can build. All the way up to intelligent responses that make your customers want to give you a warm hug and stick around for a lifetime. Pyramids are incredibly solid structures that have withstood the test of time.

Of course nowadays nobody seems to have the time (or the slaves!) to build one of these beauties anymore. Instead, we often rush to the skies with abandon, forgetting that our great conclusions can only be as good as the data that supports them.

Like everyone else, I am torn between meeting a deadline and passing my own sanity checker. If I had all the time in the world, I would make sure that every spike and every dip is real before I hit 'send' on any significant bit of analysis.

But I don't have all the time in the world. So often times I choose to trust the data, for the time being, and ask questions later.

Later is a slippery slope, and soon enough I find myself staring at some pretty unbelievable numbers going back weeks and even months. I look down to find that my pyramid has no solid footing in the ground.

When I joined the QA2L team, I had already tried a number of tools promising to make the drudgery of data collection QA better. Some touted simplicity but turned out to just deliver basics, all the stuff I already knew. Some were clearly capable, but their interfaces looked like what I imagine NASA scientists are using to design tests for the International Space Station. Who has the time to master yet another tech-y UI?

I thought, maybe with the right dose of ingenuity you could have a tool that marries simplicity and complexity to pull off the dull part of making sure the structure is anchored in the ground. And now I think it's happening. And it's pretty exciting stuff.

We live in an age of intelligent automation. Well, we're about to, anyway.

Subscribe to our quarterly data quality newsletter