Dear Madison Avenue: Set My Data Scientists Free
Posted in: UncategorizedBack in 2008, when I was 31 years young and a freshly minted Ph.D., I took my first job in digital media. I was hired as a consultant for “advanced analytics” on a new mobile ad server. So I dusted off my statistics books, cracked my knuckles and logged into the machines where six months of server files were housed. And that’s when the torture began.
The data files were scattered across directories. The names were inconsistent. The formats varied wildly. Hours passed as I typed out Unix commands on the terminal, tediously stitching together billions of events into a time series of the last few months. As the graph rendered, an amazing trend unfolded: a sudden dip, then a 300% spike in ad traffic! Yet a few more keystrokes revealed the unglamorous reason: The platform had crashed, rebooted and began logging every event in triplicate.
So much for the “advanced analytics” I’d been hired to deliver. Though I’d uncovered a few troubling bugs in the new ad server, those insights weren’t gleaned through complex mathematics. It was simple counts, averages and a plain chart that got me there.
Post a Comment