What Calvin and Hobbes Can Teach Us About Big Data
Posted in: UncategorizedFans of comic strips still fondly remember “Calvin and Hobbes,” the story of a peripatetic 6-year-old and his stuffed (but oh-so-real) tiger. One of my favorite strips shows Calvin digging in his backyard, pulling up “a few dirty rocks, a weird root and some disgusting grubs.” His exclamation to pal Hobbes: “There’s treasure everywhere!”
I’ve found myself thinking more and more about that phrase as I ponder this year of “big data.” Now big data certainly has a lot of promise, and it certainly has been hyped to death. But it’s also — to many of us — pretty scary. Big data requires lots of other big things, including big computer hardware, big staff, big project timelines and big budgets. And for many of us, big data is proving daunting and confusing as well.
We’ve been working on our own big-data project here at Discovery Digital Networks for the past year. It’s not been without challenges. We discovered that one of our trusted data sources has numerous inconsistencies, while another supposedly “real-time” source really couldn’t be trusted until about 15 days after each month closes. We’ve also had to revamp our staffing assumptions and timelines. We’re starting to get some killer stuff out of that project, but it’s taken longer, cost more and exposed more warts than we ever expected.
Post a Comment