Picking the Right Data Store: Data Storage Platforms
Watch this video to learn, how data storage platforms have evolved to meet different analytic needs.
In Darwin’s day knowledge was “stored” in books and books were kept in libraries. Today, while physical libraries still exist, a lot of data is stored digitally. Of course, cost is always a consideration when choosing a data store. But beyond cost, picking the right data store depends on the type of data you need to analyze and the questions you want to ask.
Transcript
I'm Mike Allen, VP of Products for Zoomdata. I'm going be talking about picking the right data store for your data analytics needs.
Charles Darwin
So, maybe let me start by telling a story. So, imagine you're Charles Darwin, and you have an analytical question you want to answer to backup your latest whacky theory. You're thinking, so, these dinosaurs people keep finding, how many teeth do they have compared with say, my dog. That's a good analytical question.
So, where's that information stored? Well, it's in the library. Let me head to the library. Okay, it's stored there, but how do I actually get that information to answer my question?
Legacy Storage: The Library
Well, if I had to read every book in the library to find those answers, it would take me a very long time. So, fortunately, I've got my books organized in sections, I've got indices, indexes, I've got ladders to get to various shelves. I've got a quick way to get the answer. It'll only take me a few hours or days to answer that question.
What does that mean for the modern day? Today, data isn't stored in books. It's stored on magnetic media. It might be on your hard drives. It might be in the cloud. It might be in slow drives. It might be in memory. But, you know, it's up there.
What Questions Are You Trying to Answer?
So, what are the factors you need to think about? Well, what's the cost of the actual storage? It's still not zero. In 1980, the cost of a gigabyte of storage was about $500,000. Today, it's a couple of cents.
Also, the analytical software depends on the kinds of questions you want to ask and who's asking the questions. Is it boffins (British: a person engaged in scientific or technical research) who have, you know, data science libraries and all sorts of advanced software to use machine learning to find the answers to difficult questions? Is it ordinary business users who have simple questions they want simple answers to in the form of graphics. Or, is it computers? Is it actually algorithms that are mining through your data and looking for the answers, looking for the outliers, helping you solve problems?
And much as Charles Darwin's theory gave us evolution, the kinds of questions and the kinds of systems and the kinds of data have evolved over the last few decades. We started with transactional systems where the question that I was--the data I was collecting was how many widgets did I sell, and the question I wanted to answer was how many widgets did I sell in the Northeast this month?
Then they moved on to interactional type data. Oh, I've got people coming to my website. What did they click on? How many people clicked on this before they bought that - those kinds of questions. And then the next generation is observational data. I've got all these sensors. They're collecting all this data. What does it mean?
