Just last week the Obama Administration published a memorandum about Federal agencies adopting some best practices in IT.
When I read a little bit closer, a lot of it was focused on data retention.
While data retention isn’t the “sexiest” of topics these days, we can’t undermine the importance of tracking and storing our data, both for use today and in the future. Someone I once worked with said “data is expensive, space is cheap.” Getting trend data on human behavior is hard to do, and when we have large repositories of data we should make the most of them.
Think about where our computing power was 10 years ago, and now consider the capabilities and insights that we have today. Now, imagine if we could take the data of today, and apply the analysis techniques of ten years from now. We could create deep insights into the patterns of human behaviors.
When I was an undergraduate student at the University of Michigan I did an internship at the Harvard Smithsonian Center for Astrophysics. My job was to work with data from the Einstein Slew Survey. This was data captured by the Einstein satellite as it was shifting between coordinates. Scientists paid lots of money to gather the Einstein satellite data, but I got to study the data gathered between the formal projects. I would guess that when they launched the satellite many years before that, they never considered that someone would want this “garbage data,”and more to the point that someone would make science out of this.
What is our “garbage data” of today, and how will we use it in the future?