Saturday, November 20, 2010

Big Data: Are We Ready for Our Predicted Future?


The implications of our world exploding in digital bits of big data go far beyond privacy as we think of it today; our minds cannot yet grasp the numbers themselves, much less the global economic and social ramifications of our predicted future.

Our Brains and Big Data

It's hard for the current state of our human brains to understand big numbers. We simply lack adequate frames of reference. The first time I noticed this was when I saw "Powers of Ten" in school and more recently when considering things like incremental budgets cuts to the US Federal budget.


Infographics and data visualizations can help us imagine the scale of big data but somehow relating it to our day to day existence is sometimes just out of reach. If the farthest reaches you've ever traveled in your life is 100 miles, even a trip to the moon seems fantastical.

We liken the scale of Facebook to the population of a country, but the amount of sheer data bits being generated every day far exceeds a paltry census.
... there was five exabytes of information created between the dawn of civilization through 2003, but that much information is now created every two days, and the pace is increasing ... people aren't ready for the technology revolution that's going to happen to them.  - Eric Schmidt, Google CEO on Techonomy 

It's hard enough to understand the true scale of these numbers even more so to appreciate the implications - just how big is an exabyte? A byte (in this case) is a unit of data on the order of a single character in a Notepad document, and an exabyte is 1,000,000,000,000,000,000 (10^18) bytes.

To think of it another way, an exabyte is roughly 500 billion novels that Nicholas Carr regrets no one will read - especially since Google estimates that there are only 130 million (or so) published books in all of modern history. However, machines can and do read the various bits of messages, blogs, photos, comments, checkins and status updates - the primordial ooze for the evolution of what Jeff Jonas calls accumulating context systems.

Contextual Kingdoms of Prediction

Jeff Jonas has spent the better part of the last fifteen years systematically transforming random flows of observable pixels into persistent streams of contextual insights. In short, he's a data wrangler.

This week Jeff gave one of the most compelling talks at the Defrag 2010 conference, describing what he calls the new physics of big data - the persistent and contextual accumulation of data that results in better predictions, lower compute efforts and a better sense of where to focus one's attention.

This new physics relies on data, lots of data, exabytes upon exabytes of all sorts. Exabytes of data are created by people every day - and even more so by machines, like our mobile phones which are producing "something like 600 billion geo-spatially tagged transactions every day". These bytes of space-time information alone provide data wranglers like Jeff a "super food" to fuel a future in which smart information systems will not wait for you to ask questions - but rather deliver predictable answers about your immediate future state - a burgeoning new kingdom where data finds data.

Eric Schmidt and Google are already preparing this new kingdom.
If I look at enough of your messaging and your location, and use Artificial Intelligence we can predict where you are going to go. Show us 14 photos of yourself and we can identify who you are. You think you don't have 14 photos of yourself on the internet? You've got Facebook photos! People will find it's very useful to have devices that remember what you want to do, because you forgot...But society isn't ready for questions that will be raised as result of user-generated content.  - Eric Schmidt, Google CEO 
Why are we waiting? What sorts of questions do we need to ask now? Are we ready to barter with our data for more convenience?

Convenience or Control?

The Wall Street Journal online has entire section entitled "What They Know" on Internet tracking and privacy with feature titles such as:
Each of the segments in the series outline a particular aspect of the digital trails we leave behind across the web while browsing as well as via any connected transaction such as cell phones and game consoles. The fact is that while we voluntarily publish huge amounts of data via the social web, even just lurking can generate interesting bits to feed Jonas' context accumulators.

The WSJ is simply outlining the things we know now and how we might control our own data trails. But technology is changing at a rate that most people cannot and/or do not keep pace, and we are smitten by the conveniences that social and mobile services afford us, even if it means we are trackable and ultimately predictable.

What is the personal price in the currency of privacy that I pay in exchange for that recommendation, that coupon, that next game level? And what's wrong with Facebook becoming my "identity"? Some people can make those calculations in their head, but most people don't (yet) know to even ask the question.

i need the old facebook this new one is very bad bbbbbbbbbbuuuuuuuuuuuuuuuuuuuu!

When hoards of websites adopted Facebook for Websites as part of their "social strategy" - some people could not distinguish between logging into Facebook or logging in to comment on a blog post - and became confused and outraged when they found themselves lost in the comments on the ReadWriteWeb post.

The average baby boomer is unlikely to know the difference between a web browser and a search engine. Cookies are those things Comcast repeatedly asks you to clear when your internet goes down. And a beacon? Well, maybe that's just a ray of hope.

As a society, we are falling faster and farther into the chasm of digital illiteracy - the tools are outpacing our collective understanding.

Privacy is not simply about control or convenience yet because people don't know what they don't know, nor do they know the right questions to ask.

This is the literacy gap that must be filled first before Esther Dyson can realize her dream of a personal health data marketplace.

We are not prepared for these predicted futures, but we should not sit back as Eric Schmidt implies and wait for society to "get ready". Rather, we need to, in Douglas Rushkoff's words: Program or be Programmed.


Share Some Grace:

blog comments powered by Disqus