Monday, May 25, 2015

The Millennial Gold Rush for Data

Before you start reading, I would like to stress that I am not an expert in this field.  These are my ramblings based on what I have read, patterns I have noticed, and my own personal theories.  There may or may not be truth to this, and while I have done a decent amount of research, I do not work in this field directly, and cannot specifically comment on all aspects of it.  With that said, let's begin:

With all of the "hype" in regards to big data, I've slowly begun looking into what the growth for big data is going to be like as far as the job market is concerned.  What's interesting is that this entirely depends on which part of the market you plan to go in to.  For example, if you plan to learn about processing this data, your market value is very high from what I have read, probably for the next five to ten years, until we have the process mostly automated.  But if you plan to go into the gathering sector, there really isn't a way to determine how long you'll be able to rely on your skill set.  The reason for this is that gathering data is becoming more and more like a gold rush, instead of something long term.  This is what interests me, as I believe it is a very valuable skill set to have right now, partially because I believe more "gold deposits" will be found in the future still.  Thus I have been analyzing these "gold deposits" in order to gain a better understanding of what they are, where they come from, what their value is, and how to find them.


The Rush Process

Every couple of years, a new type of data that can be extracted from a company's customers about specific things, and that data is generally very valuable for creating a better product.  A great example of this of course is Google and their collection of search histories.  This is not only a valuable way for them to improve their search engine, but also to create products based on what people search for.  Once that data is extracted, I believe it is generally sold, which is where it's "value" comes from.  At this point, I am not sure how much it "fetches on the market", but I would assume a large amount of money is given for new types of data about customers.  For those who are skeptical about whether your data is sold, my philosophy is this: "If an online service is free, you are the product."

Now that the data is sold, many companies have it, which brings on the next question: Why keep "mining"?  This is what I speculate, as there is no more data to be gathered from users in this specific field.  Once the data has spread around enough, there is no need to mine for it, as you can probably purchase it, if you don't already have it.  For example, if somehow Google managed to extract all of the possible search result info about its customers, there would be very little reason to continue gathering this data, as you could just purchase what you need from Google (obviously this is an extreme example however, as this data will constantly change, keeping the stream "fresh").  In this example however, it would be more likely that this rush would die down, and everyone would continue on to somewhere else, purchasing the data when they need to.  This brings me into my next thought:

What are the current "rushes"?

As an aspiring businessman, I believe you need to understand what the current "data rushes" are, in order to predict the future variations.  There is a ton of money in these "gold mines", and it is important to figure out where they are coming from.  In fact, I think this has more money in it than any other sub-field in computer science out there.  Don't believe me? Just look at all of the businesses that have popped up in the past twenty years in regards to data mining from customers.  Google is an obvious one, Facebook is yet another.  WhatsApp is another big one, along with Twitter, InstaGram, and more.  More recent examples that I think have yet to be found are OculusVR, along with (probably soon to be) Magic Leap.  Google hit the rush for search, Facebook for social media, WhatsApp for communication, Twitter for news, and InstaGram for photography.  What about OculusVR and Magic Leap?

Well OculusVR found its data I believe in two types of customers: Gamers and virtual reality enthusiasts.  Gamers however are not an easily calculated one I believe (our interests fluctuate, and are really not "stable" in my opinion), which leads me to think their focus will be on virtual reality enthusiasts.  So far, they have gained a large amount of data, relating to things ranging from IPDs, to what types of games people like and want, to how these experiences can affect people both physically (motion sickness), and mentally (immersion factors).  This, to me at least, perfectly explains why Facebook wants them: Facebook is gathering data, in a similar way to I'm sure how Google is trying to with Google Cardboard and their Magic Leap investment.

Magic Leap I believe is looking into the augmented reality market.  They probably already have a large amount of data for this, but I am sure they are looking into things such as the fields that could best benefit from this technology, as well as similar data to what OculusVR is gathering.  I won't do a full in depth analysis for Magic Leap now, but I think you get the point.

What are potential future "rushes"?

This is where things get tricky: It depends on what data could be useful for a product.  Home automation was a big one that I looked into, as not a whole lot of data has been gathered on how we interact with our homes.  The problem with this right now is the cost of hardware, which is slowly coming down, but is not quite at a point where it is viable.  Google has looked into this with their Nest thermostat, but I believe they have reached the same conclusion for now.  Augmented reality is another field where the potential is there, but I think either Microsoft or Magic Leap will make it into that market first.  So how can you identify these potential gold mines?

Look for things you do every day, that you don't know how others do it.  For example, how well do you sleep at night?  Do you toss and turn a lot?  Do you wake up early, and stay up late?  Maybe sleep data is the "next big thing" (I'm doubtful, but you get the point).  There are tons of potential "gold rushes" for data that could come along, but identifying the "right" ones is tough.  Eventually, I believe we will run out of gold to mine for, but how we use it is a complete separate story.  That said, if you want to "get rich quick", data mining I believe is the place to look into.

Questions, comments, thoughts?  Feel free to leave them below!