Big Data

 In digital, seasoned tech

The Big Data Movement

Big data. Sounds mysterious, maybe even mythical. But these two words are changing industries everywhere. A recent explosion in data mining and data processing has brought on the golden era of all things data. It’s time we collectively got up to speed on what this phenomenon is and how its power is affecting our lives.

Analytics have been a part of businesses for decades. Industries have been combing through any available data for hidden secrets or trends. These analytics could center around anything from customer purchasing data to machine operability and maintenance on a manufacturing floor. Everything generates data, but the key is how you utilize it. Big data is a culmination of these efforts. We now live in an age where mining data is no longer impossible and machines are more useful than ever. This has brought a renewed focus around what big data is capable of achieving.

So just what is Big Data?

Big data is essentially a massive accumulation of statistics from day to day interactions within a business, product or service. These data are collected by industries and then analyzed for trends that provide useful revelations. The hardest part is extracting the so called “useful” revelations because often they come from correlating data that may not be obviously related.

Jerry, the mouse

Let’s use an example to explain further. Say you have a mouse problem in your garage like I recently experienced. We will name him Jerry. You’ve seen Jerry sprinting around but haven’t been able to catch him. But then Jerry goes away for days. Wammo! Problem solved… Until you see him again. How did he get back in? Meanwhile, your garage door panel no longer works, you haven’t slept in days dreaming about this mouse, and the world feels like it’s going to end!


What data do we have? Jerry is getting in somehow, but you’ve kept your garage shut for days. He’s obviously finding a way in and out but you’ve found no spots after clearing your garage. No matter where you’ve set the trap, he’s avoided it. And that garage door panel is still broke!

Enter big data analysis. You, being the super computer that you are, connect the dots. A day later while fixing you garage panel, you noticed the power seemed to be dead. But after changing the battery the problem persisted. So then you looked at the wires and.. aha! The wire is in half. But how? That’s right, ole Jerry chewed through it. And lo and behold right next to the wire is a tiny gap in your garage wall. You set the trap, and a day later, you bid farewell to Jerry.

This is a much simplified version of big data usage at play but hopefully drives home the point. Multiple data can be connected, even if you believe them to be unrelated. The big data phenomenon is placing these seemingly obscure puzzle pieces together to tell a story. Take those few data points and trends from the Jerry example and explode that by millions. Then connect those dots to solve your mouse problem. Now we are talking big data.

If you’re still confused by my wild analogies, maybe this handy video will help you understand big data on some basic level:

Big Data: Why now?

It is a fair question. Why is big data suddenly gaining such momentum? After all, data gathering as a practice has been around forever. Businesses have been formulating new metrics each year to measure growth and success. So what gives?

My processor is bigger than yours

One of the more obvious shifts in data collection is around computers. I remember back in the early 2000s when a dual processor and x32 bit computing was all the rage. Even for those companies ahead of the curve (quad-core hardware, etc), the software was still catching up to the hardware.

As developers figured out how to take advantage of the additional horsepower, big data became more useful. Multiple processors, increased RAM, and x64 bit processing were just some of the contributors that unlocked the potential of big data. Suddenly, processing 1,000s to 100,000s of data points could happen in a flash. With organized data in hand, the analytics folks could now focus more on meaningful conclusions from the treasure trove of information.

As we marched through the 2000s, cloud computing then suddenly became all the rage. Over night, you were no longer beholden to what physical disk space was available to your computer. Along with expanded storage came the explosion of the virtual desktop. Those incremental shifts in processors and RAM just mentioned? Go exponential. Companies like VMware and Citrix created software to easily daisy chain super computers together. With one 7ft by 2ft rack, you now had processing power beyond anything we’d ever seen before. Big data was the biggest benefactor because now everyone could afford to collect massive amounts of data to improve their business.

Father Time

The other key factor in the rise of big data has been our old friend father time. However, not like time mentioned above where technology could mature. Time in the sense of pure data volume generation.Big Data thrives over time

In order for big data to work, you need two things. One, as the name suggests, tons of data. This doesn’t happen overnight. Even with hundreds of thousands of data points pouring into storage, big data can always use more. Two, and most importantly, big data is impactful because it provides a base to uncover trends. Data in bulk is not that impressive, but draw a trend line through it, and suddenly you have people’s attention.

Those trend lines are enabled by time. A quick example of father time at work. You shoot free throws at a basketball hoop in your driveway. Because you’re a boss, you sink 7 out of 10 free throws. A decent 70% shooter you are, only missing three. But it’s a nice day, so you keep on shooting. An hour later, you’ve made 97 out of a 100. You’ve still only missed three but now you are a 97% shooter! Time, consistency and sample size allowed this to happen and that’s what makes a useful trend. We tend to believe you are closer to a 97% shooter than a 70% shooter. This is the power of big data and what it can point out.

The Future of Big Data

So where does all this big data talk lead us? Honestly, you are feeling the impacts in your everyday life. I could list thousands of examples as to where this is being deployed today but I’m guessing you have a hunch so I’ll just mention a few.

Some uses of big data are already mature. Google and Facebook have paved the way in the social scene, with targeted advertising. The reason most ads are so seamless today is due to big data. Then retail titans like Amazon and Walmart have employed these techniques to predict what you need next. Those models have only become stronger with more data, improving convenience for the consumer each day.

Where big data is continuing to mature is in automation. The voice assistants in your cell phones or smart devices, like Google Assistant and Alexa, are all about big data. They are combining statistics around your purchase habits and those like you to simplify your life. As more people adopt such technology, it will continue to solidify it’s usefulness.

Self-driving cars are another great example of big data maturation. While the trial period has been somewhat dangerous for this technology, the collosal amount of data gathered each run will continue to improve the safety around the technology. Combine that maturing code with the billions of data around people’s driving habits, and it’s clear you will have a path to success (though not full-proof). Should be interesting to see where that technology ends up.

Lastly, big data around production equipment and automation is another key development. Robots scare alot of people but it’s true they have the potential to be programmed for high efficiency. We are not there yet, but big data will continue to drive that bus.

The future for big data is bright. The ability to generate useful data has only begun to crescendo. As machines get “smarter” they will implement this data in real time to become more efficient. In an optimistic world, this should make life easier for the masses with better, more useful products and services. It remains to be seen what impact big data and automation will have on society in the long run. Only father time will tell.

Like this column? Please follow Jarkable on the social media platform of your choosing for updates to all the latest postings!

  • 62
  • 72
Recommended Posts
Showing 18 comments
  • Waren Jean

    Big data is useful but also a bit scary. Technology has evolved so much everything has gotten convenient. I hope people uses it for good purpose.

  • Arkeylah

    This is an interesting article and thanks for the education. Big data is needed by businesses to see how well they are performing. We as bloggers always require data to monitor traffic and metric.

  • Joan

    I need to educate myself more when it comes to this, thank you for sharing.

  • Nik

    Incredible blog to learn something new completely to me. Big data is the trend, no doubt about that. People are talking about it but hardly anyone knows about it. Thanks for introducing in such an interactive way 🙂

  • Nisha

    Big Data can always be overwhelming but it is essential, must be understood to monitor traffic etc. thank you for sharing this knowledge

  • Sondra Barker

    This was really interesting. Big Data is definitely “it” right now, every business need it.
    xo, Sondra

  • Casey

    Wow, this is a very complicated topic. It seems these data can be used in a good way but at the same time it could tap into a lot of privacy issues. Anyway, thx for the info.

  • Reply

    Wow! I love the way you write! I can’t believe I was able to understand technical information so easily. Thank you! I gotta think of how I am implementing Big Data in my business. But… I feel bad for Jerry! 🙂

  • Anonymous

    Indeed, this is the core of any business marketing tactics. My husband used to work for an analyst. And any single data they get plays a big role in their research.

  • What Corinne Did

    I already knew the growing importance of big data and what it was but have not thought of it in the concept of facebook or google. targeted ads make total sense now. More than before to me at least!

  • Criz (Crizzy Kiss)

    We are definitely living at this stage where data storage, analysis etc are so important for both business and personal use. But in the same manner, I hope the technology can also do better with privacy protection. Very good article! 😉

  • Laura Dove

    Wow this is so interesting and not something I have heard a lot about before. It’ sounds like big data is really important in todays society!~

  • Merryl Rodriguez

    The net is so complex and there’s so much complex data and analytics. I’m still wrapping my head around all the information available

  • Paula Stewart

    Many don’t like that companies collect data but in my opinion, it makes them better able to provide what we want making for a better consumer experience for us all.

  • Charli Bruce

    This was a good post to read. I shall be honest and admit I normally opt out of anything collecting my data

  • Meagan Badore

    This is a really interesting read about big data. I didn’t realize how much it is used by different companies for different reasons.

  • Nicole

    This was a really interesting post. Big Data is definitely on the rise right now. Technology is continually growing and advancing every minute. It’s a scary but amazing thing!

  • Anonymous

    What a wonderful post! I liked reading this. I totally understand why it has such an impact. Thanks for sharing! 🙂

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Ping Us

Out finding Jarkable content. But you can send us an note and we'll get back to you, asap.

Not readable? Change text. captcha txt
Action Camera - GoPro | Jarkable