Contact US

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

What is BIG DATA?

What is BIG DATA?

What is BIG DATA?

What is big data?  What makes it bigger than any data we've had before?  What's the "big deal" about it - why is it such a subject of interest?  Or is it just one of the latest buzzwords?  

I'm being partly humorous, but I really don't get it.  Could someone explain the concept and why it's such a hot topic?

The trouble with doing something right the first time is that nobody appreciates how difficult it was - Steven Wright

RE: What is BIG DATA?

If you are talking about super enourmous large data sets I think it is a hot topic because there are a growing number of these sets (pun?) and because there is concern that current tools, structures, and ways of doing things may not work well at that scale.

As for what makes things bigger than anythig before, I think it used to be that few folk added millions of records a day (just to pull a point from the air) and now by comparison more folk are adding millions of records an hour.  

Hell, my home machine has a (what is normal now) terabyte drive in it.  I remember working on a Sinclair and a Commodore and then thinking an IBM Model 30 was cool.

It's just the relentless pace of change.


RE: What is BIG DATA?


because there is concern that current tools, structures, and ways of doing things may not work well at that scale.
Gently understated<g>

IMHO there is no hardware/software combination that has ever existed (or ever will) that can prevent "running out of resources". Left uncontrolled, "designers" can render any environment totally useless. . .

One of my recent clients has nearly 120 terabytes of online/realtime database storage (multiple database engines) and the term "big data" has never been used there.

From what i hear this is currently a new buzzword with no commonly accepted definition. Lots of vendors toss this in their presentations. . .


RE: What is BIG DATA?

"big data" is anything large enough to get you thinking that you might want to store it in "the cloud"


r937.com | rudy.ca
Buy my new book Simply SQL from Amazon

RE: What is BIG DATA?

Look at Wikipedia article titled "Big data"...

RE: What is BIG DATA?

I would personally say much of the problem is time based.

Look back 50 years the only issue was how to file it, and where the filing cabnet was.

Look back 20 years - well databases were starting to become very popular.  I imagine some databases have been running for well over 20 years (hopefully in a different format, on a different server) then they were on.  Data has accumalated over that time -and people either

Are scared to delete data thats not required - incase it is required.
Are scared to archive the data incase its required quickly.
Are scared to touch a system that has been working.

So data just builds and builds and builds.  I have seen systems with 15 years worth of data.  I am sure there is more - and it isnt required even on a yearly basis - but no one archives it.

So it just gets bigger and bigger.  Coupled with what other people have said about data going in quicker and quicker (internet has helped this!) you end up with BIG data!



Be who you are and say what you feel, because those who mind don't matter and those who matter don't mind - Bernard Baruch

Computer Science is no more about computers than astronomy is about telescopes - EW Dijkstra

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close