With all the buzz surrounding big data and the ways companies will leverage it, you may find yourself asking, “which types of data are we referring to?”
Well, the first thing to understand is that not all data is created equal. This means the data generated from social media apps are completely different from the data generated by point-of-sales or supply chain systems.
Some data is structured, but most is unstructured. The way this data is collected, processed, and analyzed all depends on its format.
To clear things up, we'll break down the distinct differences between structured and unstructured data.
Structured vs unstructured data
Once you have a basic understanding of qualitative vs quantitative data, you can then make sense of data structures or lack-thereof.
What is the difference between structured and unstructured data?
Structured data is highly-organized and formatted in a way so it's easily searchable in relational databases. Unstructured data has no pre-defined format or organization, making it much more difficult to collect, process, and analyze.
In addition to being collected, processed, and analyzed in different ways, structured and unstructured data will reside in completely different databases.
Structured data is most often categorized as quantitative data, and it's the type of data most of us are used to working with. Think of data that fits neatly within fixed fields and columns in relational databases and spreadsheets.
Examples of structured data include names, dates, addresses, credit card numbers, stock information, geolocation, and more.
Structured data is highly organized and easily understood by machine language. Those working within relational databases can input, search, and manipulate structured data relatively quickly. This is the most attractive feature of structured data.
The programming language used for managing structured data is called structured query language, also known as SQL. This language was developed by IBM in the early 1970s and is particularly useful for handling relationships in databases.
If it sounds confusing, the picture below should help visualize how structured data relates to each other within a database.
From the top-down, we can see that UserID 1 refers to the customer Alice, who had two OrderIDs of ‘1234’ and ‘5678’.
Next, Alice had two ProductIDs of ‘765’ and ‘987’. Finally, we can see Alice purchased two packages of potatoes and one package of dried spaghetti.
Is this data useful on the surface? Not really, but running it through analytic tools can help unveil patterns and trends about a specific customer or customer base. This type of data is commonly seen in CRM software.
Structured data revolutionized paper-based systems that companies relied on for business intelligence decades ago. While structured data is still useful, more companies are looking to deconstruct unstructured data for future opportunities.
Unstructured data is most often categorized as qualitative data, and it cannot be processed and analyzed using conventional tools and methods.
Examples of unstructured data include text, video, audio, mobile activity, social media activity, satellite imagery, surveillance imagery – the list goes on and on.
Unstructured data is difficult to deconstruct because it has no pre-defined model, meaning it cannot be organized in relational databases. Instead, non-relational, or NoSQL databases, are best fit for managing unstructured data.
Another way to manage unstructured data is to have it flow into a data lake, allowing it to be in its raw, unstructured format.
More than 80 percent of all data generated today is considered unstructured, and this number will continue to rise with the prominence of the internet of things.
Finding the insight buried within unstructured data isn’t an easy task. It requires advanced analytics and a high level of technical expertise to really make a difference. This can be an expensive shift for many companies.
Those able to harness unstructured data, however, are at a competitive advantage. While structured data gives us a birds-eye view of customers, unstructured data can give us a much deeper understanding of customer behavior and intent.
For example, data mining techniques applied to unstructured data can help companies learn buying habits and timing, patterns in purchases, sentiment toward a specific product, and much more.
Unstructured data is also key for predictive analytics software. For example, data from sensors attached to industrial machinery can alert manufacturers of strange activity ahead of time. With this information, a repair can be made before the machine suffers a costly breakdown.
The future of data
The volume of big data is continuing to rise, but soon, the importance of having large volumes will cease to exist.
Regardless if data is structured or unstructured, having the most accurate and relevant data at hand will be key for companies looking to gain an advantage.
Utilizing the right data will allow companies to:
- Reduce operational costs.
- Track current metrics and create new ones.
- Understand its customers on a far deeper level.
- Unveil smarter and more targeted marketing campaigns.
- Find new product opportunities and offerings.
Research from IDC states that companies with the right data will see an additional $430 billion in productivity gains by 2020. It’s no wonder why IBM estimates there will be roughly 2.72 million data science jobs posted over the next few years.
The more varieties of data created will lead to new and advanced algorithms – toeing the line of GDPR compliance.
Here’s an algorithm that might creep you out, courtesy of The Institute:
“Facebook last year filed a patent for an algorithm that attempts to analyze users’ emotions by how they type and compare that to their baseline. If people are tapping their phone’s keyboard harder or typing slower than usual, that could indicate they are angry or depressed.”
Of course, this is a testament to just how powerful big data can be when leveraged in unique ways. At the end of the day, it’s up to the consumer to determine how comfortable they are with the ways their data is used.
New to data analytics but want to learn more? Check out our complete beginner's guide covering basic topics from processes to types of analytics.