In its May 2011 report on big data, the McKinsey Global Institute projected a 40 percent annual growth in global data—a doubling of data volumes every two years. Much of this data is complex, unstructured data that doesn’t submit readily to analysis. In fact, big data refers to the volumes of mostly complex, unstructured data whose analysis could yield significant dividends in terms of competitive business advantage, but cannot be analyzed by the existing structured data oriented infrastructure & framework.
Therefore it lies the challenge of big data, at Three WHYs for Big Data, we explored why Big Data is Big Matter, we also further articulate Three Big WHATs to identify Big Data Challenges, and now continue the series on how to overcome the barriers to tame the Big Data?
While experts agree that Big Data is big, exactly how big is a matter of debate. IDC forecasts a roughly 50 percent annual growth rate for what it calls the world’s “digital universe,” more than 70 percent of which IDC estimates is generated by consumers and over 20 percent by enterprises. Between 2009 and 2020, the digital universe will swell by a factor of 44 to 35 zettabytes, or 35 million petabytes, IDC predicts
Big data is data that is too large to process using traditional methods. It originated with Web search companies who had the problem of querying very large distributed aggregations of loosely-structured data
If you’re creating large data sets, which most of businesses do today, you have no choice but to embrace big data management. First Steps to Managing Big Data can be overwhelming and full of confusion, especially if the organization has a lot of existing data and data continue to grow and flow exponentially.
A) Start with business Strategy/goal: As we put in the Three Big Whys for Big Data, Big Data is the means to end, not the end. Start with end: the purpose, the business goals, the strategy;
B) And then develop a Big Data Strategy/map in which you classify:
The types of data that are important to the organization.
Various aspects of your data, such as whether it changes a lot (complexity & variety) and how fast it’s growing (velocity).
Is much of your data critical information
Prioritize the business problems Big Data can help solve
C) The right Tools and Architecture: using traditional data warehouse solution to search through such vast amounts of often unstructured data can takes weeks, if not months. Hadoop, an open source ecosystem of data management tools and technologies that can help organizations tackle the broad problem of big data analytics, here is another nice article to recommend 7 top tools for taming Big Data.
D) Data Security & Governance: However, IT and business managers should keep cautious when it comes to implement Hadoop, which requires a fair amount of practices and serious attention & solutions to the security of massive amounts of data/ information stored in distributed clusters and potentially in public clouds
Extracting business value and wisdom from big data remains elusive for many organizations. For most companies today, data are abundant and readily available, but not well used. Nearly one in four survey respondents says the vast majority of its company’s data are untapped. Another 53% say they only use about half of their valuable data. Yet 73% say that data collection in their organization has increased over the last year
Big-data analytics must be business led with the right mindset, time to value, and with effective data mining tools, as not all projects will be successful at finding the needle in the haystack. The organizations that can do it successfully will out beat the competitors and become the intelligent business in 21st century.
Twitter: @Pearl_Zhu
Post a Comment