Big Data refers to data sets with exceptionally large, very complex or very rapid information which can’t be analyzed sufficiently through conventional relational databases. Big Data is high-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing to enable enhanced insight, decision making, and process automation.
Big Data is so large and complex that none of the traditional data management tools can store it or process it efficiently. Big data is when the normal application of current technology does not enable users to obtain timely, cost-effective, and quality answers to data-driven questions.
The bulk of big data generated comes from three primary sources:
- Social data: social media platforms (tweets, comments, video uploads, audios)
- Machine data: sensors like road cameras, smart meters, satellites, IoT, robots, and web logs
- Transactional data: Invoices, payment orders, storage records, delivery receipts
Big data management is the organization, administration and governance of large volumes of both structured and unstructured data. The goal of big data management is to ensure a high level of data quality and accessibility for business intelligence and big data analytics applications.