Big data is now a reality. Data is created constantly. Data from mobile phones, social media, GIS, imaging technologies for medical diagnosis, etc., all these must be stored for some purpose. Also, this data needs to be stored and processed in real time. The challenging task here is to store this vast amount of data and to manage it with meaningful patterns and traditional data structures for processing. Data sources are expanding to grow into 50X in the next 10 years. An International Data Corporation (IDC) forecast sees that big data technology and services market at a compound annual growth rate (CAGR) of 23.1% over 2014-19 period with annual spending may reach $48.6 billion in 2019. The digital universe is expected to double the data size in next two years and by 2020 we may reach 44 zettabytes (1021) or 44 trillion gigabytes. The zettabyte is a multiple of the unit byte for digital information. There is a need to design new data architecture with new analytical sandboxes and methods with an integration of multiple skills for a data scientist to operate on such large data.
