The rapid expansion of digital data through computers, mobile, video, social media, digital sensors, etc. combined with major breakthroughs in lower-cost processing power, open source database applications and wider bandwidth has sparked massive interest across the entire business world in the emerging field of Big Data science and analytics.
Big data in large unstructured volumes are too huge to be managed and analyzed through traditional methods. The sheer amount and velocity of today’s data makes capturing, filtering, storing and analyzing a real challenge. New products are developed regularly to deal with this which call for new skill sets and expertise. There’s growing need for individuals who can integrate new infrastructure, platforms and processes into the organization as well as those who can build new analytics and algorithms capable of creating enormous intelligence of great business value. For more information, read our blog post on The growing importance of Data Science and how training in this subject affects your earning potential
Relevance of Data Science in Different Industries:
Data Science & Analytics has application across all industries:
- ecommerce – Personalization & recommendation engines that increase sales.
- Advertising – Highly targeted, real-time ad delivery to consumers.
- Media & Entertainment – Customized content development that maximizes user engagement.
- Social Media – Increased site “stickiness”, user growth, ability to track fast-breaking trends based on consumer sentiments.
- Financial Services –Optimized lending practices that minimize risk and fraud.
- Pharma / Bioinformatics – Improved drug discovery, more effective treatments of threatening diseases, genetic engineering enhancements.
- Healthcare – Better scoring of medical patients for health risks as well as anticipation and early prevention of diseases.
- Power/Energy – Smart grid intelligence, usage efficiencies, energy savings and reduction of downtime.
- Information Security – Vastly improved theft detection and monitoring of valuable company information and assets.
Key Skills of Data Science Professionals:
Data Science Domain Requires Professionals who:
- Understands data analytics and decision science
- Are well versed in IT
- Have strong business acumen
- Possess the ability to communicate effectively with decision-makers
Read more: Core skills required to be a Data Scientist.
Common Technologies Associated with Data Science Practice:
- Databases
Oracle, SQL Server, Teradata
Cassandra, Hadoop, MapReduce,HBase
Aster, Greenplum, Netezza
- Languages
Ajax, C++, CSS, HTML5, Java, JavaScript, Perl, Python, Scala
Hive, Pig, Lucene, Mahout, Solr
- Statistics & Forecasting
Angoss, MATLAB, R, SAS, SPSS
ARCH, GARCH, SVAR, VAR, VEC, GAUSS
- Data Visualization
QlikView, Spotfire, Tableau, yWorks, R
- BI & Reporting
BusinessObjects, Cognos, MicroStrategy
What is Cassandra?
- Apache Cassandra is an open source distributed database management system designed to handle large amounts of data across many commodity servers.
- Cassandra provides high availability with no single point of failure.
- Cassandra offers robust support for clusters spanning multiple data centers,with asynchronous master-less replication allowing low latency operations for all clients.
For more information, read our blog post on the advantages that Cassandra has over other traditional RDBMS.
How does Data Science make use of Cassandra?
Cassandra is a distributed database for low latency, high throughput services that handle real time workloads comprising of hundreds of updates per second and tens of thousands of reads per second.
Cassandra Use Case – PROS:
PROS is a Big Data software company with prescriptive analytics in their software that facilitates their customers to analyze their data and get the insights and guidance to optimize their pricing, sales and revenue management.
They have a real-time service that computes airline availability, dynamically taking into consideration revenue control data and inventory levels that can change many hundreds of times per second.
This service is queried several thousands of times per second, which translates to tens of thousands of data lookups. Their backend storage layer for this service is Cassandra.
For their real-time solution, PROS realized a need for:
- A distributed cache that is highly available.
- Easily scalable.
- With a master-less architecture.
- With near real time data replication even across data centers.
- That can handle real time reads and writes.
PROS evaluated Cassandra against Oracle Berkeley DB, Oracle Coherence, Terracotta, Voldemort and Redis. Apache Cassandra quite easily topped the list.
PROS and Cassandra
- PROS uses Cassandra as a distributed database for low latency, high throughput services that handle real time workloads comprising of hundreds of updates per second and tens of thousands of reads per second.
- For example, they have a real-time service that computes airline availability dynamically taking into consideration revenue control data and inventory levels that can change many hundreds of times per second. This service is queried several thousands of times per second, which translates to tens of thousands of data look ups. Their backend storage layer for this service is Cassandra. Some of their SaaS offerings use Cassandra as the backend store to handle a combination of real-time and Hadoop based batch workloads.
- Talking about Hadoop and Cassandra, they take the data out of Cassandra and put it into Hadoop and run batch and analytics on that, and then that goes back into Cassandra. This is achieved through Cassandra’s Hadoop integration.
- The Hadoop jobs pull data out of Cassandra, applies job specific transformations or analysis and pushes data back into Cassandra. They are not using the Datastax (official Cassandra Maintainer) Enterprise edition for this integration; just the open source Hadoop installation with Cassandra.
Data Modelling with Cassandra:
When looking to replace a key-value store with something more capable on the real-time replication and data distribution, research on Dynamo, the CAP theorem and eventual consistency model shows Cassandra fits this model quite well. As one learns more about data modeling capabilities, we gradually move towards decomposing data.
If one is coming from a relational database background with strong ACID semantics, then one must take the time to understand the eventual consistency model.
Understand Cassandra’s architecture very well and what it does under the hood. With Cassandra 2.0 you get lightweight transaction and triggers, but they are not the same as the traditional database transactions one might be familiar with. For example, there are no foreign key constraints available – it has to be handled by one’s own application. Understanding one’s use cases and data access patterns clearly before modeling data with Cassandra and to read all the available documentation is a must.
Conclusion:
Apache Cassandra is evolving fast and we are learning and understanding its capabilities – especially on the data modeling side. We see it as a distributed NoSQL database of choice for our Big Data services and solutions.
Edureka provides a comprehensive Data Science with Python course for those who wish to become a data scientist. The course covers a range of Hadoop, R, and Machine Learning Techniques encompassing the complete Data Science study.
Also, If you are looking for online structured training in Data Science, edureka! has a specially curated Data Science Training that helps you gain expertise in Statistics, Data Wrangling, Exploratory Data Analysis, and Machine Learning Algorithms like K-Means Clustering, Decision Trees, Random Forest, Naive Bayes. You’ll learn the concepts of Time Series, Text Mining, and an introduction to Deep Learning as well. New batches for this course are starting soon!!