Top 5 Trends in Data Science

The world is swamped with Big Data. How to turn Big Data into actionable insights is a key question for business leaders at many organizations.

We analyzed data from many diverse sources, and identified 5 important trends in data science.

Trend #1: Data scientists will be in high demand in the next decade

Per McKinsey, by 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills. Our independent studies on 3 top job search sites indicated the number could be even higher than that.

Trend #2: CXOs are beginning to demand data scientists to show the impact

Over the last couple of years, many companies rushed to build their Big Data infrastructure. Now, the CEOs are demanding a return on investment on Big Data infrastructure. To create impact for business, data scientists need to work closely with business leaders to identify key areas where data science can make a big impact. And business leaders need to find the right data scientists to help deliver high-impact projects quickly and effectively.

Trend #3: Cloud-based platforms are the future for good reasons

Per Cisco, by 2020, 92% of all workloads will be processed in cloud data centers. Amazon, Microsoft, IBM, Google, and Salesforce are leading the pack in cloud-based solution offerings (source: Synergy Research data). The price for cloud solution is dropping. The security is stronger than with stand-alone company technology platforms. And the speed and flexibility of building and deploying cloud-based platforms is amazing! All those will allow top data scientists to work from remote locations.

Trend #4: Free, open-source tools are winning the minds of analytics pros

For many years, SAS has been the default tool for many data scientists to crunch Big Data. It is expensive and has been slow to adapt to new analytical algorithms. 2016 was the year when more data scientists preferred R and Python than SAS (Burtch Works survey). We are expecting the trend to speed up as more and more data scientists are working together to build the open-source community.

Trend #5: Emerging feature-discovery tools are gaining credibility

Big Data comes in many different forms. One of them is transactional-level data. Transactional-level data are massive, messy, and hard to gain insights. Traditionally, an experienced analyst spends weeks brainstorming with business, coding up variables, and conducting hypothesis tests to figure out which ones are impactful or not.

The good news is, emerging tools are addressing this challenge. We’ve seen some emerging feature-discovery tools that can systematically create hundreds of thousands of variables, test those variables, and identify the most important features. The whole process takes only a few days instead of weeks, and the results are very promising.

If you have any comments or would like to learn more about how we apply machine learning algorithms and tools to help solve business problems, please click here.

Download the Complete Whitepaper