Kunal Agarwal, Unravel co-founder and CEO, explores the trends and attitudes to data workloads for 2020. With the continued rise of Kubernetes, the increasing use of SREs and a changing view on the topic of ‘NoOps’, the year ahead is seeing a shift in conversations to modern data workloads.
Lately, fewer people use the term ‘big data’. We’re talking about how such data is actually put to use, and especially in AI and machine learning use cases. It’s because organisations have deployed big data and are actively trying to achieve value from their projects. We see this pulling into 2020 as enterprises focus more on use cases. The conversation around modern data workloads will also be impacted by the continued rise of Kubernetes, increasing use of SREs and a changing view on the topic of ‘NoOps’.
Next year will be dominated by these data trends:
Automated troubleshooting moving to data apps: Organisations have been using SREs (site reliability engineers) to ensure websites function properly by developing automated fixes for web errors. But now this approach to automated troubleshooting is now moving to data apps, with DataOps teams both adopting the general approach of SREs as well as hiring specialized SREs to solve data analytics problems. Expect major enterprises to hire SREs for data apps and to create positions that will be focused on specific technologies like Hadoop, Spark, Kafka, and so on.
NoOps falls short: Using AI to automate makes operations more efficient, but organisations can’t leverage cloud services and AI to eliminate all IT operations. DevOps and DataOps are needed in the cloud too. The cloud is an ideal destination for data workloads, but operational challenges don’t go when you get to cloud. New challenges emerge, such as re-configuring apps for improved performance and monitoring workloads for overuse and increased costs. AI tools can simplify this, but organisations need humans to leverage those tools correctly. There need to be guard rails in place to ensure cloud delivers on cost and performance. NoOps is reminiscent of nearly a decade ago when people thought the cloud was a panacea. Yet a hybrid model has won out with the cloud becoming ideal for some apps, with others remaining on-premise.
Do you remember a pre-Kubernetes?: It’s recently surpassed Docker as the most talked about container technology. In the future, every data technology will run on Kubernetes. 2020 will see Kubernetes adoption continue to rise as vendors base platforms on it. The entire big data community is behind Kubernetes and its domination is assured as kinks are worked out.
From tech to uses: Previously organisations thought that to leverage big data they had to deploy technology X. That’s changing dramatically as we move into 2020, with people paying less attention to the tech under the hood and instead focusing on delivering specialised use cases that advance their bottom line.
M&A: Expect to see more! It’s great for customers as it gives them more robust platforms and helps avoid vendor lock-in. Enterprises care very little today about which vendor they’re buying from, they just want to make sure the solutions fit their use. Everyone mixes and matches. The ongoing rollup of these companies is a result of this trend.
Big data has hit an inflection point in which data workloads are being used in full production and organisations are concerned with getting more value from them, and supporting new use cases. These trends reflect this evolution and will define the big data conversation in 2020.
