Simon Ritter, Deputy CTO at Azul, explains how optimising Java-based infrastructure can save on cloud overspend and improve efficiency.

With the cloud being the infrastructure of choice for business applications, a trend which is only anticipated to grow in this next year, CIOs are not just thinking about how they migrate to the cloud, but also how best to balance their application performance and user experience with spiralling costs.

According to our recent 2023 State of Java Survey and Report, released in October, 98% of global respondents say they use Java in software or infrastructure, and 83% state that 40% or more of their applications are Java-based. It is also clear from our report that Java plays a critical role in cloud cost optimisation – 90% of survey respondents are using Java in a cloud environment, whether it is public (48%), private (47%), or hybrid (40%).

However, businesses are not always making the best use of their cloud infrastructures: 41% of global respondents report using less than 60% of the public cloud capacity they pay for, and 69% are paying for unused cloud capacity. They are taking steps to lower their cloud costs, as the continued market volatility means that CIOs must be judicious in how they allocate precious resources. 

So, the question becomes: how do organisations prevent the cloud from becoming a money pit?  This is where the concept of FinOps is becoming increasingly relevant and sophisticated as IT teams seek to monitor and adjust their cloud usage to minimise overspend. 

One area where we believe this approach can be applied is in examining the performance of Java-based applications in the cloud. Why? Well, Java is everywhere in the enterprise. 

The power of faster Java

I recently spoke with William Fellows, Research Director at S&P Global Market Intelligence, about the topic of cloud optimisation. Fellows shared insights from S&P’s research, which showed that managing unanticipated demand and overprovisioning more resources than are required are the two biggest reasons for cloud overspend. By his estimates, Fellows suggests that spending on the public cloud was 56% over what was necessary in 2022. Fellows and I concluded that there are ways to reduce cloud waste for Java workloads.

Costs are spiralling out of control

The dilemma for CIOs is that overprovisioning is something of an insurance policy for ‘just in case’ demand peaks, and the infrastructure needs the bandwidth to cope. What if more efficient optimisation of Java-based applications could have a positive impact on the need for additional resources? 

Outlined below are some recommendations to help organisations improve cloud usage by optimising their Java-based infrastructure:

Faster Java

According to a May 2023 IDC report, effective management and optimisation of both costs and performance of Java applications are critical to the success of digital businesses to retain customers and grow revenues. IDC estimates that 750 million new applications will be created by 2025, many of those in Java. Enterprises need an easy way to optimise the performance and licensing of their new and existing Java applications without the burden of rewriting or recompiling them (which can take years to do). 

To address the challenges of controlling costs and providing better performing, more resilient applications, enterprises should look to a Java optimising platform which can speed up the start-up of Java applications, improve scaling and runtime performance, and reduce the number of cloud resources needed for the environment. This enables the most demanding Java enterprise applications to be optimised for performance and compute consumption at the runtime level.

Addressing Java warm-up

The way that Java Virtual Machine (JVM) based applications are designed, they rely on just-in-time (JIT) compilation. The time to compile all the frequently used sections of code is called warm-up time. Historically, applications have run for days, even weeks or months, so this warm-up time has been negligible compared to the time the application runs. However, in the era of microservices-based applications which offer the ability to spin up new service instances when there is a sudden increase in workload, the time to compile can become a performance issue. 

With JVM-based applications, when new service instances start, the warm-up time delays the instance’s ability to handle requests.

This leads DevOps teams to overprovision cloud systems to maintain real-time performance. They keep a pool of services running in reserve, fully warmed up and ready to use as demand changes. However, as mentioned earlier this can lead to major overspend in IT budget.

This issue with warm-up times can be addressed, alongside latency associated with garbage collection and throughput of transactions, to improve performance. For example, if you can achieve better transaction throughput you can reduce the size of the nodes in your cluster, the number of nodes, or both while still meeting Service Level Agreements (SLAs). 

Compilation-as-a-Service

It is also possible to create profiles of all frequently used codes once they have been compiled and optimised. There are tools that record a profile including all the information about a performed compilation, so that when the application or service starts again, the JVM uses the profile to ensure all code is compiled and ready-to-go before the application code starts to execute.

Taking advantage of the cloud’s distributed model, it is possible to take the compilation part of the JVM and make it into a centralised service. This offloads the work from the individual microservices, reducing the size of the instances that may need to be created for individual services as they do not have to do their own compilation. It is also possible to have a memory so that when code starts up it is not necessary to compile the code each time. Instead, it can be cached to make it much more efficient.

Squeezing more efficiencies out of Java

Another aspect of optimising JVM-based applications is understanding that Java is hardware and software agnostic. That means customers are not constrained to using one type of processor. For example, Azul’s technology works with Graviton processors, which offer a 40% price performance improvement over fifth generation Intel chips and deliver a 72% drop in power consumption. 

We are also seeing an increase in more specialised cloud computing instances for compute-intensive tasks in fields like artificial intelligence (AI) and machine learning. It is possible to optimise the JVM for these instances by using the JIT Compiler to make use of the micro-architecture instructions that are available. For example, vector processing is used to deliver higher performance for numerical-intensive applications.

Optimising your Java-based Infrastructure

Cloud adoption is moving into a new phase. Its accessibility encourages greater use, and the latest generation of tools and innovations means its potential to help organisations become more agile and responsive to customers is more attractive. 

CIOs, though, know they do not have unfettered licence to adopt cloud-based applications. They must also continue to demonstrate prudence against the backdrop of market volatility. 

Java is tuneable – and should be used to build systems with revenue and profit in mind. Cost optimisation must be measurable and tied to business impact.  Programming languages like Java provide profiling tools to analyse code performance, and while these require setup and expertise, they enable granular analyses that can lead to changes that shave milliseconds. What may seem like small optimisations accumulate into large savings at scale.

With Java so widespread in most enterprise IT infrastructures, it presents a significant opportunity to demonstrate how IT can both drive large-scale efficiencies and help organisations to maximise the value of their cloud investments.