Blog

Tuning Configuration of the Cloud Foundry Java Buildpack to Avoid Out-of-Memory Errors

Andrei Bushyk

tuning-cloud-foundry-java-buildpack-to-avoid-out-of-memory-errors-featured

Java apps created in accordance with 12-factors are supposed to run in Cloud Foundry without modifications. However, sometimes the defaults provided by the public buildpacks don’t work. In this post, we will look at two ways to resolve out-of-memory errors.

 

How memory is allocated in the Java buildpack

Most Java applications do not need any additional memory configuration to run in a Cloud Foundry environment. In general, it is enough to set the MEMORY_LIMIT environment variable or provide a manifest.yml file with the required value.

MEMORY_LIMIT specifies how much memory will be allocated for the container that will run the application. The Java buildpack uses this value to control the JRE’s use of various regions of memory: heap, metaspace, native, and stack.

Memory allocated for these regions is calculated by the memory calculator, using memory-related properties of the Java buildpack (see the diagram).

tuning-cloud-foundry-java-buildpack-to-avoid-out-of-memory-errors

By default, the Java buildpack specifies the following values for memory distribution:

  • heap: 75%
  • metaspace: 10%
  • native: 10%
  • stack: 5%

See the readme file of the Open JDK JRE for an explanation of JRE memory sizes and weights and how the Java buildpack calculates and allocates memory to the JRE for your app.

When the Java application is deployed and the native memory is not sufficient, the container will restart before it does its first full garbage collection. It is a clear sign of insufficient allocation of native memory.

There are two ways to solve this issue: a complicated one and a simple one.

 

Solution #1: Creating a custom buildpack

The first (complicated) way is to create a custom buildpack with updated memory-related JRE options and specify this buildpack in the deployment manifest. Cloud Foundry docs provide exhaustive instructions on how to do that.

This approach has some serious flaws. Creating and maintaining a separate custom version of the Java buildpack just for one application is not very efficient. It is also far from being trivial.

 

Solution #2: Overriding buildpack configuration

The second, simpler approach is to override buildpack configuration with an environment variable. The name of the variable needs to match the configuration file you want to override, but without the .yml extension and with a prefix of JBP_CONFIG. The value of the variable must be valid inline YAML.

In the case of a memory heuristics property in open_jdk_jre.yml, the variable might look like:

JBP_CONFIG_OPEN_JDK_JRE: '[memory_calculator: {memory_heuristics: {heap: 65, metaspace: 20}}]'

It can be passed to Cloud Foundry directly:

cf set-env [APPLICATION_NAME] JBP_CONFIG_OPEN_JDK_JRE: '[memory_calculator: {memory_heuristics: {heap: 65, metaspace: 20}}]’

Note that you will have to run this command each time the app is redeployed. To avoid that and make the environment reproducible, specify the variable in the application’s manifest:

env:
    JBP_CONFIG_OPEN_JDK_JRE: '[memory_calculator: {memory_heuristics: {heap: 65, metaspace: 20}}]'

That way, the environment variables won’t get lost if the app gets deployed elsewhere.

Don’t forget to restage the app after editing the files.

 

Take-aways

In a Cloud Foundry environment where apps are automatically restarted, you may not even notice that there is an issue with memory allocation—unless you are specifically looking for it. So, make sure the heap and native memory spaces are sized appropriately for the app. Specifying environment variables in application’s manifest should be the most efficient way of doing that.

 

Related reading:


About the author:

Andrei Bushyk is Senior Software Developer at Altoros. Since 2007, he has worked on high-load and distributed projects for finance, retail, and entertainment. Andrei is an expert in big data processing, NoSQL data stores, and Hadoop. In the last two years, his focus has shifted to building 12-factor apps using cloud-native platforms based on Cloud Foundry.


To stay tuned with the latest updates, subscribe to our blog or follow @altoros.

Get new posts right in your inbox!

2 Comments
  • springusr

    Question: if you can throw some light on memory vs memory_limit property in yml for cloud foundry and how they behaves.

    • Andrei Bushyk

      Hi, springusr. Actually, these properties are the same and used for one purpose, but in the different places.
      The first one, memory, is used in manifest.yml.
      Second one, memory limit, is used in the Command Line Interface (CLI) (-m argument).
      If you do the cf push using manifest, but also specify -m argument: [cf push -m 64M -f app-manifest.yml], the second (memory limit), will have precedence over the first one.

Benchmarks and Research

Subscribe to new posts

Get new posts right in your inbox!