Understanding JVM Memory, CPU, Requests, Limits, and Production Sizing
“If you’re still sizing your Java application based only on
-Xmx, you’re sizing it for a virtual machine, not for Kubernetes.”
Thursday, 2:15 PM
It was the first payroll processing day of the month.
At Global Trust Bank, this was one of the busiest days of the year.
Millions of salary payments were flowing through the Payment Service.
The Operations dashboard looked healthy.
Pods : 8
CPU : 62%
Memory : 71%
Restarts : 0
Everything appeared normal.
Then suddenly…
payment-service-7fddbbbf79-l2ktn
STATUS
OOMKilled
Within a few seconds another Pod restarted.
Then another.
Soon customer requests began timing out.
The development team immediately joined the incident bridge.
One developer confidently said,
“Impossible. We configured
-Xmx1024m. The heap can’t exceed 1 GB.”
Operations replied,
“The container limit is 1.2 GB.”
Silence.
Everyone assumed heap memory and container memory were the same thing.
They aren’t.
And understanding that difference is one of the most important skills for every Java engineer deploying to Kubernetes.
The Biggest Java Myth
Ask most Java developers:
“How much memory does your application use?”
The answer is often:
-Xmx1024m
Application uses 1 GB
Unfortunately, that’s only part of the story.
The JVM uses much more than heap.
Imagine your application as an office building.
The heap is only one department.
Many other departments also consume space.
Understanding JVM Memory
A Spring Boot application consists of multiple memory regions.
Container Memory
+------------------------------------------------------+
Java Heap
+------------------------------------------------------+
Metaspace
+------------------------------------------------------+
Thread Stacks
+------------------------------------------------------+
Direct Memory
+------------------------------------------------------+
Code Cache
+------------------------------------------------------+
Native Libraries
+------------------------------------------------------+
JVM Internal Structures
+------------------------------------------------------+
Every one of these regions counts toward the Kubernetes memory limit.
Kubernetes doesn’t care whether the memory belongs to the Java Heap or native memory.
It only sees one thing.
Container Memory Consumption
Java Heap
The Heap is where most developers spend their attention.
Objects created with new are stored here.
Examples include:
- Customer objects
- Payment requests
- JSON payloads
- Collections
- DTOs
- Business entities
If the heap fills up, the JVM performs Garbage Collection.
If memory cannot be reclaimed,
the JVM throws:
java.lang.OutOfMemoryError
This is a Java exception.
Notice something important.
This is not the same as Kubernetes OOMKilled.
Metaspace
Every loaded class consumes memory.
Spring Boot applications load thousands of classes.
Consider everything loaded during startup.
- Spring Framework
- Spring Boot
- Jackson
- Hibernate
- Tomcat
- Oracle Driver
- Redis Driver
- Kafka Client
- Solace Client
- Logging Libraries
None of these classes live inside the Java Heap.
They occupy Metaspace.
Large enterprise applications often use hundreds of megabytes of Metaspace without developers realizing it.
Thread Stacks
Now let’s examine something many teams overlook.
Every Java thread requires its own stack.
Imagine this configuration.
Tomcat Threads
200
Kafka Consumers
20
Async Executor
50
Scheduler
10
Total
280 Threads
Suppose every thread consumes approximately 1 MB.
280 Threads
≈
280 MB
That’s before processing a single customer request.
Many Spring Boot applications create far more threads than necessary.
This hidden memory consumption surprises teams during production deployments.
Direct Memory
Modern networking libraries avoid copying data repeatedly.
Instead they use native buffers.
Examples include:
- Netty
- NIO
- Kafka Client
- HTTP Clients
- gRPC
These buffers live outside the Java Heap.
Garbage Collection cannot manage them directly.
Large file uploads, streaming APIs and messaging platforms often consume significant Direct Memory.
Native Memory
The JVM itself also allocates native memory.
Examples include:
- JIT Compiler
- JVM internal structures
- Compression
- Native libraries
- SSL
- DNS
Again, Kubernetes counts all of this toward the container limit.
Total Container Memory
Now let’s revisit our Payment Service.
Java Heap
900 MB
Metaspace
180 MB
Thread Stacks
220 MB
Direct Memory
160 MB
Native Memory
90 MB
----------------------
Total
1.55 GB
The developer only configured:
-Xmx900m
Yet the container actually consumes:
1.55 GB
If Kubernetes memory limit is:
1.2 GB
The result becomes inevitable.
OOMKilled
Understanding OOMKilled
Many developers assume Kubernetes throws an exception.
It doesn’t.
The Linux Kernel terminates the container.
Imagine memory inside a node becoming exhausted.
The Linux Out Of Memory Killer chooses one or more processes to terminate.
Kubernetes simply reports what happened.
Container
↓
Killed by Linux
↓
Exit Code 137
↓
OOMKilled
Notice the JVM never gets a chance to throw an exception.
The operating system terminates it immediately.
Java OutOfMemoryError vs Kubernetes OOMKilled
These two failures are often confused.
| Java OutOfMemoryError | Kubernetes OOMKilled |
|---|---|
| JVM throws exception | Linux kills process |
| Application may continue | Container terminates immediately |
| Heap exhausted | Container memory exhausted |
| Java problem | Infrastructure limit exceeded |
Understanding the difference dramatically improves troubleshooting.
Requests and Limits
Now let’s understand another concept that confuses many developers.
Consider this Pod.
resources:
requests:
memory: 2Gi
cpu: "1"
limits:
memory: 4Gi
cpu: "2"
These values serve completely different purposes.
Requests
A request tells Kubernetes:
“Please reserve at least this much capacity.”
Think of booking a hotel room.
You’re reserving a room.
The hotel guarantees availability.
Similarly,
a memory request guarantees that the Scheduler places the Pod only on a node with sufficient available memory.
Limits
A limit tells Kubernetes:
“Do not allow this container to exceed this value.”
Memory limits are strict.
If the container exceeds them,
Linux terminates it.
CPU limits behave differently.
CPU is throttled rather than terminated.
We’ll discuss that shortly.
Why CPU Limits Behave Differently
Memory is finite.
CPU is shareable.
Imagine two people sharing a meeting room.
Memory behaves like chairs.
Once all chairs are occupied,
no additional people can sit.
CPU behaves like speaking time.
Everyone still gets an opportunity to speak.
Some people simply wait longer.
That’s why CPU exceeding its limit results in throttling rather than process termination.
CPU Throttling
Suppose the Payment Service is allocated:
CPU Limit
1 Core
Suddenly payroll traffic doubles.
The application needs:
2.5 Cores
Kubernetes doesn’t kill the Pod.
Instead,
it slows execution.
Required
2.5 CPU
Available
1 CPU
↓
Requests Wait
↓
Response Time Increases
Operations observe:
- Higher latency
- Longer response times
- Increased queue depth
The application appears slow even though CPU usage looks reasonable.
Why CPU Usage Can Be Misleading
Imagine Tomcat has:
200 Worker Threads
CPU limit:
500m
Half a CPU.
All 200 threads compete for the same limited processing time.
The result is increased context switching.
Longer request latency.
Higher response times.
Not because Java is slow,
but because Kubernetes is intentionally throttling CPU consumption.
The JVM Became Container Aware
Before Java 10,
the JVM ignored container limits.
Imagine a node with:
64 GB RAM
Container limit:
2 GB
Older JVMs saw:
64 GB
It happily allocated enormous heap sizes.
Linux immediately terminated the process.
Fortunately,
Java 17 understands containers.
It automatically detects:
- Available CPUs
- Available Memory
- Container Limits
This is one of the many reasons Java 17 is an excellent choice for Kubernetes workloads.
Choosing the Right Heap Size
A common mistake is configuring:
Container Memory
2 GB
Heap
2 GB
This leaves zero room for:
- Metaspace
- Threads
- Native Memory
- Direct Buffers
A better approach reserves space for non-heap memory.
Example:
Container
2 GB
Heap
1.2 GB
Remaining
800 MB
Metaspace
Threads
Native
Direct Buffers
This dramatically reduces unexpected OOMKilled events.
Spring Boot Thread Pools Matter
Let’s revisit our Payment Service.
Developers configure:
Tomcat
300 Threads
Async Executor
200 Threads
Kafka
40 Consumers
Scheduler
20 Threads
Total:
560 Threads
More threads do not automatically improve throughput.
Excessive threads increase:
- Memory usage
- Context switching
- CPU contention
Modern Java applications should size thread pools based on workload rather than intuition.
(We’ll revisit this topic later when discussing Virtual Threads in Java 21.)
Monitoring the Right Metrics
Many teams monitor only Heap Usage.
That’s insufficient.
Production dashboards should include:
- Heap Memory
- Non-Heap Memory
- Metaspace
- Direct Memory
- Thread Count
- Garbage Collection
- CPU Throttling
- Container Memory
- Container Restarts
- OOMKilled Events
Only then do you see the complete picture.
Production Checklist
Before deploying any Spring Boot application, verify the following.
- Container memory exceeds heap size.
- Thread pools are appropriately sized.
- Heap sizing considers non-heap memory.
- Requests reflect expected usage.
- Limits prevent noisy neighbors.
- Java 17 container awareness is enabled.
- CPU throttling is monitored.
- OOMKilled alerts are configured.
- Heap dumps are captured for investigation.
- Memory usage is validated under production load.
Memory problems rarely appear during development.
They usually appear during peak business events—the very moments when your platform needs to be most reliable.
Looking Ahead
We’ve now explored one of the most misunderstood areas of running Java applications in Kubernetes.
We understand:
- JVM memory architecture.
- Container memory.
- CPU throttling.
- Requests and limits.
- Why Pods become OOMKilled.
Yet one challenge still remains.
Our Payment Service has been allocated the correct resources.
It starts correctly.
It shuts down gracefully.
It exposes meaningful health checks.
But customer traffic doesn’t remain constant.
Every month during payroll processing, demand triples.
Late at night, transaction volume drops to almost zero.
Should Operations manually increase replicas every morning and reduce them every evening?
Or can Kubernetes automatically scale our Spring Boot application based on real-time demand?