Part 6 – Graceful Shutdown & Zero-Downtime Deployments

Why Every Spring Boot Developer Must Understand SIGTERM Before Running on Kubernetes

“Deploying a new version is easy. Deploying it without your customers noticing—that’s engineering.”


Friday Evening, 6:00 PM

It’s the last deployment window before the weekend.

The Digital Payments team at Global Trust Bank is releasing version 2.5.0 of the Payment Service.

The release notes look straightforward.

  • A new payment validation rule
  • Improved fraud checks
  • Performance optimizations
  • Minor bug fixes

Nothing unusual.

The deployment pipeline starts.

kubectl apply -f payment-service.yaml

Operations watch the deployment dashboard.

Pods begin appearing.

payment-service-v2.5.0

Everything looks healthy.

Then, within seconds, customer support receives calls.

  • “My payment failed.”
  • “The upload stopped halfway.”
  • “I received a timeout.”
  • “The payment is showing Pending forever.”

The deployment completed successfully.

No Pods crashed.

No nodes failed.

No infrastructure alarms.

So what happened?


The Invisible Problem

The answer wasn’t Kubernetes.

It was the application.

While Kubernetes was replacing the old Pods with new ones, some requests were still executing.

Those requests were terminated halfway through processing.

Imagine this timeline.

Customer starts payment

        │
        ▼

Payment Service

Validating customer

        │
        ▼

Checking balance

        │
        ▼

Kubernetes starts deployment

        │
        ▼

Old Pod terminated

        │
        ▼

Customer receives timeout

Nothing was wrong with Kubernetes.

The application simply wasn’t prepared to shut down gracefully.

This article is about solving exactly that problem.


Why Pods Are Terminated

Many developers think Pods are terminated only when something goes wrong.

In reality, Pod termination is a completely normal part of Kubernetes.

Pods are terminated during:

  • Rolling deployments
  • Scaling down
  • Node upgrades
  • Cluster maintenance
  • Resource optimization
  • Cluster autoscaling
  • Manual deployments
  • Failed health checks

In a busy production cluster, Pods may be created and destroyed hundreds of times every day.

A cloud-native application must therefore assume:

“One day, Kubernetes will ask me to stop. I need to be ready.”


What Actually Happens During a Deployment?

Let’s revisit Sarah’s deployment.

Current version:

Payment Service v2.4.1

Pod A

Pod B

Pod C

She deploys version 2.5.0.

Many developers imagine Kubernetes doing this.

Stop Old Pods

↓

Start New Pods

That would obviously cause downtime.

Instead, Kubernetes performs something much smarter.

Create New Pod

↓

Wait Until Ready

↓

Start Sending Traffic

↓

Stop Old Pod

↓

Repeat

Notice something important.

The old Pod is not immediately killed.

Kubernetes first gives the application an opportunity to shut down correctly.

How?

Using Linux signals.


Meet SIGTERM

Every operating system needs a way to tell a running process that it should stop.

Linux uses signals for this purpose.

Some common signals include:

SignalMeaning
SIGINTInterrupt (Ctrl+C)
SIGKILLStop immediately
SIGTERMPlease terminate gracefully

The important word is:

Please

SIGTERM is a polite request.

Kubernetes isn’t saying:

“I’m killing you.”

It’s saying:

“I no longer need this Pod. Please finish your work and shut down cleanly.”

That distinction is extremely important.


The Pod Termination Lifecycle

Let’s follow the exact sequence.

Deployment Starts

        │

        ▼

New Pod Created

        │

        ▼

New Pod Becomes Ready

        │

        ▼

Old Pod Removed From Service

        │

        ▼

SIGTERM Sent

        │

        ▼

Grace Period Begins

        │

        ▼

Application Stops

        │

        ▼

Container Exits

        │

        ▼

Pod Deleted

Notice something subtle.

Traffic stops before Kubernetes asks the application to terminate.

This is deliberate.

The goal is to prevent new requests from reaching a Pod that is about to disappear.


Step 1 – Remove the Pod from the Service

The first action Kubernetes performs is surprisingly not sending SIGTERM.

Instead, it removes the Pod from the Service endpoints.

Before deployment:

Service

↓

Pod A

Pod B

Pod C

After Pod B enters termination:

Service

↓

Pod A

Pod C

New customer requests immediately stop flowing to Pod B.

This process is often called connection draining.

Existing requests may still be running.

New requests are directed elsewhere.


Step 2 – Kubernetes Sends SIGTERM

Now Kubernetes sends the SIGTERM signal to the main process inside the container.

For most Spring Boot applications, that process is the JVM.

kubelet

↓

SIGTERM

↓

Java Process

The JVM receives the signal.

Spring Boot now begins its graceful shutdown sequence.

This is where many developers mistakenly believe their work is done.

Unfortunately, that’s not always true.


What Happens Inside Spring Boot?

Modern Spring Boot versions understand graceful shutdown.

If enabled, the application doesn’t immediately terminate.

Instead, it begins shutting down in phases.

A simplified sequence looks like this.

Receive SIGTERM

↓

Stop Accepting New HTTP Requests

↓

Continue Processing Existing Requests

↓

Close Web Server

↓

Shutdown Beans

↓

Close Database Connections

↓

Exit JVM

This behavior gives in-flight requests a chance to complete successfully.

Without it, the JVM simply exits, abruptly terminating every active request.


Enabling Graceful Shutdown

Fortunately, Spring Boot makes this extremely easy.

server.shutdown=graceful

spring.lifecycle.timeout-per-shutdown-phase=30s

These two lines dramatically improve deployment behavior.

Let’s understand what they actually do.


server.shutdown=graceful

This tells the embedded web server (Tomcat, Jetty, or Undertow):

“When shutdown begins, stop accepting new requests.”

Imagine a restaurant closing for the evening.

The manager locks the entrance.

No new customers enter.

But customers already eating are allowed to finish their meals.

Graceful shutdown follows exactly the same principle.

Current requests continue.

New requests are rejected because Kubernetes has already routed them elsewhere.


Shutdown Timeout

Consider a payment request taking 12 seconds.

If Kubernetes immediately terminates the JVM after sending SIGTERM, the request never finishes.

The timeout property gives Spring Boot time to complete ongoing work.

spring.lifecycle.timeout-per-shutdown-phase=30s

This means:

SIGTERM

↓

Wait

↓

Maximum 30 Seconds

↓

Shutdown Completes

If every request finishes within 30 seconds, customers never notice the deployment.


What If the Application Never Stops?

Suppose a thread becomes stuck.

Or a database call hangs forever.

Should Kubernetes wait indefinitely?

Absolutely not.

Every Pod has a termination grace period.

By default:

terminationGracePeriodSeconds: 30

Timeline:

SIGTERM

↓

30 Seconds

↓

Still Running?

↓

Yes

↓

SIGKILL

Unlike SIGTERM, SIGKILL cannot be ignored.

The operating system immediately terminates the process.

This prevents faulty applications from blocking deployments forever.


The Difference Between SIGTERM and SIGKILL

Think of a retail store closing.

SIGTERM is the manager announcing:

“The store closes in 30 minutes.”

Customers finish shopping.

Employees clean up.

Cash registers close.

Everyone leaves calmly.

SIGKILL is very different.

It’s equivalent to cutting the electricity.

Lights off.

Registers stop.

Customers remain inside.

Nothing is saved.

That’s why applications should always complete their work during the SIGTERM window.

SIGKILL should be treated as an emergency fallback—not the normal shutdown mechanism.


Long-Running Requests

Let’s revisit our banking platform.

Suppose a customer uploads a 750 MB financial document.

Upload Begins

↓

65% Complete

↓

Deployment Starts

↓

SIGTERM Received

What should happen?

Without graceful shutdown:

Upload Aborted

↓

Customer Retries

With graceful shutdown:

Finish Upload

↓

Store Document

↓

Respond Success

↓

Shutdown

This single improvement significantly enhances customer experience during deployments.


Background Tasks Matter Too

HTTP requests are only part of the story.

Enterprise applications also execute:

  • @Async methods
  • CompletableFuture
  • Kafka consumers
  • Solace consumers
  • Scheduled jobs
  • Batch processing

Suppose our Payment Service is consuming payment events from Solace.

Receive Payment Event

↓

Validate

↓

Update Oracle

↓

Publish Confirmation

Halfway through processing, Kubernetes sends SIGTERM.

If the consumer stops immediately:

  • Oracle may be updated.
  • Confirmation may never be published.

The system becomes inconsistent.

Background workers should therefore stop accepting new work while allowing current processing to complete before shutdown.

This is one of the most overlooked aspects of graceful shutdown.


Database Transactions

Consider this code.

@Transactional
public void processPayment() {

    debitAccount();

    creditMerchant();

    updateLedger();

}

Suppose shutdown occurs after:

Debit Account

✓

Credit Merchant

✓

Update Ledger

❌

What happens?

Fortunately, properly designed transactions roll back automatically.

But only if the application is allowed to complete or fail gracefully.

Abrupt termination can leave external systems in inconsistent states, particularly when multiple systems participate in the workflow.

For distributed transactions, patterns such as Saga and Outbox become even more important—topics we’ll cover later in this series.


Graceful Shutdown Is a Partnership

One of the biggest misconceptions is:

“Kubernetes provides zero-downtime deployments.”

Not exactly.

Zero-downtime deployments are a partnership.

Kubernetes does its part by:

  • Creating replacement Pods
  • Waiting for Readiness
  • Removing old Pods from Services
  • Sending SIGTERM
  • Respecting the grace period

Your application must do its part by:

  • Stopping new requests
  • Completing existing work
  • Closing resources cleanly
  • Exiting within the configured timeout

If either side fails, deployments become disruptive.


Production Best Practices

Over the years, several patterns have consistently proven valuable in production environments.

Always enable graceful shutdown.

There is almost never a reason not to.

Measure your longest-running request.

Don’t guess the shutdown timeout.

Measure it using production traffic.

Avoid very long synchronous operations.

Large file uploads, report generation, and bulk processing are often better handled asynchronously.

Design idempotent operations.

If a request must be retried after a deployment, repeating it should not create duplicate business actions.

Test deployments under load.

Many graceful shutdown issues only appear when hundreds or thousands of requests are active.


Production Readiness Checklist

Before deploying any Spring Boot service to Kubernetes, verify the following:

  • Graceful shutdown is enabled.
  • Readiness probes remove Pods before shutdown.
  • Shutdown timeout matches real workloads.
  • Long-running requests complete successfully.
  • Background consumers stop accepting new work.
  • Database transactions complete or roll back safely.
  • Connection pools close cleanly.
  • Logging continues until shutdown completes.
  • The application exits within the termination grace period.

If every item on this checklist is satisfied, rolling deployments become almost invisible to end users.

That’s exactly what modern cloud-native systems should achieve.


Looking Ahead

We’ve now followed the lifecycle of a Spring Boot application from deployment to graceful termination.

Our application starts correctly.

It scales correctly.

It shuts down correctly.

But another challenge remains.

As customer traffic grows throughout the day, the Payment Service needs more capacity.

Late at night, demand falls dramatically.

Should Operations manually increase and decrease replicas throughout the day?

Or can Kubernetes make those decisions automatically?

In the next article, we’ll explore Horizontal Pod Autoscaling (HPA) and learn how Kubernetes uses real-time metrics to automatically scale Spring Boot microservices while maintaining performance, resilience, and infrastructure efficiency.

Editorial suggestion for the series

This article is a pivotal point in the series. Before moving to autoscaling, I’d be inserting one more application-focused article:

Part 7 – Startup, Readiness, Liveness & Application Lifecycle

It would connect naturally with this post and cover:

  • The complete Spring Boot lifecycle inside Kubernetes
  • Why RunningReady
  • Startup vs Readiness vs Liveness probes
  • Spring Boot Actuator health groups
  • Database, Redis, Solace, Kafka health indicators
  • Probe design best practices
  • CrashLoopBackOff, ImagePullBackOff, and OOMKilled
  • Common enterprise probe mistakes

That sequence—Design → Graceful Shutdown → Health Probes—forms a complete guide for building production-ready Spring Boot applications on Kubernetes before diving deeper into Kubernetes-specific features like HPA, ConfigMaps, networking, and OpenShift.

Leave a Reply

Your email address will not be published. Required fields are marked *