Production Troubleshooting | Spring Boot | Microservices | Kubernetes | SQL | Java | System Design

Duration: 20 Minutes

Experience: 6–12 Years

Unlike the previous sets, this assessment focuses on real production issues that senior Java backend engineers encounter. The goal is to evaluate debugging ability, architectural thinking, performance optimization, and practical decision-making rather than memorized APIs.

---

Section A – Production Scenario MCQs (5 Questions)

---

Question 1 – High API Response Time

A Spring Boot REST API that normally responds in 200 ms suddenly starts taking 8–10 seconds after deployment.

What should be your first troubleshooting step?

A. Increase JVM heap size.

B. Restart all Kubernetes pods.

C. Analyze application logs, metrics, and thread dumps to identify the bottleneck.

D. Increase the number of replicas immediately.

---

Question 2 – Duplicate Message Processing

Your microservice consumes events from Kafka/SNS/SQS. Due to retries, the same message is occasionally processed more than once.

What is the best solution?

A. Disable retries.

B. Ignore duplicate processing.

C. Make the consumer idempotent using a unique message identifier.

D. Restart the consumer whenever duplicates occur.

---

Question 3 – Database Connection Pool Exhausted

Users begin receiving:

Cannot get JDBC Connection

What is the most likely production cause?

A. Spring Boot version mismatch.

B. Database connection pool exhaustion due to long-running or leaked connections.

C. JVM garbage collection.

D. Missing @Service annotation.

---

Question 4 – Kubernetes Rolling Deployment

During deployment, users continue accessing the application without downtime.

Which Kubernetes strategy provides this behavior by default?

A. Recreate

B. RollingUpdate

C. Blue-Green

D. Canary

---

Question 5 – Distributed Cache

Multiple pods maintain local caches. One pod updates the database, but other pods continue serving stale data.

Which approach best solves this?

A. Restart all pods after every update.

B. Reduce JVM heap.

C. Publish cache invalidation events through a messaging system (Kafka, SNS/SQS, Solace, etc.) or use a centralized distributed cache.

D. Increase CPU limits.

---

Section B – SQL Challenge

Question 6

Table

EMPLOYEE

EMP_ID	NAME	SALARY
1	John	6000
2	Steve	8000
3	Mary	7000
4	David	9000
5	Lisa	8000

Write an SQL query to retrieve the second highest distinct salary.

---

Section C – Programming Questions

---

Question 7 – LRU Cache Design

Design an LRU (Least Recently Used) Cache supporting the following operations:

• get(key)
• put(key, value)

Requirements:

• Average time complexity: O(1) for both operations.
• Explain the data structures used.
• (Bonus) Mention whether Java provides a built-in implementation.

---

Question 8 – Parallel Processing

Given:

List<Employee> employees;

Each employee requires an external REST API call that takes approximately 500 ms.

There are 100 employees.

How would you improve performance?

Requirements:

• Use Java 8 features.
• Prevent excessive thread creation.
• Handle failures gracefully.
• Preserve scalability.

---

Question 9 – System Design Scenario

Design a File Upload Service with the following requirements:

• Upload files up to 100 MB.
• Store metadata in a relational database.
• Store file contents in cloud object storage (Amazon S3 or equivalent).
• Publish an event after successful upload for downstream processing.
• Support retries for transient failures.
• Ensure duplicate uploads are not processed twice.
• Scale horizontally across multiple pods.

Describe:

1. High-level architecture.
2. Components involved.
3. Database schema.
4. API flow.
5. Error handling strategy.
6. Idempotency approach.

---

Answer Key & Detailed Explanations

---

Question 1

Correct Answer: C

Explanation

Before making infrastructure changes, determine where the delay occurs.

Typical investigation order:

1. Application logs.
2. APM tools (Dynatrace, AppDynamics, New Relic).
3. Thread dumps.
4. Database query analysis.
5. JVM metrics.
6. External dependency latency.
7. Kubernetes metrics (CPU, Memory, Network).

Increasing memory or replicas without identifying the root cause often masks the problem rather than solving it.

---

Question 2

Correct Answer: C

Explanation

Distributed messaging systems generally provide at-least-once delivery.

Therefore, duplicate messages are expected.

Recommended approaches:

• Store a unique message ID.
• Use an idempotency table or cache.
• Ignore already processed messages.
• Design downstream operations to be idempotent.

---

Question 3

Correct Answer: B

Explanation

Common causes include:

• Long-running SQL queries.
• Connection leaks.
• Insufficient pool size.
• Database slowdown.
• Transactions not closing promptly.

Useful tools:

• HikariCP metrics.
• Database session monitoring.
• Thread dumps.
• SQL execution plans.

---

Question 4

Correct Answer: B

Explanation

Kubernetes Deployments use RollingUpdate by default.

Characteristics:

• Gradually replaces old pods.
• Maintains service availability.
• Supports rollback if deployment fails.

Interview discussion:

• Rolling Update vs Blue-Green.
• Rolling Update vs Canary.

---

Question 5

Correct Answer: C

Explanation

Each pod has its own in-memory cache.

To maintain consistency:

• Publish cache invalidation events through Kafka, SNS/SQS, Solace, etc.
• Alternatively, use Redis, Hazelcast, or another distributed cache.

This ensures all application instances observe updates without restarting.

---

Question 6 – SQL Solution

Oracle

SELECT MAX(SALARY)
FROM EMPLOYEE
WHERE SALARY <
(
    SELECT MAX(SALARY)
    FROM EMPLOYEE
);

ANSI SQL

SELECT DISTINCT SALARY
FROM EMPLOYEE
ORDER BY SALARY DESC
OFFSET 1 ROW
FETCH NEXT 1 ROW ONLY;

Concepts Tested

• Aggregate functions.
• DISTINCT.
• Ordering.
• Subqueries.
• Pagination.

---

Question 7 – Sample Solution

Expected answer:

Use:

• HashMap
• Doubly Linked List

Operations:

• get() → Move node to front.
• put() → Insert/update node.
• Remove least recently used node from the tail when capacity is exceeded.

Complexity

Operation	Time
get	O(1)
put	O(1)

Bonus

Java provides:

LinkedHashMap

with accessOrder=true, which can be extended to implement an LRU cache efficiently.

---

Question 8 – Sample Solution

Preferred approach:

ExecutorService executor =
Executors.newFixedThreadPool(10);

CompletableFuture

Key points:

• Use a bounded thread pool.
• Submit REST calls using CompletableFuture.
• Combine results using CompletableFuture.allOf().
• Configure timeouts.
• Apply retries with exponential backoff for transient failures.
• Use Circuit Breakers (e.g., Resilience4j) for downstream resilience.

Expected discussion:

• Avoid creating 100 raw threads.
• Tune pool size based on CPU and I/O characteristics.
• Handle partial failures and cancellation.

---

Question 9 – Expected Design

A strong answer should include:

Architecture

Client
   │
API Gateway
   │
Spring Boot Upload Service
   │
 ├── Store Metadata → Database
 ├── Upload File → Object Storage (S3)
 └── Publish Event → Kafka / SNS / Solace
                      │
                Downstream Consumers

Database

Example table:

Column	Description
FILE_ID	Unique identifier
FILE_NAME	Original filename
CONTENT_TYPE	MIME type
FILE_SIZE	File size
STORAGE_PATH	Object storage location
CHECKSUM	SHA-256 hash
STATUS	Uploaded / Failed / Processing
CREATED_AT	Timestamp

Idempotency

• Client-generated request ID or checksum.
• Unique database constraint.
• Ignore duplicate upload requests.

Error Handling

• Retry transient storage failures.
• Dead-letter queue for failed events.
• Compensating cleanup if metadata is stored but upload fails.

Scalability

• Stateless application pods.
• Shared object storage.
• Event-driven downstream processing.
• Horizontal scaling behind a load balancer.

---

Evaluation Guide

Score	Recommendation
9/9	Outstanding – Senior Java Backend Engineer
7–8	Strong – Ready for System Design Round
5–6	Good Technical Foundation – Needs More Production Experience
Below 5	Requires Additional Practical Exposure

---

Interviewer’s Notes

This final assessment measures how candidates think under real production conditions rather than how many APIs they remember.

A strong candidate should be able to:

• Follow a structured troubleshooting methodology instead of guessing.
• Explain idempotency in event-driven systems.
• Diagnose database connection pool issues.
• Compare Kubernetes deployment strategies (Rolling Update, Blue-Green, Canary).
• Design distributed cache invalidation mechanisms.
• Write efficient SQL for ranking queries.
• Choose appropriate data structures for O(1) cache operations.
• Use CompletableFuture, bounded thread pools, retries, and circuit breakers for concurrent I/O.
• Design scalable, fault-tolerant microservices with messaging, object storage, retries, idempotency, and horizontal scalability.

Overall Series Summary

Across these 10 screening sets, candidates are evaluated on:

• Java Core: OOP, Collections, Generics, Exceptions, Concurrency.
• Java 8+: Streams, Lambdas, Functional Interfaces, CompletableFuture.
• Spring Boot: Dependency Injection, Bean Lifecycle, Transactions, Profiles, REST.
• Persistence: Spring Data JPA, Hibernate, Fetch Strategies, Locking, Performance.
• Microservices: REST, Event-Driven Architecture, Saga, Circuit Breakers, Retries, Idempotency.
• Cloud Native: Docker, Kubernetes, Health Checks, ConfigMaps, Secrets, Observability.
• Production Engineering: Performance tuning, troubleshooting, caching, messaging, SQL optimization, and system design.

A candidate who consistently scores 8/9 or higher across all ten sets is typically well-prepared for senior Java backend and microservices interview rounds at organizations building modern cloud-native systems.