AI for Java Architects – Part 5

Posted on by Rahul Mittal

Prompt Engineering for Enterprise AI: The New API Design

“In traditional software, APIs define behavior. In AI systems, prompts define behavior.”

Introduction

In our previous articles, we learned:

  • Part 1: LLM Fundamentals
  • Part 2: Embeddings and Vector Databases
  • Part 3: RAG Architecture
  • Part 4: Building RAG Applications with Spring AI

At this point, many developers assume:

“The AI model is everything.”

In reality:

The prompt is often more important than the model.

A poorly written prompt can make GPT-5 perform badly.

A well-designed prompt can make a smaller model perform surprisingly well.

As Java developers, we can think of prompts as:

  • API contracts
  • Business rules
  • Configuration
  • Runtime instructions

Prompt engineering is rapidly becoming one of the most important skills for AI engineers.

What is a Prompt?

A prompt is simply:

Instructions provided to the model.

Example:

Explain Spring Boot.

This is a prompt.

But enterprise prompts are much richer.

Example:

You are a senior Java architect.

Explain Spring Boot for a team migrating
from monolithic applications to microservices.

Provide examples and best practices.

The response changes dramatically.

Why Prompts Matter

The model itself is fixed.

Prompts control:

  • Tone
  • Depth
  • Format
  • Audience
  • Accuracy
  • Constraints
  • Behavior

Java Analogy

Consider:

calculator.add(10, 20);

The same method behaves differently based on input.

Similarly:

Explain Kafka.

vs.

Explain Kafka to a Java architect designing event-driven systems.

The same model produces very different answers.

Anatomy of a Prompt

A good prompt often contains:

Role
Task
Context
Constraints
Output Format
Examples

Example

You are a senior software architect.

Explain Redis caching for Spring Boot.

Limit the answer to 300 words.

Provide code examples.

Output as markdown.

System Prompt

The system prompt defines behavior.

Example:

You are an enterprise architecture assistant.
Provide concise technical answers.
Never invent information.

Think of this as:

@Configuration
public class AIBehaviorConfig {
}

It defines the application personality.

User Prompt

The user prompt changes per request.

Example:

Explain Redis eviction policies.

Full Prompt Structure

System Prompt

Conversation History

Retrieved Documents

User Question

This entire package becomes the context.

Role Prompting

Assign a role.

Examples:

You are a Java architect.

You are a database expert.

You are a DevOps engineer.

You are a security consultant.

This often improves responses significantly.

Example

Prompt:

Explain Kubernetes.

Response:

Generic.

Prompt:

You are a cloud architect.

Explain Kubernetes to a Spring Boot team.

Response:

Much more relevant.

Context Prompting

Provide additional information.

Example:

Our application consists of:

- Spring Boot
- Kafka
- Redis
- PostgreSQL

Recommend a caching strategy.

The AI now answers within your environment.

Constraints

Constraints improve reliability.

Example:

Answer in less than 200 words.

Provide only valid JSON.

Do not speculate.

Use bullet points.

Example

Bad:

Explain AWS.

Better:

Explain AWS ECS deployment for a Java microservice.

Limit to 5 bullet points.

Output Formatting

Enterprise systems often require:

  • JSON
  • XML
  • Markdown
  • Tables

Example:

Return valid JSON.

{
  "summary": "",
  "risks": [],
  "recommendations": []
}

Structured Output

Instead of:

The project has several risks...

Return:

{
  "risk": "Database bottleneck",
  "severity": "HIGH"
}

This enables:

  • APIs
  • Dashboards
  • Automation

Spring AI Structured Output

record RiskAnalysis(
    String risk,
    String severity) {
}

The AI can populate Java objects directly.

Few-Shot Prompting

Provide examples.

Example:

Input:
Redis

Output:
An in-memory cache.

Input:
Kafka

Output:
An event streaming platform.

Input:
RabbitMQ

Output:
?

The model learns the pattern.

Zero-Shot Prompting

No examples.

Explain Kafka.

One-Shot Prompting

One example.

Few-Shot Prompting

Several examples.

This usually improves consistency.

Chain of Thought Prompting

Ask the model to reason step by step.

Example:

Analyze the problem step by step.

Or:

Think through the solution carefully.

Example

Question:

A service processes 100 requests per second. Each request takes 200ms. How many threads are required?

Without reasoning:

Incorrect answer.

With reasoning:

100 requests/sec

Each request takes 0.2 sec

Concurrency = 100 × 0.2

20 threads

Deliberate Reasoning

Example:

Evaluate multiple solutions.

Compare advantages and disadvantages.

Recommend the best approach.

Excellent for architecture discussions.

Prompt Templates

Instead of:

String prompt =
        "Explain " + technology;

Use templates.

You are a senior architect.

Explain {technology}.

Provide:
- Advantages
- Disadvantages
- Use cases

Spring AI PromptTemplate

PromptTemplate template =
        new PromptTemplate("""
Explain {technology}
for enterprise systems.
""");

Guardrails

Guardrails prevent unwanted responses.

Examples:

Only answer from the provided documents.

If uncertain, say "I don't know."

Do not provide legal advice.

Hallucination Prevention

Prompt:

If the answer is unavailable in the context,
respond with:

"I cannot find this information."

Very important in RAG systems.

Prompt Chaining

Instead of one large prompt:

Step 1:

Summarize the document.

Step 2:

Extract risks.

Step 3:

Recommend actions.

This often improves quality.

Enterprise Example

Architecture Review Assistant.

Prompt 1:

Identify components.

Prompt 2:

Identify risks.

Prompt 3:

Recommend improvements.

Temperature

Controls creativity.

TemperatureBehavior
0Deterministic
0.2Stable
0.5Balanced
1.0Creative

Recommended Settings

Use CaseTemperature
JSON0
RAG0.2
Summaries0.3
Code0.1
Creative Writing0.9

Prompt Versioning

Just like APIs:

Prompt v1
Prompt v2
Prompt v3

Many companies now store prompts in:

  • Git
  • Databases
  • Configuration services

Prompt Testing

Questions:

  • Is the output consistent?
  • Is the JSON valid?
  • Does it hallucinate?
  • Does it follow instructions?

Prompts require testing.

Prompt Anti-Patterns

Too Short

Explain Redis.

Too Vague

Help me.

Too Many Instructions

Explain Redis, Kafka, Kubernetes,
AWS, Java, and databases.

Conflicting Instructions

Be concise.

Explain in detail.

Real Enterprise Prompt

You are a senior Java architect.

Answer only from the supplied documents.

If the answer is unavailable, say:
"I do not know."

Provide:

1. Summary
2. Risks
3. Recommendations

Limit the answer to 300 words.

Java Analogy

JavaAI
API ContractPrompt
DTOStructured Output
ValidationGuardrails
Business RulesInstructions
ConfigurationSystem Prompt

Interview Questions

What is prompt engineering?

Designing effective instructions for AI models.

Why are prompts important?

They influence output quality.

What is few-shot prompting?

Providing examples.

What is chain of thought?

Encouraging reasoning.

What are guardrails?

Instructions that constrain behavior.

Hands-On Exercises

Exercise 1

Ask:

Explain Kafka.

Then:

Explain Kafka to a Java architect.

Compare.

Exercise 2

Ask:

Provide the answer in JSON.

Exercise 3

Add:

Think step by step.

Observe reasoning.

Enterprise Project

Build:

Architecture Review Assistant

Input:

  • HLD
  • LLD

Output:

{
  "components": [],
  "risks": [],
  "recommendations": []
}

Key Takeaways

✔ Prompts are the new APIs.

✔ Role prompting improves quality.

✔ Constraints improve consistency.

✔ Few-shot prompting teaches patterns.

✔ Chain of Thought improves reasoning.

✔ Guardrails reduce hallucinations.

✔ Structured outputs enable automation.

✔ Prompt engineering is an essential AI skill.

Coming Next

Part 6 — Spring AI Deep Dive: ChatClient, Advisors, Memory and Tool Calling

We will learn:

  • ChatClient
  • PromptTemplate
  • Advisors
  • Memory
  • Function calling
  • Structured output
  • Tool execution
  • Production architecture

For Java developers, this is where AI development starts feeling like Spring Boot.

“The model provides intelligence. The prompt provides direction.”

Leave a Reply

Your email address will not be published. Required fields are marked *