Deep Dive: Spring Boot with Apache Tomcat — Architecture and Request Handling
Table of Contents
- Introduction
- Architecture Overview
- Component Deep Dive
- Request Processing Pipeline
- Common Scenarios and Solutions
- Monitoring and Troubleshooting
Introduction
Spring Boot comes with an embedded Apache Tomcat server, which serves as the default servlet container. This article explores how Spring Boot and Tomcat work together to handle web requests, manage resources, and scale applications.
Architecture Overview
Key Components
Client Request
↓
Tomcat Connector (TCP/Connection Management)
↓
Thread Pool (Request Processing)
↓
Spring Boot Application
Core Numbers
- Default max connections: 10,000 (Maximum number of simultaneous connections the server can handle; exceeding this drops new connections.)
- Default max threads: 200 (Maximum number of worker threads available to process requests concurrently; additional requests are queued.)
- Default min spare threads: 10 (Minimum number of idle threads kept ready to handle incoming requests for smoother performance.)
- Default accept queue: 100 (Number of incoming connections that can be queued if all threads are busy; beyond this, connections may be rejected.)
Component Deep Dive
1. Tomcat Connector
The connector is like a reception desk in a busy office:
Responsibilities:
- TCP connection management
- Keep-alive connection handling
- Network I/O buffering
- Protocol handling (HTTP/1.1, HTTP/2)
Connection States
10,000 max connections: The total number of “guests” the desk can serve at once:
- Keep-Alive: Guests relaxing in the lounge, waiting to make a new request (idle but connected).
- Queued: Guests waiting at the desk because all staff are busy (waiting for a thread).
2. Thread Pool
Think of threads as meeting rooms where actual work happens:
Characteristics:
- Fixed number of worker threads (default 200)
- Each thread ≈ 1MB memory
- One request per thread model
- Configurable queue for pending requests
Thread States
Out of the 200 threads (meeting rooms), each can be in one of the following states:
- Active: A meeting is in progress (processing a request).
- Idle: The room is clean and waiting for the next meeting (ready for work).
- Starting/Stopping: The room is being set up or cleaned up (threads being initialized or terminated).
Request Processing Pipeline
Detailed Flow
Client HTTP Request
↓
TCP Connection Request
↓
Tomcat Connector (Max 10,000 connections)
↓
Connection State Check
╭────┴────╮
↓ ↓
New Connection Keep-alive Connection
↓ ↓
╰────┬────╯
↓
Thread Pool Check (Max 200 threads)
╭────┴────╮
↓ ↓
Thread Available No Thread Available
↓ ↓
Assign Thread → Accept Queue
↓ |
← ← ← ← ← ← ← ←╯ (When thread becomes available)
↓
Request Processing Pipeline
↓
Spring Security Filters
↓
DispatcherServlet
↓
Handler Mapping
↓
Controller Layer
↓
Service Layer
↓
Response Generation
↓
Connection Handling
╭────┴────╮
↓ ↓
Close Keep-alive
Connection (Ready for next request)
Key Points:
- Connector manages 10,000 max TCP connections
- Thread Pool handles 200 max concurrent requests
- Keep-alive connections stay open but don’t use threads
- Accept Queue holds requests when all threads are busy
- Each request goes through the complete Spring processing pipeline
- After processing, connection can be reused or closed
Why Do We Need Both Thread Pools and Tomcat Connectors?
When designing scalable web applications, it’s important to understand the roles of the Tomcat Connector and the Thread Pool. These components work together to efficiently handle client requests, manage resources, and ensure high performance. Let’s break it down step by step with a real-world example to clarify why both are essential.
Example Scenario: What Happens When Clients Connect?
Imagine you have 1,000 clients connected to your server. Here’s how the resources are divided:
- 50 clients are actively being processed — they need threads.
- 900 clients are in a keep-alive state — they only need connection slots, not threads.
- 50 new requests are waiting for threads to be available for processing.
This distribution highlights why it’s important to separate connection management (handled by the Connector) from request processing (handled by the Thread Pool).
Real-World Example: A Browser’s Parallel Connections
Here’s a practical illustration:
- Your browser makes 6 parallel connections to a server.
- Each connection corresponds to a separate TCP connection managed by the Connector.
If the browser requests multiple images:
- Connection 1: Fetching
image1.jpg
(requires an active thread). - Connection 2: Fetching
image2.jpg
(requires an active thread). - Connections 3–6: In a keep-alive state, waiting for the next request (no thread required yet).
This shows how the Connector allows the server to maintain open connections for idle clients without wasting threads, while the Thread Pool handles requests from active connections.
What Happens Without the Connector?
Without a Connector, the server faces serious challenges:
- ❌ No connection pooling: Every new request requires a fresh connection, increasing overhead.
- ❌ No keep-alive support: Clients can’t reuse connections, leading to additional latency.
- ❌ High latency: Each request requires a new TCP handshake, slowing down response times.
What Happens Without the Thread Pool?
On the other hand, without a Thread Pool:
- ❌ No request processing: Incoming requests can’t be handled effectively.
- ❌ No concurrency: The server becomes incapable of managing multiple requests simultaneously.
- ❌ No resource management: Threads can’t be reused, which wastes CPU and memory resources.
The Perfect Duo: How They Work Together
When combined, the Connector and Thread Pool complement each other to create an efficient and scalable system:
Tomcat Connector: Connection Management
- ✅ Supports keep-alive connections for idle clients.
- ✅ Provides connection pooling to reuse connections.
- ✅ Manages network buffers for data transfer.
Thread Pool: Request Processing
- ✅ Handles concurrent requests efficiently.
- ✅ Manages resources like CPU and memory usage.
- ✅ Executes requests using reusable threads.
Common Scenarios and Solutions
1. High Concurrency
Symptom: Connection refused errors
Solution: Increase max-connections and accept-count
2. Slow Processing
Symptom: High response times
Solution:
- Increase thread pool size
- Implement async processing
- Optimize database queries
3. Memory Issues
Symptom: OutOfMemoryError
Solution:
- Reduce max threads
- Increase heap size
- Enable GC logging
Monitoring and Troubleshooting
- Active Connections
- Thread Pool Utilization
- Request Processing Time
- Error Rates
- Queue Size
Common Issues and Diagnosis
1. Connection Issues
Symptom: Connection timeouts
Check:
- max-connections
- connection-timeout
- network issues
2. Performance Issues
Symptom: High latency
Check:
- thread pool utilization
- GC logs
- CPU usage
- Database connections
Best Practices
1. Configuration
- Set appropriate timeouts
- Configure thread pool based on workload
- Enable access logging
- Use compression for responses
2. Development
- Use async processing for long operations
- Implement circuit breakers
- Handle exceptions properly
- Use connection pooling
Understanding how Spring Boot works with Tomcat is crucial for building and maintaining high-performance web applications. The key is finding the right balance between connections, threads, and resources based on your specific use case.
Remember:
- Connections ≠ Threads
- More threads ≠ Better performance
- Monitor and tune based on metrics
- Consider async processing for scaling
Thank you for being a part of the community
Before you go:
- Be sure to clap and follow the writer ️👏️️
- Follow us: X | LinkedIn | YouTube | Newsletter | Podcast
- Check out CoFeed, the smart way to stay up-to-date with the latest in tech 🧪
- Start your own free AI-powered blog on Differ 🚀
- Join our content creators community on Discord 🧑🏻💻
- For more content, visit plainenglish.io + stackademic.com