API Load Testing: Performance Testing with k6, JMeter, and Gatling

Types of Load Tests

Different load test types answer different questions about API performance. Running only one type gives an incomplete picture, so mature teams incorporate all four into their testing strategy.

Load testing simulates expected traffic levels to verify that the API meets its performance requirements under normal conditions. If your application expects 200 concurrent users during peak hours, a load test ramps up to 200 virtual users making realistic request patterns and measures whether response times stay under your thresholds (typically under 200ms for simple reads, under 500ms for writes, under 1 second for complex operations). Load tests run for a sustained period (10 to 30 minutes) to verify the API maintains performance over time, not just for a brief moment.

Stress testing pushes the API beyond its expected capacity to find the breaking point. Start at normal load and gradually increase concurrent users until the API starts degrading: response times exceed thresholds, error rates climb above zero, or the server becomes unresponsive. The stress test reveals the maximum capacity of the current infrastructure, providing the number you need for capacity planning. If your API breaks at 400 concurrent users and you expect to reach 350 within six months, you know infrastructure upgrades are needed soon.

Spike testing measures how the API handles sudden traffic surges. Instead of gradually ramping up, a spike test jumps from normal load to peak load instantly, simulating events like a product launch, a viral social media post, or a marketing campaign driving traffic. The test measures how quickly response times recover after the spike, whether the API serves errors during the surge, and whether any requests are lost entirely. APIs with auto-scaling infrastructure should scale up in response to the spike and scale back down afterward; the spike test verifies this scaling behavior works correctly.

Soak testing (endurance testing) runs sustained load for extended periods, typically 4 to 24 hours, to detect problems that only appear over time. Memory leaks that consume an additional 10MB per hour are invisible in a 10-minute test but crash the server after 8 hours. Connection pool exhaustion happens when connections are not properly returned after use, slowly depleting the pool until new requests cannot be served. Database query degradation occurs when table statistics become stale or indexes fragment under sustained write load. Soak tests find these time-dependent failures that shorter tests miss.

Key Performance Metrics

Load test results produce several metrics that together tell the full performance story. No single metric is sufficient on its own; you need to analyze them as a group to understand the API's behavior under load.

Response time is the most important metric for user experience. Measure it at multiple percentiles: the median (p50) tells you what a typical user experiences, the 95th percentile (p95) tells you what the slowest 5% of users experience, and the 99th percentile (p99) catches outlier requests that take much longer than normal. Averages are misleading because a small number of very slow requests can mask the experience of the majority. An API with a 100ms average might have a p99 of 5 seconds, meaning 1 in 100 users waits 50 times longer than typical. Set performance thresholds on p95 and p99, not averages.

Throughput measures how many requests per second (RPS) the API processes. High throughput with low response times indicates a healthy API. Throughput that plateaus while response times increase indicates a bottleneck: the API is saturated and cannot serve additional requests any faster. The maximum throughput before degradation begins is your API's effective capacity. For capacity planning, target 50% to 70% of maximum throughput for your expected peak load, leaving headroom for traffic spikes.

Error rate tracks the percentage of requests that return error responses (5xx status codes, timeouts, connection refused). Under normal load, the error rate should be zero or very near zero. A rising error rate under increasing load indicates the API is exceeding its capacity. Some APIs degrade gracefully by returning 503 (Service Unavailable) with a Retry-After header when overloaded. Others degrade poorly by timing out, returning 500 errors with stack traces, or dropping connections silently. The error rate pattern tells you how your API handles overload.

Concurrent users (virtual users, VUs) tracks how many simulated users are active during the test. Combined with throughput and response time, this metric reveals the relationship between load and performance. Plot response time against concurrent users to find the "knee of the curve," the point where response time starts increasing non-linearly. Below the knee, the API handles load well. Above the knee, performance degrades and capacity expansion is needed.

Resource utilization metrics from the server (CPU usage, memory consumption, disk I/O, network bandwidth, connection pool usage, thread count) correlate load test results with infrastructure constraints. If response times spike when CPU hits 90%, the bottleneck is compute. If errors appear when the connection pool is exhausted, the bottleneck is database connections. These correlations guide optimization: a CPU bottleneck needs code optimization or more servers, while a connection pool bottleneck needs pool size tuning or connection reuse improvements.

k6: The Modern Standard

k6, developed by Grafana Labs, has become the most popular open-source API load testing tool. Tests are written in JavaScript, making them accessible to any development team familiar with modern web development. k6 is a compiled Go binary that generates load efficiently, with a single machine capable of simulating thousands of concurrent users consuming minimal resources compared to JVM-based tools.

A basic k6 test defines a default function that runs once per virtual user per iteration. Inside this function, you send HTTP requests using k6's built-in http module and check responses with the check function. The options object configures the load profile: vus sets the concurrent user count and duration sets how long the test runs. A simple test with 50 virtual users running for 30 seconds sends thousands of requests and reports response time percentiles, throughput, error rate, and check pass rates.

Ramping stages create realistic load profiles. Instead of jumping to full load immediately, stages gradually increase and decrease virtual users: start with 0, ramp to 50 over 2 minutes (warm-up), hold at 50 for 5 minutes (sustained load), ramp to 200 over 1 minute (spike), hold at 200 for 3 minutes (peak load), ramp down to 0 over 2 minutes (cool-down). This profile reveals how the API handles each phase: warm-up shows cold-start behavior, sustained load shows steady-state performance, the spike shows scaling reaction time, and cool-down shows recovery behavior.

Thresholds define pass/fail criteria for the load test. Set thresholds on any metric: http_req_duration p(95) under 500ms, http_req_failed rate under 1%, http_reqs rate above 100 per second. If any threshold is violated during the test, k6 exits with a non-zero code, which CI pipelines interpret as a test failure. Thresholds turn load tests from informational reports into automated quality gates that prevent performance regressions from reaching production.

k6 integrates with monitoring platforms through output extensions. Send real-time metrics to Grafana Cloud, Datadog, Prometheus, InfluxDB, or New Relic for visualization on dashboards alongside application metrics. Seeing load test results overlaid on application CPU, memory, and database metrics in the same dashboard makes bottleneck identification immediate. k6 Cloud provides managed distributed load generation from multiple geographic regions for teams that need to simulate global traffic patterns.

JMeter and Gatling

Apache JMeter is the most established open-source load testing tool, maintained since 1998. Its GUI lets teams build test plans visually by assembling samplers (HTTP, JDBC, JMS, LDAP, SMTP), assertions, timers, and listeners into a hierarchical tree. JMeter's strength is protocol breadth: it handles virtually any network protocol, making it the tool of choice for teams testing complex systems that mix HTTP APIs with database queries, message queues, and LDAP authentication. Its weaknesses are significant resource consumption (JVM-based, requiring substantial memory for high concurrency), XML-based test plan files that are difficult to version control meaningfully, and a user interface that shows its age.

JMeter test plans define thread groups (analogous to virtual users) that execute samplers in sequence. Each sampler sends a request to the system under test. Assertions verify response content, status codes, and timing. Timers add realistic delays between requests. Listeners collect and display results in tables, graphs, and summary reports. For CI/CD integration, run JMeter in non-GUI mode from the command line: jmeter -n -t testplan.jmx -l results.jtl -e -o report/. This generates an HTML report with response time charts, throughput graphs, and error analysis.

Gatling, built on Scala, generates high concurrency with significantly lower resource consumption than JMeter. A single Gatling instance can simulate tens of thousands of concurrent users because it uses an asynchronous, event-driven architecture instead of JMeter's thread-per-user model. Gatling test scripts are written in Scala (or Java/Kotlin via the Gatling Java DSL), which makes them fully versionable and reviewable in Git. Gatling's HTML reports are the most visually polished in the load testing space, with interactive charts for response time distribution, throughput over time, active users, and request details.

Locust, written in Python, lets teams define load test scenarios as Python code, making it the natural choice for Python-heavy organizations. Users write a Python class that defines tasks (HTTP requests the virtual user performs), and Locust distributes these tasks across workers. Locust provides a web-based UI for real-time monitoring during test execution, showing request statistics, response time charts, and failure details. Its distributed mode runs across multiple machines for higher load generation.

Artillery, a Node.js tool, uses YAML-based scenario definitions for quick test creation. Define scenarios with phases (ramp-up, sustained, spike), request sequences, and response assertions in a YAML file, then run with artillery run scenario.yml. Artillery is particularly good at testing WebSocket and Socket.IO APIs alongside HTTP, making it suitable for real-time applications that use multiple protocols. Its simplicity makes it a good choice for teams that want basic load testing without learning a new programming language or GUI tool.

Designing Effective Load Tests

Realistic load profiles produce meaningful results. A load test that hammers a single endpoint with identical requests measures raw throughput but does not represent actual user behavior. Real users navigate through multiple endpoints, with varying request patterns, think times, and data payloads. Model your load test on actual traffic data from analytics: what percentage of requests go to the homepage versus the product API versus the search endpoint, what is the average session duration, how many requests does a typical session generate, and what is the peak concurrent user count.

Think time is the pause between requests that simulates a real user reading a page, filling out a form, or making a decision before the next action. Without think time, a virtual user sends requests as fast as the server responds, which generates unrealistically high load per user. Add randomized think time between 1 and 5 seconds (adjustable based on your application type) to simulate realistic user pacing. A 100-user test with think time generates roughly 20 to 100 RPS. The same 100-user test without think time might generate 500 to 2000 RPS, representing a scenario that never occurs in production.

Data variation prevents caching from masking performance issues. If every virtual user requests the same product (GET /api/products/1), the database and application caches serve the same data repeatedly, producing response times that are artificially fast. Vary request data by selecting from a pool of product IDs, user accounts, search queries, and filter combinations that match your production data distribution. If 80% of your traffic hits the top 100 products and 20% hits the long tail, replicate that distribution in your test data.

Baseline before load testing. Before running any concurrent load, measure single-user response times for every endpoint in the test. These baselines establish the best possible performance with zero contention. Compare load test results against baselines to identify which endpoints degrade under load (indicating shared resource contention) versus which are slow even without load (indicating algorithmic or query optimization opportunities).

Test the database, not just the API. Many APIs are thin layers over database queries, and the performance bottleneck is usually the database, not the API code. Include write operations (POST, PUT, DELETE) in your load test alongside reads (GET) because writes take locks that block reads. Include complex queries (filtered searches, aggregations, reports) that stress query plans differently than simple primary key lookups. If your load test consists entirely of simple GET requests, it tests cache performance more than database performance.

CI/CD Integration for Performance Testing

Running load tests in CI/CD pipelines catches performance regressions automatically. This is distinct from the comprehensive load tests described above, which run against production-like environments for capacity planning. CI pipeline load tests are lighter, faster, and focused on regression detection rather than capacity measurement.

A CI-friendly load test runs for 1 to 3 minutes with a moderate number of virtual users (10 to 50) and strict thresholds on response time and error rate. The goal is not to measure absolute capacity but to detect whether the latest code change made anything significantly slower. If the p95 response time for the /api/search endpoint was 150ms last week and is now 800ms, the threshold fails and the developer investigates before merging.

Store performance baselines in version control. After each release, record the performance metrics as baseline values in a JSON file or database. CI load tests compare current results against these baselines and fail if any metric has regressed beyond a tolerance threshold (for example, more than 20% slower than baseline). This approach detects gradual degradation that no single change's absolute threshold would catch: if each of 50 changes adds 2% latency, the accumulated 100% increase triggers the baseline comparison even though each individual change is within tolerance.

Separate performance tests from functional tests in the pipeline. Functional API tests should block merging. Performance tests might send notifications rather than blocking, because performance can vary between CI runs due to shared infrastructure and noisy neighbors. Alternatively, run performance tests on dedicated infrastructure where results are reproducible. k6, Gatling, and JMeter all output results in formats that CI tools can parse for status reporting.