Skip to content

What to load test for

If you’re a big company with a platform that regularly supports tons of users, you have to know how incremental traffic impacts the performance of your systems. Maybe you’re getting ready for a seasonal surge in traffic like Black Friday or Cyber Monday, and don’t want to lose out on serious revenue in case a critical process fails.

For example, how does your system behave when a large number of users are accessing your servers? How long does it take for a user to receive a response? As more users access your platform, does a response take significantly longer or does a request timeout? Is this effect magnified at certain times of the day or week?

Load testing is the practice of sending simulated HTTP traffic to a server in order to measure performance. Load testing is performed by running load testing software on one machine (or a cluster of machines) to generate a large amount of requests to a web server on a second machine (or other more complex web serving infrastructure).

Load testing falls under a broader umbrella of performance testing. During load testing, you will subject an API to a workload that approaches and possibly exceeds the limits of its specifications.

Sometimes load testing is completed along with stress testing, to determine the system’s ability to handle a heavier than normal workload. Sometimes it can include endurance testing, to determine the system’s ability to handle a specified work load for a prolonged period of time.

Load Testing helps us:

  • Determine our application’s capabilities by measuring its response time, throughput, CPU utilization, latency, etc. during average and heavy user load. This will eventually help in determining the infrastructure needs as the system scales upward.
  • It gives us an opportunity to find strange behavior or surprises when we subject an application to an insane amount of load (stress testing). Strange behaviors include request timeouts, IO Exceptions, memory leaks, or any security issues.
  • It also helps us understand if any new code path or database connection introduced any performance bottlenecks.

Load testing with Continuous Integration

Load testing can be done as a regular part of your Continuous Integration process. At these early stages of the development lifecycle, you can identify bottlenecks and improve the stability of your application before it is released to production.It’s best to catch issues and errors in these early stages when the cost of debugging is the lowest.

How to load test

Before doing load testing you should have some metrics in your mind which you want to achieve for the application.

We should try to achieve metrics for 1 instance/machine(app server) for example:

  • Throughput: 100 RPS(requests per second) or 6000 RPM(request per minute)
  • Latency: < 100ms(time taken by the server to send back the result), but for best performance, it should be less than 50ms
  • CPU: < 50%
  • Memory: < 50%

These are some common measures of API performance. Depending on your service level objectives and requirements, you may want to measure other indicators too.

  • Response time — total processing time, latency
  • Throughput — Requests Per Second, request payloads, maximum operating capacity
  • Traffic composition — average and peak concurrent users
  • Database — number of concurrent connections, CPU utilization, read and write IOPS, memory consumption, disk storage requirements
  • Errors — handling, failure rates

Deciding the Load testing framework/tool

The basic and essential part of load testing is deciding the right tool for you, which can be used. They are plenty of loads testing tool out there like locust.io, AB(apache benchmark), Jmeter, Gatlin, Vegeta etc If you just want to fire the load on your system easily, you can use AB. But if you want to generate traffic like real-time application will have, custom weight, scenario etc, use locust, Vegeta, Jmeter etc.

My personal choice is the locust, as it is written in python with easy installation, and you can write your scenarios easily.

Setup a server and app level monitoring service

Before you start doing your load testing, you must set up a good server monitoring service. Without this, you will not be able to find the CPUs, memory usages etc. And also setup app monitoring, this will help you to find out the bottlenecks in your application.

Find your load pattern

You should have a lot of end-points in your backend. But every endpoint will have its own characteristic. Means, Endpoints can be hit at the different rate like lower, average, high. Then there will be some type of flow for the using the endpoints from your client. And at last every endpoint will have different payloads too. Knowing these factors will help to design the test cases. An easy way to start is to find out the critical endpoints. And then do the load test on them. Later combine all the endpoints for generating production like traffic. Because sometimes single load test will give the good response, but when we combine all these services, they behave differently.

Generate your load data

This is an essential part of getting the best results. Because when you do your load testing without generating the load/seed data, your test case will not give you the exact output, because sometimes you missed the indexes or unnecessary have used indexes. These things are hard to find out in a small dataset. So I will suggest for generating the data use your system APIs only so that you can also identify the problem with them too. But sometimes you do not have that much writes in your application, in that case, you can insert the data directly into the DB, and can test the APIs too. It depends on the use case. But I will suggest inserting at least 1 million records in your tables/collections for getting the good insights for your platform.

Simulate different network speed

When we have done with our load testing. We should also evaluate the different network band because our customers will have a different kind of network.

Run your test for longer time

Also run your load test for at least 1 hour. Because the performance of system can be reduce over the time. As there can be memory leak orthreads are releasing the resource fast enough etc, as over the period there are chances that the result can vary. So it’s a good practice to run your load test for a longer period.

Types of Performance Testing

The different types of performance testing are:

  • Load testing — It checks the application’s ability to perform under anticipated user loads. The objective is to identify performance bottlenecks before the software application goes live.
  • Stress testing — This involves testing an application under extreme workloads to see how it handles high traffic or data processing. The objective is to identify the breaking point of an application.
  • Endurance testing — It is done to make sure the software can handle the expected load over a long period of time.
  • Spike testing — This tests the software’s reaction to sudden large spikes in the load generated by users.
  • Volume testing — Under Volume Testing large no. of. Data is populated in a database and the overall software system’s behavior is monitored. The objective is to check the software application’s performance under varying database volumes.
  • Scalability testing — The objective of scalability testing is to determine the software application’s effectiveness in scaling up to support an increase in user load. It helps plan capacity addition to your software system.