Skip to content

How to do a Network benchmark test

Estimated time to read: 6 minutes

We will explain how you can test and compare the network performance of our instance flavors. We will use an Ubuntu VM and test the networking with iperf3. You can repeat these tests on any similar platform and compare the results. You can also test this on a Kubernetes debug pod for Kubernetes clusters.

Network

A network test always measures the whole path between two endpoints: your instance, our network, the internet in between and the remote side. We have redundant, fast network links and uplinks, and we're connected to several fast transit providers, but we're not in control of the rest of the internet. In practice a result is usually limited by the path to the remote location or by the remote side itself.

That's why we recommend iperf3. You choose both endpoints yourself, so you know exactly what you're measuring: for example between two of your instances, or between your instance and a server at another provider you want to compare with. Don't have a second machine? You can also test against a public iperf3 server.

If you are seeing issues with traffic between applications in our cloud and somewhere else the network is only a small part of it. This benchmark will help you rule out the network as a bottleneck and lets you focus on application, storage or Kubernetes network components as the cause.

iperf3

Installation

Install iperf3 on the instance you want to test (the client). If you test against your own server, install it there as well.

sudo apt update && sudo apt install -y iperf3

Choose a server

You can test against your own server, or against a public iperf3 server.

Option 1: your own server

On the server side, start iperf3 in server mode. It listens on port 5201.

iperf3 -s

Make sure TCP port 5201 is reachable on the server. On a Cyso Cloud instance, add a rule to its security group that allows TCP port 5201 from the public IP of your client only. Remove the rule again when you're done.

Option 2: a public iperf3 server

If you don't have a second machine, or you want to test towards the internet, you can use one of the public iperf3 servers listed on iperf.fr. You don't need to set anything up: run the client commands below against the server's hostname. Pick a server close to your instance, for example in the Netherlands, Germany or France for instances in AMS.

Many public servers listen on a range of ports instead of only 5201. Add the port with -p, for example:

iperf3 -c iperf3.moji.fr -p 5200 -t 30 -O 2

Keep in mind that a public server is shared with other users and its own connection is also part of the test. If the result is lower than you expect, try a different server or test against your own server to compare.

Run the tests

Run these commands on the client. Replace <server-ip> with the address or hostname of your server, and add -p <port> if you use a public server that doesn't listen on 5201.

Upload (client sends to the server): 

iperf3 -c <server-ip> -t 30 -O 2

Download (server sends to the client): 

iperf3 -c <server-ip> -t 30 -O 2 -R

The same tests with 8 parallel connections: 

iperf3 -c <server-ip> -t 30 -O 2 -P 8

iperf3 -c <server-ip> -t 30 -O 2 -P 8 -R

Option Meaning
-c <server-ip> Run as client and connect to this server
-p <port> Connect to this port instead of 5201 (needed for many public servers)
-t 30 Run the test for 30 seconds
-O 2 Leave out the first 2 seconds, while TCP is still speeding up
-R Reverse the direction: the server sends, the client receives
-P 8 Use 8 parallel connections

Tip

Run each test a few times. A single run can be influenced by other traffic on the path, especially when you test against a public iperf3 server that other people use at the same time. Public servers often accept only one test at a time and run on several ports (for example 5200–5209); if a server reports that it's busy, try another port or try again later.

Result

The results are shown in Megabits or Gigabits per second. One Megabit per second is 0.125 MB per second, so 2.45 Gbit/s is around 306 MB/s.

This is an example of an upload test from an instance in cloud region AMS to the public server iperf3.moji.fr (iperf3 -c iperf3.moji.fr -p 5200 -t 30 -O 2), shortened in the middle:

Connecting to host iperf3.moji.fr, port 5200
[  5] local 198.51.100.20 port 50742 connected to 45.147.210.189 port 5200
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   269 MBytes  2.26 Gbits/sec    0   8.03 MBytes       (omitted)
[  5]   1.00-2.00   sec   291 MBytes  2.44 Gbits/sec    0   8.03 MBytes       (omitted)
[  5]   0.00-1.00   sec   292 MBytes  2.45 Gbits/sec    0   8.03 MBytes
[  5]   1.00-2.00   sec   291 MBytes  2.44 Gbits/sec    0   8.03 MBytes
[  5]   2.00-3.00   sec   289 MBytes  2.42 Gbits/sec    0   8.03 MBytes
...
[  5]   7.00-8.00   sec   299 MBytes  2.51 Gbits/sec    0   8.03 MBytes
[  5]   8.00-9.00   sec   291 MBytes  2.44 Gbits/sec  1238   8.03 MBytes
[  5]   9.00-10.00  sec   288 MBytes  2.42 Gbits/sec  810   8.03 MBytes
[  5]  10.00-11.00  sec   293 MBytes  2.46 Gbits/sec    0   8.03 MBytes
[  5]  11.00-12.00  sec   292 MBytes  2.45 Gbits/sec  865   8.03 MBytes
[  5]  12.00-13.00  sec   300 MBytes  2.51 Gbits/sec    0   8.03 MBytes
...
[  5]  28.00-29.00  sec   287 MBytes  2.41 Gbits/sec    0   8.03 MBytes
[  5]  29.00-30.00  sec   292 MBytes  2.45 Gbits/sec    0   8.03 MBytes
  • Bitrate is the speed in that second. Lines marked (omitted) are the first seconds left out by -O 2. In this test the speed stays steady around 2.45 Gbit/s.
  • When the test finishes, iperf3 prints a sender and a receiver line with the average over the whole test. That average is your result.
  • Retr is the number of packets that had to be sent again. Judge it against the speed: at 2.45 Gbit/s about 210,000 packets are sent every second, so the 1238 retransmissions in second 8 are about 0.6%. That's normal over the internet. What matters is whether the bitrate drops at the same time; here it doesn't.
  • Cwnd is the amount of data TCP keeps underway. When packets get lost, TCP usually makes it smaller and the speed drops for a moment.

Interpreting the results

  • A single connection is limited by the latency. The further away the server, the lower the speed one TCP connection reaches, even on a fast network. Compare results against servers at a similar distance, and test against a server close to your instance when you want to know what the instance can do.
  • Compare 1 connection with 8 connections. If 8 connections together are much faster than one, the limit is per connection (latency or packet loss on the path), not the bandwidth of your instance.
  • Compare both directions. Upload and download can follow different paths through the internet and give different results.