iperf2 vs iperf3: What’s the difference?

At first glance, you might be tempted to use iperf3 simply because it is one more than iperf2 (don’t worry, I’m guilty of this crime as well). It’s not an unfair assumption to think that iperf3 is the most recent version of the software, because of the name. It’s common to have two different versions of software in parallel existence, so the new one can take hold while the older version slowly dies away. Python2 and Python3 come to mind. This is not the case with iperf, however.

I recently wrote a post on how to use iperf3 to test bandwidth. Shortly after that one of the authors of iperf2, Bob McMahon, reached out to me. He pointed out that iperf2 is very much actively developed with some cool new features having been added recently. Under the surface they are very different projects, maintained by different teams with different goals.

Today we’ll take a look at some of the differences between the two.

Topology

Ubuntu 20.04 and Rocky Linux 8.5 VM’s in GNS3

We have a really basic topology here, Ubuntu 20.04 and Rocky Linux 8.5 connected on a single link with IP subnet 10.0.0.0/30. Both VM’s have iperf2 and iperf3 installed.

Bandwidth Test

For a bandwidth test, the two are almost identical. You can perform a bandwidth test using either with the same commands. For this test, the Ubuntu VM will be the client, and Rocky the server. Start the server on Rocky like this:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 85.3 KByte (default) ------------------------------------------------------------
iperf -s

------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------

And from Ubuntu perform a test like this:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
iperf -c 10.0.0.2
------------------------------------------------------------
Client connecting to 10.0.0.2, TCP port 5001
TCP window size: 238 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.1 port 36528 connected with 10.0.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1.90 GBytes 1.63 Gbits/sec
iperf -c 10.0.0.2 ------------------------------------------------------------ Client connecting to 10.0.0.2, TCP port 5001 TCP window size: 238 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.0.1 port 36528 connected with 10.0.0.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.90 GBytes 1.63 Gbits/sec
iperf -c 10.0.0.2
------------------------------------------------------------
Client connecting to 10.0.0.2, TCP port 5001
TCP window size:  238 KByte (default)
------------------------------------------------------------
[  3] local 10.0.0.1 port 36528 connected with 10.0.0.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.90 GBytes  1.63 Gbits/sec

These commands will work using iperf2 or iperf3, however it should be noted you can’t use an iperf2 client with an iperf3 server, and vice-versa. Also, they use different TCP ports by default. Even if you used an iperf3 client with an iperf2 server and manually set the TCP port to be the same, you will get an error. They are not compatible:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
iperf3 -c 10.0.0.2 -p 5001
iperf3: error - received an unknown control message
iperf3 -c 10.0.0.2 -p 5001 iperf3: error - received an unknown control message
iperf3 -c 10.0.0.2 -p 5001
iperf3: error - received an unknown control message

Supported Operating Systems

iperf2 is the clear winner here, primarily because it has up-to-date Windows packages available for easy download right on the sourceforge page. I avoid Windows when I can, but it has a tendency to be unavoidable due to it’s sheer installation base. iperf3 apparently had some unofficial builds a while back but nothing officially supported. You’ll need to compile it yourself to work on Windows which can be an inconvenience at best.

iperf2 downloads page

For Linux, many operating systems come with iperf2 preinstalled, Ubuntu 20.04 is one such example. iperf3 is just a command away though, with package managers. For macOS, the Homebrew package manager can quickly get you iperf2 or iperf3.

Feature: iperf3 authentication (not encryption)

Description of authentication features in iperf3

iperf3 supports authenticating clients to the server using public key/private key as well as a users file. I decided to try it out. To avoid a hassle I just used the exact commands they provided in the man file. You first generate a public key and private key on the server:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
openssl genrsa -des3 -out private.pem 2048
openssl rsa -in private.pem -outform PEM -pubout -out public.pem
openssl rsa -in private.pem -out private_not_protected.pem -outform PEM
openssl genrsa -des3 -out private.pem 2048 openssl rsa -in private.pem -outform PEM -pubout -out public.pem openssl rsa -in private.pem -out private_not_protected.pem -outform PEM
openssl genrsa -des3 -out private.pem 2048
openssl rsa -in private.pem -outform PEM -pubout -out public.pem
openssl rsa -in private.pem -out private_not_protected.pem -outform PEM

Then create a “credentials.csv” file with hashed passwords. The following commands will get a hashed password for you:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
S_USER=james S_PASSWD=james
echo -n "{$S_USER}$S_PASSWD" | sha256sum | awk '{ print $1 }'
----
0b0c98028105e9e4d3f100280eac29bba90af614d1c75612729228e4d160c601 #This is the hash of "james"
S_USER=james S_PASSWD=james echo -n "{$S_USER}$S_PASSWD" | sha256sum | awk '{ print $1 }' ---- 0b0c98028105e9e4d3f100280eac29bba90af614d1c75612729228e4d160c601 #This is the hash of "james"
S_USER=james S_PASSWD=james
echo -n "{$S_USER}$S_PASSWD" | sha256sum | awk '{ print $1 }'
----
0b0c98028105e9e4d3f100280eac29bba90af614d1c75612729228e4d160c601 #This is the hash of "james"

Then create a “credentials.csv” file that looks like this:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
username,sha256
james,0b0c98028105e9e4d3f100280eac29bba90af614d1c75612729228e4d160c601
username,sha256 james,0b0c98028105e9e4d3f100280eac29bba90af614d1c75612729228e4d160c601
username,sha256
james,0b0c98028105e9e4d3f100280eac29bba90af614d1c75612729228e4d160c601

Now start the server:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
iperf3 -s --rsa-private-key-path ./private_not_protected.pem --authorized-users-path ./credentials.csv
iperf3 -s --rsa-private-key-path ./private_not_protected.pem --authorized-users-path ./credentials.csv
iperf3 -s --rsa-private-key-path ./private_not_protected.pem --authorized-users-path ./credentials.csv

Then from the client, copy the public key over:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
scp james@10.0.0.1:public.pem .
scp james@10.0.0.1:public.pem .
scp james@10.0.0.1:public.pem .

Then run the client:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
iperf3 -c 10.0.0.1 --rsa-public-key-path ./public.pem --username james
iperf3 -c 10.0.0.1 --rsa-public-key-path ./public.pem --username james
iperf3 -c 10.0.0.1 --rsa-public-key-path ./public.pem --username james

You’ll be asked for the password. If you get it right, the server will display a message that authentication succeeded:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Authentication successed for user 'james' ts 1639396545
Accepted connection from 10.0.0.2, port 32784
[ 5] local 10.0.0.1 port 5201 connected to 10.0.0.2 port 32786
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 194 MBytes 1.63 Gbits/sec
[ 5] 1.00-2.00 sec 204 MBytes 1.71 Gbits/sec
----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Authentication successed for user 'james' ts 1639396545 Accepted connection from 10.0.0.2, port 32784 [ 5] local 10.0.0.1 port 5201 connected to 10.0.0.2 port 32786 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 194 MBytes 1.63 Gbits/sec [ 5] 1.00-2.00 sec 204 MBytes 1.71 Gbits/sec
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Authentication successed for user 'james' ts 1639396545
Accepted connection from 10.0.0.2, port 32784
[  5] local 10.0.0.1 port 5201 connected to 10.0.0.2 port 32786
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   194 MBytes  1.63 Gbits/sec                  
[  5]   1.00-2.00   sec   204 MBytes  1.71 Gbits/sec

Feature: iperf2 isochronous mode

One of the coolest features of iperf2 is its “isochronous” option. This option is designed to simulate video streaming network traffic. You can hear Bob McMahon explain it himself on his youtube video on this feature.

Using the parameters and commands he describes in his video, we’ll run on a test. The Ubuntu server will be the iperf2 server:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
iperf -s -e -i 1
iperf -s -e -i 1
iperf -s -e -i 1

Then on Rocky Linux we’ll run the client test:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
[james@localhost ~]$ iperf -c 10.0.0.1 -i 1 --isochronous=60:40m,10m
------------------------------------------------------------
Client connecting to 10.0.0.1, TCP port 5001 with pid 1640
UDP isochronous: 60 frames/sec mean=40.0 Mbit/s, stddev=10.0 Mbit/s, Period/IPG=16.67/0.005 ms
TCP window size: 340 KByte (default)
------------------------------------------------------------
[ 3] local 10.0.0.2 port 49150 connected with 10.0.0.1 port 5001 (ct=1.44 ms)
[ ID] Interval Transfer Bandwidth Write/Err Rtry Cwnd/RTT NetPwr
[ 3] 0.00-1.00 sec 214 MBytes 1.79 Gbits/sec 1708/0 0 67K/562 us 398346.93
[ 3] 1.00-2.00 sec 217 MBytes 1.82 Gbits/sec 1738/0 230 145K/608 us 374676.21
[ 3] 2.00-3.00 sec 205 MBytes 1.72 Gbits/sec 1640/0 427 142K/583 us 368710.26
[ 3] 3.00-4.00 sec 212 MBytes 1.78 Gbits/sec 1697/0 575 118K/920 us 241770.85
[ 3] 4.00-5.00 sec 200 MBytes 1.68 Gbits/sec 1599/0 371 134K/853 us 245702.38
[ 3] 5.00-6.00 sec 200 MBytes 1.68 Gbits/sec 1598/0 423 117K/529 us 395941.50
[james@localhost ~]$ iperf -c 10.0.0.1 -i 1 --isochronous=60:40m,10m ------------------------------------------------------------ Client connecting to 10.0.0.1, TCP port 5001 with pid 1640 UDP isochronous: 60 frames/sec mean=40.0 Mbit/s, stddev=10.0 Mbit/s, Period/IPG=16.67/0.005 ms TCP window size: 340 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.0.2 port 49150 connected with 10.0.0.1 port 5001 (ct=1.44 ms) [ ID] Interval Transfer Bandwidth Write/Err Rtry Cwnd/RTT NetPwr [ 3] 0.00-1.00 sec 214 MBytes 1.79 Gbits/sec 1708/0 0 67K/562 us 398346.93 [ 3] 1.00-2.00 sec 217 MBytes 1.82 Gbits/sec 1738/0 230 145K/608 us 374676.21 [ 3] 2.00-3.00 sec 205 MBytes 1.72 Gbits/sec 1640/0 427 142K/583 us 368710.26 [ 3] 3.00-4.00 sec 212 MBytes 1.78 Gbits/sec 1697/0 575 118K/920 us 241770.85 [ 3] 4.00-5.00 sec 200 MBytes 1.68 Gbits/sec 1599/0 371 134K/853 us 245702.38 [ 3] 5.00-6.00 sec 200 MBytes 1.68 Gbits/sec 1598/0 423 117K/529 us 395941.50
[james@localhost ~]$ iperf -c 10.0.0.1 -i 1 --isochronous=60:40m,10m
------------------------------------------------------------
Client connecting to 10.0.0.1, TCP port 5001 with pid 1640
UDP isochronous: 60 frames/sec mean=40.0 Mbit/s, stddev=10.0 Mbit/s, Period/IPG=16.67/0.005 ms
TCP window size:  340 KByte (default)
------------------------------------------------------------
[  3] local 10.0.0.2 port 49150 connected with 10.0.0.1 port 5001 (ct=1.44 ms)
[ ID] Interval        Transfer    Bandwidth       Write/Err  Rtry     Cwnd/RTT        NetPwr
[  3] 0.00-1.00 sec   214 MBytes  1.79 Gbits/sec  1708/0          0       67K/562 us  398346.93
[  3] 1.00-2.00 sec   217 MBytes  1.82 Gbits/sec  1738/0        230      145K/608 us  374676.21
[  3] 2.00-3.00 sec   205 MBytes  1.72 Gbits/sec  1640/0        427      142K/583 us  368710.26
[  3] 3.00-4.00 sec   212 MBytes  1.78 Gbits/sec  1697/0        575      118K/920 us  241770.85
[  3] 4.00-5.00 sec   200 MBytes  1.68 Gbits/sec  1599/0        371      134K/853 us  245702.38
[  3] 5.00-6.00 sec   200 MBytes  1.68 Gbits/sec  1598/0        423      117K/529 us  395941.50

On the server we get our output:

Plain text
Copy to clipboard
Open code in new window
EnlighterJS 3 Syntax Highlighter
james@u20vm:~$ iperf -s -e -i 1
------------------------------------------------------------
Server listening on TCP port 5001 with pid 3045
Read buffer size: 128 KByte
TCP window size: 128 KByte (default)
------------------------------------------------------------
[ 4] local 10.0.0.1 port 5001 connected with 10.0.0.2 port 49150
[ ID] Interval Transfer Bandwidth Reads Dist(bin=16.0K)
[ 4] 0.0000-1.0000 sec 213 MBytes 1.79 Gbits/sec 4631 503:1500:1008:577:276:191:138:438
[ 4] 1.0000-2.0000 sec 217 MBytes 1.82 Gbits/sec 4018 570:838:812:502:255:231:164:646
[ 4] 2.0000-3.0000 sec 204 MBytes 1.71 Gbits/sec 5074 590:1537:1637:511:316:152:115:216
[ 4] 3.0000-4.0000 sec 212 MBytes 1.78 Gbits/sec 3924 599:805:717:464:266:264:246:563
[ 4] 4.0000-5.0000 sec 200 MBytes 1.68 Gbits/sec 3876 575:953:672:462:258:242:188:526
[ 4] 5.0000-6.0000 sec 200 MBytes 1.68 Gbits/sec 4046 656:1040:687:476:258:242:238:449
james@u20vm:~$ iperf -s -e -i 1 ------------------------------------------------------------ Server listening on TCP port 5001 with pid 3045 Read buffer size: 128 KByte TCP window size: 128 KByte (default) ------------------------------------------------------------ [ 4] local 10.0.0.1 port 5001 connected with 10.0.0.2 port 49150 [ ID] Interval Transfer Bandwidth Reads Dist(bin=16.0K) [ 4] 0.0000-1.0000 sec 213 MBytes 1.79 Gbits/sec 4631 503:1500:1008:577:276:191:138:438 [ 4] 1.0000-2.0000 sec 217 MBytes 1.82 Gbits/sec 4018 570:838:812:502:255:231:164:646 [ 4] 2.0000-3.0000 sec 204 MBytes 1.71 Gbits/sec 5074 590:1537:1637:511:316:152:115:216 [ 4] 3.0000-4.0000 sec 212 MBytes 1.78 Gbits/sec 3924 599:805:717:464:266:264:246:563 [ 4] 4.0000-5.0000 sec 200 MBytes 1.68 Gbits/sec 3876 575:953:672:462:258:242:188:526 [ 4] 5.0000-6.0000 sec 200 MBytes 1.68 Gbits/sec 4046 656:1040:687:476:258:242:238:449
james@u20vm:~$ iperf -s -e -i 1
------------------------------------------------------------
Server listening on TCP port 5001 with pid 3045
Read buffer size:  128 KByte
TCP window size:  128 KByte (default)
------------------------------------------------------------
[  4] local 10.0.0.1 port 5001 connected with 10.0.0.2 port 49150
[ ID] Interval            Transfer    Bandwidth       Reads   Dist(bin=16.0K)
[  4] 0.0000-1.0000 sec   213 MBytes  1.79 Gbits/sec  4631    503:1500:1008:577:276:191:138:438
[  4] 1.0000-2.0000 sec   217 MBytes  1.82 Gbits/sec  4018    570:838:812:502:255:231:164:646
[  4] 2.0000-3.0000 sec   204 MBytes  1.71 Gbits/sec  5074    590:1537:1637:511:316:152:115:216
[  4] 3.0000-4.0000 sec   212 MBytes  1.78 Gbits/sec  3924    599:805:717:464:266:264:246:563
[  4] 4.0000-5.0000 sec   200 MBytes  1.68 Gbits/sec  3876    575:953:672:462:258:242:188:526
[  4] 5.0000-6.0000 sec   200 MBytes  1.68 Gbits/sec  4046    656:1040:687:476:258:242:238:449

Unfortunately the version of iperf that is available in Ubuntu 20.04 repositories (2.0.13) doesn’t support isochronous TCP mode mentioned in the video. You would need to compile from source or use Windows for that. A newer version will be included (probably already has been by the time you’re reading this) in Ubuntu 22.04 LTS.

Various smaller differences

There are many other spots that iperf2 and iperf3 are different.

  • iperf2 supports an “enhanced output mode” using -e that is totally revamped (used it above in the isochronous section).
  • iperf3 supports json output using the -j option.
  • iperf2 supports a bidirectional test which performs tests from the client and server simultaneously using -d
  • iperf2 uses a multi-threaded architecture, while iperf3 uses single-threaded. To be honest, I haven’t seen any way that this actually affects performance of the application. I’d be really curious if anyone has some input on this.

I hope this was helpful, and I hope I did both of these cool programs a small amount of justice. I’m really curious to see if anyone has any other input or differences they know about. Please fee free to comment or reach out directly.