What are VPN protocols and why do you need to understand the different options?
With most VPN providers offering a variety of VPN protocols to choose from, it is good to know the pros and cons of these different options so you can select the best fit for your unique needs.
In this guide we will compare the two most popular VPN protocols – OpenVPN vs IPSec – as well as L2TP/IPSec, IKEv2/IPSec, WireGuard, PPTP, and SSTP. This is meant to give you a brief overview of the pros and cons of each VPN protocol.
So let’s dive in.
What is a VPN protocol?
A VPN protocol is a set of instructions to establish a secure and encrypted connection between your device and a VPN server for the transmission of data.
Most commercial VPN providers offer a variety of different VPN protocols that you can use within the VPN client. For example, in the screenshot below, I am testing ExpressVPN and have the option to select OpenVPN UDP, OpenVPN TCP, SSTP, L2TP/IPSec, and PPTP.
Now we will take a closer look at various VPN protocols.
OpenVPN is a versatile, open source VPN protocol developed by OpenVPN Technologies. It is arguably the most secure and most popular VPN protocol in use today and has passed various third-party security audits.
OpenVPN is generally considered to be the industry standard when it is properly implemented and uses SSL/TLS for key exchange. It provides full confidentiality, authentication, and integrity and is also very flexible with various use cases.
Setup: OpenVPN requires special client software to use, rather than being built into different operating systems. Most VPN services provide custom OpenVPN apps, which can be used on different operating systems and devices. Installation is usually fast and simple. OpenVPN can be used on all major platforms through third-party clients: Windows, Mac OS, Linux, Apple iOS, Android, and various routers (check the firmware for compatibility).
Encryption: OpenVPN uses the OpenSSL library and TLS protocols to provide encryption. OpenSSL supports a number of different algorithms and ciphers, including AES, Blowfish, Camellia, and ChaCha20.
Security: OpenVPN is considered to be the most secure VPN protocol available, provided that it is properly implemented. It does not have any known major vulnerabilities.
Performance: OpenVPN offers good performance, especially if run over UDP (User Datagram Protocol), rather than TCP (Transmission Control Protocol). OpenVPN is also very stable and reliable whether used over wireless or cellular networks. If you are having connection problems you can use OpenVPN with TCP, which will confirm all packets sent, but it will be slower.
Ports: OpenVPN can be used on any port using UDP or TCP.
Verdict: Highly recommended.
What is IPSec?
IPSec is a secure network protocol suite that authenticates and encrypts data packets sent over an IP network. It stands for Internet Protocol Security (IPSec) and was developed by the Internet Engineering Task Force. Unlike with SSL, which works on the application level, IPSec operates on the network level and can be used natively with many operating systems. Because most operating systems support IPSec natively, it can be used without third-party apps (unlike OpenVPN).
IPSec has become a very popular protocol to use with VPNs when paired with L2TP or IKEv2, which we will discuss more below.
IPSec encrypts the entire IP packet using:
- Authentication Header (AH), which places a digital signature on each packet; and
- Encapsulating Security Protocol (ESP), which confidentiality, integrity, and authentication of the packet in transmission.
Leaked NSA presentation – A discussion of IPSec would not be complete without referencing a leaked NSA presentation that discusses the NSA compromising IPSec protocols (L2TP and IKE). It’s difficult to come to any concrete conclusions based on vague references in this dated presentation. Nonetheless, if your threat model includes targeted surveillance from sophisticated state-level actors, you may want to consider a more secure protocol, such as OpenVPN. Nonetheless, IPSec protocols are still widely considered to be secure if they are implemented properly.
Now we will examine how IPSec is used with VPNs when paired with L2TP and IKEv2.
What is IKEv2/IPSec?
IKEv2 is a tunneling protocol that is standardized in RFC 7296 and it stands for Internet Key Exchange version 2 (IKEv2). It was developed as a joint project between Cisco and Microsoft. To be used with VPNs for maximum security, IKEv2 is paired with IPSec.
The first version of IKE (Internet Key Exchange) came out in 1998, with version 2 being released seven years later in December 2005. In comparison to other VPN protocols, IKEv2 offers advantages in terms of speed, security, stability, CPU usage, and the ability to re-establish a connection. This makes it a good choice for mobile users, particularly with iOS (Apple) devices which natively support IKEv2.
Setup: Setup is generally quick and easy, requiring you to import the configuration files for the servers you want to use from your VPN provider. (See this setup example with Perfect Privacy.) IKEv2 is natively supported on Windows 7+, Mac OS 10.11+, Blackberry, and iOS (iPhone and iPad), and some Android devices. Some operating systems also support an “always on” function, which forces all internet traffic through the VPN tunnel, therefore ensuring no data leaks.
Encryption: IKEv2 uses a large selection of cryptographic algorithms, including AES, Blowfish, Camellia, and 3DES.
Security: One drawback with IKEv2/IPSec is that it is closed source and was developed by Cisco and Microsoft (but open source versions do exist). On a positive note, IKEv2 is widely-considered to be among the fastest and most secure protocols available, making it a popular choice with VPN users.
Performance: In many cases IKEv2 is faster than OpenVPN since it is less CPU-intensive. There are, however, numerous variables that affect speed, so this may not apply in all use cases. From a performance standpoint with mobile users, IKEv2 may be the best option because it does well establishing a reconnection.
Ports: IKEv2 uses the following ports: UDP 500 for the initial key exchange and UDP 4500 for NAT traversal.
Layer 2 Tunneling Protocol (L2TP) paired with IPSec is also a popular VPN protocol that is natively supported by many operating systems. L2TP/IPSec is standardized in RFC 3193 and provides confidentiality, authentication, and integrity.
Setup: Setting up L2TP/IPSec is generally fast and easy. It is natively supported on many operating systems, including Windows 2000/XP+, Mac OS 10.3+, as well as most Android operating systems. Just like with IKEv2/IPSec, you simply need to import the configuration files from your VPN provider.
Encryption: L2TP/IPSec encapsulates data twice with encryption coming via the standard IPSec protocol.
Security: L2TP/IPSec is generally considered secure and does not have any major known issues. Just like with IKEv2/IPSec, however, L2TP/IPSec was also developed by Cisco and Microsoft, which raises questions about trust.
Performance: In terms of performance L2TP/IPSec can really vary. One the one hand encryption/decryption occurs in the kernel and it also supports multi-threading, which should improve speeds. But on the other hand, because it double-encapsulates data, it may not be as fast as other options.
Ports: L2TP/IPSEC uses UDP 500 for the initial key exchange as well as UDP 1701 for the initial L2TP configuration and UDP 4500 for NAT traversal. Because of this reliance on fixed protocols and ports, it is easier to block than OpenVPN.
Verdict: L2TP/IPSec is not a bad choice, but you may want to opt for IKEv2/IPSec or OpenVPN if available.
WireGuard is a new and experimental VPN protocol that seeks to provide better performance and more security over existing protocols.
As we covered in the main WireGuard VPN guide, the protocol has some interesting benefits in terms of performance, but it also comes with a few noteworthy drawbacks. The main drawbacks are as follows:
- WireGuard remains under heavy development and has not yet been audited.
- Some VPN services have raised concerns over WireGuard’s ability to be used without logs (privacy drawbacks).
- Very limited adoption by the VPN industry (at least for now).
- No support for TCP.
Setup: WireGuard is not included in any operating system. This will likely change over time when it is included in the kernel for Linux, Mac OS, and mobile devices. A very limited number of VPNs support WireGuard – check with the provider for setup instructions.
Security: The major security issue with WireGuard is that it is not yet audited and remains under heavy development. There are a handful of VPNs already offering WireGuard to their users for “testing” purposes, but given the state of the project, WireGuard should not be used when privacy and security are important.
Performance: WireGuard should theoretically offer excellent performance in terms of speed, reliability, and also battery consumption. It may be the ideal protocol for mobile users because it allows you to switch between network interfaces without losing the connection. Re-connecting is also supposed to happen much faster than with OpenVPN and IPSec.
Ports: WireGuard uses UDP and can be configured on any port. Unfortunately, there is no support for TCP, which makes it easier to block.
Verdict: Not (yet) recommended, but we’ll keep an eye on the project’s development.
PPTP stands for Point-to-Point Tunneling Protocol and is one of the oldest VPN protocols still in use today. It runs on TCP port 1723 and was initially developed by Microsoft.
PPTP is now essentially obsolete due to serious security vulnerabilities. We won’t spend too much time discussing PPTP because most people are not using it anymore.
PPTP is supported natively on all versions of Windows and most operating systems. While it is relatively fast, PPTP is not as reliable and does not recover as quickly from a dropped connections as OpenVPN.
Overall, PPTP should not be used in any situation where security and privacy are important. If you are just using a VPN to unblock content, PPTP may not be a bad choice, but there are more secure options worth considering.
Verdict: Not recommended
Like PPTP, SSTP is not widely used in the VPN industry, but unlike PPTP, it does not have major known security issues.
SSTP stands for Secure Socket Tunneling Protocol and is a Microsoft product that is available for Windows only. The fact that it is a closed source product from Microsoft is an obvious drawback, although SSTP is also considered to be quite secure.
SSTP transports traffic through the SSL (Secure Socket Layer) protocol over TCP port 443. This makes it a useful protocol to use in restricted network situations, such as in China. There is also support for other operating systems, aside from Windows, but it is not widely used.
Because SSTP is closed source and remains entirely under the ownership and maintenance of Microsoft, you may want to consider other options. Of course, SSTP may still be the best option if all other protocols are getting blocked on your network.
In terms of performance, SSTP does well and is fast, stable, and secure.
Verdict: SSTP may be useful if other VPN protocols are getting blocked, but OpenVPN would be a better choice (if available).
OpenVPN UDP vs OpenVPN TCP
With OpenVPN being the most popular VPN protocol, you can usually select between two varieties: OpenVPN UDP or OpenVPN TCP. So which to choose? Below I’m testing out NordVPN, which gives me the option to select TCP or UDP protocols.
Here’s a brief overview of both protocols:
- TCP (Transmission Control Protocol): TCP is the more reliable option of the two, but it comes with some performance drawbacks. With TCP, packets are sent only after the last packet is confirmed to have arrived, therefore slowing things down. If confirmation is not received, a packet will simply be resent – what is known as error-correction.
- UDP (User Datagram Protocol): UDP is the fastest of the two options. Packets are sent without any confirmation, which improves speed but also may not be as reliable.
By default, OpenVPN UDP would be the better choice because it offers superior performance over OpenVPN TCP. If you are having connection problems, however, switch to TCP for more reliability.
TCP is often used for obfuscating VPN traffic to look like regular HTTPS traffic. This can be done by using OpenVPN TCP on port 443, with the traffic routed in TLS encryption. Many VPN providers offer various forms of obfuscation to defeat VPN blocks, and most utilize OpenVPN TCP.
What is the best VPN protocol?
As we discussed in the best VPN service guide, there is no one-size-fits all solution for every person, whether this is with choosing a VPN service or selecting a VPN protocol. The best protocol for your situation will depend on a few different factors:
- The device you are using – different devices support different protocols.
- Your network – if you are in a restricted network situation, such as in China or with school and work networks, some protocols may not get through. Some VPN providers offer designated VPN protocols for these situations – see the VPN for China guide for more of a discussion on this topic.
- Performance – Some protocols offer big advantages in terms of performance, especially on mobile devices that go in and out of connectivity.
- Threat model – Some protocols are weaker and less secure than others. Choose the best VPN protocol for your security and privacy needs, given your threat model.
As a general rule of thumb, however, OpenVPN is arguably the best all-around VPN protocol. It is very secure, trusted, widely-used in the industry, and it offers good speed and reliability. If OpenVPN is not an option for your situation, simply consider the alternatives.
With the majority of VPN services, OpenVPN is generally the default protocol used in their apps, although L2TP/IPSec and IKEv2/IPSec are common with mobile VPN clients.
VPN protocols conclusion
This VPN protocols guide is meant to serve as a basic overview of the main VPN protocols in use today: OpenVPN, L2TP/IPSec, IKEv2/IPSec, WireGuard, PPTP, and SSTP.
For more in-depth information on each protocol, you can examine references from the respective developers.
This guide will continue to be updated as development continues with these different VPN protocols.