What is IPsec (Internet Protocol Security)?

IPsec is a set of Internet Engineering Task Force (IETF) open standards for building secure Virtual Private Networks (VPNs) that use a variety of protocols and technologies to secure data traffic over public networks and the Internet.


IPsec Security Services

IPsec provides the following security services to protect IP traffic:

  1. Peer Authentication – verifies the identity of the Virtual Private Network (VPN) peer. The peer will authenticate each other using PSK or digital certificates to ensure they communicate with one another.
  2. Data Confidentiality – uses encryption algorithms, DES, 3DES, or AES, to protect data from eavesdropping attacks. It converts plaintext to encrypted ciphertext, and only the sender and receiver can read it.
  3. Data Integrity – minimizes Man-in-the-Middle (MitM) attacks by guaranteeing that data isn’t tampered with while traversing an insecure network.  The peers can check if the IP packet has been altered by calculating the hash value using the Hash Message Authentication Code (HMAC) functions; MD5 and SHA.
  4. Replay Detection – reduces MitM attacks where an attacker captures VPN traffic and replays it to a VPN peer to establish an unauthorized VPN tunnel even if the packet is encrypted and authenticated. IPsec will not send duplicate IP packets. Each packet is assigned a unique sequence number, and a VPN device tracks the sequence number and will not accept a packet with a processed sequence number.


To provide security services, IPsec uses the following encryption, hashing, and keying methods:

  • Data Encryption Standard (DES): a 56-bit symmetric data encryption algorithm used to encrypt data sent over a VPN, not recommended.
  • Triple DES (3DES): runs the DES encryption algorithm three times with varying three 56-bit keys, not recommended.
  • Advanced Encryption Standard (AES): replaces DES and 3DES encryption algorithms, and it supports 128 bits, 192 bits, and 256 bits key lengths. It is based on the Rijndael algorithm.
  • Message Digest 5 (MD5): a 128-bit hash algorithm used for one-way data authentication. It adds an extra layer of security against MitM attacks. It is not recommended, and SHA should be used instead.
  • Secure Hash Algorithm (SHA): a 160-bit hash algorithm used for one-way data authentication. It provides an extra layer of security against MitM attacks.
  • Diffie-Hellman (DH): an asymmetric key exchange protocol that allows peers to generate a shared secret key used by VPN peers for symmetrical algorithms over insecure communication channels. A DH group is the modulus size or key length used for a DH key exchange. The bigger the modulus, the more secure it is.The DH exchange is asymmetrical and CPU intensive, but the generated shared secret keys are symmetrical. Cisco advises not using DH groups 1, 2, and 5 and using DH groups 14 and above instead.
  • RSA Signatures: a cryptographic mechanism based on public keys or digital certificates utilized to authenticate peers mutually.
  • Pre-Shared Key: a security method that employs a locally configured key as a credential to authenticate peers mutually.


IPsec Protocols

IPsec utilizes two packet headers, Authentication Header and Encapsulating Security Payload:

Authentication Header (AH)

The IP authentication header ensures data integrity, authentication, and security against packet replays. Using protocol number 51 in the IP header, AH generates a digital signature to guarantee that the packet has not been altered. Using AH is not recommended since it doesn’t support encryption or NAT traversal (NAT-T), so it’s only used for authentication.

Encapsulating Security Payload (ESP)

ESP offers data confidentiality, authentication, and security against packet replays. With ESP, the payload means the original packet’s component encapsulated inside the IPsec headers. ESP encrypts the payload and adds a new set of headers during transmission across a public network to assure that the original payload retains data confidentiality. It utilizes protocol number 50 in the IP header, provides data confidentiality, and is NAT-T compatible.


IPSec Packet Transport Modes

Traditional IPsec supports two packet transport modes:

Tunnel Mode

The IPsec tunnel mode encrypts the whole original packet while adding a new set of IPsec headers utilized to route packets and offer overlay functionality. It is often used for site-to-site VPNs where the initial IP packet must be encapsulated since they are usually private IP addresses that can’t be sent over the Internet.

Transport Mode

The IPsec transport mode encrypts and authenticates the packet payload. It doesn’t include any overlay functionalities or routes that are based on the original IP headers. It is commonly used between devices that intend to secure certain traffic, like Telnet.

The primary difference between the modes is that a new IP header is used in tunnel mode, while in transport mode, the original IP header is used. The image below shows an example of an original packet, the packet in transport mode and tunnel mode.



Transform Sets

A transform set combines security protocols and methods to secure certain data flows. During the IPsec Security Association (SA) negotiation, peers agree to implement a specific transform set. SA is the collection of a peer’s parameters. When a transform set is detected, it is applied to both peers’ SAs. The supported transform set combinations are:


  1. Authentication Header Transform
  • ah-md5-hmac: with MD5 authentication (not recommended)
  • ah-sha-hmac: with SHA authentication
  • ah-sha256-hmac: with 256-bit SHA authentication
  • ah-sha384-hmac: with 384-bit SHA authentication
  • ah-sha512-hmac: with 512-bit SHA authentication


  1. ESP Encryption Transform
  • esp-aes: with 128-bit AES encryption
  • esp-gcm: with 128-bit (default) or 256-bit authenticated encryption
  • esp-gmac: with 128-bit (default) or 256-bit authentication without encryption
  • esp-aes 192: with 192-bit AES encryption
  • esp-aes 256: with 256-bit AES encryption
  • esp-des: with 168-bit DES encryption (not recommended)
  • esp-3des: with 56-bit DES encryption (not recommended)
  • esp-seal: with 160-bit SEAL encryption
  • esp-null: with null encryption


  1. ESP Authentication Transform
  • esp-md5-hmac: with MD5 authentication (not recommended)
  • esp-sha-hmac: with SHA authentication


  1. IP Compression Transform
  • comp-lzs: with Lempel-Ziv-Stac (LZS)

Download our Free CCNA Study Guide PDF for complete notes on all the CCNA 200-301 exam topics in one book.

We recommend the Cisco CCNA Gold Bootcamp as your main CCNA training course. It’s the highest rated Cisco course online with an average rating of 4.8 from over 30,000 public reviews and is the gold standard in CCNA training: