TLS Handshake

To protect data transferred across a public network from eavesdropping or modification, security protocols combine two distinct branches of cryptography:

• Asymmetric Encryption (Public Key Cryptography): Uses a mathematically linked key pair. Data encrypted with a public key can only be decrypted by the corresponding private key, and vice versa. Because asymmetric operations require intensive CPU calculations, they are reserved for identity validation and session key negotiation.
• Symmetric Encryption (Bulk Encryption): Uses a single session key shared between the client and server to both encrypt and decrypt data. Once established, symmetric encryption is highly efficient, processing gigabytes of network traffic with minimal CPU overhead.

The Transport Layer Security (TLS) protocol combines these tools. It uses asymmetric cryptography during the handshake phase to authenticate the server and securely establish a shared secret.

Once the handshake completes, both sides switch to symmetric encryption to protect the application data stream.

A key challenge when securing web connections is minimizing the latency introduced by negotiating security settings. The evolution from TLS 1.2 to TLS 1.3 significantly reduced this latency.

The TLS 1.2 Handshake (2 RTT)

In TLS 1.2, establishing a secure connection requires two full network round-trips (Round Trip Times, or RTT) after the initial TCP connection is established:

1. ClientHello / ServerHello: The client sends its supported version, algorithms, and a random number. The server responds with its selection, certificate, and key parameters.
2. Key Exchange: The client validates the certificate, generates key parameters, and sends them to the server. Both sides compute the master secret.
3. Finish: Both sides exchange encrypted validation messages to verify the handshake has not been tampered with.

The TLS 1.3 Handshake (1 RTT)

TLS 1.3 optimizes this process by combining messages. It assumes the client will likely use modern cryptographic algorithms:

• In the first message (ClientHello), the client sends its supported algorithms along with its Elliptic Curve Diffie-Hellman key share guess.
• The server processes the request, selects the matching algorithm, returns its key share (ServerHello), and immediately begins sending encrypted certificates and handshakes.
• The handshake completes in a single round-trip, saving one full network transition.

The primary mechanism for session key exchange in modern TLS implementations is Elliptic Curve Diffie-Hellman Ephemeral (ECDHE).

• Diffie-Hellman: A mathematical method that allows two parties to agree on a shared secret over an insecure channel without transmitting the secret itself.
• Elliptic Curve (EC): Uses the algebraic structure of elliptic curves over finite fields to achieve the same security as classical Diffie-Hellman but with much smaller key sizes (e.g., a 256-bit EC key provides similar security to a 3,072-bit RSA key).
• Ephemeral (E): Means the key pair generated for the handshake is temporary and discarded immediately after the session concludes.

Using ephemeral key exchanges ensures forward secrecy. If a server's long-term private signing key is compromised in the future, the attacker cannot decrypt historical traffic recordings because each session used its own unique, temporary key pair.

To prevent man-in-the-middle attacks, the client must verify that the public key it receives during the handshake actually belongs to the target domain name. This identity verification relies on the Public Key Infrastructure (PKI).

A domain owner requests a digital certificate from a Certificate Authority (CA). The CA verifies the owner's domain control and issues a cryptographically signed certificate.

       ┌────────────────────────┐
       │   Root CA Certificate  │ (Pre-installed in OS/Browser root store)
       └───────────┬────────────┘
                   │
                   ▼ (Signs)
       ┌────────────────────────┐
       │ Intermediate CA Cert   │ (Used to sign domain certificates)
       └───────────┬────────────┘
                   │
                   ▼ (Signs)
       ┌────────────────────────┐
       │   Domain Certificate   │ (Sent by server during TLS Handshake)
       └────────────────────────┘

Operating systems and web browsers ship with pre-installed root certificates from trusted CAs in their root stores.

When the server sends its certificate, the client builds and verifies the intermediate signing chain up to a trusted root certificate in its local store.

To validate a certificate during the handshake, the client performs several checks:

1. Signature Chain Verification: The client validates each certificate signature in the chain using the issuer's public key.
2. Domain Name Matching: The client checks that the host name in the request matches the Subject Alternative Name (SAN) field in the certificate.
3. Validity Period: The client verifies that the current date falls between the certificate's activation and expiration dates.
4. Revocation Status: The client checks if the certificate has been revoked before its expiration date. This is done via a Certificate Revocation List (CRL) or by querying an Online Certificate Status Protocol (OCSP) responder. In high-performance setups, servers use OCSP Stapling to fetch the revocation status from the CA and attach it to the handshake, saving the client a round-trip query.

To avoid handshake overhead during subsequent connections, TLS supports two session resumption methods:

• Session IDs: The server caches session keys under a unique ID. If the client reconnects using this ID, the server can resume the session, bypassing the full handshake. This requires the server to maintain a state cache.
• Session Tickets (NST): The server encrypts the session state and sends it to the client as a ticket. When reconnecting, the client sends this ticket back to the server, which decrypts it to restore the session. This model is stateless for the server.

In TLS 1.3, session tickets enable 0-RTT (Zero-Round Trip Time) reconnection. The client sends encrypted application data (like an HTTP GET request) in its first handshake message along with the session ticket.

While 0-RTT eliminates latency, it is vulnerable to replay attacks, as an attacker could capture and replay the packet. To mitigate this risk, 0-RTT should only be used for safe, idempotent requests.

During the handshake, the client and server negotiate a cipher suite: a structured combination of cryptographic algorithms that will secure the session.

In TLS 1.2, a cipher suite specifies four algorithms:

        TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
            │      │          │       │
            │      │          │       └─ Hash (MAC/KDF)
            │      │          └─ Bulk Cipher (Symmetric)
            │      └─ Authentication (Asymmetric Signature)
            └─ Key Exchange (Asymmetric Negotiation)

In TLS 1.3, this structure is simplified to specify only the symmetric encryption parameters:

        TLS_AES_256_GCM_SHA384
            │       │     │
            │       │     └─ Hash (HKDF)
            │       └─ Authenticated Symmetric Cipher Mode
            └─ TLS Protocol Identifier

The authentication and key exchange algorithms are negotiated separately, reducing the number of cipher suite combinations.

Modern applications often multiplex different protocols over the same TCP port (such as serving both HTTP/1.1 and HTTP/2 over port 443). To negotiate which protocol to use without adding round-trips, TLS uses the Application-Layer Protocol Negotiation (ALPN) extension.

During the initial ClientHello, the client appends a list of supported application protocols (e.g., ["h2", "http/1.1"]). The server processes this list and declares its selection in the ServerHello.

This negotiation completes during the TLS handshake, allowing the client and server to begin exchanging application-specific frames immediately.

• The Transport Layer Security (TLS) Protocol Version 1.3 (RFC 8446) — The official IETF specification for TLS 1.3.
• High Performance Browser Networking — Chapter 4 by Ilya Grigorik covers TLS performance, resumption, and optimization.
• How HTTPS Works — A visual explanation of TLS handshakes and PKI.
• OCSP Stapling (RFC 6066) — Explains extensions for client certificate status request operations.