HPlogo Sendmail 8.13.3: Secure Mailing Solution

TLS/SSL Support

» 

Technical documentation

Complete book in PDF

 » Table of Contents

Sendmail 8.13.3 uses the Transport Layer Security (TLS) and the Secure Socket Layer (SSL) to encrypt not only the user name and password, but the entire mail transmission. To signal the beginning of an encrypted TLS conversation, Sendmail 8.13.3 uses the STARTTLS command within an SMTP conversation.

STARTTLS feature is an extension of the SMTP service that enables an SMTP server and client to use the transport layer security in providing private and authenticated communication over the Internet. It enables the SMTP agents to protect some or all of their communications from eavesdroppers and attackers.

The STARTTLS feature offers the following benefits:

  • Verifies the identity of the client and server in a mail transmission.

  • Authenticates a user for relaying through a mail server.

  • Encrypts mail transmissions.

  • Encrypts transmissions between two mail servers over the Internet.

Sendmail 8.13.3 relies on the OpenSSL implementation for cryptographic algorithms. The cryptographic algorithms used for encrypting messages are completely transparent to Sendmail 8.13.3.

OpenSSL is an open source implementation of the SSL and TLS protocols. The version of OpenSSL available on the HP-UX 11i v1 and HP-UX 11i v2 operating systems is OpenSSL A.00.09.07l. The version of OpenSSL available on the HP-UX 11i v3 operating system is OpenSSL A.00.09.08d. Both OpenSSL A.00.09.07l and A.00.09.08d include a general-purpose cryptography library and implementation of the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols.

This section addresses the following topics:

Cryptography Algorithm

The TLS subsystem uses the following components to provide services, such as integrity checking, authentication, and confidentiality:

  • Private key algorithms, or symmetrical cryptography. This component uses a shared secret and the key, for both encryption and decryption of a message. Input data is mathematically processed using the algorithm and the key, to produce the ciphertext output that must be decrypted for the recipient. Commonly used private key algorithms include DES, Blowfish, AES, and IDEA.

  • Public key algorithms. These algorithms use two mathematically related keys to separate the processes of encryption and decryption. By using functions that are easy to perform in one direction but difficult to perform in the opposite direction, the two keys provide a high level of security if large numbers are used. Commonly used public key algorithms include RSA, El Gamal, and Diffie-Hellman.

    While establishing a TLS session, you can use public key cryptography to exchange a session key that is used in a private key algorithm. You can also use these public keys to authenticate the server and, if required, the client, and to provide session-level encryption and confidentiality for the entire session.

  • Hash algorithms. These algorithms are a set of one-way functions that accept a variable length input, and, after mathematical processing, produce a fixed length output. The transformations of the data produce a fingerprint of the input. The minor changes to the input appear as large changes in the output. Popular hash algorithms include SHA-1, MD5, and RIPEMD.

    Hash algorithms are used for integrity checking; that is, to ensure that data is not tampered during transmission.

Certificates and Authorities

A certificate is a collection of information that uniquely identifies a client or a server. It includes descriptive fields, such as the name of an organization and its location, as well as cryptographic information, such as keys and signatures.

The private key of an asymmetrical key pair can be used to sign the content that, when decrypted using the public key, establishes the signature. This signature can be used to offer proof of identity. The public key infrastructures (PKI) use a hierarchy of trustworthiness for the validation of identities, in addition to signing certificates and keys. This is in contrast to the web of trust used in pretty good protection (PGP), which has no central authority.

The central authority in a PKI issues a Certificate Authority (CA), a definitive certificate that contains the information and the public key of the server. This CA can be used to sign other certificates, by signing the public key of a requesting body, such as your server, with the private key. The trust in identity is transitive, because the CA is recognized by all the involved parties as authoritative: "I trust the CA, and the CA says that it is you, so it must be true."

Certificates can be revoked because of expiration or compromise in security. To do this, the issuing body provides a certificate revocation list (CRL) that identifies the certificates to be invalidated. This is also trusted because strong proof is provided through the trust mechanisms.

Certificates are available in different formats, though PEM is the most widely used format. The PEM encoding is an ASCII text representation of the binary data in the ASN.1 format. The X.509 standard defines the distinguished name (DN) format used in these certificates.

A certificate contains the following information that accompanies the cryptographic keys:

  • Common name (CN) being certified

  • Organization (O) associated

  • Organizational unit (OU), such as a department within an organization

  • City or location (L) where an organization is located

  • State or province (SP) where the city is located

  • Country (C) in the ISO (International Organization for Standardization) format (such as U.S.)

The DN is a combination of the different certificate information. The PEM-encoded certificate contains this information along with the DN of the issuer, the validity period of the certificate, various administration information, such as a serial number of the certificate, and any other required information, such as Netscape-specific tags. These certificates are used to establish the identity and trustworthiness of the presenter, such as a server or a client. These certificates are also used to authenticate the connecting party and to take appropriate action, such as allowing a connection to proceed, and mail relaying, or entry into a network. You can either use the commercial TLS/SSL certificates (certs) to verify the identity of the Sendmail 8.13.3 server, or create your own certificates for the Sendmail 8.13.3 servers.

RFC 2487 (SMTP Service Extension for Secure SMTP over TLS) describes how to use the method for using TLS/SSL as a popular mechanism for enhancing TCP communications with privacy and authentication. In most cases, the communication passes through one or more routers that are not controlled or trusted by either entity. Such untrusted routers can be a serious security threat. If SMTP agents are able to authenticate each others' identities, the server can allow only communications from other SMTP agents it is aware of or it can treat messages received from the known agents differently, compared to messages received from unknown agents.

© Hewlett-Packard Development Company, L.P.