Headers and Body
From ASSPSMTP
Although it is common to refer to the "headers" as the entire raw version of an Internet e-mail, the header is actually the first of two major sections that comprise the entirety of an e-mail. The second section, commonly referred to as the body, is separated from the header by a single blank line.
On This Page |
The Format of Internet E-mail Messages
The format of Internet e-mail messages is defined in RFC 2822 and a series of RFCs: RFC 2045 through RFC 2049, collectively called Multipurpose Internet Mail Extensions (MIME). Although as of July 13 2005, RFC 2822 is technically a proposed IETF standard and the MIME RFCs are draft IETF standards, these documents are the de-facto standards for the format of Internet e-mail. See the RFC Index for a complete list.
Prior to the introduction of RFC 2822 in 2001 the format described by RFC 822 was the de facto standard for Internet e-mail for nearly two decades; it is still the official IETF standard. The IETF reserved the numbers RFC 2821 and RFC 2822 for the updated versions of RFC 821 (SMTP) and RFC 822, honoring the extreme importance of these two RFCs. RFC 822 was published in 1982 and was based on the earlier RFC 733.
Example "Raw" Message
The basics of an e-mail are structured as follows:
| Received: from X.X.X.X ([X.X.X.X] helo=smtp.domain.tld) by assp.domain.tld; 13 Dec 2006 15:31:25 -0500 X-MimeOLE: Produced By MTA Software Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: This is the subject line Date: Wed, 13 Dec 2006 15:31:25 -0500 Message-ID: <F545499C1A1D1F4AB54C@server.domain.tld> From: "Display Name" <sender@domain.tld> To: <recipient@domain.tld> | The Header is structured into fields such as summary, sender, receiver, and other information about the e-mail. There is one header field per line, and there are no blank lines in between header fields. | |
| A single blank line separator. | ||
| Hello world, This is the message body. It is comprised of unstructured text - meaning that almost anything goes. The SMTP session does not recognize the end of the message body until it encounters a "return-dot-return". | The Body of the message is unstructured text; and can include text, encapsulated pictures, HTML, etc. It sometimes contains a signature block at the end. | |
| . | The end of the body is indicated by concluding the message with "<CR><LF>.<CR><LF>". |
The Internet E-mail Header
The message header consists of various fields. Each header field has a name and a value. RFC 2822 specifies the precise syntax. Informally, the field name starts in the first character of a line, followed by a ":", followed by the value which is continued on non-null subsequent lines that have a space or tab as their first character. Field names and values are restricted to 7-bit ASCII characters. Non-ASCII values may be represented using MIME encoded words. Messages usually have at least four fields in the header:
- From: The e-mail address, and optionally the name of the sender of the message
- To: The e-mail addresses, and optionally the name(s) of the receivers(s) of the message
- Subject: A brief summary of the contents of the message
- Date: The local time and date when the message was originally sent
Note however that the "To" field in the header is not necessarily related to the addresses to which the message is delivered. The actual delivery list is supplied in the SMTP protocol, not extracted from the header content. The "To" field is similar to the greeting at the top of a conventional letter which is delivered according to the address on the outer envelope, regardless of what the greeting says. Also, note that the "From" field does not have to be the real sender of the e-mail message. It is very easy to fake the "From" field so that a message seems to be from a different e-mail address. It is possible to digitally sign e-mail, which is much harder to fake. Some Internet service providers do not relay e-mail claiming to come from a domain not hosted by them, but very few (if any) check to make sure that the person or even the e-mail address named in the "From" field is the one associated with the sender domain.
Other common header fields include:
- Cc: Courtesy copy (See also carbon copy.)
- Received: Tracking information generated by mail servers that have previously handled a message
- Content-Type: Information about how the message has to be displayed, usually a MIME type
Many e-mail clients present "Bcc" (Blind courtesy copy, recipients not visible in the "To" field) as a header field. Since the entire header is visible to all recipients, "Bcc" is not included in the message header. Addresses added as "Bcc" are only added to the SMTP delivery list, and do not get included in the message data.
Content Encoding
E-mail was originally designed for 7-bit ASCII. Much e-mail software is 8-bit clean but must assume it will be communicating with 7-bit servers and mail readers. The MIME standard introduced charset specifiers and two content transfer encodings to encode 8-bit data for transmission: quoted printable for mostly 7-bit content with a few characters outside that range and base64 for arbitrary binary data. The 8BITMIME extension was introduced to allow transmission of mail without the need for these encodings, but many mail transport agents still don't support it fully. For international character sets, Unicode is growing in popularity.
Headers Added by ASSP
ASSP adds its own set of headers to processed messages in order to help administrators determine the reasons for how an e-mail was processed. This information, along with details obtains from ASSP's log, can be invaluable tools for troubleshooting ASSP. Most of ASSP's headers are prefixed with "X-Assp", and include:
X-Assp-Envelope-From: The true designated sender X-Assp-PB-Score: Penalty Box valence score X-Assp-Received-DNSBL: DNS-based Block List (DNSBL) lookup response X-Assp-Received-SPF: Sender Policy Framework (SPF) lookup response X-Assp-Spam-Prob: Bayesian probability X-Assp-Spam-Reason: Verbose reason for spam classification X-Assp-Spam: Binary spam indicator (YES/NO) X-Intended-For: The true designated recipient
Standards Violations
Standards violations are a very common indicator of poorly crafted spam. ASSP has the ability to verify adherence to many of these standards, as well the ability to block e-mail that it finds is in violation of these standards. Features that check for various standards violations are:
How to Reveal the Full Headers and Body of an E-mail
The folks at SpamCop.net maintain an excellent resource addressing the common question of "How do I get my email program to reveal the full, unmodified email?" from many popular e-mail clients for numerous operating systems.
How to Use the Headers and Body to Determine If an E-mail is Spam
Aside from letting ASSP do its duty by proxying the e-mail sent through it, ASSP has a Mail Analyzer page for showing how an email would be analyzed and pre-processed in order to come up with an assigned spam probability. This can be an invaluable tool to determine how your configuration is working, and possibly to help determine what has been mis-configured.
