6587
HISTORIC
Transmission of Syslog Messages over TCP
Authors: R. Gerhards, C. Lonvick
Date: April 2012
Working Group: NON WORKING GROUP
Stream: IETF
Abstract
There have been many implementations and deployments of legacy syslog over TCP for many years. That protocol has evolved without being standardized and has proven to be quite interoperable in practice. This memo describes how TCP has been used as a transport for syslog messages. This document defines a Historic Document for the Internet community.
RFC 6587
HISTORIC
Internet Engineering Task Force (IETF) R. Gerhards
Request for Comments: 6587 Adiscon GmbH
Category: Historic C. Lonvick
ISSN: 2070-1721 Cisco Systems, Inc.
April 2012
<span class="h1">Transmission of Syslog Messages over TCP</span>
Abstract
There have been many implementations and deployments of legacy syslog
over TCP for many years. That protocol has evolved without being
standardized and has proven to be quite interoperable in practice.
This memo describes how TCP has been used as a transport for syslog
messages.
Status of This Memo
This document is not an Internet Standards Track specification; it is
published for the historical record.
This document defines a Historic Document for the Internet community.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see <a href="./rfc5741#section-2">Section 2 of RFC 5741</a>.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
<a href="https://www.rfc-editor.org/info/rfc6587">http://www.rfc-editor.org/info/rfc6587</a>.
IESG Note
The IESG does not recommend implementing or deploying syslog over
plain tcp, which is described in this document, because it lacks the
ability to enable strong security [<a href="./rfc3365" title=""Strong Security Requirements for Internet Engineering Task Force Standard Protocols"">RFC3365</a>].
Implementation of the TLS transport [<a href="./rfc5425" title=""Transport Layer Security (TLS) Transport Mapping for Syslog"">RFC5425</a>] is recommended so that
appropriate security features are available to operators who want to
deploy secure syslog. Similarly, those security features can be
turned off for those who do not want them.
<span class="grey">Gerhards & Lonvick Historic [Page 1]</span><span id="page-2" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal
Provisions Relating to IETF Documents
(<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/license-info</a>) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
<a href="#section-1">1</a>. Introduction ....................................................<a href="#page-3">3</a>
<a href="#section-2">2</a>. Conventions Used in This Document ...............................<a href="#page-5">5</a>
<a href="#section-3">3</a>. Message Transmission ............................................<a href="#page-5">5</a>
<a href="#section-3.1">3.1</a>. Character Encoding Scheme ..................................<a href="#page-5">5</a>
<a href="#section-3.2">3.2</a>. Session ....................................................<a href="#page-6">6</a>
<a href="#section-3.3">3.3</a>. Session Initiation .........................................<a href="#page-6">6</a>
<a href="#section-3.4">3.4</a>. Message Transfer ...........................................<a href="#page-6">6</a>
<a href="#section-3.4.1">3.4.1</a>. Octet Counting ......................................<a href="#page-7">7</a>
<a href="#section-3.4.2">3.4.2</a>. Non-Transparent-Framing .............................<a href="#page-7">7</a>
<a href="#section-3.4.3">3.4.3</a>. Method Change .......................................<a href="#page-8">8</a>
<a href="#section-3.5">3.5</a>. Session Closure ............................................<a href="#page-8">8</a>
<a href="#section-4">4</a>. Applicability Statement .........................................<a href="#page-8">8</a>
<a href="#section-5">5</a>. Security Considerations .........................................<a href="#page-9">9</a>
<a href="#section-6">6</a>. Acknowledgments .................................................<a href="#page-9">9</a>
<a href="#section-7">7</a>. References .....................................................<a href="#page-10">10</a>
<a href="#section-7.1">7.1</a>. Normative References ......................................<a href="#page-10">10</a>
<a href="#section-7.2">7.2</a>. Informative References ....................................<a href="#page-10">10</a>
<span class="grey">Gerhards & Lonvick Historic [Page 2]</span><span id="page-3" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>
The Standards-Track documents in the syslog series recommend using
the syslog protocol [<a href="./rfc5424" title=""The Syslog Protocol"">RFC5424</a>] with the TLS transport [<a href="./rfc5425" title=""Transport Layer Security (TLS) Transport Mapping for Syslog"">RFC5425</a>] for
all event messages. The authors of this document wholeheartedly
support that position and only offer this document to describe what
has been observed with legacy syslog over TCP, which appears to still
be widely used.
Two primary format options have been observed with legacy syslog
being transported over TCP. These have been called "non-transparent-
framing" and "octet-counting". The non-transparent-framing mechanism
has some inherent problems.
Diagram 1 shows how all of these syslog transports relate to each
other. In this diagram, three originators are seen, labeled A, B,
and C, along with one collector. Originator A is using the TCP
transport that is described in this document. Originator B is using
the UDP transport, which is described in [<a href="./rfc5426" title=""Transmission of Syslog Messages over UDP"">RFC5426</a>]. Originator C is
using the TLS transport, which is described in [<a href="./rfc5425" title=""Transport Layer Security (TLS) Transport Mapping for Syslog"">RFC5425</a>]. The
collector is shown with the capability to accept all three
transports.
<span class="grey">Gerhards & Lonvick Historic [Page 3]</span><span id="page-4" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
+---------------------+
| Originator A |
|---------------------|
| syslog application |
| |
|---------------------|
| syslog transport |
| TCP |
|---------------------|
v
|
/ +---------------------+
/ | Originator B |
/ |---------------------|
/ +----------------------+ | syslog application |
/ | Collector | | |
| |----------------------| |---------------------|
| | syslog application | | syslog transport |
| | | | UDP |
| |----------------------| |---------------------|
| | syslog transport | v
| | TCP | TLS | UDP | |
| |----------------------| |
| ^ ^ ^ |
| | | | |
\ / | \ /
--------- | ------------------
|
|
| +---------------------+
| | Originator C |
| |---------------------|
| | syslog application |
| | |
| |---------------------|
| | syslog transport |
| | TLS |
| |---------------------|
| v
\ /
---------------
Diagram 1. Syslog Layers
<span class="grey">Gerhards & Lonvick Historic [Page 4]</span><span id="page-5" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Conventions Used in This Document</span>
The terminology defined in <a href="./rfc5424#section-3">Section 3 of [RFC5424]</a> is used throughout
this specification. The reader should be familiar with that to
follow this discussion.
This document also references devices that use the syslog message
format as described in [<a href="./rfc3164" title=""The BSD Syslog Protocol"">RFC3164</a>]. Devices that continue to use that
message format (regardless of transport) will be described as "legacy
syslog devices". Similarly, devices that use the message format as
described in [<a href="./rfc5424" title=""The Syslog Protocol"">RFC5424</a>] will be described as "standardized syslog
devices".
<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Message Transmission</span>
Syslog is simplex in nature. It has been observed that
implementations of syslog over TCP also do not use any back-channel
mechanism to convey information to the transport sender and,
consequently, do not use any application-level acknowledgement for
syslog signaling from receiver to sender. Message receipt
acknowledgement, reliability, and flow control are provided by the
capabilities of TCP.
<span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. Character Encoding Scheme</span>
Syslog over TCP messages contain no indication of the coded character
set (e.g., [<a href="#ref-US-ASCII" title=""Coded Character Set -- 7-bit American Standard Code for Information Interchange, ANSI X3.4-1986"">US-ASCII</a>] or [<a href="#ref-UNICODE" title=" 2011. ISBN 978-1-936213-01-6)">UNICODE</a>] ) or character encoding scheme
(e.g., so-called "7-bit ASCII" or UTF-8 [<a href="./rfc3629" title=""UTF-8, a transformation format of ISO 10646"">RFC3629</a>]) in use. In these
messages, the predominant approach has been to include characters
only from the ASCII repertoire (i.e., %d32 to %d126 inclusive) using
the "Network Virtual Terminal" (NVT) encoding [<a href="./rfc5198" title=""Unicode Format for Network Interchange"">RFC5198</a>].
The message header usually contains characters only from the ASCII
repertoire, in the NVT encoding. This has been observed even in
cases where a different encoding (e.g., UTF-8) has been used for the
MSG part. However, characters outside the ASCII range have been seen
inside the header. In that case, some syslog applications have been
known to experience problems processing those messages.
In some cases, it has been observed that characters outside of the
ASCII range are often being transformed by receivers in an effort to
"escape control characters". Some receiver implementations simply
drop those characters. This is considered to be a poor practice, as
it causes problems with coded character sets other than ASCII and
character encodings other than NVT, most notably the UTF-8 encoding
of Unicode.
<span class="grey">Gerhards & Lonvick Historic [Page 5]</span><span id="page-6" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
It has also been observed that relays will forward messages using the
character encoding schemes of messages they receive. In the case
where two different senders are using different character encoding
schemes, the relay will forward each message to a collector in that
character encoding. The collector of these messages will have to be
prepared to receive messages from the same relay with different
encodings.
<span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. Session</span>
Like most other protocols, the syslog transport sender is the TCP
host that initiates the TCP session. After initiation, messages are
sent from the transport sender to the transport receiver. No
application-level data is transmitted from the transport receiver to
the transport sender. The roles of transport sender and receiver
seem to be fixed once the session is established.
When it has been observed, if an error occurs that cannot be
corrected by TCP, the host detecting the error gracefully closes the
TCP session. There have been no application-level messages seen that
were sent to notify the other host about the state of the host syslog
application.
<span class="h3"><a class="selflink" id="section-3.3" href="#section-3.3">3.3</a>. Session Initiation</span>
The TCP host acting as a syslog transport receiver listens to a TCP
port. The TCP transport sender initiates a TCP session to the syslog
transport receiver as specified in [<a href="./rfc0793" title=""Transmission Control Protocol"">RFC0793</a>].
This protocol has no standardized port assignment. In practice,
network administrators generally choose something that they feel will
not conflict with anything else active in their networks. This has
most often been either TCP/514, which is actually allocated to
another protocol, or some variant of adding 514 to a multiple of
1000. Please see <a href="#section-4">Section 4</a> for more information.
<span class="h3"><a class="selflink" id="section-3.4" href="#section-3.4">3.4</a>. Message Transfer</span>
Syslog over TCP has been around for a number of years. Just like
legacy syslog over UDP, different implementations exist. The older
method of non-transparent-framing has problems. The newer method of
octet-counting is reliable and has not been seen to cause problems
noted with the non-transparent-framing method.
In both of these methods, during the message transfer phase, the
syslog transport sender sends a stream of messages to the transport
receiver. These are sent in sequence and one message is encapsulated
<span class="grey">Gerhards & Lonvick Historic [Page 6]</span><span id="page-7" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
inside each TCP frame. Either of the TCP hosts may initiate session
closure at any time as specified in <a href="./rfc0793#section-3.5">Section 3.5 of [RFC0793]</a>. In
practice, this is often seen after a prolonged period of inactivity.
<span class="h4"><a class="selflink" id="section-3.4.1" href="#section-3.4.1">3.4.1</a>. Octet Counting</span>
This framing allows for the transmission of all characters inside a
syslog message and is similar to the method used in [<a href="./rfc5425" title=""Transport Layer Security (TLS) Transport Mapping for Syslog"">RFC5425</a>]. A
transport receiver uses the defined message length to delimit a
syslog message. As noted in [<a href="./rfc3164" title=""The BSD Syslog Protocol"">RFC3164</a>], the upper limit for a legacy
syslog message length is 1024 octets. That length has been expanded
for standardized syslog.
It can be assumed that octet-counting framing is used if a syslog
frame starts with a digit.
All syslog messages can be considered to be TCP "data" as per the
Transmission Control Protocol [<a href="./rfc0793" title=""Transmission Control Protocol"">RFC0793</a>]. The syslog message stream
has the following ABNF [<a href="./rfc5234" title=""Augmented BNF for Syntax Specifications: ABNF"">RFC5234</a>] definition:
TCP-DATA = *SYSLOG-FRAME
SYSLOG-FRAME = MSG-LEN SP SYSLOG-MSG ; Octet-counting
; method
MSG-LEN = NONZERO-DIGIT *DIGIT
NONZERO-DIGIT = %d49-57
SYSLOG-MSG is defined in the syslog protocol [<a href="./rfc5424" title=""The Syslog Protocol"">RFC5424</a>] and may
also be considered to be the payload in [<a href="./rfc3164" title=""The BSD Syslog Protocol"">RFC3164</a>]
MSG-LEN is the octet count of the SYSLOG-MSG in the SYSLOG-FRAME.
<span class="h4"><a class="selflink" id="section-3.4.2" href="#section-3.4.2">3.4.2</a>. Non-Transparent-Framing</span>
The non-transparent-framing method inserts a syslog message into a
frame and terminates it with a TRAILER character. The TRAILER has
usually been a single character and most often is ASCII LF (%d10).
However, other characters have also been seen, with ASCII NUL (%d00)
being a prominent example. Some devices have also been seen to emit
a two-character TRAILER, which is usually CR and LF.
The problem with non-transparent-framing comes from the use of a
TRAILER character. In that, the traditional TRAILER character is not
escaped within the message, which causes problems for the receiver.
<span class="grey">Gerhards & Lonvick Historic [Page 7]</span><span id="page-8" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
For example, a message in the style of [<a href="./rfc3164" title=""The BSD Syslog Protocol"">RFC3164</a>] containing one or
more LF characters may be misinterpreted as multiple messages by the
receiving syslog application.
The ABNF for this is shown here:
TCP-DATA = *SYSLOG-FRAME
SYSLOG-FRAME = SYSLOG-MSG TRAILER ; non-transparent-framing
; method
TRAILER = LF / APP-DEFINED
APP-DEFINED = 1*2OCTET
SYSLOG-MSG is defined in the syslog protocol [<a href="./rfc5424" title=""The Syslog Protocol"">RFC5424</a>] and may
also be considered to be the payload in [<a href="./rfc3164" title=""The BSD Syslog Protocol"">RFC3164</a>]
A transport receiver can assume that non-transparent-framing is used
if a syslog frame starts with the ASCII character "<" (%d60).
<span class="h4"><a class="selflink" id="section-3.4.3" href="#section-3.4.3">3.4.3</a>. Method Change</span>
In legacy implementations, it has been observed that the framing may
change on a frame-by-frame basis. This is probably not a good idea,
but it's been seen.
<span class="h3"><a class="selflink" id="section-3.5" href="#section-3.5">3.5</a>. Session Closure</span>
The syslog session is closed when one of the TCP hosts decides to do
so. It then initiates a local TCP session closure. Following TCP
[<a href="./rfc0793" title=""Transmission Control Protocol"">RFC0793</a>], it doesn't need to notify the remote TCP host of its
intention to close the session, nor does it accept any messages that
are still in transit.
<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Applicability Statement</span>
Again it must be emphasized that the Standards-Track documents in the
syslog series recommend using the TLS transport [<a href="./rfc5425" title=""Transport Layer Security (TLS) Transport Mapping for Syslog"">RFC5425</a>] to
transport syslog messages. This document does not recommend that new
implementations or deployments use syslog over TCP except for the
explicit purpose of interoperating with existing deployments.
One of the major problems with interoperability with this protocol is
that there is no consistent TCP port assigned. Most of the
successful implementations have made the selection of a port a user-
configurable option. The most frequently observed port for this has
been TCP/514, which is actually assigned to the Shell protocol.
<span class="grey">Gerhards & Lonvick Historic [Page 8]</span><span id="page-9" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
Operators must carefully select which port to use in their deployment
and be prepared to encounter different default port assignments in
implementations.
There are several advantages to using TCP: flow control, error
recovery, and reliability, to name a few. These reasons, and the
ease of programming, have led people to use this transmission
protocol to transmit syslog.
One potential disadvantage is the buffering mechanism used by TCP.
Ordinarily, TCP decides when enough data has been received from the
application to form a segment for transmission. This may be adjusted
through timers; but still, some application data may wait in a buffer
for a relatively long time. Syslog data is not normally time-
sensitive, but if this delay is a concern, the syslog transport
sender may utilize the PUSH Flag as described in [<a href="./rfc0793" title=""Transmission Control Protocol"">RFC0793</a>] to have
the sending TCP immediately send all buffered data.
<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Security Considerations</span>
This protocol makes no meaningful provisions for security. It lacks
authentication, integrity checking, and privacy. It makes no
provision for flow control or end-to-end confirmation of receipt,
relying instead on the underlying TCP implementations to approximate
these functions. It should not be used if deployment of [<a href="./rfc5425" title=""Transport Layer Security (TLS) Transport Mapping for Syslog"">RFC5425</a>] on
the systems in question is feasible.
<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. Acknowledgments</span>
The authors wish to thank David Harrington, Tom Petch, Richard
Graveman, and all other people who commented on various versions of
this proposal. We would also like to thank Peter Saint-Andre for
clarifying character encodings.
The authors would also like to thank Randy Presuhn for being our
reviewer and document shepherd, and a special thanks to Dan Romascanu
for his support and guidance.
<span class="grey">Gerhards & Lonvick Historic [Page 9]</span><span id="page-10" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. References</span>
<span class="h3"><a class="selflink" id="section-7.1" href="#section-7.1">7.1</a>. Normative References</span>
[<a id="ref-RFC0793">RFC0793</a>] Postel, J., "Transmission Control Protocol", STD 7,
<a href="./rfc793">RFC 793</a>, September 1981.
[<a id="ref-RFC3365">RFC3365</a>] Schiller, J., "Strong Security Requirements for Internet
Engineering Task Force Standard Protocols", <a href="https://www.rfc-editor.org/bcp/bcp61">BCP 61</a>,
<a href="./rfc3365">RFC 3365</a>, August 2002.
[<a id="ref-RFC5198">RFC5198</a>] Klensin, J. and M. Padlipsky, "Unicode Format for Network
Interchange", <a href="./rfc5198">RFC 5198</a>, March 2008.
[<a id="ref-RFC5234">RFC5234</a>] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, <a href="./rfc5234">RFC 5234</a>, January 2008.
[<a id="ref-RFC5424">RFC5424</a>] Gerhards, R., "The Syslog Protocol", <a href="./rfc5424">RFC 5424</a>,
March 2009.
[<a id="ref-RFC5425">RFC5425</a>] Miao, F., Ma, Y., and J. Salowey, "Transport Layer
Security (TLS) Transport Mapping for Syslog", <a href="./rfc5425">RFC 5425</a>,
March 2009.
[<a id="ref-US-ASCII">US-ASCII</a>] ANSI, "Coded Character Set -- 7-bit American Standard
Code for Information Interchange, ANSI X3.4-1986", 1986.
<span class="h3"><a class="selflink" id="section-7.2" href="#section-7.2">7.2</a>. Informative References</span>
[<a id="ref-RFC3164">RFC3164</a>] Lonvick, C., "The BSD Syslog Protocol", <a href="./rfc3164">RFC 3164</a>,
August 2001.
[<a id="ref-RFC3629">RFC3629</a>] Yergeau, F., "UTF-8, a transformation format of ISO
10646", STD 63, <a href="./rfc3629">RFC 3629</a>, November 2003.
[<a id="ref-RFC5426">RFC5426</a>] Okmianski, A., "Transmission of Syslog Messages over
UDP", <a href="./rfc5426">RFC 5426</a>, March 2009.
[<a id="ref-UNICODE">UNICODE</a>] The Unicode Consortium. The Unicode Standard, Version
6.0.0, (Mountain View, CA: The Unicode Consortium,
2011. ISBN 978-1-936213-01-6),
<<a href="http://www.unicode.org/versions/Unicode6.0.0/">http://www.unicode.org/versions/Unicode6.0.0/</a>>.
<span class="grey">Gerhards & Lonvick Historic [Page 10]</span><span id="page-11" ></span>
<span class="grey"><a href="./rfc6587">RFC 6587</a> Transmission of Syslog Messages over TCP April 2012</span>
Authors' Addresses
Rainer Gerhards
Adiscon GmbH
Mozartstrasse 21
Grossrinderfeld, BW 97950
Germany
EMail: [email protected]
Chris Lonvick
Cisco Systems, Inc.
12515 Research Blvd.
Austin, TX 78759
USA
EMail: [email protected]
Gerhards & Lonvick Historic [Page 11]
Annotations
Select text to annotate