|
By Corey Nachreiner, Network Security Analyst, WatchGuard Technologies Logging provides us with the bread crumb trail we need when tracking down pesky yet elusive computer problems. Without good logs, IT workers everywhere would quickly turn into Hansel and Gretel, lost in a dark forest of malfunctioning PCs. Accurate and detailed logs are even more important from a security perspective. While Crime Scene Investigators (CSIs) might use strands of hair, tire markings, or even insects to solve physical crimes, computer forensic experts rely almost entirely on the messages recorded in various log files. If anyone breaks into your network, log entries can tell you how they did it, and how to prevent them from doing it again. With such great arguments for logging, I'm surprised at how poorly some device manufacturers implement it. Many logging approaches use proprietary messaging formats that don't play well with others. Having to learn ten different log message formats is a pain, let alone having to find where each application stores them. Add a network environment with tens to hundreds of computers, and you can spend the whole day running from computer to computer, troubleshooting a network problem. Why can't someone just invent a standard, centralized logging service that works with most applications? Good news: someone has! It's called syslog, a standardized logging service you can use on almost any computer platform. This article introduces you to the basics of syslog, including its pros and cons, and how to read its log entries. Who knows? After reading this article, you might even decide to use it as your centralized logging protocol. Born from the Founder of E-mail According to Wikipedia, Eric Allman concocted syslog as a logging service initially for his program, Sendmail. However, other developers liked it so much, they used it in their own applications until it eventually became the de facto logging standard for Unix and Linux. The Internet Engineering Task Force (IETF) cemented the syslog standard in 2001, when they released the first syslog Request for Comment (RFC). If you wish to explore syslog in technical detail, you can read its official description in RFC 3164 and 3195. Today, almost all well-developed network services, applications and devices can use the syslog protocol to send log messages. Syslog provides local and network-based logging services with fine control over how you log, without eating up heavy system resources. Because it's been adopted widely and implemented on lots of platforms, it also offers excellent centralized logging capabilities. The Pros and Cons of Syslog A quick list of the advantages of syslog includes the following. - Many operating systems (OS) and applications directly support syslog or have third-party applications that can help you use syslog.
- The syslog protocol is dead simple. Developers can easily understand and implement its message format and packet design.
- Syslog sends network messages using User Datagram Protocol UDP (typically on port 514), a very resource-friendly network protocol. No muss no fuss; it's quite lightweight.
- Despite its simplicity, syslog provides flexibility. Where do you want your logs to show up? Syslog enables applications to send log messages to a computer's console; to a user-selectable file; over a network; and even to individual users or groups.
- Applications can give each log message one of eight levels of priority. You can set up applications to do different things with a log message, depending on the message's severity. Low priority messages can be relegated to some obscure folder for later review, while highly suspicious messages can pop up on your most-used console.
That said, syslog does have a few problems -- many inversely related to its benefits. The old maxim says that simplicity and security rarely travel hand-in-hand, and syslog follows suit as a simple yet insecure protocol. All syslog messages transmit in clear text, without authentication or verification. This means anyone sniffing your network traffic can see your network-based log messages. By exploiting this weakness, a local hacker could gather tons of intelligence for a network attack. Since syslog doesn't verify or authenticate messages, attackers can easily inject fake messages into your logs. An attacker might inject such spoofed logs to confuse you and cover up her real attack. In fact, the logs store as simple text files. Although you can protect your log files from unauthorized users by restricting their read access, an attacker who elevated his privileges could easily read, modify or delete these log files using a simple text editor. Finally, since syslog uses the UDP protocol, log delivery isn't guaranteed. If a network problem interrupts or corrupts a log message, UDP-based logging packets never arrive at your log server and are lost forever. Don't sweat these negatives too much, though. You can overcome most of them fairly easily. For example, you can force encryption on your log messages by port forwarding them through SSH or by transmitting them through applications like Stunnel. You can also encrypt the text-based files whenever you rotate your logs, protecting them from idle snooping. Finally, syslog RFC 3195 describes how to send syslog messages over TCP, which ensures more reliable delivery than UDP. Many newer syslog daemons, such as syslog-ng, support this new TCP functionality, which guarantees none of your syslog messages get lost over your network. All Syslog Messages Read the Same One of the coolest things about syslog is that all syslog messages use the same log format, regardless of the platform the message comes from or the application that sends it. Syslog's standard log line is formatted like this: [date] [time] [system] [tag]: [message] Here's what each of those fields mean: - I'm sure you find the date and time sections of this format self-explanatory. They act as the log message's timestamp.
- The system section specifies which computer or device sent the syslog message. It may show an IP address or a hostname. For instance, local messages might display "localhost" here.
- The tag section tells which application or process generated the message. For instance, it could display "sendmail," "httpd," or even "doom." Technically, the syslog protocol considers this tag as part of the message's content. Nonetheless, it displays separately in individual log lines.
- Finally, the message section is the meat of the log message. This message can say anything a developer wants it to, restricted only by length. Syslog message content combined with the packet headers can't exceed a maximum of 1024 bytes.
That's it! You just learned how to read syslog messages. Now, let's practice interpreting a few quick examples: Jun 2 21:41:18 localhost ufs: alloc /: file system full In this message, the Unix/Universal File System (UFS) warns you that at 9:48pm on June 2, the root file system (/) ran out of space when trying to write a file. Another example: Apr 21 14:23:52 192.168.2.34 BrthDyApp: Happy Birthday Dorkwad! This message, generated on April 21, got sent from a program named "BrthDyApp" running on a computer with the IP address, 192.168.2.34. As you can see, the actual message can say whatever the application's developer wants it to. Anyone who has learned English knows that there is an exception to nearly every rule. Now that you know what normal syslog messages look like, I need to point out one small exception. Some implementations of syslog support a feature called ID Generation which, when enabled, injects extra information into each syslog log message. Syslog messages with ID Generation enabled appear as follows: [date] [time] [system] [tag]: [ID] [message] The ID section of a syslog line includes attributes such as the message's Facility and Severity, which I explain in an upcoming article about syslog configuration. For now, when you read syslog messages you can generally tell what's going on even if you ignore whatever appears between the tag and the message fields. Simple, Standardized, Centralized Logging! Sign Me Up If you crave an easy-to-use, affordable technology that can centralize log files between networked, cross-platform computers and many different applications, you really ought to try syslog. With its simple, lightweight design, its flexibility, all the flavors it comes in, and its huge global user base, syslog provides one of the few viable possibilities for unified logging. You can overcome its few security weaknesses fairly easily using popular (often, free) third party applications. If this article interested you, it gets better from here. In upcoming articles, I'll explain how to install syslog, and finally, how it can interact with your Firebox. Whether you want to become the all-knowing, all-seeing guru of your computer network, or you simply want to find your way home through a dark forest of confusing log formats -- give syslog a try! ## References Where to get syslog - Different Unix, Linux and Solaris distributions package their own implementations of syslog. See your distro for details.
- Syslog-ng. Next Generation (NG) syslog server with reliable network delivery.
- Kiwi Syslog Daemon. (Popular and free Windows syslog server).
Useful syslog articles
Related Items:
|
