You Don’t Mess With The System Logs

wooden logs

The wooden logs to the left have nothing to do with system logs, other than being a metaphoric reference to the subject matter of this post; ensuring that your system logs are not messed with. In other words, protecting the integrity and confidentiality of system and event log messages, in a distributed logging architecture, and ensuring their central persistence for auditing, debugging, compliance monitoring and myriad other purposes.

The context of this post is a generic problem I had recently been grappling with, concerning multiple users having system administration rights on mission critical servers, with the mission in question involving the management of data of significant value. In my mind all relevant system actions, in particular those carried out by admin users, should be logged for auditing purposes. In fact, on second thought, keeping logs of all pertinent system messages and actions  on mission critical servers seems prudent, as a rule, regardless of who might or might not have administrative rights on a server.

My interest in syslog, and the underlying protocol, as defined in RFC 5424, stems from a Computer Science project I co-supervised involving the large scale monitoring of hundreds to thousands of routers, in what amounts to a distributed system logging architecture, with a central logging server. In this project, we used the open source edition of the syslog-ng logging system, created by BalaBit, with UDP as the transport layer protocol. In the cited project, the transmitted information, such as noise levels and system uptime, where not deemed to be sufficiently sensitive to warrant using Transport Layer Security (TLS) encyption nor mutual host and server authentication using X.509 certificates. When it comes the nature of the systems I am currently dealing with, a different approach is warranted and the technical white paper by BalaBit, entitled “Regulatory compliance and system logging”, appears to answer the questions I have had, and also raised issues I was not even thinking about.

One of the first thoughts I had when reading the white paper is that the cited regulations may be worth considering in terms of compliance regardless of whether one may be required to do so or not. For example U.S. or E.U. regulations do not apply in South Africa, but considering key regulations in those global power houses may be prudent in a general sense, as a best practice, and in my mind the primary motivation should not necessarily be the desire to do business in say the U.S., rather because the regulations may just be prudent to adhere to because they contain plain old good advice gained from hard lessons in the field.

BalaBit, in the cited white paper, lists these regulations and standards for possible compliance:

  1. Sarbanes-Oxley Act (SOX) – US legislation in force since 2002 concerning the regulation of financial practice and corporate governance.
  2. Basel II Accord – international banking standard for regulators to guard against financial and operational risks faced by banks.
  3. Health Insurance Portability and Accountability Act (HIPAA) – US standards for electronic health care transactions and national identifiers for health care providers / institutions.
  4. Payment Card Industry Data Security Standard (PCI-DSS) – Requirements for payment account data security by PCI Security Standards Council (lead by Visa, American Express, MasterCard et. al.).

It is worth noting that since starting to write this post, which I’ve been busy with for a while, I have encountered two of the above in separate business contexts, heightening my interest in syslog-ng and logging best practices in general. The first encounter involves SOX Section 404, with the context being a statement of compliance requested as a part of a mobile telecommunications operator RFP (request for proposal). The second encounter involves  the fourth standard in the above mentioned list, PCI-DSS, with the context being compulsory compliance due later this year relating to an e-commerce application that I have maintained for over three years.

So how and where to get started with protecting the integrity and confidentiality of system log messages? That is how does one best design and implement a system logging architecture? Furthermore, as a case-study, what should be done in terms of logging to ensure compliance with say PCI-DSS?

There is no short answer to the above. With regards to PCI-DSS and sections relating to log management and auditing, in terms of implementation in the first instance one would have the know the e-commerce application and host configuration very well. The cited Balabit white paper seems to offer good guidance when it comes to PCI-DSS, and so I won’t repeat their content here.

So how much does the commercial / premium edition of say syslog-ng, or alternatives, cost? I contacted BalaBit regarding the former, in short it depends on how many hosts you want to monitor and the pricing is between a couple of hundred US dollars and tens of thousands, it is best you contact them directly in this regard. They naturally also offer commercial support. A possible alternative to syslog-ng is Rsyslog, this slightly dated comparison page may be handy.

4 thoughts on “You Don’t Mess With The System Logs

  1. Hello Nico,
    I am really glad that you find the whitepaper useful, it was really fun to write it. Do you have similar regulations in South Africa? It seems that local, country-specific regulations are becoming increasingly common.

    BTW, the rsyslog-syslog-ng comparison you cited is way outdated – I think it applies to syslog-ng 2.0, and the more recent versions (3.0 and 3.1) have many new features. Though from the look of it, the rsyslog part of the comparison hasn’t been updated in a while either.



  2. Hi Robert,

    Thanks for the comment and article.

    About similar regulations in South Africa, PCI-DSS is being enforced, but PCI-DSS is a global standard.

    I am not familiar with South African regulations that are similar to SOX, Base II and HIPAA, however there may be some overlap with e.g. SOX and the South African Companies Act of 2008. The South African Electronic Communications and Transactions Act of 2002 may also be of interest.

    When it comes to local regulations, a more authoritative source on the e-regulatory side may be


Leave a Reply to nicodewet Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s