Logging vulnerabilities

Logging is the process of creating output logs based on actions within an application or service with the idea to provide information to understand possible issues like crashes or performance problems. Logging vulnerabilities are simply security vulnerabilities that arise from the process of logging. Some common examples include:

• Publicly exposed log files
• Logging of sensitive information
• Insufficient logging
• Ability to poison log entries
• Blocking (or overloading) logging systems

In this lesson, you will learn about vulnerabilities stemming from bad logging practices and how to protect your applications against them. We will step into the shoes of a developer-turned-hacker, Jessica, who gains access to credit card information by gaining access to a third-party logging service.

Jessica is an extremely talented developer who has just been made redundant by her previous employer. The whole process has been very stressful!

Late one night, Jessica was scrolling through JobberTheHutt, a popular job board for remote tech workers, in search of a new opportunity. She often uses an intercept proxy application like Burp Suite or OWASP ZAP while browsing websites, because it allows her to see how other web applications work on a lower level. This often gives her ideas for her own development projects.

JobberTheHutt had a premium membership which allowed you to view additional details about the jobs being advertised.

First, let's break down what happened in the story above:

• Sensitive information was being logged to a third-party logging service
• An attacker breached that service
• The attacker is now able to view the sensitive information

Of course, if the information being logged was not sensitive, this attack would not have been nearly as severe.

An example of a logging solution that may inadvertently log sensitive information using NodeJS Express:

An example of a logging solution that may inadvertently log sensitive information using Python Express:

The offending line is:

Because the developer has just opted to log the whole request body, the credit card details will be included in the logs.

An example of a logging solution that may inadvertently log sensitive information using PHP:

The offending line is:

Because the developer has just opted to log the whole POST, the credit card details will be included in the logs.

In an application where the logging is purposely created, it would feel unintuitive for a developer to write code that logs credit card numbers or passwords.There are many widely-used third-party application monitoring suites today, many of which will offer a solution to log everything that is submitted to an application by a user to provide maximum insight into the way that your users are navigating through your applications. Most of these offerings will include some way to block the capture of personally identifiable information (PII), but these tools are not aware of how your application functions, so PII can often end up being logged regardless. It is the responsibility of the application developer to ensure that PII is explicitly excluded from logging.

A common vulnerability in the wild is the public exposure of log files. The most obvious example of this is when web server logs are being stored in a file that is inside the public web root. For example, when configuring the Apache2 configuration file, someone might specify the location of the error log with the following line:

ErrorLog /var/www/html/logs/error.log

If this was the case, an attacker could easily view the access logs by simply navigating to example.com/logs/error.log in their browser. Even simple information like IP address data and frequency of visits could have implications like providing business competitors with an advantage, but basic access logs will typically contain even more sensitive information like password reset tokens.

Another common way to expose log files is through misconfigurations in cloud storage buckets. For larger-scale, high-traffic applications, logs are often sent to cloud storage buckets instead of the web server's disk because it is easier, more efficient, and more scalable. Unfortunately, these storage buckets are commonly misconfigured, leaving them publicly accessible.

Visibility and timely response are two of the most fundamental components of a good security program. While insufficient logging does not present much exploitability by itself, you could argue that it is partly to blame for nearly every major security incident because if comprehensive logging and monitoring was taking place, it could mitigate or reduce the extent of an attack by facilitating an immediate response. For this reason, sufficient logging is important.

All sensitive or high-value transactions should be logged. The exact things that should be logged will vary based on the function of the application, but some common examples include:

• Logins
• Failed logins
• Password changes
• Forgot password requests
• Account updates
• Money transfers
• Purchases
• Account creations

Logs are often referred to as a source of absolute truth when auditing or investigating a security incident. For this reason, advanced attackers will often cover their tracks by tampering with log entries, or abuse the trust in log entries by creating fake ones.

A basic example of this is the ability to inject new line characters into a basic log file that would normally have a single log entry on each line. For example, let's say that we are logging payment transactions, the log file might look something like this:

Where the number is the payment ID, and the last column is true/false depending on whether the payment was successful or not. Let's take a look at what some basic code might look like to produce this log:

Now consider what might happen if an attacker manually tampered with their orderNumber and added some new lines:

Of course, instead of adding the new lines like this, we'd need to URL encode them:

2884%0a%0dPAYMENT%201337%20true%0a%0dPAYMENT%20

In this case, a fake line would be added to the log file without any transaction actually taking place, with the payment ID of 1337.

Some general advice for ensuring good logging practices:

• Ensure that all sensitive or high-value events are logged and monitored for suspicious activity
• Ensure that the method used for logging can not result in log entries being faked, tampered with, or removed
• Ensure that logs are not ever accessible except to those who need to see them, like your security team
• If you're using a third-party logging/analytics platform, double check that sensitive details like PII are not being logged

Resolving the vulnerable code example above is also quite simple. Currently, this line logs the whole req.body:

log("payment", success, JSON.stringify(req.body)

Of course, this includes sensitive details which should not be logged. We can easily avoid this by hand-picking the data that should be logged, like this:

log(JSON.stringify({payment: success, orderNumber: req.body.orderNumber}))

Currently, the vulnerable code example appends text to a file, including user input:

There are many ways to resolve this, one of them would be to store the line as JSON data, similar to above:

Now when someone tries to inject newlines into the log file, it will display as a literal "\n".

Of course, a better solution for logging is to use an existing NodeJS logging library, rather than rolling your own. Some examples of popular logging libraries include Winston, Bunyan and Pino.

Resolving the vulnerable code example above is also quite simple. Currently, this line logs the whole request.data:

log("payment", success, json.dumps(request.data))

Of course, this includes sensitive details which should not be logged. We can easily avoid this by hand-picking the data that should be logged, like this:

log(json.dumps({'payment': success, 'orderNumber': request.form.get('orderNumber')}))

Currently, the vulnerable code example appends text to a file, including user input:

There are many ways to resolve this, one of them would be to store the line as JSON data, similar to above:

Now when someone tries to inject newlines into the log file, it will display as a literal "\n".

The default logging library (named logging) in Python allows CRLF characters in the logging values. This means that even when using the default library instead of making your own, you are vulnerable to log injection.

There are several solutions to this:

• You can encode the values, either using the encode() function or by encoding it using e.g. JSON (as shown earlier)
• You can use the syslog library which is safe around CRLF characters
• You can use a third party library which does the encoding for you. But unfortunately there do not seem to be many of these libraries

Resolving the vulnerable code example above is also quite simple. Currently, this line logs the whole POST data:

logPayment($success, $_POST);

Of course, this includes sensitive details which should not be logged. We can easily avoid this by hand-picking the data that should be logged, like this:

Currently, the vulnerable code example appends text to a file, including user input:

There are many ways to resolve this, one of them would be to store the line as JSON data, similar to above:

Now when someone tries to inject newlines into the log file, it will display as a literal \n.

PSR-3 is a set of standards for logging in PHP applications defined by the PHP Framework Interoperability Group (FIG). It aims to standardize logging across different PHP projects and frameworks by defining different log levels and a logging interface.

Here are some key things to know about PSR-3 logging:

• Defines log levels like debug, info, notice, warning, error, critical, alert, and emergency. This allows classifying log messages by severity.
• Defines a LoggerInterface with methods like log(), debug(), info() etc. Libraries implementing PSR-3 should follow this interface.
• Specifies key-value pairs for context data that should be passed to logging methods, like datetime, file, line, function, and message.
• Allows libraries to throw LoggerException if logging fails.

Some benefits of using PSR-3 compliant logging are:

• Standardized logging approach across different libraries and frameworks. Developers don't have to learn a new logging style when switching projects.
• Interoperability between PSR-3 compliant libraries. Can substitute one logging library with another.
• Better organization of log messages by severity levels. Can configure applications to only log above a certain threshold.
• Context data provides more useful information for debugging errors or tracking events.
• Framework-agnostic. Can use PSR-3 logging with any PHP framework.

There are many PSR-3 compliant logging libraries you can choose from such as:

1. Monolog
2. Klogger
3. Analog

Some of these loggers are more lightweight than others, so depending on your needs you will have to research which logging solution fits. However, when choosing a logging library, it is recommended that you choose one that is PSR-3 compliant.

• The time from attack to detection can take up to 200 days, or sometimes longer. This window gives cyber thieves plenty of time to tamper with servers, corrupt databases, steal confidential information, and plant malicious code if sufficient logging and monitoring is not in place
• Learn about insufficient logging as part of the OWASP top 10
• Take a look at a CWE for "Insertion of Sensitive Information into Log File"
• Another CWE for "Insufficient Logging"