Logging vulnerabilities: the basics

What are logging vulnerabilities?

Logging is the process of creating output logs based on actions within an application or service with the idea to provide information to understand possible issues like crashes or performance problems. Logging vulnerabilities are simply security vulnerabilities that arise from the process of logging. Some common examples include:

  • Publicly exposed log files
  • Logging of sensitive information
  • Insufficient logging
  • Ability to poison log entries
  • Blocking (or overloading) logging systems

About this lesson

In this lesson, you will learn about vulnerabilities stemming from bad logging practices and how to protect your applications against them. We will step into the shoes of a developer-turned-hacker, Jessica, who gains access to credit card information by gaining access to a third-party logging service.

FUN FACT

Lost logs

In 2018, it was reported that a major Australian bank lost backups of logs containing private data of about 12-20 million customers. The best theory is that these records "fell off the back of a truck" while on the way to being destroyed by a contractor. Read the true story.

Logging vulnerabilities in action

Jessica is an extremely talented developer who has just been made redundant by her previous employer. The whole process has been very stressful!

Late one night, Jessica was scrolling through JobberTheHutt, a popular job board for remote tech workers, in search of a new opportunity. She often uses an intercept proxy application like Burp Suite or OWASP ZAP while browsing websites, because it allows her to see how other web applications work on a lower level. This often gives her ideas for her own development projects.

JobberTheHutt had a premium membership which allowed you to view additional details about the jobs being advertised.

Logging in action

  • STEP 1
  • STEP 2
  • STEP 3
  • STEP 4
  • STEP 5
  • STEP 6

Setting the stage

Let's jump in and see a logging vulnerability. Jessica will be using our proxy inspector browser add-on to look at requests. Let's start!

logging-start.svg

Logging vulnerabilities under the hood

Post mortem

First, let's break down what happened in the story above:

  • Sensitive information was being logged to a third-party logging service
  • An attacker breached that service
  • The attacker is now able to view the sensitive information

Of course, if the information being logged was not sensitive, this attack would not have been nearly as severe.

An example of a logging solution that may inadvertently log sensitive information using NodeJS Express:

Capture the credit card details from the user.

A note on third-party logging solutions

In an application where the logging is purposely created, it would feel unintuitive for a developer to write code that logs credit card numbers or passwords.There are many widely-used third-party application monitoring suites today, many of which will offer a solution to log everything that is submitted to an application by a user to provide maximum insight into the way that your users are navigating through your applications. Most of these offerings will include some way to block the capture of personally identifiable information (PII), but these tools are not aware of how your application functions, so PII can often end up being logged regardless. It is the responsibility of the application developer to ensure that PII is explicitly excluded from logging.

Publicly exposed log files

A common vulnerability in the wild is the public exposure of log files. The most obvious example of this is when web server logs are being stored in a file that is inside the public web root. For example, when configuring the Apache2 configuration file, someone might specify the location of the error log with the following line:

ErrorLog /var/www/html/logs/error.log

If this was the case, an attacker could easily view the access logs by simply navigating to example.com/logs/error.log in their browser. Even simple information like IP address data and frequency of visits could have implications like providing business competitors with an advantage, but basic access logs will typically contain even more sensitive information like password reset tokens.

Another common way to expose log files is through misconfigurations in cloud storage buckets. For larger-scale, high-traffic applications, logs are often sent to cloud storage buckets instead of the web server's disk because it is easier, more efficient, and more scalable. Unfortunately, these storage buckets are commonly misconfigured, leaving them publicly accessible.

Insufficient logging

Visibility and timely response are two of the most fundamental components of a good security program. While insufficient logging does not present much exploitability by itself, you could argue that it is partly to blame for nearly every major security incident because if comprehensive logging and monitoring was taking place, it could mitigate or reduce the extent of an attack by facilitating an immediate response. For this reason, sufficient logging is important.

What should be logged?

All sensitive or high-value transactions should be logged. The exact things that should be logged will vary based on the function of the application, but some common examples include:

  • Logins
  • Failed logins
  • Password changes
  • Forgot password requests
  • Account updates
  • Money transfers
  • Purchases
  • Account creations

Log poisoning

Logs are often referred to as a source of absolute truth when auditing or investigating a security incident. For this reason, advanced attackers will often cover their tracks by tampering with log entries, or abuse the trust in log entries by creating fake ones.

A basic example of this is the ability to inject new line characters into a basic log file that would normally have a single log entry on each line. For example, let's say that we are logging payment transactions, the log file might look something like this:

PAYMENT 2883 true
PAYMENT 2884 true
PAYMENT 2887 true
PAYMENT 2886 true
PAYMENT 2885 true
PAYMENT 2891 false
PAYMENT 2889 true

Where the number is the payment ID, and the last column is true/false depending on whether the payment was successful or not. Let's take a look at what some basic code might look like to produce this log:

Now consider what might happen if an attacker manually tampered with their orderNumber and added some new lines:

2884
PAYMENT 1337 true
PAYMENT

Of course, instead of adding the new lines like this, we'd need to URL encode them:

2884%0a%0dPAYMENT%201337%20true%0a%0dPAYMENT%20

In this case, a fake line would be added to the log file without any transaction actually taking place, with the payment ID of 1337.

Scan your code & stay secure with Snyk - for FREE!

Did you know you can use Snyk for free to verify that your code
doesn't include this or other vulnerabilities?

Scan your code

Logging vulnerabilities mitigation

Some general advice for ensuring good logging practices:

  • Ensure that all sensitive or high-value events are logged and monitored for suspicious activity
  • Ensure that the method used for logging can not result in log entries being faked, tampered with, or removed
  • Ensure that logs are not ever accessible except to those who need to see them, like your security team
  • If you're using a third-party logging/analytics platform, double check that sensitive details like PII are not being logged

Resolving the logging of sensitive data

Resolving the vulnerable code example above is also quite simple. Currently, this line logs the whole req.body:

log("payment", success, JSON.stringify(req.body)

Of course, this includes sensitive details which should not be logged. We can easily avoid this by hand-picking the data that should be logged, like this:

log(JSON.stringify({payment: success, orderNumber: req.body.orderNumber}))

Resolving log poisoning

Currently, the vulnerable code example appends text to a file, including user input:

fs.appendFile('log.txt', "PAYMENT " + orderNumber + " " + success + "\n", function (err) {
   if (err) return console.log(err);
});

There are many ways to resolve this, one of them would be to store the line as JSON data, similar to above:

fs.appendFile('log.txt', JSON.stringify({payment: success, orderNumber: orderNumber}) + "\n", function (err) {
   if (err) return console.log(err);
});

Now when someone tries to inject newlines into the log file, it will display as a literal "\n".

Logging libraries

Of course, a better solution for logging is to use an existing NodeJS logging library, rather than rolling your own. Some examples of popular logging libraries include Winston, Bunyan and Pino.

Keep learning

  • The time from attack to detection can take up to 200 days, or sometimes longer. This window gives cyber thieves plenty of time to tamper with servers, corrupt databases, steal confidential information, and plant malicious code if sufficient logging and monitoring is not in place
  • Learn about insufficient logging as part of the OWASP top 10
  • Take a look at a CWE for "Insertion of Sensitive Information into Log File"
  • Another CWE for "Insufficient Logging"

Congratulations

You have taken your first step into learning about logging vulnerabilities, how they work, what the impacts are, and how to protect your own applications. We hope that you will apply this knowledge to make your applications safer. We'd really appreciate it if you could take a minute to rate how valuable this lesson was for you and provide feedback to help us improve! Also, make sure to check out our lessons on other common vulnerabilities.