Structured logging with Lambda powertools

August 13, 2020

No more throwing console.log() statements all over the place in our Lambda functions. The message is logged as a string and as such it’s difficult to search and filter on the variables or data that might be baked in. And probably there’s no need for those debug logs in every invocation anyway.

By utilizing DAZN Lambda powertools library it’s easy to log structured data as the following example will show:

const Log = require('@dazn/lambda-powertools-logger')

Log.debug('Got rows from DynamoDB', {
  count,
  tableName
})

The logger supports DEBUG, INFO, WARN and ERROR levels and setting the LOG_LEVEL environment variable to the desired level will help both decluttering our logs as well as keeping costs down.

serverless.yml

service: ...
custom:
  stage: ${opt:stage, self:provider.stage}
  logLevel:
    prod: WARN
    default: INFO

provider:
  name: aws
  runtime: nodejs12.x
  environment:
    LOG_LEVEL: ${self:custom.logLevel.${self:custom.stage}, self:custom.logLevel.default}
    SAMPLE_DEBUG_LOG_RATE: 0.05 # Here, 5%. Default setting is 1% (0.01)

The custom section above allows us to define stage-specific overrides of the log level. Here, the “prod” stage will log warnings and errors, while other stages, e.g. “dev” and “staging”, will log at info level and above.

The environment variable SAMPLE_DEBUG_LOG_RATE defines the percentage of debug logs (Log.debug( ... ) statements) that will make it to CloudWatch. By samling the debug logs we have a better chance of troubleshooting issues in produduction without having to redeploy to ajust the log level.

And there are even more advantages to using these libraries such as getting error details and stack trace for WARN and ERROR logs, and handling and logging of any correlation ID’s that may be present in the invocation. Also it ensures that an error is logged if an invocation times out which make debugging time out errors easier.

I’ve put together a runnable example here.

References

I learned this by attending Yan Cui’s “Production-Ready Serverless” workshop. I really recommend this workshop for anyone wanting to up their AWS serverlesss skills.