Hey dev friends! 💻
I'm back with a new article to share what I've recently learned. Today, we're going to talk about logging in microservices and how structured logs and correlation IDs make a request’s journey easier to investigate. 🚀
If you're working with microservices, you know the pain: something breaks in production, and you're jumping between different services trying to figure out what happened. Sound familiar? Let's fix that!
🤔 The Problem
Picture this: You have 5 microservices running. A user reports an error. You start investigating.
You check Service A → Nothing obvious.
You check Service B → Maybe something?
You check Service C → Still not sure.
Instead of clear answers, you're piecing together a puzzle from scattered log files, trying to match timestamps and hoping you'll find the connection. Hours pass. 😰
This is a common reality. Many teams have logs, but they aren't useful. Each service logs independently with no way to track a request's journey through the system.
💡 The Solution: Correlation IDs
The game changer? Correlation IDs.
Think of correlation IDs like a tracking number for your package 📦. Just like you can track a package through every shipping center it passes through, a correlation ID lets you track a request through every microservice it touches.
Every request gets a unique ID that follows it through your entire system. When something fails, you:
- Search that one ID
- See the complete request journey
- Identify the exact failure point
Simple, powerful, effective.
🛠️ Keep Request Context Separate
A correlation ID is useful only if every log carries the correct one. One important correction: NestJS Scope.TRANSIENT means a new instance per consumer, not per HTTP request. Mutating a correlationId property on a logger injected into a singleton controller can mix IDs between concurrent requests.
For this example, let’s use Node’s AsyncLocalStorage to carry a separate context through each request’s asynchronous work, and Pino for structured logs. This example targets NestJS 11 with the Express adapter and a supported Node.js LTS release.
Step 1: Install Pino
In an existing NestJS application:
npm install pino
Pino includes its own TypeScript types. Keep production output as JSON; a pretty-print transport can be added separately for local development.
Step 2: Create the request context
// request-context.ts
import { AsyncLocalStorage } from 'node:async_hooks';
import { randomUUID } from 'node:crypto';
import type { Request, Response, NextFunction } from 'express';
export const requestContext = new AsyncLocalStorage<{
correlationId: string;
}>();
export function correlationIdMiddleware(
req: Request,
res: Response,
next: NextFunction,
) {
const incoming = req.headers['x-correlation-id'];
// This example accepts a UUID-shaped ID; replace anything else.
const valid = typeof incoming === 'string'
&& /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(incoming);
const correlationId = valid ? incoming : randomUUID();
res.setHeader('x-correlation-id', correlationId);
requestContext.run({ correlationId }, () => next());
}
The format check keeps arbitrary header text out of the log field. The ID is for tracing, not authentication or authorization. run() scopes the context to this request and the asynchronous operations created inside it.
Step 3: Create a logger without mutable request state
// logger.service.ts
import { Injectable } from '@nestjs/common';
import pino from 'pino';
import { requestContext } from './request-context';
@Injectable()
export class LoggerService {
private readonly logger = pino({
level: process.env.LOG_LEVEL || 'info',
redact: ['authorization', 'cookie', 'password', 'token'],
});
info(message: string, fields: Record<string, unknown> = {}) {
this.logger.info({
...fields,
correlationId: requestContext.getStore()?.correlationId,
}, message);
}
error(message: string, fields: Record<string, unknown> = {}) {
this.logger.error({
...fields,
correlationId: requestContext.getStore()?.correlationId,
}, message);
}
}
This assumes the usual NestJS TypeScript configuration with esModuleInterop enabled. The singleton logger reads the current request context each time it logs, so it does not store one request’s ID for another to overwrite.
The redaction list is only a small safeguard for those top-level keys. Prefer an allowlist of safe metadata such as operation names and status codes. Don’t log complete request bodies, credentials or personal data; review nested fields and error objects separately.
Step 4: Register and use it
// users.controller.ts
import { Controller, Get } from '@nestjs/common';
import { LoggerService } from './logger.service';
@Controller('users')
export class UsersController {
constructor(private readonly logger: LoggerService) {}
@Get()
async getUsers() {
this.logger.info('Fetching users', { context: 'UsersController' });
// Replace this empty result with your authorized application query.
const users: unknown[] = [];
await Promise.resolve();
this.logger.info('Users fetched', { count: users.length });
return users;
}
}
// app.module.ts
import { Module } from '@nestjs/common';
import { LoggerService } from './logger.service';
import { UsersController } from './users.controller';
@Module({
controllers: [UsersController],
providers: [LoggerService],
})
export class AppModule {}
// main.ts
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { correlationIdMiddleware } from './request-context';
async function bootstrap() {
const app = await NestFactory.create(AppModule);
app.use(correlationIdMiddleware);
await app.listen(3000);
}
void bootstrap();
Only calls through this custom logger carry this context; Nest’s bootstrap logs are not automatically replaced. Background jobs need their own explicit context. An integration such as nestjs-pino is another option if you want request logging and Nest logger integration together; check its version compatibility table.
Step 5: Propagate the ID to the next service
// downstream.service.ts
import { Injectable } from '@nestjs/common';
import { requestContext } from './request-context';
@Injectable()
export class DownstreamService {
async getStatus() {
const url = process.env.SERVICE_B_URL;
if (!url) throw new Error('SERVICE_B_URL is required');
const correlationId = requestContext.getStore()?.correlationId;
const response = await fetch(url, {
headers: correlationId ? { 'x-correlation-id': correlationId } : {},
signal: AbortSignal.timeout(5000),
});
if (!response.ok) {
throw new Error(`Service B returned ${response.status}`);
}
return response.json();
}
}
Configure the URL on the server, propagate the ID only to services you intend to trace, and apply the receiving middleware there too. For queues, carry the ID in message metadata and establish a new context in the consumer. A correlation ID is not a distributed trace: use OpenTelemetry and trace/span IDs when you need timing and parent-child relationships.
Step 6: Centralize logs and add error tracking
Structured JSON logs alone do not centralize anything. Collect stdout with your hosting platform or log collector, index the correlation ID, and configure retention, access control and alerts.
Sentry can complement those logs. Use the current NestJS SDK setup for your installed version, initialize instrumentation as documented, and attach the correlation ID as a tag in the request’s isolation scope. Avoid sharing mutable global Sentry tags between requests or copying sensitive payloads into events. The old Sentry.Handlers middleware example is not the current integration.
Before relying on this pattern, test overlapping requests with different IDs, invalid incoming headers, downstream propagation and log redaction. That catches the context mix-ups a sequential happy-path test misses.
📊 What This Looks Like in Practice
Before (illustrative):
[2025-01-15 10:30:45] Request received
[2025-01-15 10:30:46] Processing data
[2025-01-15 10:30:47] Error: Operation failed
Which request? Which user? No idea. 🤷♀️
After (illustrative log records):
{
"level": 30,
"msg": "Request received",
"context": "ApiController",
"correlationId": "550e8400-e29b-41d4-a716-446655440000",
"time": 1736937045123
}
{
"level": 30,
"msg": "Processing data",
"context": "DataService",
"correlationId": "550e8400-e29b-41d4-a716-446655440000",
"time": 1736937046456
}
{
"level": 50,
"msg": "Operation failed: Validation error",
"context": "DataService",
"correlationId": "550e8400-e29b-41d4-a716-446655440000",
"time": 1736937047789
}
Now you can search for 550e8400-e29b-41d4-a716-446655440000 and see the entire request journey across all services! 🎯
✨ What Changes in an Investigation
Instead of manually matching timestamps from several services, you can search one correlation ID in the log system and inspect the related events. This can shorten the search, but it does not guarantee a fixed debugging time: missing propagation, incomplete logs and the underlying bug still matter.
🎯 Key Takeaways
What separates good logging from great logging:
Structure → JSON over plain text (searchable and parseable)
Context → Correlation IDs tracking every request
Proper levels → Debug/info/error used correctly, not everything as info
Centralization → One place to search everything
Alerts → Configure error reporting and actionable thresholds; detection is not automatic for every failure
💭 Final Thoughts
Setting up useful logging takes effort, but it makes production investigations easier to follow. Start with safe structured fields, isolated request context and consistent propagation, then verify the behavior under concurrency.
Your future self will thank you! 😊
🔗 Resources
- Pino Documentation
- Sentry for NestJS
- Node.js AsyncLocalStorage
- NestJS provider scopes
- NestJS Logging
What about you? 🤔 Have you implemented correlation IDs in your microservices? What challenges did you face? Share your experiences below!
And if you found this helpful, follow me for more articles about web development, NestJS, and DevOps practices. Let's learn together ❤️
Top comments (0)