General Logger: A Complete Guide for Developers

5. Sensitive data handling and redaction

• Never log secrets: Exclude passwords, tokens, credit card numbers, PII.
• Automated scrubbing: Use middleware to redact known sensitive patterns before logging.
• Masking policy: Replace sensitive values with placeholders (REDACTED) and log hashes if needed for lookups.

6. Log sampling and volume control

• Sampling strategies: Use head-based sampling for high-volume events (e.g., TRACE) and probabilistic sampling for repetitive logs.
• Rate limiting: Protect logging backends from floods during incidents.
• Dynamic levels: Allow raising or lowering log verbosity at runtime per service or per request.

7. Contextual and actionable messages

• Include context: Attach IDs, state, configuration, and inputs relevant to the event.
• Actionable messages: Write messages that suggest next steps or indicate probable causes.
• Avoid blame: Describe failures objectively; include function names and parameters where helpful.

8. Centralized logging and observability

• Aggregation: Ship logs to a central platform (Elasticsearch, Splunk, Loki, Datadog, or S3+Parquet for batch analysis).
• Indexing and retention: Index common query fields and set retention tiers (hot for recent, cold for archive).
• Alerting: Build alerts on error rates, increased latency, or unusual patterns rather than individual messages.

9. Format, codecs, and storage considerations

• Compression & batching: Use gzip/snappy and batching to reduce network and storage costs.
• Schema evolution: Plan for adding/removing fields; consumers should tolerate missing/extra fields.
• Cost awareness: Monitor storage and query costs; summarize or roll up frequent logs.

10. Testing, validation, and deployment

• Local dev defaults: Provide verbose, readable logs for developers and structured outputs for CI.
• Log contract tests: Include tests to ensure required fields and formats are present.
• Graceful failures: Ensure logging failures (e.g., remote backend down) degrade to local disk or stdout without crashing the app.

11. Practical configuration examples

Node.js (winston) — simple JSON logger

javascript
 ffON2NH02oMAcqyoh2UU MQCbz04ET5EljRmK3YpQ CPXAhl7VTkj2dHDyAYAf” data-copycode=“true” role=“button” aria-label=“Copy Code”>Copy CodeCopied
const { createLogger, transports, format } = require(‘winston’);

const logger = createLogger({
level: process.env.LOG_LEVEL || ‘info’,
  format: format.combine(
    format.timestamp(),
    format.errors({ stack: true }),
    format.json()
  ),
  defaultMeta: { service: ‘my-service’, environment: process.env.NODEENV },
  transports: [
    new transports.Console()
  ]
});

module.exports = logger;

python
 ffON2NH02oMAcqyoh2UU MQCbz04ET5EljRmK3YpQ CPXAhl7VTkj2dHDyAYAf” data-copycode=“true” role=“button” aria-label=“Copy Code”>Copy CodeCopied
import logging, structlog from structlog.stdlib import LoggerFactory 
logging.basicConfig(format=”%(message)s”, level=logging.INFO)
structlog.configure(logger_factory=LoggerFactory(), processors=[
    structlog.processors.TimeStamper(fmt=“iso”),
    structlog.processors.JSONRenderer()
])

logger = structlog.get_logger(“my-service”)
logger = logger.bind(environment=“production”)
logger.info(“service_started”, version=“1.2.3”)