Skip to content

feat(logger): introduce log key reordering functionality #2736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 34 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
3a78c9a
feat: define `logRecordOrder` option in Logger types
arnabrahman Jul 4, 2024
a8dde6b
feat: pass the `logRecordOrder` option to `LogFormatter`
arnabrahman Jul 4, 2024
4f3967f
feat: order attributes if logRecordOrder is available
arnabrahman Jul 4, 2024
c743eb0
feat: use `lodash.merge` for merging ordered & base order attributes
arnabrahman Jul 4, 2024
e1539ac
test: `logRecordOrder` option during formatting
arnabrahman Jul 4, 2024
6115983
Merge branch 'main' into 1568-log-message-ordering
dreamorosi Jul 8, 2024
94ed8fe
Merge branch 'main' into 1568-log-message-ordering
dreamorosi Jul 8, 2024
7a6a7f1
feat: merge with main
arnabrahman Sep 2, 2024
e7be7a2
refactor: `LogRecordOrder` type union of `UnformattedAttributes` & `…
arnabrahman Sep 2, 2024
22ab7b9
test: logRecordOrder option during formatting
arnabrahman Sep 2, 2024
94b647f
refactor: replace `lodash.merge` wih native JS
arnabrahman Sep 2, 2024
e8014f4
test: `logRecordOrder` takes `additionalLogAttributes` into considera…
arnabrahman Sep 2, 2024
80a2333
test: formatter when `logRecordOrder` is not set
arnabrahman Sep 2, 2024
2dd0cc5
feat: during formatting, apply ordering for `additionalLogAttributes`…
arnabrahman Sep 2, 2024
31efcec
test: `logRecordOrder` orders correctly when key doesn't even exist
arnabrahman Sep 2, 2024
85ec8aa
test: `logRecordOrder` should be passed down to child logger
arnabrahman Sep 2, 2024
986245c
doc: reordering log keys position
arnabrahman Sep 3, 2024
bd63350
doc: fix reorderLogKeys example highlight line
arnabrahman Sep 3, 2024
da1d0ee
feat: merge with main
arnabrahman Sep 3, 2024
f0ba54e
Merge branch '1568-log-message-ordering' of github.com:arnabrahman/aw…
arnabrahman Sep 3, 2024
e3bac06
doc: remove @returns
arnabrahman Sep 3, 2024
1c6b2f6
Update packages/logger/src/Logger.ts
arnabrahman Sep 7, 2024
07ba540
doc: update `setLogFormatter` function doc block
arnabrahman Sep 7, 2024
bfe9510
refactor: return early if `logRecordOrder` is not set
arnabrahman Sep 7, 2024
da62495
refactor: make `logRecordOrder` & `logFormatter` mutually exclusive
arnabrahman Sep 8, 2024
4ae836c
refactor: use `logRecordOrder` only inside `PowertoolsLogFormatter` c…
arnabrahman Sep 8, 2024
0a50ee5
test: refactor test to match the key orders
arnabrahman Sep 8, 2024
37b43b2
refactor: extract unformattedAttributes to the top level
arnabrahman Sep 8, 2024
ce2cb90
test: update test to add uncovered line
arnabrahman Sep 8, 2024
8f90b90
test: set `unformattedAttributes.timestamp` inside `beforeEach`
arnabrahman Sep 8, 2024
735c650
doc: add callout
arnabrahman Sep 8, 2024
62f0aef
style: doc block & afterAll cleanup for `unformattedAttributes`
arnabrahman Sep 8, 2024
74f614f
test: refactor `logRecordOrder` test for child logger
arnabrahman Sep 8, 2024
f4d4220
Merge branch 'main' into 1568-log-message-ordering
dreamorosi Sep 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/core/logger.md
Original file line number Diff line number Diff line change
Expand Up @@ -542,6 +542,22 @@ We prioritise log level settings in this order:

In the event you have set a log level in Powertools to a level that is lower than the ACL setting, we will output a warning log message informing you that your messages will be discarded by Lambda.

### Reordering log keys position

You can change the order of [standard Logger keys](#standard-structured-keys) or any keys that will be appended later at runtime via the `logRecordOrder` parameter.

=== "reorderLogKeys.ts"

```typescript hl_lines="5 10"
--8<-- "examples/snippets/logger/reorderLogKeys.ts"
```

=== "reorderLogKeysOutput.json"

```json hl_lines="2-3"
--8<-- "examples/snippets/logger/reorderLogKeysOutput.json"
```

### Setting timestamp to custom Timezone

By default, Logger emits records with the default Lambda timestamp in **UTC**, i.e. `2016-06-20T12:08:10.000Z`
Expand Down
12 changes: 12 additions & 0 deletions examples/snippets/logger/reorderLogKeys.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import { Logger } from '@aws-lambda-powertools/logger';

const logger = new Logger({
serviceName: 'serverlessAirline',
logRecordOrder: ['timestamp', 'additionalKey'],
});

export const handler = async (): Promise<void> => {
logger.info('Hello, World!', {
additionalKey: 'additionalValue',
});
};
9 changes: 9 additions & 0 deletions examples/snippets/logger/reorderLogKeysOutput.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"timestamp": "2024-09-03T02:59:06.603Z",
"additionalKey": "additionalValue",
"level": "INFO",
"message": "Hello, World!",
"sampling_rate": 0,
"service": "serverlessAirline",
"xray_trace_id": "1-66d67b7a-79bc7b2346b32af01b437cf8"
}
15 changes: 12 additions & 3 deletions packages/logger/src/Logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import type {
LogFunction,
LogItemExtraInput,
LogItemMessage,
LogRecordOrder,
LoggerInterface,
PowertoolsLogData,
} from './types/Logger.js';
Expand Down Expand Up @@ -1081,11 +1082,18 @@ class Logger extends Utility implements LoggerInterface {
*
* @private
* @param {LogFormatterInterface} logFormatter - The log formatt er
* @param {LogRecordOrder} logRecordOrder
*/
private setLogFormatter(logFormatter?: LogFormatterInterface): void {
private setLogFormatter(
logFormatter?: LogFormatterInterface,
logRecordOrder?: LogRecordOrder
): void {
this.logFormatter =
logFormatter ??
new PowertoolsLogFormatter({ envVarsService: this.getEnvVarsService() });
new PowertoolsLogFormatter({
envVarsService: this.getEnvVarsService(),
logRecordOrder,
});
}

/**
Expand Down Expand Up @@ -1119,6 +1127,7 @@ class Logger extends Utility implements LoggerInterface {
persistentLogAttributes, // deprecated in favor of persistentKeys
environment,
jsonReplacerFn,
logRecordOrder,
} = options;

if (persistentLogAttributes && persistentKeys) {
Expand All @@ -1140,7 +1149,7 @@ class Logger extends Utility implements LoggerInterface {
this.setInitialSampleRate(sampleRateValue);

// configurations that affect how logs are printed
this.setLogFormatter(logFormatter);
this.setLogFormatter(logFormatter, logRecordOrder);
this.setConsole();
this.setLogIndentation();
this.#jsonReplacerFn = jsonReplacerFn;
Expand Down
8 changes: 7 additions & 1 deletion packages/logger/src/formatter/LogFormatter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import type {
LogFormatterInterface,
LogFormatterOptions,
} from '../types/Log.js';
import type { UnformattedAttributes } from '../types/Logger.js';
import type { LogRecordOrder, UnformattedAttributes } from '../types/Logger.js';
import type { LogItem } from './LogItem.js';

/**
Expand All @@ -19,8 +19,14 @@ abstract class LogFormatter implements LogFormatterInterface {
*/
protected envVarsService?: EnvironmentVariablesService;

/**
* An array of keys that defines the order of the log record.
*/
protected logRecordOrder?: LogRecordOrder;

public constructor(options?: LogFormatterOptions) {
this.envVarsService = options?.envVarsService;
this.logRecordOrder = options?.logRecordOrder;
}

/**
Expand Down
34 changes: 32 additions & 2 deletions packages/logger/src/formatter/PowertoolsLogFormatter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,38 @@ class PowertoolsLogFormatter extends LogFormatter {
timestamp: this.formatTimestamp(attributes.timestamp),
xray_trace_id: attributes.xRayTraceId,
};
const powertoolsLogItem = new LogItem({ attributes: baseAttributes });
powertoolsLogItem.addAttributes(additionalLogAttributes);

const orderedAttributes = {} as PowertoolsLog;

// If logRecordOrder is set, order the attributes in the log item
for (const key of this.logRecordOrder || []) {
if (key in baseAttributes && !(key in orderedAttributes)) {
orderedAttributes[key] = baseAttributes[key];
} else if (
key in additionalLogAttributes &&
!(key in orderedAttributes)
) {
orderedAttributes[key] = additionalLogAttributes[key];
}
}

// Add remaining attributes from baseAttributes
for (const key in baseAttributes) {
if (!(key in orderedAttributes)) {
orderedAttributes[key] = baseAttributes[key];
}
}

// Add remaining attributes from additionalLogAttributes
for (const key in additionalLogAttributes) {
if (!(key in orderedAttributes)) {
orderedAttributes[key] = additionalLogAttributes[key];
}
}

const powertoolsLogItem = new LogItem({
attributes: orderedAttributes,
});

return powertoolsLogItem;
}
Expand Down
4 changes: 3 additions & 1 deletion packages/logger/src/types/Log.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import type { EnvironmentVariablesService } from '../config/EnvironmentVariablesService.js';
import type { LogLevel as LogLevelList } from '../constants.js';
import type { LogItem } from '../formatter/LogItem.js';
import type { UnformattedAttributes } from './Logger.js';
import type { LogRecordOrder, UnformattedAttributes } from './Logger.js';

type LogLevel =
| (typeof LogLevelList)[keyof typeof LogLevelList]
Expand Down Expand Up @@ -125,6 +125,8 @@ type LogFormatterOptions = {
* If set, it gives the LogFormatter access to environment variables.
*/
envVarsService?: EnvironmentVariablesService;

logRecordOrder?: LogRecordOrder;
};

/**
Expand Down
4 changes: 4 additions & 0 deletions packages/logger/src/types/Logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ type BaseConstructorOptions = {
* This allows you to customize the serialization while still benefiting from the default behavior.
*/
jsonReplacerFn?: CustomJsonReplacerFn;
logRecordOrder?: LogRecordOrder;
};

/**
Expand Down Expand Up @@ -157,6 +158,8 @@ type UnformattedAttributes = PowertoolsLogData & {
message: string;
};

type LogRecordOrder = Array<keyof UnformattedAttributes | keyof LogAttributes>;

type LogItemMessage = string | LogAttributesWithMessage;
type LogItemExtraInput = [Error | string] | LogAttributes[];

Expand Down Expand Up @@ -197,4 +200,5 @@ export type {
ConstructorOptions,
InjectLambdaContextOptions,
CustomJsonReplacerFn,
LogRecordOrder,
};
Loading