Skip to content

Feat/masking input types support #6412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

VatsalGoel3
Copy link
Contributor

Issue number: #3473

Summary

Changes

This PR adds support to the DataMasking utility to handle complex Python input types such as:

  • Pydantic models
  • Dataclasses
  • Standard Python classes with .dict() method

To support this, a new prepare_data function was introduced, which performs type introspection and converts the input data into a dictionary before processing.

This function is now invoked at the beginning of the erase, encrypt, and decrypt methods, allowing these methods to seamlessly accept structured objects in addition to primitive types like dict, str, list, etc.

User experience

Before:

from aws_lambda_powertools.utilities.data_masking import DataMasking
from pydantic import BaseModel

class MyModel(BaseModel):
    name: str
    age: int

data = MyModel(name="powertools", age=5)
masker = DataMasking()
masked = masker.erase(data, fields=["age"])  # ❌ This raised errors or did not work

After:

# ✅ Now works correctly and returns: {'name': 'powertools', 'age': '*****'}
masked = masker.erase(data, fields=["age"])

This allows customers to use the utility directly with modern application architectures that use type-safe data structures.

Checklist

Is this a breaking change?

RFC issue number: N/A

Checklist:

  • Migration process documented
  • Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@VatsalGoel3 VatsalGoel3 requested a review from a team as a code owner April 6, 2025 09:08
@boring-cyborg boring-cyborg bot added documentation Improvements or additions to documentation tests labels Apr 6, 2025
@pull-request-size pull-request-size bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 6, 2025
@pull-request-size pull-request-size bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 6, 2025
Copy link

sonarqubecloud bot commented Apr 6, 2025

@VatsalGoel3
Copy link
Contributor Author

@leandrodamascena, for now I applied the prepare_data() function as you suggested in the Issue, but I have an idea for making the function more robust and covering more edge cases, it would be like

"""
Recursively convert complex objects into dictionaries (or simple types) so that they can be
processed by the data masking utility. This function handles:

- Dataclasses (using dataclasses.asdict)
- Pydantic models (using model_dump)
- Custom classes with a dict() method
- Fallback to using __dict__ if available
- Recursively traverses dicts, lists, tuples, and sets
- Guards against circular references

Parameters
----------
data : Any
    The input data which may be a complex type.
_visited : set, optional
    Internal set of visited object IDs to prevent infinite recursion on cyclic references.

Returns
-------
Any
    A primitive type, or a recursively converted structure (dict, list, etc.)
"""

If that is more relevant, will implement this one.

@VatsalGoel3 VatsalGoel3 closed this Apr 6, 2025
@VatsalGoel3 VatsalGoel3 deleted the feat/masking-input-types-support branch April 6, 2025 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants