Skip to content

Support base64 encoded data (type: string + format: byte) #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
czechboy0 opened this issue May 25, 2023 · 2 comments · Fixed by #326
Closed

Support base64 encoded data (type: string + format: byte) #11

czechboy0 opened this issue May 25, 2023 · 2 comments · Fixed by #326
Assignees
Labels
area/openapi Adding/updating a feature defined in OpenAPI. kind/feature New feature. size/S Small task. (A couple of hours of work.)
Milestone

Comments

@czechboy0
Copy link
Contributor

Currently, only type: string + format: binary is supported, and represents raw data (an octet stream).

OpenAPI also supports base64-encoded data, represented as the following JSON schema:

type: string
format: byte

The generated type for it should also be Foundation.Data (same as for format: binary), but as part of serialization/deserialization, the extra step of base64 coding needs to be added.

Testing: verify that both a whole request/response body being application/json with the contents of a base64-encoded string (use case 1), but also as a nested field in a JSON object (use case 2), both are pretty common.

@czechboy0 czechboy0 added 🆕 semver/minor Adds new public API. area/openapi Adding/updating a feature defined in OpenAPI. kind/feature New feature. size/M Medium task. (A couple of days of work.) labels May 25, 2023
@czechboy0 czechboy0 removed the 🆕 semver/minor Adds new public API. label Aug 9, 2023
@czechboy0 czechboy0 added this to the 1.0 milestone Aug 25, 2023
czechboy0 added a commit that referenced this issue Aug 30, 2023
[Generator] Integrate the new URI and String coders

### Motivation

Depends on runtime changes from apple/swift-openapi-runtime#45.

Up until now, we relied on a series of marker and helper protocols `_StringConvertible` and `_AutoLosslessStringConvertible` to handle converting between various types and their string representation.

This has been very manual and required a non-trivial amount of work to support any extra type, especially `Date` and generated string enums.

Well, turns out this was an unnecessarily difficult way to approach the problem - we had a better solution available for a long time - `Codable`.

Since all the generated types and all the built-in types we reference are already `Codable`, there is no need to reinvent a way to serialize and deserialize types, and we should just embrace it.

While a JSON encoder and decoder already exists in Foundation, we didn't have one handy for encoding to and from URIs (used by headers, query and path parameters), and raw string representation (using `LosslessStringConvertible`). We created those in the runtime library in PRs apple/swift-openapi-runtime#44 and apple/swift-openapi-runtime#41, and integrated them into our helper functions (which got significantly simplified this way) in apple/swift-openapi-runtime#45.

Out of scope of this PR, but this also opens the door to supporting URL form encoded bodies (#182), multipart (#36), and base64 (#11).

While this should be mostly invisible to our adopters, this refactoring creates space for implementing more complex features and overall simplifies our serialization story.

### Modifications

- Updated the generator to use the new helper functions.
- Updated the article about serialization, shows how we reduced the number of helper functions by moving to `Codable`.
- Set the `lineLength` to 120 on the formatter configuration, it was inconsistent with our `.swift-format` file, and lead to the soundness script trying to update the reference files, but then the reference tests were failing. Since we're planning to sync these in #40, this is a step closer to it, but it means that it's probably best to review this PR's diff with whitespace ignored.

### Result

Now the generated code uses the new helper functions, allowing us to delete all the deprecated helpers in 0.2.0.

### Test Plan

Updated file-based reference, snippet, and unit tests.


Reviewed by: glbrntt

Builds:
     ✔︎ pull request validation (5.8) - Build finished. 
     ✔︎ pull request validation (5.9) - Build finished. 
     ✔︎ pull request validation (docc test) - Build finished. 
     ✔︎ pull request validation (integration test) - Build finished. 
     ✔︎ pull request validation (nightly) - Build finished. 
     ✔︎ pull request validation (soundness) - Build finished. 

#226
@czechboy0
Copy link
Contributor Author

czechboy0 commented Sep 8, 2023

Note that since this isn't a content type, but a way to encode and decode any schema, including a nested one, this'll likely justify creating a type looking like this in the runtime library:

struct Base64EncodedData: Sendable, Codable, Hashable {
    var data: Foundation.Data // or [UInt8] or ArraySlice<UInt8>
    init(data: Foundation.Data) {
        self.data = data
    }
    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        let base64EncodedString = try container.decode(String.self)
        guard let data = Data(base64Encoded: base64EncodedString) else {
            throw RuntimeError.invalidBase64String(base64EncodedString)
        }
        self.init(data: data)
    }
    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        let base64String = data.base64EncodedString()
        try container.encode(base64String)
    }
}

And then for e.g. the following JSON Schema snippet

MyObj:
  type: object
  properties:
    stuff:
      type: string
      format: byte
  required: [stuff]

generate a struct like:

struct MyObj: Sendable, Codable, Hashable {
  var stuff: OpenAPIRuntime.Base64EncodedData
  // ...
}

@czechboy0 czechboy0 added size/S Small task. (A couple of hours of work.) and removed size/M Medium task. (A couple of days of work.) labels Sep 8, 2023
rnro added a commit to rnro/swift-openapi-runtime that referenced this issue Sep 26, 2023
Motivation

OpenAPI supports base64-encoded data but to this point OpenAPI Generator
has not (apple/swift-openapi-generator#11).

Modifications

Introduce the `Base64EncodedData` codable type to allow users in the
generator to describe byte types which must be en/de-coded.

Result

Users will be able to describe base64-encoded data as
`OpenAPIRuntime.Base64EncodedData` e.g.
```
public typealias MyData = OpenAPIRuntime.Base64EncodedData
```

Test Plan

Added a round-trip encode/decode test `testEncodingDecodingRoundTrip_base64_success`
rnro added a commit to rnro/swift-openapi-runtime that referenced this issue Sep 26, 2023
Motivation

OpenAPI supports base64-encoded data but to this point OpenAPI Generator
has not (apple/swift-openapi-generator#11).

Modifications

Introduce the `Base64EncodedData` codable type to allow users in the
generator to describe byte types which must be en/de-coded.

Result

Users will be able to describe base64-encoded data as
`OpenAPIRuntime.Base64EncodedData` e.g.
```
public typealias MyData = OpenAPIRuntime.Base64EncodedData
```

Test Plan

Added a round-trip encode/decode test `testEncodingDecodingRoundTrip_base64_success`
rnro added a commit to rnro/swift-openapi-runtime that referenced this issue Oct 4, 2023
Motivation

OpenAPI supports base64-encoded data but to this point OpenAPI Generator
has not (apple/swift-openapi-generator#11).

Modifications

Introduce the `Base64EncodedData` codable type to allow users in the
generator to describe byte types which must be en/de-coded.

Result

Users will be able to describe base64-encoded data as
`OpenAPIRuntime.Base64EncodedData` e.g.
```
public typealias MyData = OpenAPIRuntime.Base64EncodedData
```

Test Plan

Added a round-trip encode/decode test `testEncodingDecodingRoundTrip_base64_success`
rnro added a commit to rnro/swift-openapi-runtime that referenced this issue Oct 5, 2023
Motivation

OpenAPI supports base64-encoded data but to this point OpenAPI Generator
has not (apple/swift-openapi-generator#11).

Modifications

Introduce the `Base64EncodedData` codable type to allow users in the
generator to describe byte types which must be en/de-coded.

Result

Users will be able to describe base64-encoded data as
`OpenAPIRuntime.Base64EncodedData` e.g.
```
public typealias MyData = OpenAPIRuntime.Base64EncodedData
```

Test Plan

Added a round-trip encode/decode test `testEncodingDecodingRoundTrip_base64_success`
rnro added a commit to rnro/swift-openapi-generator that referenced this issue Oct 6, 2023
Motivation

OpenAPI supports base64-encoded data but to this point Swift OpenAPI Generator
has not (apple#11).

Modifications

A data type specified as `type: string, format: byte` will now result in
a generated type which is `Codable` and backed by a
`OpenAPIRuntime.Base64EncodedData` type which knows how to encode and
decode base64 data.

Result

Users will be able to specify request/response payloads as
base64-encoded data which will be encoded and decoded transparently

Test Plan

Unit tested locally.
rnro added a commit to rnro/swift-openapi-generator that referenced this issue Oct 6, 2023
OpenAPI supports base64-encoded data but to this point Swift OpenAPI Generator has not (apple#11).

Modifications

A data type specified as `type: string, format: byte` will now result in a generated type which is `Codable` and backed by a `OpenAPIRuntime.Base64EncodedData` type which knows how to encode and decode base64 data.

Result

Users will be able to specify request/response payloads as base64-encoded data which will be encoded and decoded transparently

Test Plan

Unit tested locally.
rnro added a commit to rnro/swift-openapi-generator that referenced this issue Oct 6, 2023
Motivation

OpenAPI supports base64-encoded data but to this point Swift OpenAPI Generator has not (apple#11).

Modifications

A data type specified as `type: string, format: byte` will now result in a generated type which is `Codable` and backed by a `OpenAPIRuntime.Base64EncodedData` type which knows how to encode and decode base64 data.

Result

Users will be able to specify request/response payloads as base64-encoded data which will be encoded and decoded transparently

Test Plan

Unit tested locally.
czechboy0 pushed a commit to apple/swift-openapi-runtime that referenced this issue Oct 10, 2023
### Motivation

OpenAPI supports base64-encoded data but to this point OpenAPI Generator
has not (apple/swift-openapi-generator#11).

### Modifications

Introduce the `Base64EncodedData` codable type to allow users in the
generator to describe byte types which must be en/de-coded.

### Result

Users will be able to describe base64-encoded data as
`OpenAPIRuntime.Base64EncodedData` e.g.
```
public typealias MyData = OpenAPIRuntime.Base64EncodedData
```

### Test Plan

Added a round-trip encode/decode test
`testEncodingDecodingRoundTrip_base64_success`
@czechboy0 czechboy0 linked a pull request Oct 10, 2023 that will close this issue
czechboy0 pushed a commit that referenced this issue Oct 10, 2023
This change accompanies
apple/swift-openapi-runtime#55 and relies on it
for the `OpenAPIRuntime.Base64EncodedData` type.

### Motivation

OpenAPI supports base64-encoded data but to this point Swift OpenAPI
Generator has not
(#11).

### Modifications

A data type specified as `type: string, format: byte` will now result in
a generated type which is `Codable` and backed by a
`OpenAPIRuntime.Base64EncodedData` type which knows how to encode and
decode base64 data.

### Result

Users will be able to specify request/response payloads as
base64-encoded data which will be encoded and decoded transparently

### Test Plan

Unit tested locally.
@czechboy0
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/openapi Adding/updating a feature defined in OpenAPI. kind/feature New feature. size/S Small task. (A couple of hours of work.)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants