-
Notifications
You must be signed in to change notification settings - Fork 49
[Multipart] Add a frames -> raw parts parsing sequence #74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
czechboy0
merged 6 commits into
apple:main
from
czechboy0:hd-multipart-frames-to-raw-parts
Nov 17, 2023
Merged
[Multipart] Add a frames -> raw parts parsing sequence #74
czechboy0
merged 6 commits into
apple:main
from
czechboy0:hd-multipart-frames-to-raw-parts
Nov 17, 2023
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
czechboy0
commented
Nov 16, 2023
czechboy0
commented
Nov 16, 2023
czechboy0
commented
Nov 16, 2023
Sources/OpenAPIRuntime/Multipart/MultipartFramesToRawPartsSequence.swift
Outdated
Show resolved
Hide resolved
czechboy0
commented
Nov 16, 2023
Tests/OpenAPIRuntimeTests/Multipart/Test_MultipartBytesToFramesSequence.swift
Show resolved
Hide resolved
czechboy0
commented
Nov 16, 2023
Tests/OpenAPIRuntimeTests/Multipart/Test_MultipartFramesToBytesSequence.swift
Show resolved
Hide resolved
simonjbeaumont
approved these changes
Nov 17, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me. Did you want more eyes on the state machine logic?
Sources/OpenAPIRuntime/Multipart/MultipartFramesToRawPartsSequence.swift
Show resolved
Hide resolved
Happy to get feedback even after landing this, but the unit tests cover at least the common codepaths. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Next sequence, parses
MultipartRawPart
values from a sequence ofMultipartFrame
values.Modifications
Moved the MultipartParser/Serializer into the files where their respective sequences live, it's better organized this way. But it's just a move so doesn't need to be re-reviewed.
This sequence is a bit unique, as this is where we need to handle the two sequences pulling from a common source of truth.
On the consumer side, this allows you to do:
The outer sequence has the element of
MultipartRawType
, which has header fields, and anHTTPBody
(which itself is an async sequence, of byte chunks). This sequence vendsMultipartRawType
values, and needs to demultiplex the frames appropriately to the right async sequence (either the outer one vending parts, or the inner body one of the last-vended part).It is a requirement that the part's body chunks (inner sequence) must be fully consumed before the outer sequence is asked for the next part. The state machine enforces this.
The solution is by using an actor that both iterators (outer and each of the inner ones) have a reference to and can ask for frames.
Also, had to disable strict concurrency checking on 5.10, because they added a warning when you iterate an async sequence from an actor; it's known, no workaround AFAIK. The alternative here would be to turn off warnings-as-errors for 5.10.
Result
A parsing sequence that turns a sequence of frames into a sequence of parts, each of which has a nested sequence of body chunks.
Test Plan
Unit tests added at all 3 layers: