Skip to content

Support NIO Transport Services - part 2 #184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Apr 18, 2020
Merged

Conversation

adam-fowler
Copy link
Member

@adam-fowler adam-fowler commented Mar 28, 2020

This is the continuation of the good work of @Yasumoto and @weissi in #135

The following code adds support for NIO Transport services. When the ConnectionPool asks for a connection bootstrap it is returned a NIOClientTCPBootstrap which wraps either a NIOTSConnectionBootstrap or a ClientBootstrap depending on whether the EventLoop we are running on is NIOTSEventLoop.

If you initialize an HTTPClient with eventLoopGroupProvider set to .createNew then if you are running on iOS, macOS 10.14 or later it will provide a NIOTSEventLoopGroup instead of a EventLoopGroup.

Currently a number of tests are failing. 4 of these are related to the NIOSSLUncleanShutdown error the others all seem related to various race conditions which are being dealt with on other PRs. I have tested this code with aws-sdk-swift and it is working on both macOS and iOS.

Things look into:

  • The aws-sdk-swift NIOTS HTTP client had issues with on Mojave. We should check if this is the case for async-http-client as well.

@swift-server-bot
Copy link

Can one of the admins verify this patch?

2 similar comments
@swift-server-bot
Copy link

Can one of the admins verify this patch?

@swift-server-bot
Copy link

Can one of the admins verify this patch?

@adam-fowler
Copy link
Member Author

adam-fowler commented Mar 28, 2020

Here's a list of all the failing tests. Some of them fail both with TS and without.

With Transport Services

  • All the TLS tests. Fixed by adding temporary certificate disabling code. A more complete version of this code should probably be in NIOTS.
  • SSL Unclean shutdown functions fail but they are expecting an NIOSSL error.
  • testUploadStreamingBackpressure cannot test this as maxMessagesPerRead is not available.
  • testAvoidLeakingTLSHandshakeCompletionPromise fixed, throws a connection refused error now
  • testStressGetClose: appears to be HTTPClient.Task.fail() being called on the same request twice.
  • testResponseFutureIsOnCorrectEL have a fix but it is on branch @adtrevor is working on. Crashing in the same place as testStressGetClose.

Without

  • testProxyTLS fixed. crashes inside the test server code (this happens with TS as well)
  • testUDS... fixed. Was trying to create NIOSSLClientTLSProvider with unix hostname.
  • testStressGetHttpsSSLError fixed. fail returning the wrong error

@adam-fowler
Copy link
Member Author

fyi Set Environment variable "ENABLE_TS_TESTS" to "true" to test NIOTS.

let group = MultiThreadedEventLoopGroup(numberOfThreads: 4)
let group = getDefaultEventLoopGroup(numberOfThreads: 4)
defer {
XCTAssertNoThrow(try group.syncShutdownGracefully())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be executed by the tests teardown?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case the eventLoopGroup has been created inside the test because it required a different number of threads

@weissi weissi requested review from a team and removed request for a team March 30, 2020 13:04
@adam-fowler adam-fowler force-pushed the niots branch 3 times, most recently from 14a3a63 to e57d8bb Compare March 31, 2020 06:39
@adam-fowler adam-fowler force-pushed the niots branch 2 times, most recently from 55e3c18 to c090008 Compare April 3, 2020 08:45
Added NWTLSError, NWDNSError and NWPOSIXError to wrap Network.framework errors. The reason to do this was to make it easier to read Errors thrown from the Network.framework.
Added NWError tests
Added macOS 10.14 fallbacks for min/max TLS protocol
Enabled applicationProtocols code, now uses `String.withCString`
Removed setting option `waitForActivity` to false
If running with a proxy don't setup NIOTSClientTLSProvider
While running NIOTS ignore SSL unclean shutdown tests
@artemredkin
Copy link
Collaborator

@swift-server-bot test this please

Copy link
Contributor

@weissi weissi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! This looks like a great start, I left some comments regarding some potential issues.

Also don't need to create a tlsConfiguration if there is a proxy so moved its initialisation down a bit

/// Are we testing NIO Transport services
func isTestingNIOTS() -> Bool {
return ProcessInfo.processInfo.environment["ENABLE_TS_TESTS"] == "true"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adam-fowler one last question: Could we flip the default here and have the tests run on TS by default? Otherwise I'm sure nobody will ever run them :|

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

@weissi weissi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you so much!!!

@weissi
Copy link
Contributor

weissi commented Apr 17, 2020

@swift-server-bot add to whitelist

@weissi
Copy link
Contributor

weissi commented Apr 18, 2020

hit #175

@weissi
Copy link
Contributor

weissi commented Apr 18, 2020

retrying

@weissi
Copy link
Contributor

weissi commented Apr 18, 2020

@swift-server-bot test this please

@weissi weissi merged commit 5298f20 into swift-server:master Apr 18, 2020
@weissi weissi added the 🆕 semver/minor Adds new public API. label Apr 18, 2020
@adam-fowler
Copy link
Member Author

@weissi thank you.

I do have a fix for #175, but @adtrevor has his own version of the fix in one of his branches I believe. Do you want me to put up a PR to compare?

@PopFlamingo
Copy link
Contributor

@adam-fowler I think the already opened PR #192 should fix #175 while also making the code simpler

@adam-fowler
Copy link
Member Author

@adtrevor as I mentioned in #175 if it protects against releaseAssociatedConnection being called multiple times then we're cool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🆕 semver/minor Adds new public API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants