Skip to content

fail if user tries writing bytes after request is sent #270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
3 changes: 3 additions & 0 deletions Sources/AsyncHTTPClient/HTTPClient.swift
Original file line number Diff line number Diff line change
Expand Up @@ -967,6 +967,7 @@ public struct HTTPClientError: Error, Equatable, CustomStringConvertible {
case traceRequestWithBody
case invalidHeaderFieldNames([String])
case bodyLengthMismatch
case writeAfterRequestSent
}

private var code: Code
Expand Down Expand Up @@ -1019,4 +1020,6 @@ public struct HTTPClientError: Error, Equatable, CustomStringConvertible {
public static func invalidHeaderFieldNames(_ names: [String]) -> HTTPClientError { return HTTPClientError(code: .invalidHeaderFieldNames(names)) }
/// Body length is not equal to `Content-Length`.
public static let bodyLengthMismatch = HTTPClientError(code: .bodyLengthMismatch)
/// Body part was written after request was fully sent.
public static let writeAfterRequestSent = HTTPClientError(code: .writeAfterRequestSent)
}
19 changes: 15 additions & 4 deletions Sources/AsyncHTTPClient/HTTPHandler.swift
Original file line number Diff line number Diff line change
Expand Up @@ -876,12 +876,10 @@ extension TaskHandler: ChannelDuplexHandler {
let promise = self.task.eventLoop.makePromise(of: Void.self)
// All writes have to be switched to the channel EL if channel and task ELs differ
if channel.eventLoop.inEventLoop {
self.actualBodyLength += part.readableBytes
context.writeAndFlush(self.wrapOutboundOut(.body(part)), promise: promise)
self.writeBodyPart(context: context, part: part, promise: promise)
} else {
channel.eventLoop.execute {
self.actualBodyLength += part.readableBytes
context.writeAndFlush(self.wrapOutboundOut(.body(part)), promise: promise)
self.writeBodyPart(context: context, part: part, promise: promise)
}
}

Expand All @@ -901,6 +899,19 @@ extension TaskHandler: ChannelDuplexHandler {
}
}

private func writeBodyPart(context: ChannelHandlerContext, part: IOData, promise: EventLoopPromise<Void>) {
switch self.state {
case .idle:
self.actualBodyLength += part.readableBytes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think (would need a test here too) that we're policing the body length too late here, no? We seem to just add the readableBytes but I can't see that we check that it's actually less than or equal the number of body bytes allowed.

What happens if we do say content-length: 1 and then send XABCDEFG, aren't we then still sending ABCDEFG? I think we are and that'd be a security vulnerability because we could smuggle a request (say we did XGET /something HTTP/1.1\r\n\r\n

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thats a great point, yeah, we only check in the end, let me see how I can handle it here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I added an early fail here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@artemredkin this looks good but given that it's security relevant, could we add a test specifically for that case (one that'd fail without the new condition)

Copy link
Collaborator Author

@artemredkin artemredkin Jun 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have it, by accident, we have two tests for content-length/body-length checks:

  • testContentLengthTooLongFails
  • testContentLengthTooShortFails
    Last one will fail at the end of the request, before we send .end, but the first one will now be failed by this new check

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potentially we can split the error message, just to be sure

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@artemredkin I think we need a new one that tests that we do not send the bytes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because as soon as the bytes are on the wire, we created a security vulnerability. Even if we detect later that something's wrong, it'll be too late.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Lukasa same here, I've added a test to check that we do not send more bytes then body length, although I'm not 100% sure its foolproof

context.writeAndFlush(self.wrapOutboundOut(.body(part)), promise: promise)
default:
let error = HTTPClientError.writeAfterRequestSent
promise.fail(error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can't be quite right (and I think we should have a test for this). We're calling out to the user here before we change our state. So we may get user code "sandwiched in" here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've re-ordered things, not sure yet how to test it though, I'll have to get access to a handlers internal state... Could you elaborate on "sandwiched" code a bit?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@artemredkin sure. If you do promise.fail(...), then the promise's whenFailure and other code will run inline here. So we detected an error (ie. we should be in state .endOrError) but we call out before we entered the .endOrError state. So if the user for example uses another method that does a state check, we potentially have an issue because their code runs before the self.state = .endOrError so they won't see it. That could lead us to make wrong decisions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, that is so obvious now :) I'll think about how to test this

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Lukasa could you pick up from Johannes here? I've added a test to verify that if delegate is called here, state is changed. I haven't figured out how to attach anything to that promise though... It seems that I need to have access to channel.write(HTTPClient.request, promise), and I don't think anything in the code can attach to it

self.state = .endOrError
self.failTaskAndNotifyDelegate(error: error, self.delegate.didReceiveError)
}
}

public func read(context: ChannelHandlerContext) {
if self.mayRead {
self.pendingRead = false
Expand Down
22 changes: 13 additions & 9 deletions Tests/AsyncHTTPClientTests/HTTPClientInternalTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -424,11 +424,10 @@ class HTTPClientInternalTests: XCTestCase {
let randoEL = group.next()

let httpClient = HTTPClient(eventLoopGroupProvider: .shared(group))
let promise: EventLoopPromise<Channel> = httpClient.eventLoopGroup.next().makePromise()
let httpBin = HTTPBin(channelPromise: promise)
let server = NIOHTTP1TestServer(group: MultiThreadedEventLoopGroup(numberOfThreads: 1))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aren't we leaking the group here? It needs to be stopped.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, we do

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

defer {
XCTAssertNoThrow(try server.stop())
XCTAssertNoThrow(try httpClient.syncShutdown(requiresCleanClose: true))
XCTAssertNoThrow(try httpBin.shutdown())
}

let body: HTTPClient.Body = .stream(length: 8) { writer in
Expand All @@ -439,21 +438,26 @@ class HTTPClientInternalTests: XCTestCase {
}
}

let request = try Request(url: "http://127.0.0.1:\(httpBin.port)/custom",
let request = try Request(url: "http://127.0.0.1:\(server.serverPort)/custom",
body: body)
let delegate = Delegate(expectedEventLoop: delegateEL, randomOtherEventLoop: randoEL)
let future = httpClient.execute(request: request,
delegate: delegate,
eventLoop: .init(.testOnly_exact(channelOn: channelEL,
delegateOn: delegateEL))).futureResult

let channel = try promise.futureResult.wait()
XCTAssertNoThrow(try server.readInbound()) // .head
XCTAssertNoThrow(try server.readInbound()) // .body
XCTAssertNoThrow(try server.readInbound()) // .end

// Send 3 parts, but only one should be received until the future is complete
let buffer = channel.allocator.buffer(string: "1234")
try channel.writeAndFlush(HTTPServerResponsePart.body(.byteBuffer(buffer))).wait()
XCTAssertNoThrow(try server.writeOutbound(.head(.init(version: .init(major: 1, minor: 1),
status: .ok,
headers: HTTPHeaders([("Transfer-Encoding", "chunked")])))))
let buffer = ByteBuffer(string: "1234")
XCTAssertNoThrow(try server.writeOutbound(.body(.byteBuffer(buffer))))
XCTAssertNoThrow(try server.writeOutbound(.end(nil)))

try channel.writeAndFlush(HTTPServerResponsePart.end(nil)).wait()
let (receivedMessages, sentMessages) = try future.wait()
XCTAssertEqual(2, receivedMessages.count)
XCTAssertEqual(4, sentMessages.count)
Expand Down Expand Up @@ -488,7 +492,7 @@ class HTTPClientInternalTests: XCTestCase {

switch receivedMessages.dropFirst(0).first {
case .some(.head(let head)):
XCTAssertEqual(["transfer-encoding": "chunked"], head.headers)
XCTAssertEqual(head.headers["transfer-encoding"].first, "chunked")
default:
XCTFail("wrong message")
}
Expand Down
1 change: 1 addition & 0 deletions Tests/AsyncHTTPClientTests/HTTPClientTests+XCTest.swift
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ extension HTTPClientTests {
("testDelegateCallinsTolerateRandomEL", testDelegateCallinsTolerateRandomEL),
("testContentLengthTooLongFails", testContentLengthTooLongFails),
("testContentLengthTooShortFails", testContentLengthTooShortFails),
("testBodyUploadAfterEndFails", testBodyUploadAfterEndFails),
]
}
}
35 changes: 35 additions & 0 deletions Tests/AsyncHTTPClientTests/HTTPClientTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2502,4 +2502,39 @@ class HTTPClientTests: XCTestCase {
XCTAssertEqual(info.connectionNumber, 1)
XCTAssertEqual(info.requestNumber, 1)
}

func testBodyUploadAfterEndFails() {
let url = self.defaultHTTPBinURLPrefix + "post"

func uploader(_ streamWriter: HTTPClient.Body.StreamWriter) -> EventLoopFuture<Void> {
let done = streamWriter.write(.byteBuffer(ByteBuffer(string: "X")))
done.recover { error -> Void in
XCTFail("unexpected error \(error)")
}.whenSuccess {
// This is executed when we have already sent the end of the request.
done.eventLoop.execute {
streamWriter.write(.byteBuffer(ByteBuffer(string: "BAD BAD BAD"))).whenComplete { result in
switch result {
case .success:
XCTFail("we succeeded writing bytes after the end!?")
case .failure(let error):
XCTAssertEqual(HTTPClientError.writeAfterRequestSent, error as? HTTPClientError)
}
}
}
}
return done
}

XCTAssertThrowsError(
try self.defaultClient.execute(request:
Request(url: url,
body: .stream(length: 1, uploader))).wait()) { error in
XCTAssertEqual(HTTPClientError.writeAfterRequestSent, error as? HTTPClientError)
}

// Quickly try another request and check that it works. If we by accident wrote some extra bytes into the
// stream (and reuse the connection) that could cause problems.
XCTAssertNoThrow(try self.defaultClient.get(url: self.defaultHTTPBinURLPrefix + "get").wait())
}
}