[1.6.0] Precondition Failure in HTTPRequestStateMachine+Demand

[t089]

Hello,

since upgrading to 1.6.0 of AHC my server app is experiencing a new crash that seems to be related to a precondition failure in AHC. I haven't dug deeper into it, but have a stack trace for you (see below).

I am using AHC through SOTO to download files from AWS S3.

{
        "package": "async-http-client",
        "repositoryURL": "https://github.com/swift-server/async-http-client.git",
        "state": {
          "branch": null,
          "revision": "16ad7588b1bb1a3000524785b9217e238f0c2ea8",
          "version": "1.6.0"
        }
}

If you need anything else, let me know.

Swift: swift:5.4-focal
Build: swift build -c release -Xswiftc -static-stdlib
System: Docker FROM ubuntu:20.04

Received signal 4. Backtrace:
0x55e95079a862, Backtrace.(printBacktrace in _B82A8C0ED7C904841114FDF244F9E58E)(signal: Swift.Int32) -> () at /app/.build/checkouts/swift-backtrace/Sources/Backtrace/Backtrace.swift:66
0x7f9c864de3bf
0x55e950749ce6, Swift runtime failure: precondition failure at /app/.build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTPRequestStateMachine+Demand.swift:0
0x55e950749ce6, AsyncHTTPClient.HTTPRequestStateMachine.ResponseStreamState.channelReadComplete() -> Swift.Optional<NIOCore.CircularBuffer<NIOCore.ByteBuffer>> at /app/.build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTPRequestStateMachine+Demand.swift:98
0x55e9507132ba, closure #1 (inout AsyncHTTPClient.HTTPRequestStateMachine.(State in _7B06F6A0870FE3257BBA6A8046D00B43)) -> AsyncHTTPClient.HTTPRequestStateMachine.Action in AsyncHTTPClient.HTTPRequestStateMachine.channelReadComplete() -> AsyncHTTPClient.HTTPRequestStateMachine.Action at .build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTPRequestStateMachine.swift:438
0x55e9507132ba, reabstraction thunk helper from @callee_guaranteed (@inout AsyncHTTPClient.HTTPRequestStateMachine.(State in _7B06F6A0870FE3257BBA6A8046D00B43)) -> (@owned AsyncHTTPClient.HTTPRequestStateMachine.Action) to @escaping @callee_guaranteed (@inout AsyncHTTPClient.HTTPRequestStateMachine.(State in _7B06F6A0870FE3257BBA6A8046D00B43)) -> (@out AsyncHTTPClient.HTTPRequestStateMachine.Action) at /app/<compiler-generated>:0
0x55e9507132ba, generic specialization <AsyncHTTPClient.HTTPRequestStateMachine.Action> of AsyncHTTPClient.HTTPRequestStateMachine.(avoidingStateMachineCoW in _7B06F6A0870FE3257BBA6A8046D00B43)<A>((inout AsyncHTTPClient.HTTPRequestStateMachine.(State in _7B06F6A0870FE3257BBA6A8046D00B43)) -> A) -> A at .build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTPRequestStateMachine.swift:683
0x55e9507132ba, AsyncHTTPClient.HTTPRequestStateMachine.channelReadComplete() -> AsyncHTTPClient.HTTPRequestStateMachine.Action at .build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTPRequestStateMachine.swift:437
0x55e9507132ba, closure #1 (inout AsyncHTTPClient.HTTP1ConnectionStateMachine.(State in _B4B57A38397CC16E2BA1CEE9E9DDA370)) -> AsyncHTTPClient.HTTP1ConnectionStateMachine.Action in AsyncHTTPClient.HTTP1ConnectionStateMachine.channelReadComplete() -> AsyncHTTPClient.HTTP1ConnectionStateMachine.Action at /app/.build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTP1.1/HTTP1ConnectionStateMachine.swift:287
0x55e95070c5e7, reabstraction thunk helper from @callee_guaranteed (@inout AsyncHTTPClient.HTTP1ConnectionStateMachine.(State in _B4B57A38397CC16E2BA1CEE9E9DDA370)) -> (@owned AsyncHTTPClient.HTTP1ConnectionStateMachine.Action) to @escaping @callee_guaranteed (@inout AsyncHTTPClient.HTTP1ConnectionStateMachine.(State in _B4B57A38397CC16E2BA1CEE9E9DDA370)) -> (@out AsyncHTTPClient.HTTP1ConnectionStateMachine.Action) at /app/<compiler-generated>:0
0x55e95070c5e7, generic specialization <AsyncHTTPClient.HTTP1ConnectionStateMachine.Action> of AsyncHTTPClient.HTTP1ConnectionStateMachine.(avoidingStateMachineCoW in _B4B57A38397CC16E2BA1CEE9E9DDA370)<A>((inout AsyncHTTPClient.HTTP1ConnectionStateMachine.(State in _B4B57A38397CC16E2BA1CEE9E9DDA370)) -> A) -> A at .build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTP1.1/HTTP1ConnectionStateMachine.swift:345
0x55e95070c5e7, AsyncHTTPClient.HTTP1ConnectionStateMachine.channelReadComplete() -> AsyncHTTPClient.HTTP1ConnectionStateMachine.Action at .build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTP1.1/HTTP1ConnectionStateMachine.swift:286
0x55e95070c5e7, AsyncHTTPClient.HTTP1ClientChannelHandler.channelReadComplete(context: NIOCore.ChannelHandlerContext) -> () at /app/.build/checkouts/async-http-client/Sources/AsyncHTTPClient/ConnectionPool/HTTP1.1/HTTP1ClientChannelHandler.swift:126
0x55e950ac317a, NIOCore.ChannelHandlerContext.(invokeChannelReadComplete in _F5AC316541457BD146E3694279514AA3)() -> () at /app/.build/checkouts/swift-nio/Sources/NIOCore/ChannelPipeline.swift:1698
0x55e950ab8519
0x55e950ab8a75, protocol witness for NIOCore._ChannelInboundHandler.channelReadComplete(context: NIOCore.ChannelHandlerContext) -> () in conformance NIOCore.ByteToMessageHandler<A> : NIOCore._ChannelInboundHandler in NIOCore at /app/<compiler-generated>:0
0x55e950ac317a, NIOCore.ChannelHandlerContext.(invokeChannelReadComplete in _F5AC316541457BD146E3694279514AA3)() -> () at /app/.build/checkouts/swift-nio/Sources/NIOCore/ChannelPipeline.swift:1698
0x55e950ac3193, NIOCore.ChannelHandlerContext.(invokeChannelReadComplete in _F5AC316541457BD146E3694279514AA3)() -> () at /app/.build/checkouts/swift-nio/Sources/NIOCore/ChannelPipeline.swift:1700
0x55e950ac0259
0x55e950c64df6, NIOSSL.NIOSSLHandler.(doFlushReadData in _4C55B9A85907C0CB3F4E7FBD2C1C5493)(context: NIOCore.ChannelHandlerContext, receiveBuffer: NIOCore.ByteBuffer, readOnEmptyBuffer: Swift.Bool) -> () at /app/.build/checkouts/swift-nio-ssl/Sources/NIOSSL/NIOSSLHandler.swift:435
0x55e950c638ea, NIOSSL.NIOSSLHandler.channelReadComplete(context: NIOCore.ChannelHandlerContext) -> () at /app/.build/checkouts/swift-nio-ssl/Sources/NIOSSL/NIOSSLHandler.swift:163
0x55e950ac317a, NIOCore.ChannelHandlerContext.(invokeChannelReadComplete in _F5AC316541457BD146E3694279514AA3)() -> () at /app/.build/checkouts/swift-nio/Sources/NIOCore/ChannelPipeline.swift:1698
0x55e950ac4182
0x55e950c35f5a
0x55e950c3bab7, generic specialization <NIOPosix.Socket> of NIOPosix.BaseSocketChannel.readEOF0() -> () at .build/checkouts/swift-nio/Sources/NIOPosix/BaseSocketChannel.swift:988
0x55e950c3bab7, generic specialization <NIOPosix.Socket> of NIOPosix.BaseSocketChannel.readEOF() -> () at .build/checkouts/swift-nio/Sources/NIOPosix/BaseSocketChannel.swift:974
0x55e950c3bab7, generic specialization <NIOPosix.Socket> of protocol witness for NIOPosix.SelectableChannel.readEOF() -> () in conformance NIOPosix.BaseSocketChannel<A> : NIOPosix.SelectableChannel in NIOPosix at /app/<compiler-generated>:965
0x55e950c3bab7, function signature specialization <Arg[2] = Dead> of generic specialization <NIOPosix.SocketChannel> of NIOPosix.SelectableEventLoop.handleEvent<A where A: NIOPosix.SelectableChannel>(_: NIOPosix.SelectorEventSet, channel: A) -> () at /app/.build/checkouts/swift-nio/Sources/NIOPosix/SelectableEventLoop.swift:378
0x55e950c3b522, reabstraction thunk helper from @callee_guaranteed (@guaranteed NIOPosix.SelectorEvent<NIOPosix.NIORegistration>) -> (@error @owned Swift.Error) to @escaping @callee_guaranteed (@in_guaranteed NIOPosix.SelectorEvent<NIOPosix.NIORegistration>) -> (@error @owned Swift.Error) at /app/<compiler-generated>:0
0x55e950c3b522, partial apply forwarder for reabstraction thunk helper from @callee_guaranteed (@guaranteed NIOPosix.SelectorEvent<NIOPosix.NIORegistration>) -> (@error @owned Swift.Error) to @escaping @callee_guaranteed (@in_guaranteed NIOPosix.SelectorEvent<NIOPosix.NIORegistration>) -> (@error @owned Swift.Error) at /app/<compiler-generated>:0
0x55e950c389ac, generic specialization <NIOPosix.NIORegistration> of NIOPosix.Selector.whenReady0(strategy: NIOPosix.SelectorStrategy, onLoopBegin: () -> (), _: (NIOPosix.SelectorEvent<A>) throws -> ()) throws -> () at .build/checkouts/swift-nio/Sources/NIOPosix/SelectorEpoll.swift:252
0x55e950c3633f, generic specialization <NIOPosix.NIORegistration> of NIOPosix.Selector.whenReady(strategy: NIOPosix.SelectorStrategy, onLoopBegin: () -> (), _: (NIOPosix.SelectorEvent<A>) throws -> ()) throws -> () at .build/checkouts/swift-nio/Sources/NIOPosix/SelectorGeneric.swift:286
0x55e950c3633f, closure #2 () throws -> () in NIOPosix.SelectableEventLoop.run() throws -> () at /app/.build/checkouts/swift-nio/Sources/NIOPosix/SelectableEventLoop.swift:447
0x55e950c3633f, reabstraction thunk helper from @callee_guaranteed () -> (@error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out (), @error @owned Swift.Error) at /app/<compiler-generated>:0
0x55e950c3633f, generic specialization <()> of NIOPosix.withAutoReleasePool<A>(() throws -> A) throws -> A at /app/.build/checkouts/swift-nio/Sources/NIOPosix/SelectableEventLoop.swift:28
0x55e950c3633f, NIOPosix.SelectableEventLoop.run() throws -> () at /app/.build/checkouts/swift-nio/Sources/NIOPosix/SelectableEventLoop.swift:446
0x55e950c21627, static NIOPosix.MultiThreadedEventLoopGroup.(runTheLoop in _C2B1528F4FBA68A3DBFA89DBAEBE9D4D)(thread: NIOPosix.NIOThread, canEventLoopBeShutdownIndividually: Swift.Bool, selectorFactory: () throws -> NIOPosix.Selector<NIOPosix.NIORegistration>, initializer: (NIOPosix.NIOThread) -> (), _: (NIOPosix.SelectableEventLoop) -> ()) -> () at /app/.build/checkouts/swift-nio/Sources/NIOPosix/MultiThreadedEventLoopGroup.swift:87
0x55e950c21627, closure #1 (NIOPosix.NIOThread) -> () in static NIOPosix.MultiThreadedEventLoopGroup.(setupThreadAndEventLoop in _C2B1528F4FBA68A3DBFA89DBAEBE9D4D)(name: Swift.String, selectorFactory: () throws -> NIOPosix.Selector<NIOPosix.NIORegistration>, initializer: (NIOPosix.NIOThread) -> ()) -> NIOPosix.SelectableEventLoop at /app/.build/checkouts/swift-nio/Sources/NIOPosix/MultiThreadedEventLoopGroup.swift:107
0x55e950c23dd5, partial apply forwarder for closure #1 (NIOPosix.NIOThread) -> () in static NIOPosix.MultiThreadedEventLoopGroup.(setupThreadAndEventLoop in _C2B1528F4FBA68A3DBFA89DBAEBE9D4D)(name: Swift.String, selectorFactory: () throws -> NIOPosix.Selector<NIOPosix.NIORegistration>, initializer: (NIOPosix.NIOThread) -> ()) -> NIOPosix.SelectableEventLoop at /app/<compiler-generated>:0
0x55e950c240c9, reabstraction thunk helper from @escaping @callee_guaranteed (@guaranteed NIOPosix.NIOThread) -> () to @escaping @callee_guaranteed (@in_guaranteed NIOPosix.NIOThread) -> (@out ()) at /app/<compiler-generated>:0
0x55e950c23ded, partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed (@guaranteed NIOPosix.NIOThread) -> () to @escaping @callee_guaranteed (@in_guaranteed NIOPosix.NIOThread) -> (@out ()) at /app/<compiler-generated>:0
0x55e950c507da, closure #1 (Swift.Optional<Swift.UnsafeMutableRawPointer>) -> Swift.Optional<Swift.UnsafeMutableRawPointer> in static NIOPosix.ThreadOpsPosix.run(handle: inout Swift.Optional<Swift.UInt>, args: NIOPosix.Box<(body: (NIOPosix.NIOThread) -> (), name: Swift.Optional<Swift.String>)>, detachThread: Swift.Bool) -> () at /app/.build/checkouts/swift-nio/Sources/NIOPosix/ThreadPosix.swift:105
0x7f9c864d2608
0x7f9c85928292
0xffffffffffffffff

[fabianfett]

Hi @t089, thanks so much for reporting this. I'll prepare a patch now and I think we should be able to release a hotfix today.

[t089]

Thanks for the quick fix, @fabianfett!

[fabianfett]

New version 1.6.1 released!