Failed to build with custom build command

[inflation]

This crate is a wrapper over a C library. So I have a build.rs to run bindgen and other stuffs.

Normal cargo works fine. But strangely, something aborted the process when using cg_clif/cargo.sh:

   Compiling jpegxl-sys v0.3.2
error: failed to run custom build command for `jpegxl-sys v0.3.2`

Caused by:
  process didn't exit successfully: `/path/to/project/target/debug/build/jpegxl-sys-9bc55f14172fa98e/build-script-build` (signal: 6, SIGABRT: process abort signal)

Freshly built from master on macOS Big Sur 11.2.1. Is there any way to find out more?

[bjorn3]

Thanks for the bug report! I will try if I can reproduce it on Linux tomorrow.

[inflation]

Alright, I hooked up lldb to see what's happened. Apparently some locking issue:

* thread #2, stop reason = signal SIGUSR1
  * frame #0: 0x00007fff204dd0c6 libsystem_kernel.dylib`__psynch_mutexwait + 10
    frame #1: 0x00007fff2050d2c5 libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_wait + 81
    frame #2: 0x00007fff2050b1bc libsystem_pthread.dylib`_pthread_mutex_firstfit_lock_slow + 211
    frame #3: 0x00007fff2050ff16 libsystem_pthread.dylib`_pthread_cond_wait + 1421
    frame #4: 0x000000010087813e cargo`jobserver::HelperState::for_each_request::h527c20b942320c00 + 478
    frame #5: 0x00000001008782dc cargo`std::sys_common::backtrace::__rust_begin_short_backtrace::h20f43433e2154b3c + 60
    frame #6: 0x0000000100878e5e cargo`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h15c7879371c90873 + 142
    frame #7: 0x00000001008c3acd cargo`std::sys::unix::thread::Thread::new::thread_start::h67646ed36c745e2d + 45
    frame #8: 0x00007fff2050f950 libsystem_pthread.dylib`_pthread_start + 224
    frame #9: 0x00007fff2050b47b libsystem_pthread.dylib`thread_start + 15

Another attempt show the same crush but on CondVar:

  thread #1, queue = 'com.apple.main-thread'
    frame #0: 0x00007fff204dd8e2 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff2050fe6f libsystem_pthread.dylib`_pthread_cond_wait + 1254
    frame #2: 0x00000001008bc83e cargo`std::sys::unix::condvar::Condvar::wait_timeout::h83d9c60d3c4c5082 + 254
    frame #3: 0x0000000100876ed4 cargo`jobserver::imp::Helper::join::h84ff51737f6aff7e + 212
    frame #4: 0x0000000100877f05 cargo`_$LT$jobserver..HelperThread$u20$as$u20$core..ops..drop..Drop$GT$::drop::hb000f6b82fc74c0a + 149
    frame #5: 0x0000000100299a63 cargo`_$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h65f6e0509eb5df5b + 131
    frame #6: 0x00000001001d1e2b cargo`crossbeam_utils::thread::scope::h12b8e662b9c59058 + 139
    frame #7: 0x00000001002862a7 cargo`cargo::core::compiler::job_queue::JobQueue::execute::h8bd3e59c0e7b79ce + 3207
    frame #8: 0x00000001001040b0 cargo`cargo::core::compiler::context::Context::compile::h3ffd8d06ec00baae + 3056
    frame #9: 0x00000001003cacee cargo`cargo::ops::cargo_compile::compile_ws::h04f465ced689a88a + 574
    frame #10: 0x00000001003caa73 cargo`cargo::ops::cargo_compile::compile::h17570933a9ba1f61 + 147
    frame #11: 0x00000001000676ed cargo`cargo::commands::build::exec::h73d589279770c2c6 + 701
    frame #12: 0x0000000100064c2e cargo`cargo::cli::main::h746d90044d75d6b5 + 7710
    frame #13: 0x000000010007279d cargo`cargo::main::h2457145b4a592557 + 125
    frame #14: 0x000000010005f7fa cargo`std::sys_common::backtrace::__rust_begin_short_backtrace::ha24b4c0b939aa407 + 10
    frame #15: 0x00000001000699fc cargo`std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::h13f0e50e0712c5a0 + 12
    frame #16: 0x00000001008babe4 cargo`std::rt::lang_start_internal::h6cd9161d564e26bc + 644
    frame #17: 0x0000000100074b89 cargo`main + 41
    frame #18: 0x0000000100003034 cargo`start + 52
* thread #2, stop reason = signal SIGUSR1
  * frame #0: 0x00007fff204dd8e2 libsystem_kernel.dylib`__psynch_cvwait + 10
    frame #1: 0x00007fff2050fe6f libsystem_pthread.dylib`_pthread_cond_wait + 1254
    frame #2: 0x000000010087813e cargo`jobserver::HelperState::for_each_request::h527c20b942320c00 + 478
    frame #3: 0x00000001008782dc cargo`std::sys_common::backtrace::__rust_begin_short_backtrace::h20f43433e2154b3c + 60
    frame #4: 0x0000000100878e5e cargo`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h15c7879371c90873 + 142
    frame #5: 0x00000001008c3acd cargo`std::sys::unix::thread::Thread::new::thread_start::h67646ed36c745e2d + 45
    frame #6: 0x00007fff2050f950 libsystem_pthread.dylib`_pthread_start + 224
    frame #7: 0x00007fff2050b47b libsystem_pthread.dylib`thread_start + 15

I'll try a debug build to find anything else.

[inflation]

Reproduced on Linux as well:

* thread #2, name = 'cargo', stop reason = signal SIGUSR1
  * frame #0: 0x00007ffff7f889ba libpthread.so.0`__futex_abstimed_wait_common64 + 202
    frame #1: 0x00007ffff7f82260 libpthread.so.0`pthread_cond_wait@@GLIBC_2.3.2 + 512
    frame #2: 0x0000555555e655d1 cargo`jobserver::HelperState::for_each_request::h9f498704c1043438 + 449
    frame #3: 0x0000555555e6574c cargo`std::sys_common::backtrace::__rust_begin_short_backtrace::hc5a834bc9bfca3dc (.llvm.3913707323791418439) + 44
    frame #4: 0x0000555555e6620f cargo`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h50869e0f00442260 + 127
    frame #5: 0x0000555555eac58a cargo`std::sys::unix::thread::Thread::new::thread_start::hba22c7961aa0e2f8 [inlined] _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h321523073ee11258 at boxed.rs:1546:9
    frame #6: 0x0000555555eac584 cargo`std::sys::unix::thread::Thread::new::thread_start::hba22c7961aa0e2f8 [inlined] _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::hf0e85e12c83b99b3 at boxed.rs:1546
    frame #7: 0x0000555555eac57b cargo`std::sys::unix::thread::Thread::new::thread_start::hba22c7961aa0e2f8 at thread.rs:71
    frame #8: 0x00007ffff7f7c299 libpthread.so.0`start_thread + 233
    frame #9: 0x00007ffff7d59053 libc.so.6`__clone + 67

(Small note when you building the crate: use --features without-build to prevent from building the underlying C library and save you some hassle 😄)

[bjorn3]

Can you try continuing execution a few times? SIGUSR1 is not the original crash signal (SIGABRT). I believe it is an implementation detail of jobserver and completely normal to happen.

[inflation]

Well, I guess that's not the problem. I checked the crash log and it's a different story:

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff204e1462 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff2050f610 pthread_kill + 263
2   libsystem_c.dylib             	0x00007fff20462720 abort + 120
3   build-script-build            	0x0000000106413184 panic_abort::__rust_start_panic::abort::h4ff5becbe3799dd8 + 13
4   build-script-build            	0x0000000106413165 __rust_start_panic + 9
5   build-script-build            	0x00000001063a7908 rust_panic + 35
6   build-script-build            	0x00000001063a7713 std::panicking::rust_panic_with_hook::hb14edfa4a40cb7b7 + 402
7   build-script-build            	0x0000000106312013 std::panicking::begin_panic::_$u7b$$u7b$closure$u7d$$u7d$::h08e8844377b06123 + 60
8   build-script-build            	0x0000000106311cf6 std::sys_common::backtrace::__rust_end_short_backtrace::h05d2952a8ef36f47 + 47
9   build-script-build            	0x0000000106311fc5 std::panicking::begin_panic::hdea40defdc8d92d4 + 64
10  build-script-build            	0x00000001062d4485 proc_macro::bridge::client::_$LT$impl$u20$proc_macro..bridge..Bridge$GT$::with::_$u7b$$u7b$closure$u7d$$u7d$::h7865e4a32c67fc40 + 136
11  build-script-build            	0x00000001062d204c proc_macro::bridge::client::BridgeState::with::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h9e3a3864131874f1 + 23
12  build-script-build            	0x00000001062d6fe7 proc_macro::bridge::scoped_cell::ScopedCell$LT$T$GT$::replace::h0716d1476032a228 + 412
13  build-script-build            	0x000000010630608d proc_macro::bridge::client::BridgeState::with::_$u7b$$u7b$closure$u7d$$u7d$::had9b12a2c83879e8 + 33
14  build-script-build            	0x000000010630279d std::thread::local::LocalKey$LT$T$GT$::try_with::h8caa966aad541787 + 232
15  build-script-build            	0x00000001062fd4f3 std::thread::local::LocalKey$LT$T$GT$::with::ha2abb9cdf04cbf0f + 9
16  build-script-build            	0x000000010630cc69 proc_macro::bridge::client::Span::call_site::h964675f608b8b68c + 20
17  build-script-build            	0x00000001062f2e7d proc_macro::Span::call_site::h36c2761bbaa58a3b + 13
18  build-script-build            	0x00000001062c4810 core::ops::function::FnOnce::call_once::h9113fbd478eeb47d + 17
19  build-script-build            	0x00000001062cb983 std::panicking::try::do_call::hfc095444d73c74b3 + 135
20  build-script-build            	0x00000001062cb731 std::panicking::try::h4ceae983e22f028b + 53
21  build-script-build            	0x00000001062b40f1 std::panic::catch_unwind::h49d78fe4d8329064 + 24
22  build-script-build            	0x00000001062b17aa proc_macro2::detection::initialize::hf3c508d42b16d1d1 + 254
23  build-script-build            	0x00000001062c47fa core::ops::function::FnOnce::call_once::h3028266123316c24 + 13
24  build-script-build            	0x00000001062b40d4 std::sync::once::Once::call_once::_$u7b$$u7b$closure$u7d$$u7d$::he337727b8313de2a + 284
25  build-script-build            	0x00000001063e9547 std::sync::once::Once::call_inner::h0848e284d000a73e + 507
26  build-script-build            	0x00000001062b3fab std::sync::once::Once::call_once::hc6f7b703cefc9041 + 194
27  build-script-build            	0x00000001062b165f proc_macro2::detection::inside_proc_macro::hfe043a228f46dea1 + 152
28  build-script-build            	0x00000001062b979c proc_macro2::imp::Span::call_site::h4a218b42fb4df2f5 + 17
29  build-script-build            	0x00000001062c0337 proc_macro2::Span::call_site::hd42c8d1abece4c5e + 17
30  build-script-build            	0x0000000105cb1e1a bindgen::ir::context::BindgenContext::rust_ident_raw::h4c7878b5f6934433 + 68
31  build-script-build            	0x0000000105cb1c08 bindgen::ir::context::BindgenContext::rust_ident::h818e0714641189dd + 88
32  build-script-build            	0x0000000105e43fe7 _$LT$bindgen..ir..var..Var$u20$as$u20$bindgen..codegen..CodeGenerator$GT$::codegen::h098c6135a240b50c + 545
33  build-script-build            	0x0000000105d7b9bd _$LT$bindgen..ir..item..Item$u20$as$u20$bindgen..codegen..CodeGenerator$GT$::codegen::h6c5b1428544d89b6 + 1301
34  build-script-build            	0x0000000105d9541d _$LT$bindgen..ir..module..Module$u20$as$u20$bindgen..codegen..CodeGenerator$GT$::codegen::_$u7b$$u7b$closure$u7d$$u7d$::hffe90da3a4886477 + 1115
35  build-script-build            	0x0000000105e43daf _$LT$bindgen..ir..module..Module$u20$as$u20$bindgen..codegen..CodeGenerator$GT$::codegen::h488b92b0c3181a94 + 2911
36  build-script-build            	0x0000000105d7b92a _$LT$bindgen..ir..item..Item$u20$as$u20$bindgen..codegen..CodeGenerator$GT$::codegen::h6c5b1428544d89b6 + 1154
37  build-script-build            	0x0000000105da1b32 bindgen::codegen::codegen::_$u7b$$u7b$closure$u7d$$u7d$::h9f800f99e3c3b7bc + 1925
38  build-script-build            	0x0000000105cb49c6 bindgen::ir::context::BindgenContext::gen::h5c81be0adb5b674a + 285
39  build-script-build            	0x0000000105da1398 bindgen::codegen::codegen::h2e4365ceb8a8249d + 92
40  build-script-build            	0x0000000105dce59c bindgen::Bindings::generate::hf3fcb1dce2ffec91 + 3134
41  build-script-build            	0x0000000105dcac38 bindgen::Builder::generate::hfdf7721fd5202a9d + 942
42  build-script-build            	0x00000001059b6294 build_script_build::main::ha8551c308cab0279 + 1376
43  build-script-build            	0x00000001059b537a core::ops::function::FnOnce::call_once::h5c667e9ca9072815 + 10
44  build-script-build            	0x00000001059b59bc std::sys_common::backtrace::__rust_begin_short_backtrace::h36efbcb584851abb + 17
45  build-script-build            	0x00000001059b50ca std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::h2401841dd054f7ef + 20
46  build-script-build            	0x00000001063c897e core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnOnce$LT$A$GT$$u20$for$u20$$RF$F$GT$::call_once::hfd1ff8297a328933 + 10
47  build-script-build            	0x00000001063a6ea3 std::panicking::try::do_call::h12a22371e6593e24 + 70
48  build-script-build            	0x00000001063a6db0 std::panicking::try::hbb82910c171fd482 + 61
49  build-script-build            	0x00000001063a7a4f std::rt::lang_start_internal::hf1bcf7e1504067fc + 186
50  build-script-build            	0x00000001059b50b1 std::rt::lang_start::hb2698138a224f95f + 44
51  build-script-build            	0x00000001059b67d8 main + 64
52  libdyld.dylib                 	0x00007fff2052a621 start + 1

The panic should happen here

    fn with<R>(f: impl FnOnce(&mut Bridge<'_>) -> R) -> R {
        BridgeState::with(|state| match state {
            BridgeState::NotConnected => {
                panic!("procedural macro API is used outside of a procedural macro");
            }
            BridgeState::InUse => {
                panic!("procedural macro API is used while it's already in use");
            }
            BridgeState::Connected(bridge) => f(bridge),
        })
    }

I know nothing about proc macros. Might be hitting some limitations here.

[bjorn3]

proc_macro2::detection::inside_proc_macro

Looks like bindgen uses an older proc_macro2 version that checks if it is running inside a proc macro by calling a proc_macro function and checking if it panics (which happens when it is not running inside a proc macro) cg_clif doesn't yet support unwinding on panics, so the panic results in an abort. Newer proc_macro2 versions use a dedicated method of proc_macro which doesn't panic.

I will close this in favor of https://github.com/bjorn3/rustc_codegen_cranelift/issues/1101 which is caused by the same proc_macro2 problem.

[inflation]

Cool thanks.