Failed to build datafusion-python 0.8.0 because of substrait build command

[cclienti]

Describe the bug
Failed to build datafusion-python 0.8.0 because of the run custom build command of substrait v0.4.0

error: failed to run custom build command for `substrait v0.4.0`

Caused by:
  process didn't exit successfully: `./datafusion-0.8.0/target/release/build/substrait-236e8658a3f78ffe/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=FORCE_REBUILD
  cargo:rerun-if-changed=substrait/text/simple_extensions_schema.yaml

  --- stderr
  thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: NoRustfmt', .cargo/registry/src/gb.xjqchip.workers.dev-1ecc6299db9ec823/typify-impl-0.0.10/src/lib.rs:614:47
  stack backtrace:
     0:     0x56469a53ea4a - std::backtrace_rs::backtrace::libunwind::trace::h79937bc171ada62c
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
     1:     0x56469a53ea4a - std::backtrace_rs::backtrace::trace_unsynchronized::h2292bca8571cb919
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
     2:     0x56469a53ea4a - std::sys_common::backtrace::_print_fmt::h9c461f248e4ae90d
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/sys_common/backtrace.rs:65:5
     3:     0x56469a53ea4a - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::he9fe6bf1a39182e1
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/sys_common/backtrace.rs:44:22
     4:     0x56469a56530e - core::fmt::write::h032658c119c720d7
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/fmt/mod.rs:1208:17
     5:     0x56469a53b325 - std::io::Write::write_fmt::h299fc90dfae41c0d
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/io/mod.rs:1682:15
     6:     0x56469a53e815 - std::sys_common::backtrace::_print::heb70d25df9937e3f
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/sys_common/backtrace.rs:47:5
     7:     0x56469a53e815 - std::sys_common::backtrace::print::had745c0a76b8b521
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/sys_common/backtrace.rs:34:9
     8:     0x56469a53ffff - std::panicking::default_hook::{{closure}}::h1ea782cdfa2fd097
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:267:22
     9:     0x56469a53fd3b - std::panicking::default_hook::h1cc3af63455a163c
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:286:9
    10:     0x56469a54070c - std::panicking::rust_panic_with_hook::h5cafdc4b3bfd5528
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:688:13
    11:     0x56469a5404a9 - std::panicking::begin_panic_handler::{{closure}}::hf31c60f40775892c
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:579:13
    12:     0x56469a53eefc - std::sys_common::backtrace::__rust_end_short_backtrace::h28a5c7be595826cd
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/sys_common/backtrace.rs:137:18
    13:     0x56469a5401b2 - rust_begin_unwind
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:575:5
    14:     0x564699f646c3 - core::panicking::panic_fmt::h8fa27a0b37dd98b7
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/panicking.rs:64:14
    15:     0x564699f64b53 - core::result::unwrap_failed::h62abf61e411aaa08
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/result.rs:1791:5
    16:     0x56469a0f406b - core::result::Result<T,E>::unwrap::h9fad5175f68c3e6b
    17:     0x56469a0507f9 - <typify_impl::TypeSpace as alloc::string::ToString>::to_string::h5e656dd62addd117
    18:     0x564699fdee41 - build_script_build::text::hc1db7c7902a70157
    19:     0x564699fdf5c5 - build_script_build::main::h52aab4b7bbdc8d16
    20:     0x564699fb9d43 - core::ops::function::FnOnce::call_once::h60900042eff19fbe
    21:     0x564699fc0769 - std::sys_common::backtrace::__rust_begin_short_backtrace::h2c738c90f1fe1f91
    22:     0x564699f8e319 - std::rt::lang_start::{{closure}}::h5f2c7336d2b77417
    23:     0x56469a536a2c - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::hfa1c3687c9a20bb8
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/ops/function.rs:606:13
    24:     0x56469a536a2c - std::panicking::try::do_call::h0497133ebe1341e5
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:483:40
    25:     0x56469a536a2c - std::panicking::try::h6c3de05c7ca5d07f
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:447:19
    26:     0x56469a536a2c - std::panic::catch_unwind::h969058ecb5334b30
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panic.rs:137:14
    27:     0x56469a536a2c - std::rt::lang_start_internal::{{closure}}::h0de6d98cafb42a58
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/rt.rs:148:48
    28:     0x56469a536a2c - std::panicking::try::do_call::h77301dcf43953993
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:483:40
    29:     0x56469a536a2c - std::panicking::try::h45b05523a2fe135a
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:447:19
    30:     0x56469a536a2c - std::panic::catch_unwind::hd95cea544b2b9bd2
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panic.rs:137:14
    31:     0x56469a536a2c - std::rt::lang_start_internal::hd16e6ff7bf05a444
                                 at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/rt.rs:148:20
    32:     0x564699f8e2f7 - std::rt::lang_start::h04902ebcc3a311b6
    33:     0x564699fdfa75 - main
    34:     0x7f53dd16dc05 - __libc_start_main
                                 at /usr/src/debug/glibc-2.17-c758a686/csu/../csu/libc-start.c:274
    35:     0x564699f64d32 - <unknown>
    36:                0x0 - <unknown>
warning: build failed, waiting for other jobs to finish...

maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `"cargo" "rustc" "--release" "--manifest-path" "./datafusion-0.8.0/Cargo.toml" "--message-format" "json" "--locked" "--lib" "--crate-type" "cdylib"`

To Reproduce
redhat-7 + conda environment with:

• python=3.8
• pip
• maturin>=0.14.14
• rust>=1.64.0

Expected behavior
Maturin should succeed in producing the "wheel" package

Additional context
I use python 3.8 and rust-1.64.0 or 1.67.1

[mbrobbel]

Do you have rustfmt installed? (It's this error)

[cclienti]

Thank you for your response :)

Yes rustfmt is installed, but I found some interesting information here: https://aur.archlinux.org/packages/ztui

The rustc/cargo are installed using a conda package, not using rustup. When I try to build datafusion-python with a compiler installed with rustup everything works fine (in the meantime I discovered that protobuf was missing)!

I will try to understand what is missing in the rust conda package or if it is only related to the missing protobuf package.

[jdye64]

I don't have a solution yet but wanted to voice that I too see this with a similar environment setup on an x86 server. I know we have seen it before on mac arm based chips but the x86 was a new one.

[cclienti]

My initial problem was due to the missing protobuf package, but for an unknown reason the error output is different when the rust toolchain is installed with conda versus installed by rustup (in both cases rustfmt is intalled).