Skip to content

[Flang] Implicitly add clang if flang enabled. #123964

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 26, 2025

Conversation

Meinersbur
Copy link
Member

Clang is required to compile Flang. Instead of erroring-out if Clang is enabled, for convinience implicitly add it to LLVM_ENABLE_PROJECTS, consistent with how the MLIR dependency is handled.

This is mainly intended as a discussion about whether LLVM_ENABLE_RUNTIMES=flang-rt (#110217) is to be enabled implicitly as well (https://discourse.llvm.org/t/buildbot-changes-with-llvm-enable-runtimes-flang-rt/83571/2). If yes, not implicitly enabling Clang would be inconsistent with how the MLIR and flang-rt dependency are enforced.

I'd prefer to not do so implicitly:

  • LLVM_ENABLE_PROJECTS and LLVM_ENABLE_RUNTIMES are user-controlled. Overwriting them takes away from their decision. I would expect to have only those projects/runtimes to be built that I have specified.

  • The global variable LLVM_ENABLE_PROJECTS and the CMakeCache.txt entry for LLVM_ENABLE_PROJECTS are different entities. How it is implemented, the global LLVM_ENABLE_PROJECTS shadows the cache entry, so they have two different values. This adds to confusion. Overwriting the cache value with FORCE option would cause CMake to re-run a second time. Better get it right the first time.

  • The buildbots use the settings depends_on_projects=[..] (which the values of LLVM_ENABLE_PROJECTS and LLVM_ENABLE_RUNTIMES is derived from). If clang, mlir, or flang-rt is not specified there, the buildbot will not trigger a rebuild if files in those directories have been changed.

  • For LLVM_ENABLE_RUNTIMES, including flang-rt, it is possible to build the runtime separately/out-of-tree/standalone, or not for the default target for whatever reason, in which case the implicit addition of LLVM_ENABLE_RUNTIMES=flang-rt must be again disabled somehow. This adds to the complexity to the build system.

  • LLVM_ENABLE_PROJECTS=clang also does not implicitly add any project/runtime that is technically required for a complete toolchain, such as lld, compiler-rt, libc, libcxx, libcxxabi, unwind.

@cor3ntin cor3ntin added cmake Build system in general and CMake in particular flang:build labels Jan 23, 2025
@DavidTruby
Copy link
Member

DavidTruby commented Feb 5, 2025

I would like this but I think some downstreams don't depend on the clang-based driver so might have objections.

The difference between this/flang-rt and e.g. clang not adding other things is that clang can use alternatives for all of its dependencies, and does by default on most platforms. There's no alternative runtime to flang and it's not designed to support that in future; the runtime is considered a part of the compiler that must always be in sync with it.

@Meinersbur
Copy link
Member Author

Meinersbur commented Feb 5, 2025

If some downstreams don't need Clang, how do they build Flang? CMake currently aborts with FATAL_ERROR. If you try a standalone build, you get:

CMake Error at CMakeLists.txt:110 (find_package):
  Could not find a package configuration file provided by "Clang" with any of
  the following names:

    ClangConfig.cmake
    clang-config.cmake

  Add the installation prefix of "Clang" to CMAKE_PREFIX_PATH or set
  "Clang_DIR" to a directory containing one of the above files.  If "Clang"
  provides a separate development package or SDK, be sure it has been
  installed.

@DavidTruby
Copy link
Member

Not sure; I'm not involved personally in any downstream that doesn't use the clang-based driver. I just remember this as the justification previously that we add mlir but don't add clang.

@Meinersbur
Copy link
Member Author

Meinersbur commented Feb 5, 2025

The difference between this/flang-rt and e.g. clang not adding other things is that clang can use alternatives for all of its dependencies, and does by default on most platforms. There's no alternative runtime to flang and it's not designed to support that in future; the runtime is considered a part of the compiler that must always be in sync with it.

You may still compile the runtime separately, using a standalone build, or you don't even need it for the host platform, only for the target platform (cross-compile). In either case, you need a workaround to not add "flang-rt" implicitly. There isn't one for "clang" and it seems nobody requested one.

The position

  1. "clang" should be added explicitly by the user
  2. "mlir" and "flang-rt" should be added implicitly

feels inconsistent to me.

@DavidTruby
Copy link
Member

I agree. I personally think it should all be added implicitly, with flags for not doing so.

@DavidTruby
Copy link
Member

That or the cmake should be fixed so that we don't need clang to be in ENABLE_PROJECTS. We don't actually need all of clang, just specific driver stuff that could be built as dependencies in the cmake without requiring clang in its entirety.

@Meinersbur Meinersbur merged commit e5129b7 into llvm:main Mar 26, 2025
11 checks passed
@h-vetinari
Copy link
Contributor

How does this work when using a pre-built clang (say for a matching version and/or commit)? We need a way to build flang independently of clang, not least because of resource constraints.

@llvm-ci
Copy link
Collaborator

llvm-ci commented Mar 26, 2025

LLVM Buildbot has detected a new failure on builder lldb-aarch64-ubuntu running on linaro-lldb-aarch64-ubuntu while building llvm at step 6 "test".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/59/builds/14954

Here is the relevant piece of the build log for the reference
Step 6 (test) failure: build (failure)
...
UNSUPPORTED: lldb-api :: functionalities/ptrauth_diagnostics/BLRAA_error/TestPtrauthBLRAADiagnostic.py (563 of 2110)
UNSUPPORTED: lldb-api :: functionalities/ptrauth_diagnostics/BRAA_error/TestPtrauthBRAADiagnostic.py (564 of 2110)
UNSUPPORTED: lldb-api :: functionalities/ptrauth_diagnostics/LDRAA_error/TestPtrauthLDRAADiagnostic.py (565 of 2110)
UNSUPPORTED: lldb-api :: functionalities/ptrauth_diagnostics/brkC47x_code/TestPtrauthBRKc47xDiagnostic.py (566 of 2110)
PASS: lldb-api :: functionalities/progress_reporting/TestProgressReporting.py (567 of 2110)
UNSUPPORTED: lldb-api :: functionalities/ptrauth_diagnostics/brkC47x_x16_invalid/TestPtrauthBRKc47xX16Invalid.py (568 of 2110)
PASS: lldb-api :: functionalities/process_group/TestChangeProcessGroup.py (569 of 2110)
UNSUPPORTED: lldb-api :: functionalities/rerun_and_expr_dylib/TestRerunAndExprDylib.py (570 of 2110)
PASS: lldb-api :: functionalities/rerun/TestRerun.py (571 of 2110)
PASS: lldb-api :: functionalities/reverse-execution/TestReverseContinueNotSupported.py (572 of 2110)
FAIL: lldb-api :: functionalities/rerun_and_expr/TestRerunAndExpr.py (573 of 2110)
******************** TEST 'lldb-api :: functionalities/rerun_and_expr/TestRerunAndExpr.py' FAILED ********************
Script:
--
/usr/bin/python3.10 /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./lib --env LLVM_INCLUDE_DIR=/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/include --env LLVM_TOOLS_DIR=/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin --arch aarch64 --build-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/lldb-test-build.noindex --lldb-module-cache-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin/lldb --compiler /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin/clang --dsymutil /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin --lldb-obj-root /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/tools/lldb --lldb-libs-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./lib /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/rerun_and_expr -p TestRerunAndExpr.py
--
Exit Code: -11

Command Output (stdout):
--
lldb version 21.0.0git (https://github.com/llvm/llvm-project.git revision e5129b7e2092c54e503f60cc0ac552c765d5a144)
  clang revision e5129b7e2092c54e503f60cc0ac552c765d5a144
  llvm revision e5129b7e2092c54e503f60cc0ac552c765d5a144
Skipping the following test categories: ['libc++', 'dsym', 'gmodules', 'debugserver', 'objc']

--
Command Output (stderr):
--
UNSUPPORTED: LLDB (/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang-aarch64) :: test_dsym (TestRerunAndExpr.TestRerunExpr) (test case does not fall in any category of interest for this run) 
PASS: LLDB (/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang-aarch64) :: test_dwarf (TestRerunAndExpr.TestRerunExpr)
PASS: LLDB (/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang-aarch64) :: test_dwo (TestRerunAndExpr.TestRerunExpr)
----------------------------------------------------------------------
Ran 3 tests in 0.994s

OK (skipped=1)

--

********************
UNSUPPORTED: lldb-api :: functionalities/scripted_process/TestStackCoreScriptedProcess.py (574 of 2110)
UNSUPPORTED: lldb-api :: functionalities/scripted_process_empty_memory_region/TestScriptedProcessEmptyMemoryRegion.py (575 of 2110)
PASS: lldb-api :: functionalities/return-value/TestReturnValue.py (576 of 2110)
UNSUPPORTED: lldb-api :: functionalities/set-data/TestSetData.py (577 of 2110)
PASS: lldb-api :: functionalities/show_location/TestShowLocationDwarf5.py (578 of 2110)
PASS: lldb-api :: functionalities/scripted_process/TestScriptedProcess.py (579 of 2110)
PASS: lldb-api :: functionalities/signal/handle-abrt/TestHandleAbort.py (580 of 2110)
PASS: lldb-api :: functionalities/reverse-execution/TestReverseContinueWatchpoints.py (581 of 2110)
PASS: lldb-api :: functionalities/signal/handle-segv/TestHandleSegv.py (582 of 2110)
PASS: lldb-api :: functionalities/signal/TestSendSignal.py (583 of 2110)

@Meinersbur
Copy link
Member Author

How does this work when using a pre-built clang (say for a matching version and/or commit)? We need a way to build flang independently of clang, not least because of resource constraints.

This has already been discussed in this thread. If you don't add LLVM_ENABLE_PROJECTS=flang, then LLVM_ENABLE_PROJECTS=clang will not be added either. If you have a standalone build of flang, you are not adding LLVM_ENABLE_PROJECTS=flang to the LLVM build.

Meinersbur added a commit that referenced this pull request Mar 26, 2025
Remove the FLANG_INCLUDE_RUNTIME option which was replaced by
LLVM_ENABLE_RUNTIMES=flang-rt.

The FLANG_INCLUDE_RUNTIME option was added in #122336 which disables the
non-runtimes build instructions for the Flang runtime so they do not
conflict with the LLVM_ENABLE_RUNTIMES=flang-rt option added in #110217.
In order to not maintain multiple build instructions for the same thing,
this PR completely removes the old build instructions (effectively
forcing FLANG_INCLUDE_RUNTIME=OFF).

As per discussion in
https://discourse.llvm.org/t/buildbot-changes-with-llvm-enable-runtimes-flang-rt/83571/2
we now implicitly add LLVM_ENABLE_RUNTIMES=flang-rt whenever Flang is
compiled in a bootstrapping (non-standalone) build. Because it is
possible to build Flang-RT separately, this behavior can be disabled
using `-DFLANG_ENABLE_FLANG_RT=OFF`. Also see the discussion an
implicitly adding runtimes/projects in #123964.
Meinersbur added a commit that referenced this pull request Apr 30, 2025
This reverts commit 27539c3. Retry
with new buildbot configuration after master restart.

Original message:

Remove the FLANG_INCLUDE_RUNTIME option which was replaced by
LLVM_ENABLE_RUNTIMES=flang-rt.

The FLANG_INCLUDE_RUNTIME option was added in #122336 which disables the
non-runtimes build instructions for the Flang runtime so they do not
conflict with the LLVM_ENABLE_RUNTIMES=flang-rt option added in #110217.
In order to not maintain multiple build instructions for the same thing,
this PR completely removes the old build instructions (effectively
forcing FLANG_INCLUDE_RUNTIME=OFF).

As per discussion in
https://discourse.llvm.org/t/buildbot-changes-with-llvm-enable-runtimes-flang-rt/83571/2
we now implicitly add LLVM_ENABLE_RUNTIMES=flang-rt whenever Flang is
compiled in a bootstrapping (non-standalone) build. Because it is
possible to build Flang-RT separately, this behavior can be disabled
using `-DFLANG_ENABLE_FLANG_RT=OFF`. Also see the discussion an
implicitly adding runtimes/projects in #123964.
IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
This reverts commit 27539c3. Retry
with new buildbot configuration after master restart.

Original message:

Remove the FLANG_INCLUDE_RUNTIME option which was replaced by
LLVM_ENABLE_RUNTIMES=flang-rt.

The FLANG_INCLUDE_RUNTIME option was added in llvm#122336 which disables the
non-runtimes build instructions for the Flang runtime so they do not
conflict with the LLVM_ENABLE_RUNTIMES=flang-rt option added in llvm#110217.
In order to not maintain multiple build instructions for the same thing,
this PR completely removes the old build instructions (effectively
forcing FLANG_INCLUDE_RUNTIME=OFF).

As per discussion in
https://discourse.llvm.org/t/buildbot-changes-with-llvm-enable-runtimes-flang-rt/83571/2
we now implicitly add LLVM_ENABLE_RUNTIMES=flang-rt whenever Flang is
compiled in a bootstrapping (non-standalone) build. Because it is
possible to build Flang-RT separately, this behavior can be disabled
using `-DFLANG_ENABLE_FLANG_RT=OFF`. Also see the discussion an
implicitly adding runtimes/projects in llvm#123964.
IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
This reverts commit 27539c3. Retry
with new buildbot configuration after master restart.

Original message:

Remove the FLANG_INCLUDE_RUNTIME option which was replaced by
LLVM_ENABLE_RUNTIMES=flang-rt.

The FLANG_INCLUDE_RUNTIME option was added in llvm#122336 which disables the
non-runtimes build instructions for the Flang runtime so they do not
conflict with the LLVM_ENABLE_RUNTIMES=flang-rt option added in llvm#110217.
In order to not maintain multiple build instructions for the same thing,
this PR completely removes the old build instructions (effectively
forcing FLANG_INCLUDE_RUNTIME=OFF).

As per discussion in
https://discourse.llvm.org/t/buildbot-changes-with-llvm-enable-runtimes-flang-rt/83571/2
we now implicitly add LLVM_ENABLE_RUNTIMES=flang-rt whenever Flang is
compiled in a bootstrapping (non-standalone) build. Because it is
possible to build Flang-RT separately, this behavior can be disabled
using `-DFLANG_ENABLE_FLANG_RT=OFF`. Also see the discussion an
implicitly adding runtimes/projects in llvm#123964.
IanWood1 pushed a commit to IanWood1/llvm-project that referenced this pull request May 6, 2025
This reverts commit 27539c3. Retry
with new buildbot configuration after master restart.

Original message:

Remove the FLANG_INCLUDE_RUNTIME option which was replaced by
LLVM_ENABLE_RUNTIMES=flang-rt.

The FLANG_INCLUDE_RUNTIME option was added in llvm#122336 which disables the
non-runtimes build instructions for the Flang runtime so they do not
conflict with the LLVM_ENABLE_RUNTIMES=flang-rt option added in llvm#110217.
In order to not maintain multiple build instructions for the same thing,
this PR completely removes the old build instructions (effectively
forcing FLANG_INCLUDE_RUNTIME=OFF).

As per discussion in
https://discourse.llvm.org/t/buildbot-changes-with-llvm-enable-runtimes-flang-rt/83571/2
we now implicitly add LLVM_ENABLE_RUNTIMES=flang-rt whenever Flang is
compiled in a bootstrapping (non-standalone) build. Because it is
possible to build Flang-RT separately, this behavior can be disabled
using `-DFLANG_ENABLE_FLANG_RT=OFF`. Also see the discussion an
implicitly adding runtimes/projects in llvm#123964.
GeorgeARM pushed a commit to GeorgeARM/llvm-project that referenced this pull request May 7, 2025
This reverts commit 27539c3. Retry
with new buildbot configuration after master restart.

Original message:

Remove the FLANG_INCLUDE_RUNTIME option which was replaced by
LLVM_ENABLE_RUNTIMES=flang-rt.

The FLANG_INCLUDE_RUNTIME option was added in llvm#122336 which disables the
non-runtimes build instructions for the Flang runtime so they do not
conflict with the LLVM_ENABLE_RUNTIMES=flang-rt option added in llvm#110217.
In order to not maintain multiple build instructions for the same thing,
this PR completely removes the old build instructions (effectively
forcing FLANG_INCLUDE_RUNTIME=OFF).

As per discussion in
https://discourse.llvm.org/t/buildbot-changes-with-llvm-enable-runtimes-flang-rt/83571/2
we now implicitly add LLVM_ENABLE_RUNTIMES=flang-rt whenever Flang is
compiled in a bootstrapping (non-standalone) build. Because it is
possible to build Flang-RT separately, this behavior can be disabled
using `-DFLANG_ENABLE_FLANG_RT=OFF`. Also see the discussion an
implicitly adding runtimes/projects in llvm#123964.
Ankur-0429 pushed a commit to Ankur-0429/llvm-project that referenced this pull request May 9, 2025
This reverts commit 27539c3. Retry
with new buildbot configuration after master restart.

Original message:

Remove the FLANG_INCLUDE_RUNTIME option which was replaced by
LLVM_ENABLE_RUNTIMES=flang-rt.

The FLANG_INCLUDE_RUNTIME option was added in llvm#122336 which disables the
non-runtimes build instructions for the Flang runtime so they do not
conflict with the LLVM_ENABLE_RUNTIMES=flang-rt option added in llvm#110217.
In order to not maintain multiple build instructions for the same thing,
this PR completely removes the old build instructions (effectively
forcing FLANG_INCLUDE_RUNTIME=OFF).

As per discussion in
https://discourse.llvm.org/t/buildbot-changes-with-llvm-enable-runtimes-flang-rt/83571/2
we now implicitly add LLVM_ENABLE_RUNTIMES=flang-rt whenever Flang is
compiled in a bootstrapping (non-standalone) build. Because it is
possible to build Flang-RT separately, this behavior can be disabled
using `-DFLANG_ENABLE_FLANG_RT=OFF`. Also see the discussion an
implicitly adding runtimes/projects in llvm#123964.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmake Build system in general and CMake in particular flang:build
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants