"west flash" stopped working

[holgerschurig]

Describe the bug
When I run "west flash", I get a long backtrace:

schurig@desktop:~/d/v73$ west flash
-- west flash: rebuilding
ninja: no work to do.
-- west flash: using runner openocd
Traceback (most recent call last):
  File "/home/schurig/.local/bin/west", line 10, in <module>
    sys.exit(main())
  File "/home/schurig/.local/lib/python3.7/site-packages/west/app/main.py", line 779, in main
    app.run(argv or sys.argv[1:])
  File "/home/schurig/.local/lib/python3.7/site-packages/west/app/main.py", line 106, in run
    self.run_command(argv)
  File "/home/schurig/.local/lib/python3.7/site-packages/west/app/main.py", line 338, in run_command
    self.run_extension(args.command, argv)
  File "/home/schurig/.local/lib/python3.7/site-packages/west/app/main.py", line 408, in run_extension
    command.run(args, unknown, self.topdir, manifest=self.manifest)
  File "/home/schurig/.local/lib/python3.7/site-packages/west/commands.py", line 132, in run
    self.do_run(args, unknown)
  File "/home/schurig/d/v73/zephyr/scripts/west_commands/flash.py", line 29, in do_run
    do_run_common(self, my_args, runner_args)
  File "/home/schurig/d/v73/zephyr/scripts/west_commands/run_common.py", line 227, in do_run_common
    runner = runner_cls.create(runner_config, args)
  File "/home/schurig/d/v73/zephyr/scripts/west_commands/runners/core.py", line 462, in create
    ret = cls.do_create(cfg, args)
  File "/home/schurig/d/v73/zephyr/scripts/west_commands/runners/openocd.py", line 126, in do_create
    gdb_port=args.gdb_port)
  File "/home/schurig/d/v73/zephyr/scripts/west_commands/runners/openocd.py", line 50, in __init__
    for p in cfg.openocd_search:
TypeError: 'NoneType' object is not iterable

The issue is that build/zephy/runners.yaml now contains an empty openocd_search: entry:

...
config:
  board_dir: /home/schurig/d/v73/boards/arm/advantech_pcm8420
  # Build outputs:
  elf_file: zephyr.elf
  hex_file: zephyr.hex
  bin_file: zephyr.bin
  # Host tools:
  gdb: /usr/bin/gdb-multiarch
  openocd: /usr/bin/openocd
  openocd_search:

# Runner specific arguments
args:

Deleteing this superfluous openocd_search: entry fixes my issue. So my guess is that c3db83b has the bug.

To Reproduce
Steps to reproduce the behavior:

1. cd zephyr/samples/basic/blinky
2. west build -b nucleo_f303re
3. west flash
4. See error

Environment (please complete the following information):

• OS: Linux
• Toolchain: doesn't matter
• openocd 0.11.0-rc2 (self-made .deb package) in the path as /usr/bin/openocd
• Commit SHA or Version used: current zephyr from git, SHA e12e126

[nandojve]

Hi @holgerschurig ,

Thank you for bring this to us. I just update my env and try reproduce issue listed.

This is the resulting:

# Available runners configured by board.cmake.
runners:
- openocd
- jlink

# Default flash runner if --runner is not given.
flash-runner: openocd

# Default debug runner if --runner is not given.
debug-runner: openocd

# Common runner configuration values.
config:
  board_dir: $(HOME)/zephyr/mainline/zephyr/boards/arm/nucleo_f303re
  # Build outputs:
  elf_file: zephyr.elf
  hex_file: zephyr.hex
  bin_file: zephyr.bin
  # Host tools:
  gdb: $(HOME)/zephyr-sdk-0.13.0/arm-zephyr-eabi/bin/arm-zephyr-eabi-gdb
  openocd: $(HOME)/zephyr-sdk-0.13.0/sysroots/x86_64-pokysdk-linux/usr/bin/openocd
  openocd_search:
    - $(HOME)/zephyr-sdk-0.13.0/sysroots/x86_64-pokysdk-linux/usr/share/openocd/scripts

# Runner specific arguments
args:
  openocd:
    - --cmd-load
    - flash write_image erase
    - --cmd-verify
    - verify_image
  jlink:
    - --dt-flash=y
    - --device=STM32F303RE
    - --speed=4000

My steps were:

• cd <your zephyr home>
• git fetch upstream
• git checkout main
• git rebase upstream/main
• west update
• pip3 install --user -r scripts/requirements.txt

Can you check again and report? It will be nice if you can perform a check using Zephyr OpenOCD version too.

[holgerschurig]

git fetch upstream; git checkout main

Huh? "git fetch upstream" yields "fatal: 'upstream' does not appear to be a git repository". It might be that you have an "upstream" in your .git/config because of read/write access? I did the anonymous clone via "git clone https://github.com/zephyrproject-rtos/zephyr.git" originally.

So instead I did the common thing that is quite normal with git: "git pull". I am now at:

´``
git rev-parse HEAD
c235144

> it rebase upstream/main

I also skipped the rebasing thing, as I don't develop (change anything) inside the zephyr source code. And I don't switch branches. And the rebase would fail anyway due to the "upstream" ...

Just to be sure: "git status" is clean.

> west update

For "west update", I fail to see how this can influence what the hal_stm32 and cmsis (the only modules I am using) can have any influence on how west creates the runners.yaml file. Can you elaborate?  But in any case, these two modules are up-to-date.

I have not installed the other modules at all, I only run "west update cmsis hal_stm32", so I don't get all of the (for me!) junk.

> It will be nice if you can perform a check using Zephyr OpenOCD version too. 

Why?

* openocd is in the path, even correctly in `runners.yaml`
* it worked the last half year without any issue, I have probably flashed a hundred times
* `west flash` crashes before it has a chance to call openocd
* so it can be concluded that it isn't the culprit

So, one obvious fix is that west doesn't emit an empty openocd_search: stance. And also that it checks if this variable is nil.

> pip3 install --user -r scripts/requirements.txt

This is unfortunately incompatible with Debian Buster. The requirements pull in everything under the moon, even things that I don't use. And PyOCD from requirements-run-test.txt fails to install. I didn't care about this, as I'm happy with openocd as-is and I don'd do the run tests.

So, excluding pyocd, all my python3 packages are up-to-date, according to pip3.

And the following command sequence:

schurig@desktop:/d/v73/zephyr/samples/basic/blinky$
schurig@desktop:/d/v73/zephyr/samples/basic/blinky$ rm -rf build/
schurig@desktop:~/d/v73/zephyr/samples/basic/blinky$ west build -b nucleo_f303re
-- west build: generating a build system
Including boilerplate (Zephyr base): /home/schurig/d/v73/zephyr/cmake/app/boilerplate.cmake
CMake Deprecation Warning at /home/schurig/d/v73/zephyr/cmake/app/boilerplate.cmake:37 (cmake_policy):
The OLD behavior for policy CMP0079 will be removed from a future version
of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
/home/schurig/d/v73/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:24 (include)
/home/schurig/d/v73/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:35 (include_boilerplate)
CMakeLists.txt:4 (find_package)

-- Application: /home/schurig/d/v73/zephyr/samples/basic/blinky
-- Zephyr version: 2.6.99 (/home/schurig/d/v73/zephyr), build: zephyr-v2.6.0-3010-gc235144dc23a
-- Found Python3: /usr/bin/python3.7 (found suitable exact version "3.7.3") found components: Interpreter
-- Found west (found suitable version "0.11.1", minimum required is "0.7.1")
-- Board: nucleo_f303re
-- Cache files will be written to: /home/schurig/.cache/zephyr
-- Found dtc: /usr/bin/dtc (found suitable version "1.4.7", minimum required is "1.4.6")
-- Found toolchain: cross-compile (/usr/bin/arm-none-eabi-)
-- Found BOARD.dts: /home/schurig/d/v73/zephyr/boards/arm/nucleo_f303re/nucleo_f303re.dts
-- Generated zephyr.dts: /home/schurig/d/v73/zephyr/samples/basic/blinky/build/zephyr/zephyr.dts
-- Generated devicetree_unfixed.h: /home/schurig/d/v73/zephyr/samples/basic/blinky/build/zephyr/include/generated/devicetree_unfixed.h
-- Generated device_extern.h: /home/schurig/d/v73/zephyr/samples/basic/blinky/build/zephyr/include/generated/device_extern.h
-- Including generated dts.cmake file: /home/schurig/d/v73/zephyr/samples/basic/blinky/build/zephyr/dts.cmake
Parsing /home/schurig/d/v73/zephyr/Kconfig
Loaded configuration '/home/schurig/d/v73/zephyr/boards/arm/nucleo_f303re/nucleo_f303re_defconfig'
Merged configuration '/home/schurig/d/v73/zephyr/samples/basic/blinky/prj.conf'
Configuration saved to '/home/schurig/d/v73/zephyr/samples/basic/blinky/build/zephyr/.config'
Kconfig header saved to '/home/schurig/d/v73/zephyr/samples/basic/blinky/build/zephyr/include/generated/autoconf.h'
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/arm-none-eabi-gcc
-- Configuring done
-- Generating done
-- Build files have been written to: /home/schurig/d/v73/zephyr/samples/basic/blinky/build
-- west build: building application
[1/145] Preparing syscall dependency handling

[138/145] Linking C executable zephyr/zephyr_prebuilt.elf

[145/145] Linking C executable zephyr/zephyr.elf
Memory region Used Size Region Size %age Used
FLASH: 14144 B 512 KB 2.70%
SRAM: 4192 B 64 KB 6.40%
IDT_LIST: 0 GB 2 KB 0.00%
schurig@desktop:~/d/v73/zephyr/samples/basic/blinky$ grep -C1 openocd_search build/zephyr/runners.yaml
openocd: /usr/bin/openocd
openocd_search:

shows that still the empty openocd_search line is emitted.

My suggestion: simply applying defensive programming by:

* checking for nil in zephyr/scripts/west_commands/runners/openocd.py:50
* stop writing the empty openocd_search

It never have hurt a programmer in an interpreted language (like Python) to check everything under the sun ...

[holgerschurig]

BTW; i went to some more hoops :-)

I used

rm -rf build; west build -b nucleo_f303re; cat openocd_search build/zephyr/runners.yaml

to check for this superfluous openocd_search: line. It didn't appear before. This is the git bisect log:

schurig@desktop:~/d/v73/zephyr$ git bisect good
Bisecting: 1504 revisions left to test after this (roughly 11 steps)
[453ce7b59358e94c51afdd583d8d84df9e94855e] tests: arm: adding a new test-suite for HardFault validation
schurig@desktop:~/d/v73/zephyr$ git bisect good
Bisecting: 752 revisions left to test after this (roughly 10 steps)
[538f97f73711c73f8a6e6478d461978e3cf2c295] Bluetooth: Mesh: Clarify name length in bt_mesh_model_data_store
schurig@desktop:~/d/v73/zephyr$ git bisect good
Bisecting: 376 revisions left to test after this (roughly 9 steps)
[9917de4a1b8f50e70d1e45c0983f1a97a36ea443] pm: fully initialize pm_device on Z_DEVICE_STATE_DEFINE
schurig@desktop:~/d/v73/zephyr$ git bisect bad
Bisecting: 187 revisions left to test after this (roughly 8 steps)
[fcfb1d9872dfd538c676b99cc9453af5ecae84d2] drivers: sensors: fix of sensor channel get for z axis
schurig@desktop:~/d/v73/zephyr$ git bisect good
Bisecting: 93 revisions left to test after this (roughly 7 steps)
[66c931dbd4d4e5e6b441223742d605aee3bbbf05] tests: interrupt: fix typos in the commits
schurig@desktop:~/d/v73/zephyr$ git bisect good
Bisecting: 46 revisions left to test after this (roughly 6 steps)
[50a31d2f36fb5c75f3cf1f0d6c102cb1fd2e2d08] drivers: sensor: bmg160: check gpio calls return code
schurig@desktop:~/d/v73/zephyr$ git bisect good
Bisecting: 23 revisions left to test after this (roughly 5 steps)
[c2a01af00327ed9b167873e44670b386ff9d4215] x86: pin z_x86_set_stack_guard()
schurig@desktop:~/d/v73/zephyr$ git bisect good
Bisecting: 11 revisions left to test after this (roughly 4 steps)
[27f36bda51a68de8af542dbcda06e737a75aed1d] boards: qemu_x86_tiny: enabled for general demand paging testing
schurig@desktop:~/d/v73/zephyr$ git bisect good
Bisecting: 5 revisions left to test after this (roughly 3 steps)
[22a48e502e85816bb6dd654e1ec53704f4c5e25a] Bluetooth: Controller: Minor rename of disabled callback functions
schurig@desktop:~/d/v73/zephyr$ git bisect bad
Bisecting: 2 revisions left to test after this (roughly 2 steps)
[c3db83ba58465cacc8fcea6b7f6dfdd8757a33bd] scripts: runners: openocd: Enable thread awareness
schurig@desktop:~/d/v73/zephyr$ git bisect bad
Bisecting: 0 revisions left to test after this (roughly 1 step)
[0c2dabb4b94d7042a311660c47da09c8d95311b6] ipc: rpmsg_multi_instance: Rework instance tracking
schurig@desktop:~/d/v73/zephyr$ git bisect good
c3db83ba58465cacc8fcea6b7f6dfdd8757a33bd is the first bad commit
commit c3db83ba58465cacc8fcea6b7f6dfdd8757a33bd
Author: Gerson Fernando Budke <[email protected]>
Date:   Wed Aug 25 22:07:54 2021 -0300

    scripts: runners: openocd: Enable thread awareness
    
    Zephyr thread awareness is available for openocd but boards don't
    have debuggers configuration.  This configure OpenOCD runner
    automatically to complete configuration.
    
    User still require enable CONFIG_DEBUG_THREAD_INFO=y to visualize
    thread debug information.
    
    Signed-off-by: Gerson Fernando Budke <[email protected]>

 boards/common/openocd-nrf5.board.cmake   |  1 -
 cmake/flash/CMakeLists.txt               |  7 ++--
 scripts/west_commands/run_common.py      |  6 +--
 scripts/west_commands/runners/core.py    |  2 +-
 scripts/west_commands/runners/openocd.py | 68 +++++++++++++++++++++++++++-----
 scripts/west_commands/tests/conftest.py  |  2 +-
 6 files changed, 68 insertions(+), 18 deletions(-)

After this commit, the empty openocd_search entry started to appear.

[nandojve]

Hi @holgerschurig ,

Could you try this patch:

diff --git a/cmake/flash/CMakeLists.txt b/cmake/flash/CMakeLists.txt
index 68785f04e3..28e573f317 100644
--- a/cmake/flash/CMakeLists.txt
+++ b/cmake/flash/CMakeLists.txt
@@ -53,8 +53,8 @@ function(runners_yaml_append_config)
   endif()
   if(OPENOCD)
     runners_yaml_append("  openocd: ${OPENOCD}")
-    runners_yaml_append("  openocd_search:")
     if(OPENOCD_DEFAULT_PATH)
+      runners_yaml_append("  openocd_search:")
       runners_yaml_append("    - ${OPENOCD_DEFAULT_PATH}")
     endif()
   endif()

[nwsetec]

@nandojve I hit this issue as well and tried your patch above: #38272 (comment)
and it works for me, thanks!