[🐛 Bug]: selenium-webdriver's manager usage breaks Ruby bindings on Linux aarch64

[chadlwilson]

What happened?

Since the changes in 4.11.0 via #11356 and #12352 the previously working behaviour of selenium-webdriver seems broken on aarch64 when used by the Ruby gem, even when the driver is on the PATH. I understand now that selenium-manager is always attempted to be executed even if a driver is on the PATH and only a direct code driver_path option avoids use of selenium-manager.

This seems to be a problem because the manager doesn't work with the Ruby Gem on linux aarch64/arm64, because the gems are neither platform specific nor packaging aarch64/arm64 variants of selenium-manager for Linux. So one needs QEMU or something else - which IMHO should not be a requirement. There seems no obvious workaround here when using webdriver via some other library (e.g Capybara) but any hint would be useful.

On Mac it'll likely work if Rosetta2 is installed (see #12423), and on Windows due to the similar WOW64 emulation layer.

How can we reproduce the issue?

require "selenium-webdriver"

driver = Selenium::WebDriver.for :firefox
driver.navigate.to "http://google.com"

element = driver.find_element(name: 'q')
element.send_keys "Hello WebDriver!"
element.submit

puts driver.title

driver.quit

Relevant log output

Error Message: Unable to obtain geckodriver using Selenium Manager; Unsuccessful command executed: ["/usr/local/bundle/gems/selenium-webdriver-4.12.0/bin/linux/selenium-manager", "--browser", "firefox", "--output", "json"]
/usr/local/bundle/gems/selenium-webdriver-4.12.0/bin/linux/selenium-manager: /go/pipelines/usr/local/bundle/gems/selenium-webdriver-4.12.0/bin/linux/selenium-manager: cannot execute binary file
; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location

Operating System

Linux

Selenium version

4.11.0 / 4.12.0

What are the browser(s) and version(s) where you see this issue?

Firefox 117

What are the browser driver(s) and version(s) where you see this issue?

geckodriver

Are you using Selenium Grid?

No response

[github-actions]

@chadlwilson, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[titusfortner]

It's a balance.

The number of people inconvenienced by not being able to have geckodriver on 32 bit or aarch64 Linux automatically located on PATH, is dwarfed by the people who can be helped by having the Selenium Manager binary process PATH. We also don't want to release too many separate binaries, due to package size. Google doesn't support these architectures with official driver downloads and geckodriver versions are valid for much longer.

The current answer is to set the driver path in the service class. An example of this is in our docs at the link in the error message. Does capybara not support setting a service class? Ruby also provides a shortcut as a class accessor in the browser module, which is what the webdrivers gem uses, and can definitely be used with capybara.

We could throw a better error message here. "Not executable" is not as helpful, but again, people running obscure Linux installs are the people who would know exactly what's wrong.

[chadlwilson]

It's a balance.

The number of people inconvenienced by not being able to have geckodriver on 32 bit or aarch64 Linux automatically located on PATH, is dwarfed by the people who can be helped by having the Selenium Manager binary process PATH. We also don't want to release too many separate binaries, due to package size. Google doesn't support these architectures with official driver downloads and geckodriver versions are valid for much longer.

Yeah, I understand it's a balance.

If the size of the total Ruby package is the concern, theoretically I suppose you could either release platform specific Ruby variants which include only the correct binary version (rather than bundling all platforms); or possibly you could dynamically download the "correct" binary version for the platform within a Gem extension at install time (although there are obviously some downsides to this). Bit clunky though perhaps.

A gem I can recall that actually takes both approaches is https://rubygems.org/gems/sass-embedded which has an ext/Rakefile to download the "best" dart-sass variant for the platform.

The current answer is to set the driver path in the service class. An example of this is in our docs at the link in the error message. Does capybara not support setting a service class? Ruby also provides a shortcut as a class accessor in the browser module, which is what the webdrivers gem uses, and can definitely be used with capybara.

Yeah, I'll have a look - just was surprised by the regression in behaviour, and critical dependency on selenium-manager, and since we had already done the work to download our own correct geckodriver and ensure it was available, it's a bit strange for selenium to force use of its own manager, but then not correctly handle the platform.

We could throw a better error message here. "Not executable" is not as helpful, but again, people running obscure Linux installs are the people who would know exactly what's wrong.

I wasn't too worried about the error message, it's pretty easily googleable.

However I totally would not call aarch64 an "obscure linux install". It's in widespread use for a number of reasons on the server side, including performance, cost, even carbon footprint for some folks. Given that all newer Macs are now arm64, if you run anything Selenium inside a container or virtual machine you'll hit a similar problem without emulation on the Linux side.

In our case we are running on arm64 within CI/test automation because we produce arm64 container images. We want to validate those images by running a subset of web-driven tests. To avoid running the containers cross arch emulated (which is horribly slow and unreliable), these smoke tests run on the same arch as the target software under test, which means selenium driving firefox on arm64/aarch64.

[titusfortner]

I guess the bigger issue that I didn't discuss is that Rust doesn't cross compile the way we thought it did when we started down this path, so we can't actually generate a binary for the other architectures with our current setup. Right now we are relying on GitHub Actions Runners to build the binaries.

The limitation is that a user needs to be able to add selenium to their pom/gradle/gemfile/npm/requirements/csproj and have it "just work" without needing to specify architecture or an additional dependency. I'm open to working on separate packages for different architectures, but I'm not sure if any of the managers will filter by matching architecture as an option?

Problems we wanted to address in 4.11:

1. Each language was looking for drivers on PATH differently
2. Users have old drivers on PATH and don't know where or why things are failing

Giving responsibility to the binary lets us be more intelligent about determining the problem and eventually fixing it for the user.

Here's the Ruby shortcut I mentioned if you don't want to create a separate Service instance.

Selenium::WebDriver::Firefox::Service.driver_path = "/path/to/driver"

That value can also be a proc and not a string if you want something evaluated at runtime instead of hard coded.

With respect to "obscure," I should have said "niche." People don't browse the internet on Linux ARM64, and if you are doing browser testing on that platform, it's for specific technical reasons, and you already know this and know there might be unexpected things as a result. I'm much more interested in helping the Windows user who can't figure out environment variables have a better first-time experience. But that's where the better error message would help: "Hey person doing the not-common thing, we don't support what you are doing directly right now, here add this code."

[chadlwilson]

I guess the bigger issue that I didn't discuss is that Rust doesn't cross compile the way we thought it did when we started down this path, so we can't actually generate a binary for the other architectures with our current setup. Right now we are relying on GitHub Actions Runners to build the binaries.

Yeah, this is unfortunate. Lack of arm64 runners for GitHub Actions without self-hosting is a pain, and cross-compiling can be slow, even when it works. Also have this issue for some projects. I naively thought you probably were producing linux arm64 selenium-manager binaries already, but didn't actually look.

The limitation is that a user needs to be able to add selenium to their pom/gradle/gemfile/npm/requirements/csproj and have it "just work" without needing to specify architecture or an additional dependency. I'm open to working on separate packages for different architectures, but I'm not sure if any of the managers will filter by matching architecture as an option?

Not sure I understand you here, but it's certainly possible to produce os and arch specific versions of Gems, e.g https://rubygems.org/gems/google-protobuf/versions and such (aarch64-linux vs x86_64-linux vs arm64-darwin vs x86_64-darwin etc).

With respect to "obscure," I should have said "niche." People don't browse the internet on Linux ARM64, and if you are doing browser testing on that platform, it's for specific technical reasons, and you already know this and know there might be unexpected things as a result. I'm much more interested in helping the Windows user who can't figure out environment variables have a better first-time experience. But that's where the better error message would help: "Hey person doing the not-common thing, we don't support what you are doing directly right now, here add this code."

Hey hey, don't tell the Raspberry Pi enthusiasts that they are niche! 😉

[mapkon]

@titusfortner i see you closed this as 'not-planned'. Any particular reason why it not being planned has an effect on the legitimacy of the regression?

[titusfortner]

We don't have the ability to change it right now even if we decided it was worth the tradeoffs which we currently do not. So it's in the wontfix category. If something in our calculus changes, we don't need this issue to be open to re-address it.

[chadlwilson]

For those who come across this, another workaround available may be provided by #12752 allowing use of SE_MANAGER_PATH to override the bundled version. (4.13.0+) - with the asterisk that youd need to be able to build the selenium-manager for the other archs yourself, which may or may not not be desirable compared to changing ones test code to override just the driver path to a pre-downloaded driver.

[titusfortner]

I think the steps are:

1. Install Rust Dev Environment
2. git clone https://github.com/SeleniumHQ/selenium.git --depth 1
3. cd selenium/rust
4. cargo build --release
5. SE_MANAGER_PATH=~/selenium/rust/target/release/selenium-manager
6. Put the driver you want in a location on PATH
7. Run a Selenium test and Selenium will use the built Selenium Manager to locate the manually downloaded driver on PATH

This would allow you to avoid setting a location in the service class

Please let us know if you try it and it works.

[chadlwilson]

Thanks @titusfortner - are you/the team interested in any help to try and get the cross build working for aarch64-linux on GH Actions? (not sure what the issues are, as I see you use cross-rs which conceptually should do all the necessary binfmt_msc and container stuff to be able to cross-compile for linux-aarch64, but haven't tried yet).

I have some docker cross-arch experience and note #11357 so could play around with it to see if it can be partially unblocked; but I also do note that issues with bloating the download size would have to be addressed before it could actually be used downstream.

Nevertheless, having an "official", but "non-distributed-with-language-artifacts" build would still be useful, even if it is only published into the github repo (or github releases) like the current manager bins.

[titusfortner]

Let me think through the issues again..

1. The bindings right now are picking the binary to use based on Platform (linux/windows/mac) not on architecture; I'm not sure what the right way to do this is in all the languages
2. Only geckodriver offers an official driver for aarch64 linux, so it would have non-standard behavior when used with chrome/edge
3. We need a programmatic way to do it via Github Actions (we've not had great success with rust cross compilation attempts)
4. Package size... if we had a ~5MB executable for aarch64 linux I think we'd be ok adding it.
5. I'm still reluctant to have official binaries available for download until this tool is out of beta, because I want us to be able to make backwards incompatible changes between now and then
6. We have so many other things we're trying to push forward this particular item is not a priority.

So, yes, we're tentatively interested in help with this given the limitations and considerations above.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.