Skip to content

[🚀 Feature]: webdriver connect pool maxsize increase #14594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Liu-XinYuan opened this issue Oct 14, 2024 · 5 comments
Closed

[🚀 Feature]: webdriver connect pool maxsize increase #14594

Liu-XinYuan opened this issue Oct 14, 2024 · 5 comments
Labels
C-py Python Bindings I-enhancement Something could be better

Comments

@Liu-XinYuan
Copy link

Feature and motivation

When I am doing automated testing, I need to asynchronously monitor whether the verification code pop-up box appears. At this time, because the default connection pool size is 1, performance queue and log warnings appear.
[Warning] The connection pool is full and connections are being dropped: xxxx. Connection pool size: 1

Usage example

Asynchronously monitor pop-up boxes that appear randomly

@Liu-XinYuan Liu-XinYuan added I-enhancement Something could be better A-needs-triaging A Selenium member will evaluate this soon! labels Oct 14, 2024
Copy link

@Liu-XinYuan, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

@Liu-XinYuan
Copy link
Author

Liu-XinYuan commented Oct 14, 2024

The code involved is as follows:
class RemoteConnection

def _get_connection_manager(self):
    pool_manager_init_args = {"timeout": self.get_timeout()}
    if self._ca_certs:
        pool_manager_init_args["cert_reqs"] = "CERT_REQUIRED"
        pool_manager_init_args["ca_certs"] = self._ca_certs

    if self._proxy_url:
        if self._proxy_url.lower().startswith("sock"):
            from urllib3.contrib.socks import SOCKSProxyManager

            return SOCKSProxyManager(self._proxy_url, **pool_manager_init_args)
        if self._identify_http_proxy_auth():
            self._proxy_url, self._basic_proxy_auth = self._separate_http_proxy_auth()
            pool_manager_init_args["proxy_headers"] = urllib3.make_headers(proxy_basic_auth=self._basic_proxy_auth)
        return urllib3.ProxyManager(self._proxy_url, **pool_manager_init_args)

    return urllib3.PoolManager(**pool_manager_init_args)``

@diemol diemol added C-py Python Bindings and removed A-needs-triaging A Selenium member will evaluate this soon! labels Nov 5, 2024
@navin772
Copy link
Member

Hi @Liu-XinYuan, you can increase the poolsize via init_args_for_pool_manager from the Client config, this should work for both direct connections and proxy connections:

from selenium import webdriver
from selenium.webdriver.remote.client_config import ClientConfig

client_config = ClientConfig(
    remote_server_addr="http://ip_addr:port",

    init_args_for_pool_manager={
        "init_args_for_pool_manager": {
            "maxsize": 20,  # Set pool size
        }
    }
)

driver = webdriver.Remote(client_config=client_config)

Client config was introduced in this PR.

@Delta456
Copy link
Member

Delta456 commented Feb 3, 2025

This issue can be closed because connect pool maxsize can now be increased through ClientConfig.

@diemol diemol closed this as completed Feb 3, 2025
Copy link

github-actions bot commented Mar 5, 2025

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators Mar 5, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
C-py Python Bindings I-enhancement Something could be better
Projects
None yet
Development

No branches or pull requests

4 participants