[EventHubs] Custom ssl certificate support when using http proxy

[yunhaoling]

Summary

EventHub allows connecting to the service via http proxy.
Currently we expose http proxy setting including:

proxy_hostname (str)
proxy_port (int)
username (str)
password (str)

However, there're scenarios:

• the http proxy server has its own Server SSL certificate -- when the client wants to build tls connection to the http proxy, a custom certificate is required for authenticating the identity of the proxy server.
• Apart from the server certificate, there is also Client SSL certificate which is used by the server to authenticate the identity of a client.

We would like to provide the ability to set server/client certificate in EventHub Python SDK for authenticating the identity of the proxy server/client when connection to the service via a http proxy.

Scope of work

• Client accepts server SSL certificate and uses the certificate (path to the CA_BUNDLE file) in the case of http proxy usage for authenticating the identity of the proxy server when building tls connection.
  • (?) certificate ignored if there's no proxy setting
• Client accepts client SSL certificate and uses the certificate in the case of http proxy usage for authenticating the identity of the client when building tls connection.
  • (?) certificate ignored if there's no proxy setting
• The surface should align with the azure-core on exposing the certificate settings at the top-level client

Success Criteria

• The server ssl certificate and client certificate is supported/implemented in the underlying uamqp library
• clients accept ssl certificate and client certificate could connect to the service via http proxy which requires the certificates.

Samples

http_proxy = {
proxy_hostname (str)
proxy_port (int)
username (str)
password (str)
connection_verify: path the server certificate CA_BUNDLE file
connection_cert: path to the client side certificate CA_BUNDLE file or (key, certificate pair?)
}

References

Python request ssl cert
Python request client cert

[yunhaoling]

action items:

• investigate the "verify" feature in uamqp auth and test setting the server certificate to connect to http proxy server
• investigate the client side certification in uamqp-c library
  • whether setting client side cert is supported or not

issue opened in the azure-c-shared-utility repo asking for the certificate support: Azure/azure-c-shared-utility#501
PR for tls http proxy: Azure/azure-c-shared-utility#512

issue about the client-side certificate: Azure/azure-c-shared-utility#513

api proposal for the c lib: https://gist.github.com/yunhaoling/753677c4ee8137f50da38402c6646595

[fulii]

Hi!

I would really like this to happen.
Currently we cannot use azure service bus because our proxy server has a custom certificate.
I cannot see any workaround at the moment how to bypass this, i spent hours to figure out.
Aslo from uamqp library side there is not much information about bad certificate.
If this happens in march that is really great, until then do you have any workaround?

[yunhaoling]

hey @fulii,

I'm sorry to tell you that the currently the underlying C networking implementation for http proxy only supports basic auth (username and password) and we could do nothing until the support is being added into the C library first so that our uamqp library could take advantage of the feature and expose the settings to the upper layer.

I'll continue my work on adding support to the C library this month, but I can't guarantee you the timeline.

[yunhaoling]

uamqp PR: Azure/azure-uamqp-python#232

prototype is available here: https://github.com/yunhaoling/uamqp-tls-proxy-prototype

event hub API proposal is here: https://gist.github.com/yunhaoling/720e8bea2cdd06cdf94515c7e31dd266

API shape in requests:

https://docs.python-requests.org/en/latest/user/advanced/#ssl-cert-verification
https://docs.python-requests.org/en/latest/user/advanced/#client-side-certificates

source code: https://github.com/psf/requests/blob/master/requests/api.py#L16-L47
certificates related api shape:

'''
    :param verify: (optional) Either a boolean, in which case it controls whether we verify
            the server's TLS certificate, or a string, in which case it must be a path
            to a CA bundle to use. Defaults to ``True``.
    :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair.

sample:
'''python
requests.get('https://kennethreitz.org', verify='/path/to/certfile', cert=('/path/client.cert', '/path/client.key'))

API shape in httpx:

https://www.python-httpx.org/advanced/#ssl-certificates
https://www.python-httpx.org/advanced/#client-side-certificates

source code:
on requests: https://github.com/encode/httpx/blob/master/httpx/_api.py#L70-L77
on client: https://github.com/encode/httpx/blob/master/httpx/_client.py#L575-L582
certificates related api:

    * **verify** - *(optional)* SSL certificates (a.k.a CA bundle) used to
    verify the identity of requested hosts. Either `True` (default CA bundle),
    a path to an SSL certificate file, an `ssl.SSLContext`, or `False`
    (which will disable verification).
    * **cert** - *(optional)* An SSL certificate used by the requested host
    to authenticate the client. Either a path to an SSL certificate file, or
    two-tuple of (certificate file, key file), or a three-tuple of (certificate
    file, key file, password).

sample:

# verify/server certificates
import httpx
r = httpx.get("https://example.org", verify="path/to/client.pem")
# or pass a ssl context
import ssl
import httpx
context = ssl.create_default_context()
context.load_verify_locations(cafile="/tmp/client.pem")
# context = httpx.create_ssl_context(verify="/tmp/client.pem")
httpx.get('https://example.org', verify=context)
# or client side
client = httpx.Client(verify=False)

# client side certificates
import httpx

r = httpx.get("https://example.org", cert="path/to/client.pem")
# tuple
cert = ("path/to/client.pem", "path/to/client.key")
# cert = ("path/to/client.pem", "path/to/client.key", "password")
httpx.get("https://example.org", cert=cert)

[yunhaoling]

hey @fulii , apologize for not getting back to you sooner.

I have tried to implement the feature in the uamqp library based on my understanding.
I would like to invite you to try out my implementation to see if it works for your scenario -- uamqp wheels for linux and samples for service bus sdk could be found here: https://github.com/yunhaoling/uamqp-tls-proxy-prototype.

please let me know if you need help on testing it out, your feedbacks on the API are also welcomed!

(if you're working on Windows or macOS, I could manually build wheels for you as well)

[yunhaoling]

closing the issue as there's no active development plan for it.
besides we heard no feedback from the community.

Please create a new issue if you feel this is the feature you need, and we could discuss.