Basic support for encrypted local connections (#9); update readme

Author: simonrob

README.md

@@ -13,7 +13,7 @@ You can remove details from the sample configuration file for services you don't
 
 Next, from a terminal, install the script's requirements: `python3 -m pip install -r requirements.txt`, and start the proxy: `python3 emailproxy.py` – a menu bar/taskbar icon should appear. If instead of the icon you see an error in the terminal, it is likely that your system is missing dependencies for the `pywebview` or `pystray` packages. See the [Dependencies and setup](https://github.com/simonrob/email-oauth2-proxy#dependencies-and-setup) section below to resolve this. Once any missing dependencies have been installed, starting the proxy should create a menu bar icon.
 
-Finally, open your email client and configure your account's server details to match the ones you set in the proxy's configuration file. For example, using the sample Office 365 details, this would be `localhost` on port `1993` for IMAP and `localhost` on port `1587` for SMTP. The local connection in your email client should be configured as unencrypted to allow the proxy to operate, but the connection between the proxy and your email server is secured (SSL for IMAP; SSL or STARTTLS for SMTP).
+Finally, open your email client and configure your account's server details to match the ones you set in the proxy's configuration file. For example, using the sample Office 365 details, this would be `localhost` on port `1993` for IMAP and `localhost` on port `1587` for SMTP. The local connection in your email client should be configured as unencrypted to allow the proxy to operate, but the connection between the proxy and your email server is secured (implicit SSL/TLS for IMAP; implicit or explicit (STARTTLS) SSL/TLS for SMTP).
 
 The first time your email client makes a request you should see a notification from the proxy about authorising your account. (Note that the notification is not itself clickable, but pull requests to improve this are very welcome). Click the proxy's menu bar icon, select your account name in the `Authorise account` submenu, and then log in via the popup browser window that appears. The window will close itself once the process is complete.
 
@@ -23,7 +23,9 @@ After successful authentication and authorisation you should have IMAP/SMTP acce
 ## Starting the proxy automatically
 The simplest way to start the proxy automatically is to click its menu bar icon and then select `Start at login`, which will stop the terminal instance and restart the script, configuring it to run each time you log in. A different approach is used to achieve this depending on whether you are using macOS, Windows or Linux.
 
-On macOS, if you stop the proxy's service (i.e., `Quit Email OAuth 2.0 Proxy` from the menu bar), you can restart it using `launchctl start ac.robinson.email-oauth2-proxy` from a terminal. You can stop, disable or remove the service from your startup items either via the menu bar icon options, or using `launchctl unload ~/Library/LaunchAgents/ac.robinson.email-oauth2-proxy.plist`. On more recent macOS versions, you may find that you need to manually load the launch agent file the first time in order to trigger a permission prompt about python access. To do this, click `Start at login` from the app's menu bar icon, then run the unload command listed above followed by `launchctl load ~/Library/LaunchAgents/ac.robinson.email-oauth2-proxy.plist`. After this (and approving access via the permission prompt), the menu bar option should work as normal.
+On macOS, if you stop the proxy's service (i.e., `Quit Email OAuth 2.0 Proxy` from the menu bar), you can restart it using `launchctl start ac.robinson.email-oauth2-proxy` from a terminal. You can stop, disable or remove the service from your startup items either via the menu bar icon options, or using `launchctl unload ~/Library/LaunchAgents/ac.robinson.email-oauth2-proxy.plist`.
+
+On more recent macOS versions (10.14 and later), you may find that you need to manually load the proxy's launch agent in order to trigger a permission prompt when first running as a service. You will know if this is necessary if the proxy exits (rather than restarts) the first time you click `Start at login` from its menu bar icon. To resolve this, exit the proxy and then run `launchctl start ~/Library/LaunchAgents/ac.robinson.email-oauth2-proxy.plist` from a terminal. A permission pop-up should appear requesting access for python. Once this has been approved, the proxy's menu bar icon will appear as normal (though you may need to run the command again). In some cases — particularly when running the proxy in a python virtual environment — the permission prompt does not appear. If this happens it is worth first trying to `unload` and then `load` the service via `launchctl`. If this still does not cause the prompt to appear, the only currently-known resolution is to run the proxy outside of a virtual environment and manually grant Full Disk Access to your python executable via the privacy settings in the macOS System Preferences. You may also need to edit the proxy's launch agent plist file, which is found at the location given in the command above, to set the path to your python executable – it must be the real path rather than a symlink (the `readlink` command can help here). Fortunately this is a one-time fix, and once the proxy loads successfully via this method you will not need to adjust its startup configuration again (except perhaps when upgrading to a newer major macOS version, in which case just repeat the procedure).
 
 On Windows the auto-start functionality is achieved via a shortcut in your user account's startup folder. Pressing the Windows key and `r` and entering `shell:startup` (and then clicking OK) will open this folder – from here you can either double-click the `ac.robinson.email-oauth2-proxy.cmd` file to relaunch the proxy, or delete this file (either manually or by deselecting the option in the proxy's menu) to remove the script from your startup items.
 
@@ -64,11 +66,10 @@ Please feel free to [open an issue](https://github.com/simonrob/email-oauth2-pro
 
 
 ## Potential improvements (pull requests welcome)
-- Full feature parity on different platforms (e.g., live menu updating)
+- Full feature parity on different platforms (e.g., live menu updating; suspend on sleep)
 - Testing with different providers (currently verified only with Office 365 and Gmail)
 - STARTTLS for IMAP?
 - POP3?
-- Encrypted local connections?
 - Package as .app/.exe etc?
 
 

emailproxy.config

@@ -6,12 +6,17 @@ documentation = This is a sample Email OAuth 2.0 Proxy configuration file.
 
 [Server setup]
 documentation = Local servers are specified as demonstrated below where, for example, the section heading [IMAP-1993]
-	gives the type (which can be IMAP or SMTP) and the local port to listen on (i.e., 1993 here). The local port number
-	must be above 1023 (unless the proxy script is run via sudo), below 65536, and unique across local servers.
-	Multiple accounts can share the same server. A SSL connection to the remote server is assumed by default. If your
-	SMTP server uses the STARTTLS protocol, add starttls = True, as shown in the example, and this will be handled by
-	the proxy (assumed to be False otherwise). You should configure your email client to use an unencrypted connection
-	for both SMTP and IMAP. If the property 'local_address' is not specified, its value is assumed to be 'localhost'.
+	gives the type (which can be IMAP or SMTP) and the local port to listen on (i.e., 1993 etc). The local port number
+	must be above 1023 (unless the proxy script is run via sudo), below 65536, and unique across local servers. Multiple
+	accounts can share the same server. If the property 'local_address' is not specified, its value is assumed to be
+	'localhost'. A secure connection to the remote server is assumed by default (i.e., implicit SSL/TLS). If your SMTP
+	server uses the STARTTLS approach instead, add starttls = True, as shown in the example, and this will be handled by
+	the proxy (assumed to be False otherwise). IMAP STARTTLS is not currently supported. In the standard configuration
+	you should set up your email client to use an unencrypted connection for both SMTP and IMAP. Note that only the
+	local channel between your email client and the proxy is unencrypted, so this is normally not a concern. However, if
+	you prefer, you may provide a local_certificate_path (e.g., /etc/letsencrypt/live/mail.example.net/fullchain.pem)
+	and local_key_path (e.g., /etc/letsencrypt/live/mail.example.net/privkey.pem) for a server, and the proxy will
+	use these to set up a secure connection between itself and your email client.
 
 [IMAP-1993]
 local_address = localhost
@@ -34,7 +39,7 @@ server_port = 465
 
 [Account setup]
 documentation = Accounts are specified using your email address as the section heading (e.g., [[email protected]],
-	below). Account names (i.e., email addresses) must be unique – only one entry per account is permitted. The
+	below). Account usernames (i.e., email addresses) must be unique - only one entry per account is permitted. The
 	examples given will work for Office 365 and Gmail, but you will need to add your own client_id and client_secret.
 	See https://developers.google.com/identity/protocols/oauth2/native-app and the Microsoft link below for details.
 	Multiple accounts on the same server can use the same client_id/client_secret; just duplicate these in each

emailproxy.py

@@ -4,7 +4,7 @@
 __author__ = 'Simon Robinson'
 __copyright__ = 'Copyright (c) 2021 Simon Robinson'
 __license__ = 'Apache 2.0'
-__version__ = '2022-01-27'  # ISO 8601
+__version__ = '2022-02-07'  # ISO 8601
 
 import argparse
 import asyncore
@@ -614,7 +614,7 @@ def process_data(self, byte_data, censor_server_log=False):
         str_data = byte_data.decode('utf-8', 'replace').rstrip('\r\n')
         str_data_lower = str_data.lower()
 
-        # intercept EHLO so we can add STARTTLS (in parent class)
+        # intercept EHLO so we can add STARTTLS (in server connection class)
         if self.server_connection.ehlo is None and self.custom_configuration['starttls']:
             if str_data_lower.startswith('ehlo') or str_data_lower.startswith('helo'):
                 self.server_connection.ehlo = str_data  # save the command so we can replay later from the server side
@@ -692,7 +692,7 @@ def create_socket(self, socket_family=socket.AF_INET, socket_type=socket.SOCK_ST
         new_socket = socket.socket(socket_family, socket_type)
         new_socket.setblocking(True)
 
-        # connections can either be wrapped via the STARTTLS command, or SSL from the start
+        # connections can either be upgraded (wrapped) after setup via the STARTTLS command, or secure from the start
         if self.custom_configuration['starttls']:
             self.set_socket(new_socket)
         else:
@@ -880,8 +880,11 @@ def __init__(self, proxy_type, local_address, server_address, custom_configurati
         self.client_connections = []
 
     def info_string(self):
-        return '%s server at %s:%d proxying %s:%d' % (self.proxy_type, self.local_address[0], self.local_address[1],
-                                                      self.server_address[0], self.server_address[1])
+        secure = self.custom_configuration['local_certificate_path'] and self.custom_configuration['local_key_path']
+        return '%s server at %s:%d (%s) proxying %s:%d (%s)' % (
+            self.proxy_type, self.local_address[0], self.local_address[1], 'TLS' if secure else 'unsecured',
+            self.server_address[0], self.server_address[1],
+            'STARTTLS' if self.custom_configuration['starttls'] else 'SSL/TLS')
 
     def handle_accepted(self, connection, address):
         if MAX_CONNECTIONS <= 0 or len(self.client_connections) < MAX_CONNECTIONS:
@@ -927,6 +930,19 @@ def start(self):
         self.bind(self.local_address)
         self.listen(1)
 
+    def create_socket(self, socket_family=socket.AF_INET, socket_type=socket.SOCK_STREAM):
+        if self.custom_configuration['local_certificate_path'] and self.custom_configuration['local_key_path']:
+            new_socket = socket.socket(socket_family, socket_type)
+            new_socket.setblocking(False)
+
+            ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
+            ssl_context.load_cert_chain(
+                certfile=self.custom_configuration['local_certificate_path'],
+                keyfile=self.custom_configuration['local_key_path'])
+            self.set_socket(ssl_context.wrap_socket(new_socket, server_side=True))
+        else:
+            super().create_socket(socket_family, socket_type)
+
     def remove_client(self, client):
         if client in self.client_connections:  # remove closed clients
             self.client_connections.remove(client)
@@ -1197,7 +1213,9 @@ def get_last_activity(account):
     @staticmethod
     def edit_config():
         if sys.platform == 'darwin':
-            os.system('open %s' % CONFIG_FILE_PATH)
+            result = os.system('open %s' % CONFIG_FILE_PATH)
+            if result != 0:  # no default editor found for this file type; open as a text file
+                os.system('open -t %s' % CONFIG_FILE_PATH)
         elif sys.platform == 'win32':
             os.startfile(CONFIG_FILE_PATH)
         elif sys.platform.startswith('linux'):
@@ -1490,7 +1508,9 @@ def load_and_start_servers(self, icon=None):
                 break
 
             custom_configuration = {
-                'starttls': config.getboolean(section, 'starttls', fallback=False) if server_type == 'SMTP' else False
+                'starttls': config.getboolean(section, 'starttls', fallback=False) if server_type == 'SMTP' else False,
+                'local_certificate_path': config.get(section, 'local_certificate_path', fallback=None),
+                'local_key_path': config.get(section, 'local_key_path', fallback=None)
             }
 
             if server_address:  # all other values are checked, regex matched or have a fallback above