feat: AppArmor confinement for codejail-service (#109)

Allow codejail-service to actually run code (and run it securely) by
giving it the needed confinement. Previously it would run but refuse to
execute code since it would detect the insecure environment; now, the
startup safety checks pass and the code-exec endpoint works as expected.

- Add an AppArmor profile with fairly strict rules. It needs to be
  thoroughly vetted and to have exceptions added before it can be used in
  production, but it's be fine for devstack.
- Apply the profile to the codejail service in docker-compose.
- Add Django configs for codejail service.
- Add documentation for installing the profile so that it is available for
  use on the dev's machine.

Also:

- Add configuration and documentation for edxapp to actually call the
  codejail service, disabled by default. (Will later want to make this
  default to true, once the service is working properly.)
- Update image name in docker-compose to follow rename in
  edx/public-dockerfiles#102

Currently edxapp gets an error back from codejail-service relating
to python_path:
`No such file or directory: "/edx/var/edxapp/data/edX/DemoX/Demo_Course"`
But the integration is working otherwise.

---------

Co-authored-by: Robert Raposa <[email protected]>

Author: timmc-edx

Diff for: codejail.profile

@@ -0,0 +1,113 @@
+# AppArmor profile for running codejail-service in devstack.
+#
+#                         #=========#
+#                         # WARNING #
+#                         #=========#
+#
+# This is not a complete and secure apparmor profile! Do not use this
+# in any deployed environment (even a staging environment) without
+# careful inspection and modification to fit your needs.
+#
+# See https://manpages.ubuntu.com/manpages/noble/man5/apparmor.d.5.html
+# or `man apparmor.d` for documentation of syntax and options.
+#
+# Failure to apply a secure apparmor profile *will* likely result in a
+# compromise of your environment by an attacker.
+#
+# We may at some point make this file good enough for confinement in
+# production, but for now it is only intended to be used in devstack.
+
+
+
+# Sets standard variables used by abstractions/base, later. Controlled
+# by OS, see /etc/apparmor.d/tunables/global for contents.
+include <tunables/global>
+
+# Require that the system understands the feature set that this policy was written
+# for. If we didn't include this, then on Ubuntu >= 22.04, AppArmor might assume
+# the wrong feature set was requested, and some rules might become too permissive.
+# See https://github.com/netblue30/firejail/issues/3659#issuecomment-711074899
+abi <abi/3.0>,
+
+# This outer profile applies to the entire container, and isn't as
+# important as the inner (codejail_sandbox) profile. If the inner profile doesn't work, it's not likely that
+# the outer one is going to help. But there may be some small value in
+# defense-in-depth, as it's possible that a bug in the codejail_sandbox (inner)
+# profile isn't present in the outer one.
+profile codejail_service flags=(mediate_deleted) {
+
+    # Allow access to a variety of commonly needed, generally safe things
+    # (such as reading /dev/random, free memory, etc.)
+    #
+    # Manpage: "Includes files that should be readable and writable in all profiles."
+    include <abstractions/base>
+
+    # Filesystem access -- self-explanatory
+    file,
+
+    # netlink is needed for sudo's interprocess communication
+    network netlink raw,
+
+    # Allow all of the various network operations required to listen, accept connection, etc.
+    network tcp,
+    # But then deny making a new *outbound* connection.
+    deny network (connect) tcp,
+
+    # Required for sudoing to sandbox
+    capability setuid setgid audit_write,
+    # Allow sending a kill signal
+    capability kill,
+
+    # Allow sending a kill signal to the codejail_sandbox subprofile when the execution
+    # runs beyond time limits.
+    signal (send) set=(kill) peer=codejail_service//codejail_sandbox,
+
+    # The core of the confinement: When the sandbox Python is executed, switch to
+    # the (extremely constrained) codejail_sandbox profile.
+    #
+    # This path needs to be coordinated with the Dockerfile and Django settings.
+    #
+    # Manpage: "Cx: transition to subprofile on execute -- scrub the environment"
+    /sandbox/venv/bin/python Cx -> codejail_sandbox,
+
+    # This is the important apparmor profile -- the one that actually
+    # constrains the sandbox Python process.
+    #
+    # mediate_deleted is not well documented, but it seems to indicate that
+    # apparmor will continue to make policy decisions in cases where a confined
+    # executable has a handle to a file's inode even after the file is removed
+    # from the filesystem.
+    profile codejail_sandbox flags=(mediate_deleted) {
+
+        # This inner profile also gets general access to "safe"
+        # actions; we could list those explicitly out of caution but
+        # it could get pretty verbose.
+        include <abstractions/base>
+
+        # Read and run binaries and libraries in the virtualenv. This
+        # includes the sandbox's copy of Python as well as any
+        # dependencies that have been installed for inclusion in
+        # sandboxes.
+        #
+        # m: executable mapping, required for shared libraries used by some
+        #    Python dependencies with C compontents, eg `nltk`.
+        /sandbox/venv/** rm,
+
+        # Allow access to the temporary directories that are set up by
+        # codejail, one for each code-exec call. Each /tmp/code-XXXXX
+        # contains one execution.
+        #
+        # Codejail has a hardcoded reference to this file path, although the
+        # use of /tmp specifically may be controllable with environment variables:
+        # https://github.com/openedx/codejail/blob/0165d9ca351/codejail/util.py#L15
+        /tmp/codejail-*/ r,
+        /tmp/codejail-*/** rw,
+
+        # Allow interactive terminal in devstack.
+        /dev/pts/* rw,
+
+        # Allow receiving a kill signal from the webapp when the execution
+        # runs beyond time limits.
+        signal (receive) set=(kill) peer=codejail_service,
+    }
+}

Diff for: docker-compose.yml

@@ -726,11 +726,13 @@ services:
     hostname: codejail.devstack.edx
     stdin_open: true
     tty: true
-    image: edxops/codejail-dev:latest
+    image: edxops/codejail-service-dev:latest
     environment:
       DJANGO_SETTINGS_MODULE: codejail_service.settings.devstack
     ports:
       - "18030:8080"
+    security_opt:
+      - apparmor=codejail_service
 
   xqueue:
     container_name: "edx.${COMPOSE_PROJECT_NAME:-devstack}.xqueue"

Diff for: docs/codejail.rst

@@ -0,0 +1,43 @@
+Codejail service
+################
+
+The ``codejail`` devstack component (codejail-service) requires some additional configuration before it can be enabled. This page describes how to set it up and debug it.
+
+Background
+**********
+
+The `codejail-service <https://github.com/openedx/codejail-service>`__ webapp is a wrapper around the `codejail <https://github.com/openedx/codejail>`__ library. See the READMEs of each repo for more information on the special requirements for deploying codejail, in particular the AppArmor-based sandboxing.
+
+References to "codejail" can mean either the library or the service. In devstack, "codejail" usually refers to the service.
+
+Configuration
+*************
+
+These instructions are for Linux only. Additional research would be required to create instructions for a Mac, which probably involves accessing the Linux VM that docker is run inside of.
+
+In order to run the codejail devstack component:
+
+1. Install AppArmor: ``sudo apt install apparmor``
+2. Add the provided codejail AppArmor profile to your OS: ``sudo apparmor_parser --add -W ./codejail.profile``
+3. Configure LMS and CMS to use the codejail-service by uncommenting ``# ENABLE_CODEJAIL_REST_SERVICE = True`` in ``py_configuration_files/{lms,cms}.py``
+4. Run ``make codejail-up``
+
+The service does not need any provisioning, and does not have dependencies.
+
+Over time, the AppArmor profile may need to be updated. Changes to the file do not automatically cause changes to the version that has been installed in the OS. When significant changes have been made to the profile, you'll need to re-install the profile. This can be done by passing ``--replace`` instead of ``--add``, like so: ``sudo apparmor_parser --replace -W ./codejail.profile``
+
+Development
+***********
+
+Changes to the AppArmor profile must be coordinated with changes to the Dockerfile, as they need to agree on filesystem paths.
+
+Any time you update the profile file, you'll need to update the profile in your OS as well: ``sudo apparmor_parser --replace -W ./codejail.profile``
+
+The profile file contains the directive ``profile codejail_service``. That defines the name of the profile when it is installed into the OS, and must agree with the relevant ``security_opt`` line in ``docker-compose.yml``. This name should not be changed, as it creates a confusing situation and would require every developer who uses codejail-service to do a number of manual steps. (Profiles can't be renamed *within* the OS; they must first be removed **under the old name**, and then a new profile must be installed under the new name.)
+
+Debugging
+*********
+
+To check whether the profile has been applied, run ``sudo aa-status | grep codejail``. This won't tell you if the profile is out of date, but it will tell you if you have *some* version of it installed.
+
+If you need to debug the confinement, either because it is restricting too much or too little, a good strategy is to run ``tail -F /var/log/kern.log | grep codejail`` and watch for ``DENIED`` lines. You should expect to see several appear during service startup, as the service is designed to probe the confinement as part of its initial healthcheck.

Diff for: docs/index.rst

@@ -27,3 +27,4 @@ Contents
    troubleshoot_general_tips
    manual_upgrades
    advanced_configuration
+   codejail

Diff for: py_configuration_files/cms.py

@@ -312,6 +312,16 @@ def should_show_debug_toolbar(request):  # lint-amnesty, pylint: disable=missing
 xblock_duplicated_event_setting = EVENT_BUS_PRODUCER_CONFIG['org.openedx.content_authoring.xblock.duplicated.v1']
 xblock_duplicated_event_setting['course-authoring-xblock-lifecycle']['enabled'] = True
 
+############################ Codejail ############################
+
+# Disabled by default since codejail service needs to be configured
+# and started separately. See docs/codejail.rst for details.
+#ENABLE_CODEJAIL_REST_SERVICE = True
+
+# Note that this is exposed on port 8080 to other devstack services,
+# but 18030 outside of Docker.
+CODE_JAIL_REST_SERVICE_HOST = "http://edx.devstack.codejail:8080"
+
 
 ################# New settings must go ABOVE this line #################
 ########################################################################

Diff for: py_configuration_files/codejail.py

@@ -1,3 +1,29 @@
 """Settings for devstack use."""
 
 from codejail_service.settings.local import *  # pylint: disable=wildcard-import
+
+ALLOWED_HOSTS = [
+    # When called from outside of docker's network (dev's terminal)
+    'localhost',
+    # When called from another container (lms, cms)
+    'edx.devstack.codejail',
+]
+
+CODEJAIL_ENABLED = True
+
+CODE_JAIL = {
+    # These values are coordinated with the Dockerfile (in edx/public-dockerfiles)
+    # and the AppArmor profile (codejail.profile in edx/devstack).
+    'python_bin': '/sandbox/venv/bin/python',
+    'user': 'sandbox',
+
+    # Configurable limits.
+    'limits': {
+        # CPU-seconds
+        'CPU': 3,
+        # 100 MiB memory
+        'VMEM': 100 * 1024 * 1024,
+        # Clock seconds
+        'REALTIME': 3,
+    },
+}

Diff for: py_configuration_files/lms.py

@@ -554,6 +554,15 @@ def should_show_debug_toolbar(request):  # lint-amnesty, pylint: disable=missing
     'http://localhost:1996',  # frontend-app-learner-dashboard
 ]
 
+############################ Codejail ############################
+
+# Disabled by default since codejail service needs to be configured
+# and started separately. See docs/codejail.rst for details.
+#ENABLE_CODEJAIL_REST_SERVICE = True
+
+# Note that this is exposed on port 8080 to other devstack services,
+# but 18030 outside of Docker.
+CODE_JAIL_REST_SERVICE_HOST = "http://edx.devstack.codejail:8080"
 
 ################# New settings must go ABOVE this line #################
 ########################################################################