feat: limit /etc to readonly

[staaldraad]

What kind of change does this PR introduce?

Feature

What is the current behavior?

Please link any relevant issues here.

What is the new behavior?

https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#ReadWritePaths=

Prevent postgres, or child-process of, from writing to /etc

[linear]

SEC-197 Use NoNewPriviliges for postgres

[samrose]

@staaldraad I think I need to rebase this and get some tests (including adding to our osquery test)

[staaldraad]

@staaldraad I think I need to rebase this and get some tests (including adding to our osquery test)

rebased - and will look at the osquery test

[samrose]

Just recommending that we can expand our existing permissions testing to make sure this doesn't regress

Those tests live https://github.com/supabase/postgres/blob/develop/ansible/files/permission_check.py

Could (for instance) throw in another query related to this setting, or expand existing query

[staaldraad]

Just recommending that we can expand our existing permissions testing to make sure this doesn't regress

Those tests live https://github.com/supabase/postgres/blob/develop/ansible/files/permission_check.py

Could (for instance) throw in another query related to this setting, or expand existing query

Not finding a great way to retrieve this with osquery. Can use osquery to get the systemd_units table and then check that the unit file matches expected content (cat /etc/systemd/system/postgresql.service)

Otherwise, the mounts table doesn't contain the process mounts, so not finding it that way.

What could work is getting the pid of /usr/lib/postgresql/bin/postgres -D /etc/postgresql and then viewing the /proc/pid/mounts file contents to ensure /etc is ro (but no way of doing this directly in osquery, unless also installing the macadmins osquery pack to get access to file_lines table):

$ cat /proc/65372/mounts | grep /etc
/dev/nvme0n1p2 /etc ext4 ro,relatime,discard 0 0

[samrose]

Just recommending that we can expand our existing permissions testing to make sure this doesn't regress
Those tests live https://github.com/supabase/postgres/blob/develop/ansible/files/permission_check.py
Could (for instance) throw in another query related to this setting, or expand existing query

Not finding a great way to retrieve this with osquery. Can use osquery to get the systemd_units table and then check that the unit file matches expected content (cat /etc/systemd/system/postgresql.service)

Otherwise, the mounts table doesn't contain the process mounts, so not finding it that way.

What could work is getting the pid of /usr/lib/postgresql/bin/postgres -D /etc/postgresql and then viewing the /proc/pid/mounts file contents to ensure /etc is ro (but no way of doing this directly in osquery, unless also installing the macadmins osquery pack to get access to file_lines table):

$ cat /proc/65372/mounts | grep /etc
/dev/nvme0n1p2 /etc ext4 ro,relatime,discard 0 0

I think the way you described it is the best way to do it

1. you are checking the contents of systemd file with osquery
2. you can find process with something like SELECT pid FROM processes WHERE path LIKE '%/usr/lib/postgresql/bin/postgres%' AND cmdline LIKE '%-D /etc/postgresql%'; (or whatever works for you)
3. You can do pid=$(osqueryi --json "SELECT pid FROM processes WHERE path LIKE '%/usr/lib/postgresql/bin/postgres%' AND cmdline LIKE '%-D /etc/postgresql%'" | jq -r '.[0].pid')
   cat /proc/$pid/mounts | grep '/etc' or something like that

Basically this seems like what you described already, which seems to be the best way to accomplish this kind of test. The test could be unstable or fail if you cannot pin down the correct process to check, possibly. Thank you for doing this!

If you feel like you have a better way that works than what I describe, of course please pursue that.

[samrose]

Just recommending that we can expand our existing permissions testing to make sure this doesn't regress
Those tests live https://github.com/supabase/postgres/blob/develop/ansible/files/permission_check.py
Could (for instance) throw in another query related to this setting, or expand existing query

Not finding a great way to retrieve this with osquery. Can use osquery to get the systemd_units table and then check that the unit file matches expected content (cat /etc/systemd/system/postgresql.service)
Otherwise, the mounts table doesn't contain the process mounts, so not finding it that way.
What could work is getting the pid of /usr/lib/postgresql/bin/postgres -D /etc/postgresql and then viewing the /proc/pid/mounts file contents to ensure /etc is ro (but no way of doing this directly in osquery, unless also installing the macadmins osquery pack to get access to file_lines table):

$ cat /proc/65372/mounts | grep /etc
/dev/nvme0n1p2 /etc ext4 ro,relatime,discard 0 0

I think the way you described it is the best way to do it

1. you are checking the contents of systemd file with osquery
2. you can find process with something like SELECT pid FROM processes WHERE path LIKE '%/usr/lib/postgresql/bin/postgres%' AND cmdline LIKE '%-D /etc/postgresql%'; (or whatever works for you)
3. You can do pid=$(osqueryi --json "SELECT pid FROM processes WHERE path LIKE '%/usr/lib/postgresql/bin/postgres%' AND cmdline LIKE '%-D /etc/postgresql%'" | jq -r '.[0].pid')
   cat /proc/$pid/mounts | grep '/etc' or something like that

Basically this seems like what you described already, which seems to be the best way to accomplish this kind of test. The test could be unstable or fail if you cannot pin down the correct process to check, possibly. Thank you for doing this!

If you feel like you have a better way that works than what I describe, of course please pursue that.

....if there is a better/simpler way that does not use osquery that should be fine too, and you could still throw it into the this python file

[samrose]

Nice thank you!

[samrose]

@staaldraad do you think this is likely ready at this point?

[staaldraad]

@staaldraad do you think this is likely ready at this point?

@samrose yes. Will co-ordinate a phased rollout, but this actual change is ready

[samrose]

Ok, before we merge, let's add a suffix to these version numbers here like

postgresorioledb-17: "17.0.1.054-orioledb-etc-1"

Then sometime today we can test this in local infra, prior to merge.

[samrose]

@staaldraad do you think this is likely ready at this point?

@samrose yes. Will co-ordinate a phased rollout, but this actual change is ready

Ok, before we merge, let's add a suffix to the version numbers as described here

https://github.com/supabase/postgres/pull/1451/files#r2013958022

We're trying to make sure we get all significant changes tested in local infra before we roll out, to avoid failure incidents.

[samrose]

@staaldraad I added that version suffix here 1083e1f

If you don't yet have a local infra set up to test pause/restore, upgrade, and other scenarios yourself, I can do this sometime today.

[staaldraad]

@staaldraad I added that version suffix here 1083e1f

If you don't yet have a local infra set up to test pause/restore, upgrade, and other scenarios yourself, I can do this sometime today.

beat me too it, no wonder I couldn't push :) thanks!

yeah I can run through the testing locally again