Description
Cloud currently builds and runs a fully custom Elasticsearch image. The Stack image should provide an image suitable for direct use instead.
User / Group IDs
Cloud currently expects to be able to set the user and group IDs via environment variables. The entry point script changes the founduser user's UID and GID to those provided in the env vars, and performs a recursive chown on some directories in the container.
The Stack image takes a different approach, in that it follows the OKD image guidelines and uses GID 0 for all files.
The different in approaches may well be due to Cloud's multi-tenancy. I don't know how OKD handles this.
(There is a related open issue, "Pin USER in Dockerfile complying with Docker best practices" (#46166).)
setuid flags
Cloud ensures that there are no files with setuid, in order to mitigate "stackclash" attacks. Basically, they do this:
RUN find / -xdev -perm -4000 -exec chmod u-s {} +
This could be done in the Stack image.
Init process
Cloud runs Elasticsearch via a mini-init process in order to avoid zombie processes. There should be no harm in adopting this in the Stack image.
JVM options
Cloud comments out the heap settings and disables the HeapDumpOnOutOfMemoryError flag in jvm.options. Cloud is also passing the relevant heap options on the command line so it's not necessary to comment out the heap settings (this has been the case since 5.0). It does has the advantage that the resulting Elasticsearch command line flags is tidier. I don't know if that's important to Cloud.
Quota-aware filesystem
Cloud changes the default filesystem provider to an implementation that is aware of user quotas. This is achieved by providing a file to the container that contains the quota settings, which the provider then takes into account then answering capacity questions. The file has a default location, which can be changed via a system property. The filesystem provider is changed via a JVM option, set via ES_JAVA_OPTS.
Cloud has suggested that the Stack provide "a native and equivalent way to deal with this". There is a Stack Overflow question that suggests that Java lacks support for this at present. It ought to be possible to perform native calls on Linux (and possibly other operating systems) that return quota information, but it's unclear how feasible / time consuming that will be. Cloud can continue using their current solution without modifying the base image.
Plugins
The Cloud Docker build process downloads and adds plugins to the image, in a "plugin-archive" directory. Then, when an allocator run an Elasticsearch image, it runs the elasticsearch-plugin tool to install the active plugins from the archive directory. User-provided plugins are downloaded and installed dynamically.
There's nothing obvious that the Stack image needs to do here.