docker: Always use the official SQL Server image for testing.

Now that Docker for macOS supports using Rosetta for x86/amd64 emulation, we no longer need to use the `azure-sql-edge` image. We can always use the official `mcr.microsoft.com/mssql/server` one.

This also means that we no longer need the separate healthcheck container, because the official image ships with `sqlcmd`.

When this is merged, you will need to ensure you have enabled Rosetta emulation in the Docker settings to test against SQL Server on macOS. This requires macOS 13 (Ventura).

PR-URL: hasura/graphql-engine-mono#10139
GitOrigin-RevId: 2225bf5f4c6d3632da1f29b2229c9b04ead5e34c

Author: SamirTalwar

Diff for: .envrc

@@ -3,12 +3,6 @@
 # This file provides some tooling on an opt-in basis via `direnv`
 # (https://direnv.net/)
 
-# We cannot use the proper MSSQL Server image on arm64, as it's not available.
-# Instead, we use `mcr.microsoft.com/azure-sql-edge`.
-if [[ "$(uname -m)" == 'arm64' ]]; then
-  export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
-fi
-
 # To use the functionality here, create an `.envrc.local` file in this folder
 # that runs the functions you need.
 # There is an example in `.envrc.local.example` you can start with

Diff for: .envrc.local.example

@@ -24,10 +24,6 @@
 # HASURA_BIGQUERY_SERVICE_KEY=$(cat ../bigquery-service-account.json)
 # export HASURA_BIGQUERY_SERVICE_KEY
 
-### Enable to use the correct image for SQLServer on an M1
-### (the env var is used in our Docker Compose files)
-# export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
-
 ### Export EE license key for running pro tests locally
 ### This depends on a `ee-license-key.txt` living in the directory above
 ### the `graphql-engine` repository

Diff for: docker-compose.yaml

@@ -21,11 +21,16 @@
 #
 # Facts:
 #
-# * The SERVICE PORTS numbering start at 65001, 65002, etc. to avoid bother
+# * The SERVICE PORTS numbering start at 65001, 65002, etc. to avoid bothering
 #   existing instances of databases.
 #
 # * The login credentials are, where possible, all "hasura" to avoid unnecessary
 #   mental overhead.
+#
+# * The SQL Server image will only work on macOS if you enable the relevant
+#   settings. Open the Docker Desktop settings, enable "Use Virtualization
+#   framework" in the "General" tab, and "Use Rosetta for x86/amd64 emulation on
+#   Apple Silicon" in the "Features in development" tab.
 
 version: "3.6"
 
@@ -81,14 +86,6 @@ services:
     volumes:
       - mssql-data:/var/opt/mssql
 
-  sqlserver-healthcheck:
-    extends:
-      file: docker-compose/databases.yaml
-      service: sqlserver-healthcheck
-    depends_on:
-      sqlserver:
-        condition: service_started
-
   dc-reference-agent:
     extends:
       file: dc-agents/docker-compose.yaml

Diff for: docker-compose/databases.yaml

@@ -81,12 +81,8 @@ services:
       - /var/lib/postgresql/data
 
   sqlserver:
-    # We cannot use this image on arm64, as it's not available.
-    # Instead, we use `mcr.microsoft.com/azure-sql-edge`.
-    # This uses an environment variable so it can be overridden by scripts to
-    # provide that image instead.
-    image: ${MSSQL_IMAGE:-mcr.microsoft.com/mssql/server:2019-latest}
-    init: true # azure-sql-edge doesn't shut down properly without this
+    image: mcr.microsoft.com/mssql/server:2019-latest
+    platform: linux/amd64
     ports:
       - 1433
     environment:
@@ -95,35 +91,15 @@ services:
       MSSQL_SA_PASSWORD: "Password!"
     volumes:
       - /var/opt/mssql
-
-  # We would prefer to attach the healthcheck directly to the `sqlserver` container.
-  # However, while we can do this in the `mcr.microsoft.com/mssql/server` image,
-  # it's not possible when running on macOS, as we don't use that image, but
-  # instead `mcr.microsoft.com/azure-sql-edge`.
-  # The `mcr.microsoft.com/azure-sql-edge` image does not provide `sqlcmd`.
-  # We therefore need to run it in a separate container.
-  # We can reconsider this once the MSSQL Server image can run on arm64.
-  sqlserver-healthcheck:
-    image: mcr.microsoft.com/mssql-tools
-    platform: linux/amd64
-    command:
-      - sleep
-      - inf
-    init: true # handle signals correctly
     healthcheck:
       test:
         - CMD-SHELL
         - |
-          /opt/mssql-tools/bin/sqlcmd -S 'sqlserver,1433' -U SA -P "Password!"
+          /opt/mssql-tools/bin/sqlcmd -U SA -P "$$SA_PASSWORD"
       start_period: 5s
       interval: 5s
       timeout: 10s
       retries: 20
-    # This is commented out because this file can be extended, and dependencies don't work for extensions.
-    # Instead, ensure you also run this one.
-    # depends_on:
-    #   sqlserver:
-    #     condition: service_started
 
   mysql:
     image: mysql

Diff for: frontend/docker/README.md

@@ -21,7 +21,7 @@ You may also need to flip the [feature flags here](http://localhost:4200/setting
 
 ## MSSQL / SQL Server
 
-In order to initialize and populate a MSSQL demo database automatically, we making use of the `mssql-tools` image. This image runs a few commands against the `mssql` image to create the demo data.
+In order to initialize and populate a MSSQL demo database automatically, we making use of the `mssql-init` container. This image runs a few commands against the `mssql` image to create the demo data.
 
 This scripting is done in `docker/DataSources/mssql/run-initialization.sh`.
 

Diff for: frontend/docker/docker-compose.template.yml

@@ -35,6 +35,10 @@ services:
       - '8100:8100'
     volumes:
       - ./DataSources/sqlite.db:/sqlite.db
+  # The SQL Server image will only work on macOS if you enable the relevant
+  # settings. Open the Docker Desktop settings, enable "Use Virtualization
+  # framework" in the "General" tab, and "Use Rosetta for x86/amd64 emulation on
+  # Apple Silicon" in the "Features in development" tab.
   mssql:
     container_name: 'mssql'
     user: root
@@ -43,14 +47,23 @@ services:
     volumes:
       - mssql-database-data:/var/opt/mssql
     stdin_open: true
-    image: mcr.microsoft.com/azure-sql-edge:latest
-    # if not running Apple Silicon (M1/M2 etc), you can use this:
-    #image: mcr.microsoft.com/mssql/server:2022-latest
-    init: true # azure-sql-edge doesn't shut down properly without this
+    image: &mssql-image mcr.microsoft.com/mssql/server:2019-latest
+    platform: linux/amd64
     environment:
       ACCEPT_EULA: 'Y'
       SA_PASSWORD: 'Password!'
       MSSQL_SA_PASSWORD: 'Password!'
+  mssql-init:
+    depends_on:
+      - mssql
+    container_name: 'mssql-init'
+    image: *mssql-image
+    platform: linux/amd64
+    command:
+      - /opt/mssql_scripts/run-initialization.sh
+    stdin_open: true
+    volumes:
+      - ./DataSources/mssql:/opt/mssql_scripts
   mongo:
     image: mongo
     restart: always
@@ -62,17 +75,6 @@ services:
       - '27017:27017'
     command: mongod --setParameter enableLocalhostAuthBypass=0 --dbpath=/data/db --bind_ip_all --port 27017
 
-  # sqlcmd (tools) are not included in the arm64 image
-  mssql-tools:
-    depends_on:
-      - mssql
-    container_name: 'mssql-tools'
-    image: mcr.microsoft.com/mssql-tools:latest
-    command: bash /opt/mssql_scripts/run-initialization.sh
-    stdin_open: true
-    volumes:
-      - ./DataSources/mssql:/opt/mssql_scripts
-
   graphql-engine:
     image: hasura/graphql-engine:latest
 

Diff for: scripts/make/repl.mk

@@ -3,7 +3,7 @@
 .PHONY: repl-sqlserver
 ## repl-sqlserver: start a sqlserver docker image and connect to it using sqlcmd
 repl-sqlserver:
-	@docker compose up -d --wait sqlserver-healthcheck
+	@docker compose up -d --wait sqlserver
 	@sqlcmd -S localhost,$(shell docker compose port sqlserver 1433 | sed -e 's#.*:\(\)#\1#') -U SA -P "Password!"
 
 .PHONY: repl-postgres

Diff for: scripts/make/test-infrastructure.mk

@@ -2,16 +2,6 @@ API_TESTS_DOCKER_COMPOSE = docker compose --project-directory=./server/lib/api-t
 API_TESTS_PRO_DOCKER_COMPOSE = docker compose --project-directory=./pro/server/lib/api-tests
 PYTHON_TESTS_DOCKER_COMPOSE = docker compose --project-directory=./server/tests-py
 
-# Use the Azure SQL Edge image instead of the SQL Server image on arm64.
-# The latter doesn't work yet.
-ifeq ($(shell uname -m),arm64)
-MSSQL_IMAGE=mcr.microsoft.com/azure-sql-edge
-else
-MSSQL_IMAGE=  # allow the Docker Compose file to set the image
-endif
-
-export MSSQL_IMAGE
-
 TEST_MSSQL_CONNECTION_STRING = Driver={ODBC Driver 18 for SQL Server};Server=localhost,65003;Uid=sa;Pwd=Password!;Encrypt=optional
 TEST_POSTGRES_URL = postgres://hasura:hasura@localhost:65002/hasura
 

Diff for: scripts/make/tests.mk

@@ -147,7 +147,7 @@ test-unit: remove-tix-file
 .PHONY: test-integration-mssql
 ## test-integration-mssql: run MS SQL Server integration tests
 test-integration-mssql: remove-tix-file
-	docker compose up --build --detach --wait sqlserver-healthcheck
+	docker compose up --build --detach --wait sqlserver
 	HASURA_MSSQL_CONN_STR='$(TEST_MSSQL_CONNECTION_STRING)' \
 			cabal run graphql-engine:test:graphql-engine-test-mssql
 
@@ -162,7 +162,7 @@ test-integration-postgres: remove-tix-file
 ## test-native-queries: run all tests for the Native Query feature
 test-native-queries:
 	cabal build exe:graphql-engine-pro
-	docker compose up --build --detach --wait postgres citus cockroach sqlserver-healthcheck
+	docker compose up --build --detach --wait postgres citus cockroach sqlserver
 	HSPEC_MATCH=NativeQueries make test-unit
 	HSPEC_MATCH=NativeQueries \
 		GRAPHQL_ENGINE=$(GRAPHQL_ENGINE_PRO_PATH) \
@@ -183,7 +183,7 @@ test-native-queries-postgres:
 ## test-native-queries-sqlserver: run all sqlserver tests for the Native Query feature
 test-native-queries-sqlserver: remove-tix-file
 	cabal build exe:graphql-engine-pro
-	docker compose up --build --detach --wait postgres sqlserver-healthcheck
+	docker compose up --build --detach --wait postgres sqlserver
 	HSPEC_MATCH=${HSPEC_MATCH:-"NativeQueries"}
 	HASURA_TEST_BACKEND_TYPE=SQLServer \
 		GRAPHQL_ENGINE=$(GRAPHQL_ENGINE_PRO_PATH) \
@@ -203,7 +203,7 @@ test-native-queries-bigquery: remove-tix-file
 ## test-stored-procedures-sqlserver: run all sqlserver tests for the Stored Procedure feature
 test-stored-procedures-sqlserver: remove-tix-file
 	cabal build exe:graphql-engine-pro
-	docker compose up --build --detach --wait postgres sqlserver-healthcheck
+	docker compose up --build --detach --wait postgres sqlserver
 	HASURA_TEST_BACKEND_TYPE=SQLServer \
 		HSPEC_MATCH=StoredProcedures \
 		GRAPHQL_ENGINE=$(GRAPHQL_ENGINE_PRO_PATH) \

Diff for: server/lib/api-tests/README.md

@@ -311,43 +311,9 @@ brew install microsoft/mssql-release/mssql-tools@18
 brew unlink mssql-tools18 && brew link mssql-tools18
 ```
 
-### Microsoft SQL Server failures on Apple aarch64 chips
+### The MS SQL Server container fails to start
 
-This applies to all Apple hardware that uses aarch64 chips, e.g. the MacBook M1
-or M2.
-
-We have a few problems with Microsoft SQL Server on Apple aarch64:
-
-1.  Microsoft has not yet released SQL Server for aarch64. We need to use Azure
-    SQL Edge instead.
-
-    You don't need to do anything if you're using the `make` commands; they
-    will provide the correct image automatically.
-
-    If you run `docker compose` directly, make sure to set the environment
-    variable yourself:
-
-    ```sh
-    export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
-    ```
-
-    You can add this to your _.envrc.local_ file if you like.
-
-2.  Azure SQL Edge for aarch64 does not ship with the `sqlcmd` utility with
-    which we use to setup the SQL Server schema.
-
-    If you need it, you can instead use the `mssql-tools` Docker image, for
-    example:
-
-    ```
-    docker run --rm -it --platform=linux/amd64 --net=host mcr.microsoft.com/mssql-tools \
-      /opt/mssql-tools/bin/sqlcmd -S localhost,65003 -U SA -P <password>
-    ```
-
-    To make this easier, you might want to define an alias:
-
-    ```
-    alias sqlcmd='docker run --rm -it --platform=linux/amd64 --net=host mcr.microsoft.com/mssql-tools /opt/mssql-tools/bin/sqlcmd'
-    ```
-
-    You can also install them directly with `brew install microsoft/mssql-release/mssql-tools`.
+The SQL Server image will only work on macOS if you enable the relevant
+settings. Open the Docker Desktop settings, enable "Use Virtualization
+framework" in the "General" tab, and "Use Rosetta for x86/amd64 emulation on
+Apple Silicon" in the "Features in development" tab.

Diff for: server/lib/api-tests/docker-compose.yaml

@@ -52,14 +52,6 @@ services:
     volumes:
       - mssql-data:/var/opt/mssql
 
-  sqlserver-healthcheck:
-    extends:
-      file: ../../../docker-compose/databases.yaml
-      service: sqlserver-healthcheck
-    depends_on:
-      sqlserver:
-        condition: service_started
-
   dc-reference-agent:
     extends:
       file: ../../../dc-agents/docker-compose.yaml

Diff for: server/tests-py/docker-compose.yml

@@ -27,11 +27,3 @@ services:
     extends:
       file: ../../docker-compose/databases.yaml
       service: sqlserver
-
-  sqlserver-healthcheck:
-    extends:
-      file: ../../docker-compose/databases.yaml
-      service: sqlserver-healthcheck
-    depends_on:
-      sqlserver:
-        condition: service_started

Diff for: server/tests-py/run.sh

@@ -14,7 +14,7 @@ set -o pipefail
 
 cd -- "$(dirname -- "${BASH_SOURCE[0]}")"
 
-DATABASES=(postgres citus sqlserver sqlserver-healthcheck)
+DATABASES=(postgres citus sqlserver)
 
 (
   cd ../..
@@ -29,12 +29,6 @@ DATABASES=(postgres citus sqlserver sqlserver-healthcheck)
 # shellcheck disable=SC1091
 source .hasura-dev-python-venv/bin/activate
 
-# Use the Azure SQL Edge image instead of the SQL Server image on arm64.
-# The latter doesn't work yet.
-if [[ "$(uname -m)" == 'arm64' ]]; then
-  export MSSQL_IMAGE='mcr.microsoft.com/azure-sql-edge'
-fi
-
 echo
 echo '*** Starting databases ***'
 docker compose up -d --wait "${DATABASES[@]}"