Had a … fun thing happen in GitHub Actions today where my tests were starting before the MySQL server they needed was up and running. MySQL in this case was running in Docker compose. So I started my process with a little shell script using mysqladmin
:
#!/usr/bin/env bash
count=0
alive="no"
while [ "$count" -lt 5 ]; do
docker-compose exec -T mysql mysqladmin ping --silent > /dev/null
if [ $? -eq 0 ]; then
alive="yes"
break
fi
echo "waiting for mysql"
sleep 5
count=$((count+1))
done
if [ "$alive" = "yes" ]; then
exit 0
else
echo "MySQL did not start up in time"
exit 1
fi
But for some reason this wait_for_mysql
script would work, but the server would still go away. I thought it might be some sort of issue with docker itself, so I switched to using a healthcheck via docker compose:
mysql:
image: mysql:8
# ...
healthcheck:
test: ["CMD", "mysqladmin", "ping", "--silent"]
retries: 1
And checking the container status in my wait_for_mysql
script:
#!/usr/bin/env bash
count=0
alive="no"
while [ "$count" -lt 6 ]; do
health=$(docker inspect --format "{{.State.Health.Status}}" "$(docker-compose ps -q mysql)")
if [ "$health" == "healthy" ]; then
alive="yes"
break
fi
echo "waiting for mysql: $health"
sleep 10
count=$((count+1))
done
if [ "$alive" = "yes" ]; then
exit 0
else
echo "MySQL did not start up in time"
exit 1
fi
I like this quite a bit better, honestly, since it accounts for the start up of the container itself.
However, the same issue kept up: either I’d get a MySQL server has gone away error or my initial setup creating the database would fail, despite the health check succeeding.
It turns out that MySQL server after its first start up goes down again before coming back up again. On a CI server, this happens every time. I ended up changing my healthcheck command to a simple mysql
call with -e
flag and this seemed to catch everything only when it was truly up.
mysql:
image: mysql:8
# ...
healthcheck:
test: ["CMD", "mysql", "-u", "root", "-e", "USE changme_to_your_database_name;"]
retries: 1