Fixing Apache Superset Connection to Host PostgreSQL from Docker

TL;DR

Apache Superset failed to connect to a PostgreSQL database running on the Docker host, even after PostgreSQL was configured to listen on the Docker bridge network and firewall rules were updated. Network connectivity tests confirmed that the Superset container could reach PostgreSQL on 172.17.0.1:5432, but connection attempts still failed. The root cause turned out to be a missing PostgreSQL driver (psycopg2) inside the Superset container. This article walks through the troubleshooting process, including PostgreSQL configuration, Docker networking validation, and the temporary fix of installing psycopg2-binary directly into the running Superset container.

Environment

In this setup:

• Apache Superset runs inside a Docker container.
• PostgreSQL runs directly on the Linux host.
• The target database is completejourney.

The objective was to connect Superset to PostgreSQL using the host machine instead of running PostgreSQL inside Docker.

---

Initial Symptoms

Superset Could Not Connect Using host.docker.internal

The first attempt used:

host.docker.internal

in the Superset PostgreSQL connection form.

Superset returned:

The hostname provided can't be resolved

Inspecting the container confirmed that host.docker.internal was not defined inside the container:

cat /etc/hosts

This is a common situation on Linux, where host.docker.internal is not automatically available.

---

Superset Could Reach the Docker Host IP but Returned Another Error

The next attempt used the Docker bridge gateway IP:

172.17.0.1

This time Superset no longer reported a hostname resolution problem. Instead, it returned a different error indicating that the connection could not be established.

At this stage, the issue appeared to be related to either PostgreSQL configuration or network access.

---

PostgreSQL Was Not Listening on the Docker Bridge Interface

Although PostgreSQL was configured with:

listen_addresses = 'localhost,172.17.0.1'

checking active listeners showed:

sudo ss -ltnp | grep 5432

Output:

LISTEN 0 244 127.0.0.1:5432

This meant PostgreSQL was still only listening on localhost.

Attempting to connect using the Docker bridge IP failed:

psql -U postgres -h 172.17.0.1

could not connect to server: Connection refused

---

After PostgreSQL Restart, Superset Reported Generic Connection Failure

After correcting the PostgreSQL configuration and restarting the service, PostgreSQL successfully listened on the Docker bridge interface.

Connectivity testing confirmed that the Superset container could reach PostgreSQL:

python -c "import socket; print(socket.create_connection(('172.17.0.1',5432),5))"

Output:

<socket.socket ...>

At this point:

• PostgreSQL was reachable.
• Docker networking was functioning correctly.
• Firewall rules were not blocking access.

However, the Superset UI still displayed:

Connection failed, please check your connection settings

This suggested that the issue was no longer related to networking and required further investigation.

---

The Real Error Appeared in SQLAlchemy Mode

After switching to a SQLAlchemy connection string, Superset showed a more specific error:

ERROR: Could not load database driver: PostgresEngineSpec

This finally revealed the actual root cause: the PostgreSQL Python driver (psycopg2) was missing from the Superset container.

---

Root Cause

The PostgreSQL Python driver was missing from the Superset container.

Verification:

docker exec -it superset bash

python -c "import psycopg2; print(psycopg2.version)"

Output:

ModuleNotFoundError: No module named 'psycopg2'

Because the driver was missing, Superset could not create PostgreSQL connections even though the database was reachable.

---

Temporary Solution

Access the Container as Root

docker exec -u root -it superset bash

Install the PostgreSQL Driver

pip install psycopg2-binary

Verify Installation

python -c "import psycopg2; print(psycopg2.version)"

Expected output:

2.x.x

Restart the Container

docker restart superset

---

Superset Connection Configuration

After installing the driver, use either the PostgreSQL form or the SQLAlchemy URI.

Example SQLAlchemy URI:

postgresql+psycopg2://postgres:<PASSWORD>@172.17.0.1:5432/completejourney

Example:

postgresql+psycopg2://postgres:mypassword@172.17.0.1:5432/completejourney

---

Lessons Learned

When connecting Apache Superset running in Docker to PostgreSQL running on the host:

1. Ensure PostgreSQL listens on an address reachable from Docker containers.
2. Allow Docker network ranges in pg_hba.conf.
3. Verify connectivity from inside the container rather than relying solely on the UI.
4. If host.docker.internal does not resolve on Linux, use the Docker bridge gateway IP such as 172.17.0.1.
5. If Superset reports:

Could not load database driver: PostgresEngineSpec

check whether the PostgreSQL Python driver is installed:

python -c "import psycopg2"

and install it if necessary:

pip install psycopg2-binary

In this case, multiple networking issues were investigated, but the final blocker turned out to be a missing PostgreSQL driver inside the Superset container.