How to establish secure communication between two containers

By /

I am often asked how two containers can talk securely to one another when using Docker or Podman in CI/CD pipelines or in local development environments such as one’s laptop. “Network” is usually the answer, but the number of posts on this topic is low, so here is my attempt to shed some light on the matter. The blog post will concern itself with Podman, specifically Podman Machine, as I’m running this on my local MacBook Pro. Most of the content applies to Docker as well. However, there may be some nuances here and there where Docker does things differently. A basic understanding of how to install Podman and run containers in Podman is required for this post. If you are not there yet, check out the Podman Getting Started guide.

Table of Contents

Host access to the container

The quickest and easiest way to have something on your host talk to your container is by publishing the container’s port(s) to the host. This is done via the -p | --publish parameter and what most people are used to, for example:

gvenzl@gvenzl-mac ~ % podman run -d --name my-oracle-free -p 1521:1521 -e ORACLE_PASSWORD=LetsTest1 gvenzl/oracle-free:slim
1a141e2020efb9aa76b69bc29930225e7aac8f2ca8af1d2a14ff0bfb2e94ce39

Once the container is running, local programs, i.e., programs running directly on the host, can communicate with the container via the published port, e.g. 1521. You can verify this via the podman ps command, which shows the PORTS that are forwarded from the host into the container (the 0.0.0.0 indicates that the port is forwarded from every network interface).

gvenzl@gvenzl-mac ~ % podman ps
CONTAINER ID  IMAGE                              COMMAND     CREATED        STATUS        PORTS                   NAMES
e5f55060cb3b  docker.io/gvenzl/oracle-free:slim              3 minutes ago  Up 3 minutes  0.0.0.0:1521->1521/tcp  my-oracle-free

Hence, any program connecting to localhost:1521 on the host will automatically be forwarded into the container to port 1521:

gvenzl@gvenzl-mac ~ % sql system/LetsTest1@localhost:1521/free
🔄 Checking SQLcl version...
✅ SQLcl is current.
🚀 Launching SQLcl...

SQLcl: Release 24.3 Production on Mon Feb 10 18:51:52 2025

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

Last Successful login time: Mon Feb 10 2025 18:51:54 +01:00

Connected to:
Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.6.0.24.10

SQL 🚀 >
viins ¦ 1:0 ¦ SYSTEM ¦ localhost:1521/free ¦ /Users/gvenzl/ ¦ None ¦ None ¦ No time

However, any program in another container cannot reach the host’s port (the port forwarding is from the host to a container, not the other way around). Hence, the port forwarding doesn’t work for the other container to speak to the former one:

gvenzl@gvenzl-mac ~ % podman run --rm -ti container-registry.oracle.com/database/sqlcl
SQLcl: Release 24.3 Production on Mon Feb 10 19:28:09 2025

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

SQL> conn system/LetsTest1@localhost:1521/free
Connection failed
  USER          = system
  URL           = jdbc:oracle:thin:@localhost:1521/free
  Error Message = ORA-12541: Cannot connect. No listener at host localhost port 1521. (CONNECTION_ID=l3bey92QT6m0h1qD/qpfvg==)
https://docs.oracle.com/error-help/db/ora-12541/

Podman networking basics

The Podman documentation offers a Basic Networking Guide in their GitHub repository elaborating on the networking techniques:

Basic Network Setups

Most containers and pods being run with Podman adhere to a couple of simple scenarios.

By default, rootful Podman will create a bridged network. This is the most straightforward and preferred network setup for Podman. Bridge networking creates an interface for the container on an internal bridge network, which is then connected to the internet via Network Address Translation(NAT).

We also see users wanting to use macvlan for networking as well. The macvlan plugin forwards an entire network interface from the host into the container, allowing it access to the network the host is connected to.
With macvlan, the container is given access to a physical network interface on the host. This interface can configure multiple subinterfaces. And each subinterface is capable of having its own MAC and IP address. In the case of Podman containers, the container will present itself as if it is on the same network as the host.

And finally, the default network configuration for rootless containers is slirp4netns. The slirp4netns network mode has limited capabilities but can be run on users without root privileges. It creates a tunnel from the host into the container to forward traffic.

Communicating between containers and pods

Most users of containers have a decent understanding of how containers communicate with each other and the rest of the world. Usually each container has its own IP address and networking information. They communicate amongst each other using regular TCP/IP means like IP addresses or, in many cases, using DNS names often based on the container name. But pods are a collection of one or more containers, and with that, some uniqueness is inherited.

By definition, all containers in a Podman pod share the same network namespace. This fact means that they will have the same IP address, MAC addresses, and port mappings. You can conveniently communicate between containers in a pod by using localhost.

Bridged network communication between two containers

For two or more containers to communicate with each other, the easiest way is to add them to the same network. This can be done either at container startup via the podman run ... --network <network name> parameter or by connecting a running container to a network via the podman network connect <network name> <container name> command. Either command requires the creation of a network first, which is done via the podman network create <network name> command.

Once containers are connected, they can find each other via their container name. Podman automatically does a DNS resolution for the container name to its IP address. Note in the below example that the database container is called my-oracle-free and no port 1521 is published (no -p parameter). Yet the SQLcl container can connect to the database via @my-oracle-free:1521/free because it can reach the database container my-oracle-free and its port 1521 directly:

gvenzl@gvenzl-mac ~ % podman network create my-network
my-network
gvenzl@gvenzl-mac ~ % podman run -d --name my-oracle-free --network my-network -e ORACLE_PASSWORD=LetsTest1 gvenzl/oracle-free:slim
5b3b6e0f83ab030f4c6e6bec50e6eef6ebf2ca834b9167aeb734148e33514296
gvenzl@gvenzl-mac ~ % podman run --rm --network my-network -ti container-registry.oracle.com/database/sqlcl

SQLcl: Release 24.3 Production on Mon Feb 10 21:16:51 2025

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

SQL> conn system/LetsTest1@my-oracle-free:1521/free
Connected.
SQL> select banner from v$version;

BANNER
__________________________________________________________________________________
Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free

SQL>

However, what does not work, deliberately and hence making this a secure communication between the containers, is that no outside program can connect to either of the containers. Here is the same SQLcl command running on the host trying to connect to the database via the same connect string and failing to do so:

gvenzl@gvenzl-mac ~ % sql system/LetsTest1@my-oracle-free:1521/free
🔄 Checking SQLcl version...
✅ SQLcl is current.
🚀 Launching SQLcl...

SQLcl: Release 24.3 Production on Tue Feb 11 12:47:39 2025

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

Connection failed
  USER          = system
  URL           = jdbc:oracle:thin:@my-oracle-free:1521/free
  Error Message = ORA-17868: Unknown host specified.: my-oracle-free: nodename nor servname provided, or not known
https://docs.oracle.com/error-help/db/ora-17868/

Likewise, connecting via @localhost:1521/free also no longer works because the database container did not publish its port to the host:

gvenzl@gvenzl-mac ~ % sql system/LetsTest1@localhost:1521/free
🔄 Checking SQLcl version...
✅ SQLcl is current.
🚀 Launching SQLcl...

SQLcl: Release 24.3 Production on Tue Feb 11 12:49:18 2025

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

Connection failed
  USER          = system
  URL           = jdbc:oracle:thin:@localhost:1521/free
  Error Message = ORA-12541: Cannot connect. No listener at host localhost port 1521. (CONNECTION_ID=ekQ+0XTSTge1VJcIFLUVUg==)
https://docs.oracle.com/error-help/db/ora-12541/

You can, however, still publish any port(s) of any container, even if the container is assigned to one or more networks. For example, you could still make the database accessible to the host via port 1521 by publishing the port and also to other containers by adding the container to the network, for example:

gvenzl@gvenzl-mac ~ % podman run -d --name my-oracle-free -p 1521:1521 --network my-network -e ORACLE_PASSWORD=LetsTest1 gvenzl/oracle-free:slim
febd5155923d6ee87b987c271f4195ed4904efc297619c504d81375d7893ccd9
gvenzl@gvenzl-mac ~ % podman ps
CONTAINER ID  IMAGE                              COMMAND     CREATED         STATUS         PORTS                   NAMES
febd5155923d  docker.io/gvenzl/oracle-free:slim              10 seconds ago  Up 10 seconds  0.0.0.0:1521->1521/tcp  my-oracle-free

This makes the container accessible to software on the host using localhost, e.g.:

 gvenzl@gvenzl-mac ~ % sql system/LetsTest1@localhost:1521/free
🔄 Checking SQLcl version...
✅ SQLcl is current.
🚀 Launching SQLcl...

SQLcl: Release 24.3 Production on Tue Feb 11 14:22:42 2025

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

Last Successful login time: Tue Feb 11 2025 14:22:42 +01:00

Connected to:
Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free
Version 23.6.0.24.10

SQL 🚀 >
viins ¦ 1:0 ¦ SYSTEM ¦ localhost:1521/free ¦ /Users/gvenzl/ ¦ None ¦ None ¦ No time

And inside another container by using the container name, i.e., my-oracle-free:

gvenzl@gvenzl-mac ~ % podman run --rm --network my-network -ti container-registry.oracle.com/database/sqlcl

SQLcl: Release 24.3 Production on Tue Feb 11 13:23:22 2025

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

SQL> conn system/LetsTest1@my-oracle-free/free
Connected.
SQL> select banner from v$version;

BANNER
__________________________________________________________________________________
Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free

SQL>

While this setup seems counterintuitive at first, it’s actually rather common for application containers. These tend to publish their ports to the outside world but are also connected to a network to securely communicate with, e.g., the database:

Container architecture with two containers securely communicating over a bridged network while one container also publishes a port to the host

Network communication between pods

Podman also provides the concept of pods (go figure, given it’s called Podman :)). The best description for pods is probably the description provided by the podman pod command itself:

Pods are a group of one or more containers sharing the same network, pid and ipc namespaces.

Pods allow you to treat multiple containers as one. You can create a pod either via podman pod create <name> or you can simply have it created on the fly by adding the --pod parameter with the new: prefix to a podman run command. All containers that are in the same pod share the same network, meaning that they can connect to each other via localhost. It would be as if the software inside the containers is installed on the same machine, just that, in this case, the “machine” is the pod.

As said, a pod can easily be created via the podman pod create command and providing a pod name, e.g., oracle-pod:

gvenzl@gvenzl-mac ~ % podman pod create oracle-pod
0de0a2a914bfba7acc275c68f8d05922686a4a14ea758ba1ad5a0536f7e30013

Then, a new container can be started inside the pod via the --pod parameter:

gvenzl@gvenzl-mac ~ % podman run -d --name my-oracle-free --pod oracle-pod -e ORACLE_PASSWORD=LetsTest1 gvenzl/oracle-free:slim
a07af94ae3537b31a79c94ec02a8295a75237dc1d4a22f441258c3127d4146f7
gvenzl@gvenzl-mac ~ % podman ps --pod
CONTAINER ID  IMAGE                                    COMMAND     CREATED         STATUS        PORTS       NAMES               POD ID        PODNAME
ea8526a7d896  localhost/podman-pause:5.3.1-1732147200              19 seconds ago  Up 8 seconds              0de0a2a914bf-infra  0de0a2a914bf  oracle-pod
a07af94ae353  docker.io/gvenzl/oracle-free:slim                    8 seconds ago   Up 8 seconds              my-oracle-free      0de0a2a914bf  oracle-pod

And any other container sharing the same pod can communicate with it just via localhost:

 gvenzl@gvenzl-mac ~ % podman run --rm --pod oracle-pod -ti container-registry.oracle.com/database/sqlcl

SQLcl: Release 24.3 Production on Wed Feb 19 15:26:03 2025

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

SQL> conn system/LetsTest1@localhost:1521/free
Connected.
SQL> select banner from v$version;

BANNER
__________________________________________________________________________________
Oracle Database 23ai Free Release 23.0.0.0.0 - Develop, Learn, and Run for Free

SQL>

Another nice benefit of using pods is that multiple containers belonging to the same pod can be started and stopped together. That’s probably the main benefit of pods. Here is an example:

gvenzl@gvenzl-mac ~ % podman pod stop oracle-pod
oracle-pod
gvenzl@gvenzl-mac ~ % podman ps --pod
CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES       POD ID      PODNAME
gvenzl@gvenzl-mac ~ %

Conclusion

As containers are isolated entities by design, they cannot communicate directly with each other by default, even when they are on the same host. Networks can be used for containers to communicate securely with each other via the container name or assigned IP address. Pods can be used to group containers into a logical unit and share the same network, pid, and ipc namespaces, making the software inside multiple containers behave as if they were in one.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to Top