Setting Up Corporate Proxy On Docker For Mac

Docker Networking and DNS: The Good, The Bad, And The Ugly. We’ll create a few VMs running Ubuntu and use them to simulate a Docker Swarm cluster. Setting Up The Cluster. As the result of combining the proxy with Docker networking and DNS, we solved the public access to services, load balancing, and the internal. Setting up a Reverse-Proxy with Nginx and docker-compose. To start use docker-compose up -d. The -d specifies that it should be started in the background. Without it, the containers would be stopped when the command line is closed. Setup Nginx as a Reverse-Proxy inside Docker. For a basic setup only 3 things are needed: 1) Mapping of the.

• Setting Up Corporate Proxy On Docker For Mac Windows 10

Docker SDN ( Software Defined Network) already exists for quite some time. What is new, starting from the release 1.11, is the addition of DNS round-robin load balancing. That is both a reason for celebration and an opportunity to explore Docker networking and DNS. We’ll explore internal and external networking, see how DNS fits into the picture, discuss use cases that might be a good fit, and finish with pros and cons. Let’s start from the beginning. Since I don’t know whether you’re a Mac, Windows, or Linux user, we’ll create a few VMs running Ubuntu and use them to simulate a Docker Swarm cluster.

Setting Up The Cluster The only requirements for the examples that follow are. I chose Vagrant over so that everyone can use the same commands no matter whether host OS is Linux, OS X, or Windows. The principles explained here are the same no matter how you setup your Docker hosts.

Please make sure that both Vagrant and Git are installed on your laptop. Everything else will be provisioned automatically. Let’s get going. The commands that will create and provision the VMs are as follows. Git clone cd docker-flow vagrant plugin install vagrant-cachier vagrant up swarm-master swarm-node-1 swarm-node-2 proxy When the execution of the vagrant up command is finished, we’ll have a commonly used Docker Swarm cluster configuration. The proxy node hosts Consul that will serve as service registry.

At the moment, proxy service is missing (more on that later). The swarm-master server will host Swarm running in master mode. Finally, the two cluster nodes ( swarm-node-1 and swarm-node-2) are also running Swarm but, this time, in node mode. The cluster nodes also host Registrator that will make sure that all containers running in the cluster are registered in Consul. I won’t go deeper into the explanation of the cluster setup but concentrate on Docker networking and DNS features.

Swarm cluster setup through Vagrant Let us enter the proxy VM and start exploring different networking options. Vagrant ssh proxy cd /vagrant/articles/dns The dns directory contains all the code we’ll need throughout this article. Creating Docker Network Automatically Let’s see Docker networking in action. We’ll run a simple service consisting of two containers. They are defined as targets app and db inside the file.

Its content is as follows. Version: '2' services: app: image: vfarcic/books-ms ports: - 8080 environment: - DBHOST=books-ms-db dependson: - db db: containername: books-ms-db image: mongo environment: - SERVICENAME=books-ms-db This definition does not contain anything related to networking, so we’ll skip the explanation and run the containers. Export DOCKERHOST=tcp://swarm-master:2375 docker-compose -f docker-compose-internal.yml up -d app We specified that the DOCKERHOST variable should point to the Swarm master and run the Docker Compose target app. Since the definition of the app target states that it depends on db, the books-ms-db container was run as well. The output of the up command is as follows. Creating network 'dnsdefault' with the default driver.

Creating books-ms-db. Creating dnsapp1 Unless specified otherwise, Docker Compose always creates the default network for all containers belonging to a project. Unless specified with the -p argument, a project is equivalent to a directory Compose file reside in (in this case dns). Therefore, as seen from the output, Docker Compose created a new network called dnsdefault. Let’s confirm that the containers we just run are indeed inside the newly created network.

We can, for example, try to ping the container books-ms-db from inside the container dnsapp1. Docker exec -it dnsapp1 ping -c 1 books-ms-db The output of the exec command is as follows. PING books-ms-db (10.0.0.2): 56 data bytes 64 bytes from 10.0.0.2: icmpseq=0 ttl=64 time=0.055 ms - books-ms-db ping statistics - 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.055/0.055/0.055/0.000 ms We can see that both containers are inside the same network and that the IP 10.0.0.2 was assigned to books-ms-db. Since those two containers were deployed with Swarm, they are running inside different servers.

We don’t even know (without listing processes or querying Consul) where is each of those containers running. The point is that, in this case, we do not need to know the IP of the server where the container is running. As long as it belongs to the same network, it is accessible by its name.

We can invert the situation and ping the container dnsapp1 from inside the container books-ms-db. Docker exec -it books-ms-db ping -c 1 dnsapp1. All containers that form the same service (defined in the same docker-compose.yml file) are inside the same network Let’s see what would happen if we instruct Swarm to deploy an additional instance of the app target. Docker-compose -f docker-compose-internal.yml scale app=2 This time, we have the app target scaled to two instances, and we can repeat the ping commands for all containers. Docker exec -it dnsapp1 ping -c 1 dnsapp2 docker exec -it dnsapp2 ping -c 1 dnsapp1 docker exec -it dnsapp2 ping -c 1 books-ms-db As expected, all three containers can be accessed from each other. Since they belong to the same project (the same Docker Compose file), Docker put all of them inside the same network and assigned a different IP to each.

The result is the same no matter whether all containers are created at the same time or run later on. What would happen if we run a new container that, this time, does not belong to the same Compose project? Docker run -it -rm alpine ping books-ms-db The output of the run command is as follows. Ping: bad address 'books-ms-db' Since we run a container that was not defined inside the same Docker Compose project, it was not added to the same network and, therefore, it could not access the other containers.

All containers belonging to the same external network can access each others. Do we want to send requests to a particular instance of the target (e.g. Load Balancing Requests To Multiple Instances Sending requests to a specific instance of the target defies the purpose of scaling. We scale because we want to improve performance and provide failover in case one of the instances stops working. Therefore, we want requests to be load balanced between all instances of the target. We can do that easily with DNS load balancing and aliases. If you go back to the file we used earlier, you’ll notice that the backend network defined inside the app target has the book-ms alias.

By sending a request to the alias instead to a particular container name, Docker will perform round-robin load balancing and redirect it to one of the instances. As a way to test it, we’ll send three pings to the books-ms alias. Docker run -it -rm -net backend alpine ping -c 1 books-ms docker run -it -rm -net backend alpine ping -c 1 books-ms docker run -it -rm -net backend alpine ping -c 1 books-ms The combined output of the commands is as follows. PING books-ms (10.0.1.3): 56 data bytes PING books-ms (10.0.1.3): 56 data bytes PING books-ms (10.0.1.2): 56 data bytes The first two requests were sent to the instance with the IP 10.0.1.3 while the third one was sent to 10.0.1.2. Docker alias simplified communication between different services.

The choice which instance the request should be forwarded can be moved outside of the service that sends the request. All that service needs to know is that the destination service is accessible through the alias books-ms. Docker will make sure that it reaches one of the instances of that service.

In your case, the result might have been different. Don’t despair.

The explanation is coming a bit later. DNS load balancing through aliases Networking Use Cases When working with Docker and SDN (Software Defined Networking), some patterns emerge and we can observe different use cases. Networks created automatically by Docker Compose should be utilized for internal communication between containers that form the same project or the same service. That is useful because those containers need to be able “speak” with each other. In our example, app target stores and retrieves data from the database running in the separate container.

No one outside that network can access those containers. We can call this type internal communication. On the other hand, the app target exposes an HTTP API. We need other services to be able to invoke it. For that reason, we created the second (external) network called backend. Since other services are not supposed to access each other’s databases, they are excluded from the network. In other words, communication between different services is available only through their APIs.

We can call this type external communication. Allowing Public Access To Services Those who paid attention will notice that we didn’t mention the third case. We did not explore ways to access services from outside the cluster. After all, in most cases, we want our services to be public. We want our users to be able to use them. The reason for this omission lies in the need to use different tools for this type of access. While it is technically possible and relatively easy to accomplish this use case with Docker DNS load balancing, the solution would be sub-optimum.

Proxy services like and are battle tested solutions for load balancing, reverse proxy, and few other things. Large scale access to scaled services with potentially huge load might not be best served using methods we explored. On the other hand, continuous deployment promoted by containers require constant reconfiguration of those services together with service discovery that will provide necessary information about the cluster and everything running inside it. Dynamic proxy reconfiguration and service discovery are beyond the scope of this article, and there are plenty of tools that will allow you to build your own solution.

We’ll use one of those only as an example of the result we want to accomplish with a proxy. Since we do not, yet, have a proxy, the service we experimented with is currently not available from the public network. We can confirm that by sending an HTTP request. Curl -I proxy/api/v1/books The output of the curl command is as follows. HTTP/1.0 503 Service Unavailable Cache-Control: no-cache Connection: close Content-Type: text/html As expected, we got 503 Service Unavailable response.

We can easily change that by running a proxy and configuring it to perform load balancing among the two instances of the service. We’ll run the vfarcic/docker-flow-proxy container defined in the file. The definition is as follows. Version: '2' services: proxy: containername: docker-flow-proxy image: vfarcic/docker-flow-proxy environment: CONSULADDRESS: 10.100.198.200:8500 ports: - 80:80 - 443:443 - 8080:8080 Part of the process that created the VMs was to run the proxy, so all we have to do to is reconfigure it to load balance requests to the books-ms service.

Curl 'proxy:8080/v1/docker-flow-proxy/reconfigure?serviceName=books-ms&servicePath=/api/v1/books' Now that the proxy is configured, we can use it as a relay to our service. Curl -I proxy/api/v1/books This output of the curl command is as follows. HTTP/1.1 200 OK Server: spray-can/1.3.1 Date: Sat, 23 Apr 2016 17:47:20 GMT Access-Control-Allow-Origin:. Content-Type: application/json; charset=UTF-8 Content-Length: 2 This time, the response is 200 OK meaning that the service is indeed accessible through the proxy. As the result of combining the proxy with Docker networking and DNS, we solved the public access to services, load balancing, and the internal communication between containers.

Proxy combined with Docker networking and DSN If you’re interested in more details about the proxy we used, please read the article or jump directly to the project. The Good, The Bad, And The Ugly Let us, quickly, evaluate the practices and technology we explored in this article. The good (adopt now) Docker networking exists for several versions, and it is stable. Operating containers inside a cluster was painful and complicated without it.

Links could not span multiple containers without involving workarounds, tricks, and “witchcraft”. It was the major milestone that made Swarm a viable and, in many cases, the best solution for running containers inside a cluster. DNS round robin is very easy to define and use. Big thumbs up to Docker for this feature. It is great when we need to allow internal communication between different containers. Without it, Docker networking is truly useful only when there is a single instance of the destination service. With DNS, we can start contemplating the usage of Docker SDN on a larger scale.

The bad (potentially useful but still in its infancy) While the DNS resolver in Engine 1.11 will randomize records (i.e. Return them in a different order at each request), most DNS resolvers will sort the records. As the result, round-robin will not truly work as expected. You saw the example when we made three pings through DNS to the destination target app. Even though no one else was accessing the service, the first two requests went to the first instance, while the third went to the second (in your case the order might have been different).

True round-robin would make the first request to the first instance, the second request to the second instance, and the third request to the first instance again. The problem is not in the Docker’s implementation but in DNS itself. While in many cases this deviation should not cause problems, there are, still, better solutions we can apply. On the upside, this addition is a very important milestone, and we are yet to see the improvements next releases will bring.

The ugly (do not use it) Docker’s DNS combined with aliases has many uses, but public-facing load balancing of big traffic is not one of them. Do not be tempted to replace your proxy with it.

HAProxy and nginx, just to name a few, are battle tested and proven to work well under almost any load. With health checks, reverse proxy, URL rewriting, and many other features, they are much more than simple round-robin load balancers. The challenge when using proxies is dynamic reconfiguration based on service discovery, but that does not diminish their value nor justifies the switch to Docker’s DNS.

On the other hand, proxies do not reduce the value we got in the release 1.11. It is one more tool in the toolbelt and it is our job to know when to use one over the other. Please read the article for more information. Even if you do not choose to use Docker Flow: Proxy, it provides an insight into what we expect from proxies inside a cluster full of containers. The DevOps 2.1 Toolkit: Docker Swarm If you liked this article, you might be interested in book. Unlike the previous title in the series ( ) that provided a general overlook of some of the latest DevOps practices and tools, this book is dedicated entirely to Docker Swarm and the processes and tools we might need to build, test, deploy, and monitor services running inside a cluster.

You can get a copy from (and the other worldwide sites). It is also available as bundle. Give the book a try and let me know what you think.

Have you heard of? You probably have—everybody’s talking about it. Even my dad’s like, “what’s Docker? I saw someone twitter about it on the Facebook. You should call your mom.” Docker is a program that makes running and managing super easy.

It has the potential to change all aspects of server-side applications, from development and testing to deployment and scaling. It’s pretty cool. Recently, I’ve been working through. It’s a top notch book and I highly recommend it, but I’ve had some problems running the examples on OS X. After a certain point, the book assumes you’re using Linux and skips some of the extra configuration required to make the examples work on OS X. This isn’t the book’s fault; rather, it speaks to underlying issues with how Docker works on OS X. This post is a walkthrough of the issues you’ll face running Docker on OS X and the workarounds to deal with them.

It’s not meant to be a tutorial on Docker itself, but I encourage you to follow along and type in all the commands. You’ll get a better understanding of how Docker works in general and on OS X specifically. Plus, if you decide to dig deeper into Docker on your Mac, you’ll be saved hours of troubleshooting. Don’t say I never gave you nothing. First, let’s talk about how Docker works and why running it on OS X no work so good. How Docker Works Docker is a client-server application. The Docker server is a daemon that does all the heavy lifting: building and downloading images, starting and stopping containers, and the like.

It exposes a REST API for remote management. The Docker client is a command line program that communicates with the Docker server using the REST API. You will interact with Docker by using the client to send commands to the server.

The machine running the Docker server is called the Docker host. The host can be any machine—your laptop, a server in the Cloud™, etc—but, because Docker uses features only available to Linux, that machine must be running Linux (more specifically, the Linux kernel). Docker on Linux Suppose we want to run containers directly on our Linux laptop.

Here’s how it looks: Docking on Linux The laptop is running both the client and the server, thus making it the Docker host. Docker on OS X Here’s the thing about OS X: it’s not Linux. It doesn’t have the kernel features required to run Docker containers natively. We still need to have Linux running somewhere. Boot2docker is a “lightweight Linux distribution made specifically to run Docker containers.” Spoiler alert: you’re going to run it in a VM on your Mac.

Here’s a diagram of how we’ll use boot2docker: Docking on OS X We’ll run the Docker client natively on OS X, but the Docker server will run inside our boot2docker VM. This also means boot2docker, not OS X, is the Docker host, not OS X. Let’s install dat software. Installation Step 1: Install VirtualBox Go and do it. You don’t need my help with that.

Step 2: Install Docker and boot2docker You have two choices: the offical package from. I prefer homebrew because I like to manage my environment from the command line.

brew update brew install docker brew install boot2docker Step 3: Initialize and start boot2docker First, we need to initialize boot2docker (we only have to do this once): boot2docker init 2014/08/21 13:49:33 Downloading boot2docker ISO image. 2014/08/21 13:49:50 Done. Type `boot2docker up` to start the VM.

Next, we can start up the VM. Do like it says: boot2docker up 2014/08/21 13:51:29 Waiting for VM to be started. 2014/08/21 13:51:50 Started.

Setting Up Corporate Proxy On Docker For Mac Windows 10

2014/08/21 13:51:51 Trying to get IP one more time 2014/08/21 13:51:51 To connect the Docker client to the Docker daemon, please set: 2014/08/21 13:51:51 export DOCKERHOST=tcp://192.168.59.103:2375 Step 4: Set the DOCKERHOST environment variable The Docker client assumes the Docker host is the current machine. We need to tell it to use our boot2docker VM by setting the DOCKERHOST environment variable: export DOCKERHOST=tcp://192.168.59.103:2375.