On 17/01/17 15:00, Fabian Arrotin wrote:

On 13/01/17 22:25, Karanbir Singh wrote:

...

hi,

the docker-distribution package can act as a smart proxy for docker registries; and having a local cache inside the ci infra would help speed up a lot of the jobs I am working with at the moment there.

Do we have the space to host a cache of this nature ? I'd think a 15 to 20 GB of space is all we need.

Could we then have dns inside the ci infra use this cache for the dockerhub urls ?

Any objections to getting this in ?

regards

I'd say it's oable, and we already "override" some DNS records internally to redirect to internal node/mirror in the same network (known example : mirrorlist.centos.org)

But the only question I have is about the dockerhub urls : are those using tcp/443 and so TLS ? if so, that will not work as we can't impersonate such server (and thanks to TLS btw). If they are only using plain tcp/80 or tcp/5000 , that should work

I think the docker-proxy actually handles this case, is it possible to test once ?

On 17/01/17 19:20, Fabian Arrotin wrote:

On 17/01/17 19:11, Karanbir Singh wrote:

...

On 17/01/17 15:00, Fabian Arrotin wrote:

...

On 13/01/17 22:25, Karanbir Singh wrote:

...

hi,

the docker-distribution package can act as a smart proxy for docker registries; and having a local cache inside the ci infra would help speed up a lot of the jobs I am working with at the moment there.

Do we have the space to host a cache of this nature ? I'd think a 15 to 20 GB of space is all we need.

Could we then have dns inside the ci infra use this cache for the dockerhub urls ?

Any objections to getting this in ?

regards

I'd say it's oable, and we already "override" some DNS records internally to redirect to internal node/mirror in the same network (known example : mirrorlist.centos.org)

But the only question I have is about the dockerhub urls : are those using tcp/443 and so TLS ? if so, that will not work as we can't impersonate such server (and thanks to TLS btw). If they are only using plain tcp/80 or tcp/5000 , that should work

I think the docker-proxy actually handles this case, is it possible to test once ?

We can, but that I meant was that it will not be transparent as it it would get it from dockerhub : if one adds the internal "proxy" as "insecure" registry, itself being the docker-distribution proxy that will fetch from "upstream dockerhub" then it will work (afaik). Let me test this

So I tested two things :

= ensuring docker knows that it has to use an internal mirror registry docker daemon supports a "--registry-mirror=" option for the daemon, so it knows that it has to fetch images through that internal mirror and not from docker.io (in fact registry-1.docker.io) So we just need docker-distribution installed and configured internally (can even be on the internal CI mirror), and images are automatically pulled/cached from docker.io to that internal node.

Pros : easy Cons : need to ensure that all people modify their jobs to add that line to /etc/sysconfig/docker and restart docker before using the internal cache/mirror registry

= playing the MITM (yeah) Still same technique for the caching server, but instead of plain tcp/5000, we have apache in front, running on tcp/443 with cert for registry-1.docker.io (sic !), and using proxypass on localhost:5000 (to reach docker-distribution cache) . We can overidde registry-1.docker.io A record internally (as we do for other records)

Pros : transparent and nothing do at the docker daemon level on the ci bare-metal node (except update-ca-trust, see in the Cons section) Cons : this is "on the edge" as it's really a MITM operation, and for that one to be working, we then need to add our CA crt on each provisioned node (can be done at kickstart level) and update-ca-trust. Also we have to keep an eye on it, as if suddenly they change the default registry A record in the docker code, we'll have to adapt and regen another "fake" cert

Comments welcome before we decide which way to go :-)