See: https://pagure.io/fedora-atomic-host-continuous/issue/24
https://ci.centos.org/artifacts/sig-atomic/fahc/rdgo/build should be a symlink, but is rendering as a regular file. Some sort of nginx frontend config change?
Is the source code to the config management/kube objects/whatever for stuff like the artifacts nginx public somewhere so I could look at `git log`?
I wouldn't say we're blocked, I can figure out some other place to sync our content and point other things at it, but hopefully it's an easy fix and I won't have to.
On 24/08/18 02:13, Colin Walters wrote:
See: https://pagure.io/fedora-atomic-host-continuous/issue/24
https://ci.centos.org/artifacts/sig-atomic/fahc/rdgo/build should be a symlink, but is rendering as a regular file. Some sort of nginx frontend config change?
Is the source code to the config management/kube objects/whatever for stuff like the artifacts nginx public somewhere so I could look at `git log`?
I wouldn't say we're blocked, I can figure out some other place to sync our content and point other things at it, but hopefully it's an easy fix and I won't have to.
I confirm that there was no config change at the nginx level, but that there was an issue wrt rsync to the artifacts storage node. I just had a quick look and it's *not* a symlink on that box either, so was wondering if it's a symlink at the src side.
Due to the fact that the storage box was itself updated/restarted during the maintenance window, we suffered from a rsync 3.0 => 3.1 (landed in 7.5) issue, that was fixed by this commit :
commit 26b49f2cf0e6bf401b9248f6fa2b76dcd2020036 Author: Fabian Arrotin arrfab@centos.org Date: Thu Aug 23 13:51:05 2018 +0200
added "fake super" option for rsyncd.conf, as update to rsync 3.1.2 blocked CI projects from uploading to artifacts (see https://bugzilla.redhat.com/show_bug.cgi?id=1506234)
That solved the "operation not permitted" issue that some projects had when pushing to artifacts (discussed in #centos-devel)
Can you verify that if src is a symlink on your side and that you rsync to artifacts, that issue is still there ? (aka consider a file and not a symlink at all)
On Fri, Aug 24, 2018 at 8:31 AM, Fabian Arrotin arrfab@centos.org wrote:
That solved the "operation not permitted" issue that some projects had when pushing to artifacts (discussed in #centos-devel)
Ahh awesome, we've definitely been hitting that as well and ended up peppering chown/chmods a bit everywhere.
Can you verify that if src is a symlink on your side and that you rsync to artifacts, that issue is still there ? (aka consider a file and not a symlink at all)
I can confirm that's still the case:
``` $ rsync --delete --stats -a sig-atomic@artifacts.ci.centos.org::sig-atomic/fahc/rdgo/ rdgo/ $ file rdgo/build rdgo/build: symbolic link to `build-0' ```
And yet:
``` $ curl -I http://artifacts.ci.centos.org/sig-atomic/fahc/rdgo/build/ HTTP/1.1 404 Not Found Date: Fri, 24 Aug 2018 13:51:16 GMT Server: Apache/2.4.6 (CentOS) PHP/5.4.16 Content-Type: text/html; charset=iso-8859-1 ```
Compare with:
``` $ curl -I http://artifacts.ci.centos.org/sig-atomic/centos-continuous/rdgo/build/ HTTP/1.1 200 OK Date: Fri, 24 Aug 2018 13:51:08 GMT Server: Apache/2.4.6 (CentOS) PHP/5.4.16 Content-Type: text/html;charset=ISO-8859-1 ```
which is also a symlink, but somehow that one works.
On Fri, Aug 24, 2018, at 9:58 AM, Jonathan Lebon wrote:
$ curl -I http://artifacts.ci.centos.org/sig-atomic/fahc/rdgo/build/ HTTP/1.1 404 Not Found Date: Fri, 24 Aug 2018 13:51:16 GMT Server: Apache/2.4.6 (CentOS) PHP/5.4.16 Content-Type: text/html; charset=iso-8859-1
Yeah, I just pushed: https://pagure.io/fedora-atomic-host-continuous/c/37749456215b242012535a69c7... and that ran successfully: https://ci.centos.org/view/Atomic/job/fahc-rdgo/25128/console yet what shows up on the other side is the weird "symlink contents as regular file".
This also reproduces if I just ssh to the slave directly; I just did:
ssh slave01.ci.centos.org ... mkdir -p rsync-test cd rsync-test mkdir foo ln -s blah foo rsync --stats -av ./ sig-atomic@artifacts.ci.centos.org::sig-atomic/fahc/rsync-test/
And I notice rsync says: Number of files: 4 (dir: 3, link: 1)
But what resulted: https://ci.centos.org/artifacts/sig-atomic/fahc/rsync-test/rsync-test/foo
So this must be something related to the server right?
On 24/08/18 20:23, Colin Walters wrote:
This also reproduces if I just ssh to the slave directly; I just did:
ssh slave01.ci.centos.org ... mkdir -p rsync-test cd rsync-test mkdir foo ln -s blah foo rsync --stats -av ./ sig-atomic@artifacts.ci.centos.org::sig-atomic/fahc/rsync-test/
And I notice rsync says: Number of files: 4 (dir: 3, link: 1)
But what resulted: https://ci.centos.org/artifacts/sig-atomic/fahc/rsync-test/rsync-test/foo
So this must be something related to the server right?
Temporary downgraded rsync to previous version on artifacts node so can you verify that it now works fine ? If so, that means having someone with enough time investigating why it fails with rsync 3.1.2
On Mon, Aug 27, 2018, at 4:49 AM, Fabian Arrotin wrote:
Temporary downgraded rsync to previous version on artifacts node so can you verify that it now works fine ?
Thanks, I can confirm it works now: https://ci.centos.org/artifacts/sig-atomic/fahc/rdgo/build/
If so, that means having someone with enough time investigating why it fails with rsync 3.1.2
Both client and server are 3.1.2 right? Let's file a RH bugzilla and add in the rsyncd config?
-
Colin Walters -
Fabian Arrotin -
Jonathan Lebon