Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
On Mar 22 14:57, Ari LiVigni wrote:
I think one problem with artifacts.ci (as it currently exists) is that rsync is the only supported protocol. This makes managing files in the project spaces a little tricky.
I'd like to hear from you and the other projects there: If we were to expand the methods of accessing artifacts.ci from the duffy nodes and the workspaces, what would you like to see?
I can't make any promises at this point about what to support, but let's get some discussion going.
Cheers! --Brian
On Wed, 2017-03-22 at 14:04 -0500, Brian Stinson wrote:
Hi Brian,
nfs or gluster mount would be much better for publishing ostree images. rsync is great for duplicating but if we have to pull/update/push to make changes it will be prone to race conditions and just messy. :-)
Thanks, Bill
+1 thanks Bill!
On Wed, Mar 22, 2017 at 4:00 PM, Bill Peck bpeck@redhat.com wrote:
On 22/03/17 20:00, Bill Peck wrote:
rsync will almost always win if you need a compare and push; nfs is better if the app can use metadata ( are you going to use ostree tooling to push the repo content ? ).
the actual implementation model should also take into consideration where on the network and what part of the process you are going to hit the remote storage node. eg. you want the job worker nodes to push/compare directly, and not via the slave node.
Are there any estimates on what the data churn will look like ?
On 22/03/17 21:00, Bill Peck wrote:
NFS can be tricky as we can't restrict who can write where, as the only way to do that would be to have kerberos inside CI, which we haven't. And because some projects are pushing files directly from the duffy nodes (and so unpredictable) and not from slaves (and even for this, some projects are sharing the same VM, so same ip) we can't apply any IP ACL either
Either we can use - sshfs (reusing the pub key already known by duffy/CI env) - cifs (and maybe creating the same user/pass as credentials as the one already created for rsyncd on the artifacts node)
Just wild ideas :-)
On 03/22/2017 04:00 PM, Bill Peck wrote:
The ostree folks have some rel-eng scripts available which may help sync'ing content to a location.
I think this 'rsync-repos' script would be most helpful:
https://github.com/ostreedev/ostree-releng-scripts/blob/master/rsync-repos
On Mon, 2017-03-27 at 09:38 -0400, Micah Abbott wrote:
Thanks for the pointers everyone. My next question is what is the procedure for setting up this storage? It looks like the sig-atomic scripts are using the following:
sig-atomic@artifacts.ci.centos.org::sig-atomic/
And it appears to use the duffy password as the rsync password. So how do I go about getting this for our user which is fedora-atomic?
Thanks!
-
Ari LiVigni -
Bill Peck -
Brian Stinson -
Fabian Arrotin -
Karanbir Singh -
Micah Abbott