Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
On Mar 22 14:57, Ari LiVigni wrote:
Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
-- -== @ri ==- My PGP fingerprint is F87F1EE7CD8BEE13
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
I think one problem with artifacts.ci (as it currently exists) is that rsync is the only supported protocol. This makes managing files in the project spaces a little tricky.
I'd like to hear from you and the other projects there: If we were to expand the methods of accessing artifacts.ci from the duffy nodes and the workspaces, what would you like to see?
I can't make any promises at this point about what to support, but let's get some discussion going.
Cheers! --Brian
On Wed, 2017-03-22 at 14:04 -0500, Brian Stinson wrote:
On Mar 22 14:57, Ari LiVigni wrote:
Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
-- -== @ri ==- My PGP fingerprint is F87F1EE7CD8BEE13 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
I think one problem with artifacts.ci (as it currently exists) is that rsync is the only supported protocol. This makes managing files in the project spaces a little tricky.
I'd like to hear from you and the other projects there: If we were to expand the methods of accessing artifacts.ci from the duffy nodes and the workspaces, what would you like to see?
I can't make any promises at this point about what to support, but let's get some discussion going.
Cheers! --Brian
Hi Brian,
nfs or gluster mount would be much better for publishing ostree images. rsync is great for duplicating but if we have to pull/update/push to make changes it will be prone to race conditions and just messy. :-)
Thanks, Bill
+1 thanks Bill!
On Wed, Mar 22, 2017 at 4:00 PM, Bill Peck bpeck@redhat.com wrote:
On Wed, 2017-03-22 at 14:04 -0500, Brian Stinson wrote:
On Mar 22 14:57, Ari LiVigni wrote:
Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
-- -== @ri ==- My PGP fingerprint is F87F1EE7CD8BEE13 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
I think one problem with artifacts.ci (as it currently exists) is that rsync is the only supported protocol. This makes managing files in the project spaces a little tricky.
I'd like to hear from you and the other projects there: If we were to expand the methods of accessing artifacts.ci from the duffy nodes and the workspaces, what would you like to see?
I can't make any promises at this point about what to support, but let's get some discussion going.
Cheers! --Brian
Hi Brian,
nfs or gluster mount would be much better for publishing ostree images. rsync is great for duplicating but if we have to pull/update/push to make changes it will be prone to race conditions and just messy. :-)
Thanks, Bill
On 22/03/17 20:00, Bill Peck wrote:
nfs or gluster mount would be much better for publishing ostree images. rsync is great for duplicating but if we have to pull/update/push to make changes it will be prone to race conditions and just messy. :-)
rsync will almost always win if you need a compare and push; nfs is better if the app can use metadata ( are you going to use ostree tooling to push the repo content ? ).
the actual implementation model should also take into consideration where on the network and what part of the process you are going to hit the remote storage node. eg. you want the job worker nodes to push/compare directly, and not via the slave node.
Are there any estimates on what the data churn will look like ?
On 22/03/17 21:00, Bill Peck wrote:
On Wed, 2017-03-22 at 14:04 -0500, Brian Stinson wrote:
On Mar 22 14:57, Ari LiVigni wrote:
Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
-- -== @ri ==- My PGP fingerprint is F87F1EE7CD8BEE13 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
I think one problem with artifacts.ci (as it currently exists) is that rsync is the only supported protocol. This makes managing files in the project spaces a little tricky.
I'd like to hear from you and the other projects there: If we were to expand the methods of accessing artifacts.ci from the duffy nodes and the workspaces, what would you like to see?
I can't make any promises at this point about what to support, but let's get some discussion going.
Cheers! --Brian
Hi Brian,
nfs or gluster mount would be much better for publishing ostree images. rsync is great for duplicating but if we have to pull/update/push to make changes it will be prone to race conditions and just messy. :-)
Thanks, Bill
NFS can be tricky as we can't restrict who can write where, as the only way to do that would be to have kerberos inside CI, which we haven't. And because some projects are pushing files directly from the duffy nodes (and so unpredictable) and not from slaves (and even for this, some projects are sharing the same VM, so same ip) we can't apply any IP ACL either
Either we can use - sshfs (reusing the pub key already known by duffy/CI env) - cifs (and maybe creating the same user/pass as credentials as the one already created for rsyncd on the artifacts node)
Just wild ideas :-)
On 03/22/2017 04:00 PM, Bill Peck wrote:
On Wed, 2017-03-22 at 14:04 -0500, Brian Stinson wrote:
On Mar 22 14:57, Ari LiVigni wrote:
Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
-- -== @ri ==- My PGP fingerprint is F87F1EE7CD8BEE13 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
I think one problem with artifacts.ci (as it currently exists) is that rsync is the only supported protocol. This makes managing files in the project spaces a little tricky.
I'd like to hear from you and the other projects there: If we were to expand the methods of accessing artifacts.ci from the duffy nodes and the workspaces, what would you like to see?
I can't make any promises at this point about what to support, but let's get some discussion going.
Cheers! --Brian
Hi Brian,
nfs or gluster mount would be much better for publishing ostree images. rsync is great for duplicating but if we have to pull/update/push to make changes it will be prone to race conditions and just messy. :-)
The ostree folks have some rel-eng scripts available which may help sync'ing content to a location.
I think this 'rsync-repos' script would be most helpful:
https://github.com/ostreedev/ostree-releng-scripts/blob/master/rsync-repos
Thanks, Bill _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
On Mon, 2017-03-27 at 09:38 -0400, Micah Abbott wrote:
On 03/22/2017 04:00 PM, Bill Peck wrote:
On Wed, 2017-03-22 at 14:04 -0500, Brian Stinson wrote:
On Mar 22 14:57, Ari LiVigni wrote:
Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
-- -== @ri ==- My PGP fingerprint is F87F1EE7CD8BEE13 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
I think one problem with artifacts.ci (as it currently exists) is that rsync is the only supported protocol. This makes managing files in the project spaces a little tricky.
I'd like to hear from you and the other projects there: If we were to expand the methods of accessing artifacts.ci from the duffy nodes and the workspaces, what would you like to see?
I can't make any promises at this point about what to support, but let's get some discussion going.
Cheers! --Brian
Hi Brian,
nfs or gluster mount would be much better for publishing ostree images. rsync is great for duplicating but if we have to pull/update/push to make changes it will be prone to race conditions and just messy. :- )
The ostree folks have some rel-eng scripts available which may help sync'ing content to a location.
I think this 'rsync-repos' script would be most helpful:
https://github.com/ostreedev/ostree-releng-scripts/blob/master/rsync- repos
Thanks for the pointers everyone. My next question is what is the procedure for setting up this storage? It looks like the sig-atomic scripts are using the following:
sig-atomic@artifacts.ci.centos.org::sig-atomic/
And it appears to use the duffy password as the rsync password. So how do I go about getting this for our user which is fedora-atomic?
Thanks!
Thanks, Bill _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
On Apr 10 13:46, Bill Peck wrote:
On Mon, 2017-03-27 at 09:38 -0400, Micah Abbott wrote:
On 03/22/2017 04:00 PM, Bill Peck wrote:
On Wed, 2017-03-22 at 14:04 -0500, Brian Stinson wrote:
On Mar 22 14:57, Ari LiVigni wrote:
Is artifacts.ci a good place to store ostree and image artifacts out of CentOS CI? If so what are the steps to hook that up?
Thanks,
-- -== @ri ==- My PGP fingerprint is F87F1EE7CD8BEE13 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
I think one problem with artifacts.ci (as it currently exists) is that rsync is the only supported protocol. This makes managing files in the project spaces a little tricky.
I'd like to hear from you and the other projects there: If we were to expand the methods of accessing artifacts.ci from the duffy nodes and the workspaces, what would you like to see?
I can't make any promises at this point about what to support, but let's get some discussion going.
Cheers! --Brian
Hi Brian,
nfs or gluster mount would be much better for publishing ostree images. rsync is great for duplicating but if we have to pull/update/push to make changes it will be prone to race conditions and just messy. :- )
The ostree folks have some rel-eng scripts available which may help sync'ing content to a location.
I think this 'rsync-repos' script would be most helpful:
https://github.com/ostreedev/ostree-releng-scripts/blob/master/rsync- repos
Thanks for the pointers everyone. My next question is what is the procedure for setting up this storage? It looks like the sig-atomic scripts are using the following:
sig-atomic@artifacts.ci.centos.org::sig-atomic/
And it appears to use the duffy password as the rsync password. So how do I go about getting this for our user which is fedora-atomic?
Thanks!
Thanks, Bill _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
This directory should be in place. If you have trouble let us know.
Cheers!
--Brian
-
Ari LiVigni -
Bill Peck -
Brian Stinson -
Fabian Arrotin -
Karanbir Singh -
Micah Abbott