Hi, we'd like to migrate some of our workloads into Kubernetes jobs; see for example: https://github.com/projectatomic/papr/pull/70/commits/bdaabc975b6770e2c6826a...
What are the available resources in apps.ci versus Duffy?
A lot of our jobs want basically a "classic Docker" environment with e.g. uid 0, but not CAP_SYS_ADMIN. And with seccomp disabled, etc. I was trying to create the test pod below, but it fails. It looks like our accounts use the default SCC. Can we lift these restrictions?
BTW, I'd also like oci-kvm-hook installed, with this patch: https://github.com/stefwalter/oci-kvm-hook/pull/4
apiVersion: v1 kind: DeploymentConfig metadata: labels: run: cgwalters-shell name: cgwalters-shell spec: replicas: 1 selector: run: cgwalters-shell strategy: resources: {} template: metadata: labels: run: cgwalters-shell spec: containers: - args: - sleep - 24h image: registry.fedoraproject.org/fedora:27 name: cgwalters-shell # Run as uid 0 securityContext: runAsUser: 0
On Jan 31 15:51, Colin Walters wrote:
Hi, we'd like to migrate some of our workloads into Kubernetes jobs; see for example: https://github.com/projectatomic/papr/pull/70/commits/bdaabc975b6770e2c6826a...
What are the available resources in apps.ci versus Duffy?
A lot of our jobs want basically a "classic Docker" environment with e.g. uid 0, but not CAP_SYS_ADMIN. And with seccomp disabled, etc. I was trying to create the test pod below, but it fails. It looks like our accounts use the default SCC. Can we lift these restrictions?
BTW, I'd also like oci-kvm-hook installed, with this patch: https://github.com/stefwalter/oci-kvm-hook/pull/4
apiVersion: v1 kind: DeploymentConfig metadata: labels: run: cgwalters-shell name: cgwalters-shell spec: replicas: 1 selector: run: cgwalters-shell strategy: resources: {} template: metadata: labels: run: cgwalters-shell spec: containers: - args: - sleep - 24h image: registry.fedoraproject.org/fedora:27 name: cgwalters-shell # Run as uid 0 securityContext: runAsUser: 0 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
We have separate SCCs per-namespace for this. I'll see if I can get a proper one on your project.
As far as the oci-kvm-hook thing goes, do we know a timeline for getting that merged?
--Brian
On Thu, Feb 1, 2018, at 8:43 AM, Brian Stinson wrote:
We have separate SCCs per-namespace for this. I'll see if I can get a proper one on your project.
Anything we need to do to request this?
As far as the oci-kvm-hook thing goes, do we know a timeline for getting that merged?
It's merged in master, I also built it for C7 (assuming that's the underlying host here): https://copr.fedorainfracloud.org/coprs/walters/oci-kvm-hook/
Is there any more information about what's the infrastructure backing apps.ci? For example, are any PVs available? (This is a general OpenShift issue in that AFAIK there's no standard way for users to get a sense of backing resources; enumerating PVs is an admin permission by default) (I tried creating a 1GB test claim and it doesn't seem to be fullfilling)
Also, are things in apps.ci able to talk to Duffy?
Just following up on this thread here. We're getting really close to ready to migrate workloads to OCP.
(1) How can we request a per-namespace SCC with RunAsAny?
(2) Can we get oci-kvm-hook installed on the underlying nodes? I see Stef has now created a build for EPEL:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1058034
(3) I noticed there are no resource quotas on the namespace, though just to get a sense of what's available, are the nodes underneath sufficiently powerful to have, say, 10 running pods each with resources.requests.memory=4G?
(4) Are there any plans to upgrade the cluster to v3.7 anytime soon?
Thanks!
On Apr 05 10:05, Jonathan Lebon wrote:
Just following up on this thread here. We're getting really close to ready to migrate workloads to OCP.
(1) How can we request a per-namespace SCC with RunAsAny?
Consider it requested, but let's do a bug in bugs.centos.org for tracking purposes.
(2) Can we get oci-kvm-hook installed on the underlying nodes? I see Stef has now created a build for EPEL:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1058034
oci-kvm-hook is already installed on some of the nodes. Currently we're using the node label "oci_kvm_hook=allowed" for scheduling purposes.
(3) I noticed there are no resource quotas on the namespace, though just to get a sense of what's available, are the nodes underneath sufficiently powerful to have, say, 10 running pods each with resources.requests.memory=4G?
We can certainly have capacity for this by early next week.
(4) Are there any plans to upgrade the cluster to v3.7 anytime soon?
Our current plan is to skip 3.7 and do a full reinstall on 3.9 in the coming weeks. We'll reach out here to the ci-users list for folks willing to try a trial migration in a staging environment before we make the switch.
--Brian
On 06/04/18 01:06, Brian Stinson wrote:
(4) Are there any plans to upgrade the cluster to v3.7 anytime soon?
Our current plan is to skip 3.7 and do a full reinstall on 3.9 in the coming weeks. We'll reach out here to the ci-users list for folks willing to try a trial migration in a staging environment before we make the switch.
need to co-ord this with the PaaS SIG, at the moment there is no 3.9 - and the 3.7 is still at the 3.7 - the 3.7.2 isnt released yet.
Maybe a good topic to bring to the next paas sig meeting
On Thu, Apr 5, 2018 at 8:06 PM, Brian Stinson brian@bstinson.com wrote:
Consider it requested, but let's do a bug in bugs.centos.org for tracking purposes.
Bug created here: https://bugs.centos.org/view.php?id=14666
Thanks!
Have you tried kubevirt, Colin?
https://kubevirt.gitbooks.io/user-guide/installation/
It does require origin 3.9 or kube 1.9, but it's pretty nice for running vms under kube/openshift.
On Wed, Jan 31, 2018 at 12:51 PM, Colin Walters walters@verbum.org wrote:
Hi, we'd like to migrate some of our workloads into Kubernetes jobs; see for example: https://github.com/projectatomic/papr/pull/70/commits/bdaabc975b6770e2c6826a...
What are the available resources in apps.ci versus Duffy?
A lot of our jobs want basically a "classic Docker" environment with e.g. uid 0, but not CAP_SYS_ADMIN. And with seccomp disabled, etc. I was trying to create the test pod below, but it fails. It looks like our accounts use the default SCC. Can we lift these restrictions?
BTW, I'd also like oci-kvm-hook installed, with this patch: https://github.com/stefwalter/oci-kvm-hook/pull/4
apiVersion: v1 kind: DeploymentConfig metadata: labels: run: cgwalters-shell name: cgwalters-shell spec: replicas: 1 selector: run: cgwalters-shell strategy: resources: {} template: metadata: labels: run: cgwalters-shell spec: containers: - args: - sleep - 24h image: registry.fedoraproject.org/fedora:27 name: cgwalters-shell # Run as uid 0 securityContext: runAsUser: 0 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
3.9 was built https://cbs.centos.org/koji/taskinfo?taskID=380690
Troy can you get this into testing?
-== @ri ==-
On Thu, Apr 5, 2018 at 9:08 PM, Jason Brooks jbrooks@redhat.com wrote:
Have you tried kubevirt, Colin?
https://kubevirt.gitbooks.io/user-guide/installation/
It does require origin 3.9 or kube 1.9, but it's pretty nice for running vms under kube/openshift.
On Wed, Jan 31, 2018 at 12:51 PM, Colin Walters walters@verbum.org wrote:
Hi, we'd like to migrate some of our workloads into Kubernetes jobs; see for example: https://github.com/projectatomic/papr/pull/70/commits/
bdaabc975b6770e2c6826aa259cfd2c7fddd0b9e
What are the available resources in apps.ci versus Duffy?
A lot of our jobs want basically a "classic Docker" environment with e.g. uid 0, but not CAP_SYS_ADMIN. And with seccomp
disabled, etc.
I was trying to create the test pod below, but it fails. It looks like
our accounts
use the default SCC. Can we lift these restrictions?
BTW, I'd also like oci-kvm-hook installed, with this patch:
https://github.com/stefwalter/oci-kvm-hook/pull/4
apiVersion: v1 kind: DeploymentConfig metadata: labels: run: cgwalters-shell name: cgwalters-shell spec: replicas: 1 selector: run: cgwalters-shell strategy: resources: {} template: metadata: labels: run: cgwalters-shell spec: containers: - args: - sleep - 24h image: registry.fedoraproject.org/fedora:27 name: cgwalters-shell # Run as uid 0 securityContext: runAsUser: 0 _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
On Thu, Apr 5, 2018, at 9:08 PM, Jason Brooks wrote:
Have you tried kubevirt, Colin?
See: https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2017-Marc...
-
Ari LiVigni -
Brian Stinson -
Colin Walters -
Jason Brooks -
Jonathan Lebon -
Karanbir Singh