[Ci-users] Disabling dnf execution in background on an EC2 bare metal node

Mon Nov 28 06:52:12 UTC 2022
Anoop C S <anoopcs at disr.it>

On Tue, 2022-09-13 at 11:22 +0200, Fabian Arrotin wrote:
> On 13/09/2022 10:07, Anoop C S wrote:
> > 
> > Therefore I put forward a suggestion to disable `dnf update` as
> > part of
> > cloud-init service such that it does not interfere with other DNF
> > operations done after the node is reserved by a tenant.
> > 
> > Please feel free to correct me in any of the details mentioned
> > above
> > and let me know your thoughts.
> > 
> > 
> > Thanks,
> > Anoop C S.
> 
> (Follow-up on the thread started on #centos-ci irc channel)
> 
> As we initially wanted to hand over centos nodes that would be
> "up2date" 
> wrt rpm packages, we indeed added the "dnf update -y" operation in
> our 
> ansible deployments through the ec2 user data field, so that cloud-
> init, 
> on ec2 init , would update automatically the machine.
> If we have enough machines in the pool, it should be "transparent"
> for 
> users but I see that tenants start to default to the metal instance
> (and 
> clearly we already warned against it as tenants should start using
> the 
> classic ec2 instances - large enough but different thread), so while 
> these ec2 nodes are entering the duffy pool, they are 'given' to
> tenants 
> while cloud-init is still updating these.
> 
> I don't mind disabling completely that "dnf update" step from our ec2
> config and so each project/tenant would start with such operation in 
> their ci/test workflow/pipeline
> 
> Waiting for feedback from other tenants/projects and if so, it's just
> a 
> git commit && git push operation at centos infra side and it will be 
> reflected for newly deployed ec2 nodes (for all, so not only bare-
> metal 
> ones)

I would like to hear from other tenants for any objections around the
above suggestion. Many of our jobs performing `dnf update` or `dnf
install` started failing in a more frequent manner which feels bad
looking at their respective statuses.

If I don't get to hear any objections my recommendation would be to
disable `dnf update` from cloud-init on EC2 instances.

Thanks,
Anoop C S.