[Ci-users] Disabling dnf execution in background on an EC2 bare metal node

Mon Nov 28 07:39:05 UTC 2022
Fabian Arrotin <arrfab at centos.org>

On 28/11/2022 07:52, Anoop C S wrote:
> On Tue, 2022-09-13 at 11:22 +0200, Fabian Arrotin wrote:
>> On 13/09/2022 10:07, Anoop C S wrote:
>>>
>>> Therefore I put forward a suggestion to disable `dnf update` as
>>> part of
>>> cloud-init service such that it does not interfere with other DNF
>>> operations done after the node is reserved by a tenant.
>>>
>>> Please feel free to correct me in any of the details mentioned
>>> above
>>> and let me know your thoughts.
>>>
>>>
>>> Thanks,
>>> Anoop C S.
>>
>> (Follow-up on the thread started on #centos-ci irc channel)
>>
>> As we initially wanted to hand over centos nodes that would be
>> "up2date"
>> wrt rpm packages, we indeed added the "dnf update -y" operation in
>> our
>> ansible deployments through the ec2 user data field, so that cloud-
>> init,
>> on ec2 init , would update automatically the machine.
>> If we have enough machines in the pool, it should be "transparent"
>> for
>> users but I see that tenants start to default to the metal instance
>> (and
>> clearly we already warned against it as tenants should start using
>> the
>> classic ec2 instances - large enough but different thread), so while
>> these ec2 nodes are entering the duffy pool, they are 'given' to
>> tenants
>> while cloud-init is still updating these.
>>
>> I don't mind disabling completely that "dnf update" step from our ec2
>> config and so each project/tenant would start with such operation in
>> their ci/test workflow/pipeline
>>
>> Waiting for feedback from other tenants/projects and if so, it's just
>> a
>> git commit && git push operation at centos infra side and it will be
>> reflected for newly deployed ec2 nodes (for all, so not only bare-
>> metal
>> ones)
> 
> I would like to hear from other tenants for any objections around the
> above suggestion. Many of our jobs performing `dnf update` or `dnf
> install` started failing in a more frequent manner which feels bad
> looking at their respective statuses.
> 
> If I don't get to hear any objections my recommendation would be to
> disable `dnf update` from cloud-init on EC2 instances.
> 
> Thanks,
> Anoop C S.

Hi,

As nobody said anything about it, I just pushed the change to remove the 
"dnf update -y" part from the cloud-init configuration.
Ideally we'd let it there but if all tenants are starting themselves all 
their jobs with such dnf transaction, it's all good . Ideally we'd have 
refreshed centos 8s ami on regular basis and updating the ami id is just 
a  git commit away for us (and Duffy then provisioning these new ones 
instead)

-- 
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | twitter: @arrfab

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.centos.org/pipermail/ci-users/attachments/20221128/b8f6d651/attachment-0002.sig>