[Ci-users] Duffy API sessions mismatch (not all the time)

Wed Aug 3 15:05:30 UTC 2022
Fabian Arrotin <arrfab at centos.org>

On 03/08/2022 16:23, Fabian Arrotin wrote:
> hi,
> 
> We switched last Monday to new Duffy API and while we saw machines being 
> requested (previous seamicro pool but also now VMs from AWS/EC2) and 
> returned, from time to time tenants informed us of transient errors.
> 
> Based on some troubleshooting, it seems that Duffy api was answering in 
> the same second to either different tenant with same nodes (so nodes 
> being handed over to different tenants) or even same tenant but with 
> different session IDs (but same hostname)
> 
> Nils (Duffy code author) is busy today looking at a fix and we'll let 
> you know when we'll be able to roll it out.
> 
> PS : that means that we'll have also to stop Duffy API and proceed with 
> some DB clean-up operations to restart from a clean/fresh situation. 
> That will mean that duffy will consider deployed node as unused and so 
> will start reinstalling these (to start from clean situation). We'll let 
> you know when we'll proceed with that hotfix push
> 

Follow-up : Nils released a newer minor version 
(https://pypi.org/project/duffy/3.3.2/) that should have needed fix.

I just rolled it out and restarted the service (ansible did it) while 
all existing deployed/sessions were cleared out from DB (to restart from 
a clean state)

Hope that it fixes the issue some of you were seeing and sorry for the 
troubles.

-- 
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | twitter: @arrfab
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xA25DBAFB17F3B7A1.asc
Type: application/pgp-keys
Size: 12767 bytes
Desc: OpenPGP public key
URL: <http://lists.centos.org/pipermail/ci-users/attachments/20220803/128cb5fd/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.centos.org/pipermail/ci-users/attachments/20220803/128cb5fd/attachment-0003.sig>