Hello,
I just noticed that many of the recent builds from our Jenkins slave[0] were unexpectedly removed, even though they shouldn't be, according to the retention policy (which is set to keep builds for 90 days in all affected jobs). As it's not the first time this happened I'm curious if there's some other retention policy we should be aware of, as this is kind of annoying - the missing jobs are a few days old, some of them failed, but apart from not being able to check the logs, we can't even reschedule them, because they don't exist anymore.
Note: the retention policy is currently set unnecessarily high - that's a remnant from the last time this issue occurred, as I thought Jenkins is simply miscalculating the retention policy configuration. Cutting it to 45 or even 30 days should be an issue to conserve space on the master.
PS: sorry for the double post, Vipul, I accidentally dropped the ML from To:
Thank you, Frantisek
[0] systemd-ci-slave01
When /var/lib/jenkins gets full (or about to), I leave just the last 20 builds and prune rest to clear up some space Since we advise to copy important things (like builds to artifacts), I didn't know they were important (leaving 20 to go back and address some issues) In case you don't want me messing with them, please let me know. Apologies for those.
On Fri, Feb 7, 2020 at 10:04 PM František Šumšal frantisek@sumsal.cz wrote:
Hello,
I just noticed that many of the recent builds from our Jenkins slave[0] were unexpectedly removed, even though they shouldn't be, according to the retention policy (which is set to keep builds for 90 days in all affected jobs). As it's not the first time this happened I'm curious if there's some other retention policy we should be aware of, as this is kind of annoying - the missing jobs are a few days old, some of them failed, but apart from not being able to check the logs, we can't even reschedule them, because they don't exist anymore.
Note: the retention policy is currently set unnecessarily high - that's a remnant from the last time this issue occurred, as I thought Jenkins is simply miscalculating the retention policy configuration. Cutting it to 45 or even 30 days should be an issue to conserve space on the master.
PS: sorry for the double post, Vipul, I accidentally dropped the ML from To:
Thank you, Frantisek
[0] systemd-ci-slave01
-- PGP Key ID: 0xFB738CE27B634E4B
I see, thanks a lot for the explanation!
I guess the issue here is that in systemd we usually generate several dozens of builds per day, so leaving just last 20 cuts into the 'recent history'. However, I absolutely get the space issue, so I'll reconfigure all jobs to keep the artifacts for 30 days, and the jobs themselves for 60 days (and they shouldn't take much space and would help us to reschedule them if they're needed again).
Anyway, if the need arises and a cleanup is necessary, could you remove just the artifacts? It would leave us with the option of being able to easily reschedule the job (as doing that manually is next to impossible due to sheer amount of variables the GitHub plugin sets).
Thanks!
On 2/7/20 5:43 PM, Vipul Siddharth wrote:
When /var/lib/jenkins gets full (or about to), I leave just the last 20 builds and prune rest to clear up some space Since we advise to copy important things (like builds to artifacts), I didn't know they were important (leaving 20 to go back and address some issues) In case you don't want me messing with them, please let me know. Apologies for those.
On Fri, Feb 7, 2020 at 10:04 PM František Šumšal frantisek@sumsal.cz wrote:
Hello,
I just noticed that many of the recent builds from our Jenkins slave[0] were unexpectedly removed, even though they shouldn't be, according to the retention policy (which is set to keep builds for 90 days in all affected jobs). As it's not the first time this happened I'm curious if there's some other retention policy we should be aware of, as this is kind of annoying - the missing jobs are a few days old, some of them failed, but apart from not being able to check the logs, we can't even reschedule them, because they don't exist anymore.
Note: the retention policy is currently set unnecessarily high - that's a remnant from the last time this issue occurred, as I thought Jenkins is simply miscalculating the retention policy configuration. Cutting it to 45 or even 30 days should be an issue to conserve space on the master.
PS: sorry for the double post, Vipul, I accidentally dropped the ML from To:
Thank you, Frantisek
[0] systemd-ci-slave01
-- PGP Key ID: 0xFB738CE27B634E4B
Sure, thank you for understanding :)
On Fri, Feb 7, 2020 at 10:46 PM František Šumšal frantisek@sumsal.cz wrote:
I see, thanks a lot for the explanation!
I guess the issue here is that in systemd we usually generate several dozens of builds per day, so leaving just last 20 cuts into the 'recent history'. However, I absolutely get the space issue, so I'll reconfigure all jobs to keep the artifacts for 30 days, and the jobs themselves for 60 days (and they shouldn't take much space and would help us to reschedule them if they're needed again).
Anyway, if the need arises and a cleanup is necessary, could you remove just the artifacts? It would leave us with the option of being able to easily reschedule the job (as doing that manually is next to impossible due to sheer amount of variables the GitHub plugin sets).
Thanks!
On 2/7/20 5:43 PM, Vipul Siddharth wrote:
When /var/lib/jenkins gets full (or about to), I leave just the last 20 builds and prune rest to clear up some space Since we advise to copy important things (like builds to artifacts), I didn't know they were important (leaving 20 to go back and address some issues) In case you don't want me messing with them, please let me know. Apologies for those.
On Fri, Feb 7, 2020 at 10:04 PM František Šumšal frantisek@sumsal.cz wrote:
Hello,
I just noticed that many of the recent builds from our Jenkins slave[0] were unexpectedly removed, even though they shouldn't be, according to the retention policy (which is set to keep builds for 90 days in all affected jobs). As it's not the first time this happened I'm curious if there's some other retention policy we should be aware of, as this is kind of annoying - the missing jobs are a few days old, some of them failed, but apart from not being able to check the logs, we can't even reschedule them, because they don't exist anymore.
Note: the retention policy is currently set unnecessarily high - that's a remnant from the last time this issue occurred, as I thought Jenkins is simply miscalculating the retention policy configuration. Cutting it to 45 or even 30 days should be an issue to conserve space on the master.
PS: sorry for the double post, Vipul, I accidentally dropped the ML from To:
Thank you, Frantisek
[0] systemd-ci-slave01
-- PGP Key ID: 0xFB738CE27B634E4B
-- PGP Key ID: 0xFB738CE27B634E4B
-
František Šumšal -
Vipul Siddharth