Hello all,
My name is Zach, and I'm a member of the Foreman team. We use ci.centos.org to run some of our testing. About 30 minutes ago I was setting up a job to maintain the jobs that we use on ci.centos.org and I copied over a command that we use on ci.theforeman.org which will update jobs and delete old jobs. We use this with ci.theforeman.org as we maintain all of our jobs in the same repository, also not realizing that the foreman user would have permissions to delete other jobs. I realized a bit into the job that it was not doing this, and instead was deleting other users jobs. I stopped the job, but it had already deleted 128 jobs. I sincerely apologize for this, it was not intentional, but a big mistake. I'm attaching a list of all the deleted jobs so that users can be aware. We will be removing this command and not running this again.
Once again I sincerely apologize, for any undue stress or harm this may have caused.
Thanks,
Hello Zach,
mistakes happen - I would assume that the deleted jobs can be restored from backups, so it shouldn't be that dramatic. Not sure if that means a regular user can delete jobs belonging to other users (I hope Brian or someone else can bring some light into this).
Best regards, Laurențiu
On 04/09/18 19:28, Zach Huntington Meath wrote:
Hello all,
My name is Zach, and I'm a member of the Foreman team. We use ci.centos.org to run some of our testing. About 30 minutes ago I was setting up a job to maintain the jobs that we use on ci.centos.org and I copied over a command that we use on ci.theforeman.org which will update jobs and delete old jobs. We use this with ci.theforeman.org as we maintain all of our jobs in the same repository, also not realizing that the foreman user would have permissions to delete other jobs. I realized a bit into the job that it was not doing this, and instead was deleting other users jobs. I stopped the job, but it had already deleted 128 jobs. I sincerely apologize for this, it was not intentional, but a big mistake. I'm attaching a list of all the deleted jobs so that users can be aware. We will be removing this command and not running this again.
Once again I sincerely apologize, for any undue stress or harm this may have caused.
Thanks,
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
On Sep 04 13:28, Zach Huntington Meath wrote:
Hello all,
My name is Zach, and I'm a member of the Foreman team. We use ci.centos.org to run some of our testing. About 30 minutes ago I was setting up a job to maintain the jobs that we use on ci.centos.org and I copied over a command that we use on ci.theforeman.org which will update jobs and delete old jobs. We use this with ci.theforeman.org as we maintain all of our jobs in the same repository, also not realizing that the foreman user would have permissions to delete other jobs. I realized a bit into the job that it was not doing this, and instead was deleting other users jobs. I stopped the job, but it had already deleted 128 jobs. I sincerely apologize for this, it was not intentional, but a big mistake. I'm attaching a list of all the deleted jobs so that users can be aware. We will be removing this command and not running this again.
Once again I sincerely apologize, for any undue stress or harm this may have caused.
Thanks,
-- Zachary Huntington-Meath Red Hat Engineering (919)-619-4189 irc: zhunting
atomic-cahc-image-cloud-continuous atomic-cahc-image-cloud-smoketested atomic-cahc-installer-continuous atomic-cahc-installer-smoketested atomic-duffy-allocate atomic-duffy-gc atomic-rdgo-centos7 atomic-tree-smoketest-centos7 atomic-treecompose-centos7 atomicapp-test-docker-master atomicapp-test-docker-pr atomicapp-test-kubernetes-master atomicapp-test-kubernetes-pr atomicapp-test-openshift-master atomicapp-test-openshift-pr atomicapp-test-unittest-master atomicapp-test-unittest-pr atomictest-build-git-evtag-master atomictest-duffy-allocate atomictest-duffy-gc bodhi-bodhi bstinson-test-build bstinsonmhk-compose_tester centos-container-pipeline-service-ci-cleanup centos-container-pipeline-service-ci-master centos-container-pipeline-service-ci-pr centos-container-pipeline-service-ci-pr-debug centos-container-pipeline-service-ci-pr-openshift centos-container-pipeline-service-ci-pr-test centos-container-pipeline-service-ci-pr-unittests centos-container-pipeline-service-ci-return-failed-nodes centos-container-pipeline-service-container-index centos-container-pipeline-service-job centos-container-pipeline-service-pre-build-caddyserver-caddyserver-latest centos-container-pipeline-service-pre-build-centos-centos7-atomic-latest centos-container-pipeline-service-pre-build-centos-postgresql-94-centos7-latest centos-container-pipeline-service-pre-build-centos-postgresql-95-centos7-latest centos-container-pipeline-service-pre-build-centos-postgresql-96-centos7-latest centos-container-pipeline-service-pre-build-centos-varnish-4-centos7-latest centos-container-pipeline-service-pre-build-centos-varnish-5-centos7-latest centos-container-pipeline-service-pre-build-fabric8-services-fabric8-wit-latest centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-latest centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-testing centos-container-pipeline-service-pre-build-kbsingh-centos7-atomic-latest centos-container-pipeline-service-pre-build-kbsingh-container-prebuild-1 centos-container-pipeline-service-pre-build-service-job centos-container-pipeline-service-pre-build-user-cont-tools-latest ci-pipeline-atomic-host-tests ci-pipeline-duffy-allocate ci-pipeline-duffy-gc ci-pipeline-kernel-f26-ltp ci-pipeline-kernel-trigger ci-pipeline-ostree-boot-sanity ci-pipeline-ostree-compose ci-pipeline-ostree-image-boot-sanity ci-pipeline-ostree-image-compose ci-pipeline-rpmbuild cloudinstance-vagrant-test devtools-api-machine-stacks-f8a-build-master devtools-api-machine-stacks-fabric8-analytics devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-spring-boot devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-thorntail devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-vert.x devtools-chat-build-master devtools-chat-integrations-github-build-master devtools-chat-integrations-gitlab-build-master devtools-chat-integrations-irc-build-master devtools-chat-integrations-rssfeeds-build-master devtools-chat-push-proxy-build-master devtools-che-functional-tests-after-rh-che-build-prod-preview.openshift.io devtools-che-functional-tests-periodical-openshift.io-2 devtools-che-functional-tests-periodical-prod-preview.openshift.io-2a devtools-che-functional-tests-periodical-prod-preview.openshift.io-free-stg devtools-che-functional-tests-prcheck-openshift.io-2 devtools-che-functional-tests-prcheck-prod-preview.openshift.io-2a devtools-che-plugin-registry-build-master devtools-che-starter-build-che-credentials-master devtools-che-starter-prcheck devtools-che-tenant-maintainer-build-master devtools-che-vertx-server-build-master devtools-cvedb devtools-cvedb-build-master devtools-cvejob devtools-cvejob-build-master devtools-cvejob-fabric8-analytics-pydoc devtools-cvejob-fabric8-analytics-pylint devtools-e2e-fabric8-analytics devtools-eclipse-che-build-dockerfiles devtools-eclipse-che-promotion devtools-error-tracking devtools-error-tracking-build-master devtools-f8a-3scale-connect-api-f8a-build-master devtools-f8a-3scale-connect-api-fabric8-analytics devtools-f8a-hpf-insights-f8a-build-master devtools-f8a-hpf-insights-fabric8-analytics devtools-f8a-hpf-insights-fabric8-analytics-pydoc devtools-f8a-hpf-insights-fabric8-analytics-pylint devtools-f8a-master-deploy-e2e-test devtools-f8a-server-backbone-f8a-build-master devtools-f8a-server-backbone-fabric8-analytics devtools-f8a-server-backbone-fabric8-analytics-pydoc devtools-f8a-server-backbone-fabric8-analytics-pylint devtools-fabric8-admin-proxy devtools-fabric8-admin-proxy-build-master devtools-fabric8-analytics-api-gateway-f8a-build-master devtools-fabric8-analytics-api-gateway-fabric8-analytics devtools-fabric8-analytics-api-gateway-fabric8-analytics-pydoc devtools-fabric8-analytics-api-gateway-fabric8-analytics-pylint devtools-fabric8-analytics-auth-fabric8-analytics devtools-fabric8-analytics-auth-fabric8-analytics-pydoc devtools-fabric8-analytics-auth-fabric8-analytics-pylint devtools-fabric8-analytics-common-fabric8-analytics-pydoc devtools-fabric8-analytics-common-fabric8-analytics-pylint devtools-fabric8-analytics-cvedb-s3-dump-docker-f8a-build-master devtools-fabric8-analytics-cvedb-s3-dump-docker-fabric8-analytics devtools-fabric8-analytics-data-model-f8a-build-master devtools-fabric8-analytics-data-model-fabric8-analytics devtools-fabric8-analytics-data-model-fabric8-analytics-pydoc devtools-fabric8-analytics-data-model-fabric8-analytics-pylint devtools-fabric8-analytics-dependency-editor devtools-fabric8-analytics-dependency-editor-npm-publish-build-master devtools-fabric8-analytics-firehose-fetcher-f8a-build-master devtools-fabric8-analytics-firehose-fetcher-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics-pydoc devtools-fabric8-analytics-ingestion-fabric8-analytics-pylint devtools-fabric8-analytics-jenkins-plugin-fabric8-analytics devtools-fabric8-analytics-jobs-f8a-build-master
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Hi All,
Zach: Thanks for including a list here. We keep nightly backups of the jenkins master, and I was going to see about restoring these job before I included a note to the list but it's taking a bit longer to sync over and extract the job configs.
We should have these back in short order (if your service jobs haven't already caught up with some of them).
The Foreman account was one of our legacy accounts (we provisioned this project almost 3 years ago), and included more lax permissions than are necessary. Foreman folks: let's work together soon to get a service job going so you won't need to run this by hand any more, and we can drop permissions here.
Cheers!
-- Brian Stinson CentOS CI Infrastructure Team
On Sep 04 14:52, Brian Stinson wrote:
On Sep 04 13:28, Zach Huntington Meath wrote:
Hello all,
My name is Zach, and I'm a member of the Foreman team. We use ci.centos.org to run some of our testing. About 30 minutes ago I was setting up a job to maintain the jobs that we use on ci.centos.org and I copied over a command that we use on ci.theforeman.org which will update jobs and delete old jobs. We use this with ci.theforeman.org as we maintain all of our jobs in the same repository, also not realizing that the foreman user would have permissions to delete other jobs. I realized a bit into the job that it was not doing this, and instead was deleting other users jobs. I stopped the job, but it had already deleted 128 jobs. I sincerely apologize for this, it was not intentional, but a big mistake. I'm attaching a list of all the deleted jobs so that users can be aware. We will be removing this command and not running this again.
Once again I sincerely apologize, for any undue stress or harm this may have caused.
Thanks,
-- Zachary Huntington-Meath Red Hat Engineering (919)-619-4189 irc: zhunting
atomic-cahc-image-cloud-continuous atomic-cahc-image-cloud-smoketested atomic-cahc-installer-continuous atomic-cahc-installer-smoketested atomic-duffy-allocate atomic-duffy-gc atomic-rdgo-centos7 atomic-tree-smoketest-centos7 atomic-treecompose-centos7 atomicapp-test-docker-master atomicapp-test-docker-pr atomicapp-test-kubernetes-master atomicapp-test-kubernetes-pr atomicapp-test-openshift-master atomicapp-test-openshift-pr atomicapp-test-unittest-master atomicapp-test-unittest-pr atomictest-build-git-evtag-master atomictest-duffy-allocate atomictest-duffy-gc bodhi-bodhi bstinson-test-build bstinsonmhk-compose_tester centos-container-pipeline-service-ci-cleanup centos-container-pipeline-service-ci-master centos-container-pipeline-service-ci-pr centos-container-pipeline-service-ci-pr-debug centos-container-pipeline-service-ci-pr-openshift centos-container-pipeline-service-ci-pr-test centos-container-pipeline-service-ci-pr-unittests centos-container-pipeline-service-ci-return-failed-nodes centos-container-pipeline-service-container-index centos-container-pipeline-service-job centos-container-pipeline-service-pre-build-caddyserver-caddyserver-latest centos-container-pipeline-service-pre-build-centos-centos7-atomic-latest centos-container-pipeline-service-pre-build-centos-postgresql-94-centos7-latest centos-container-pipeline-service-pre-build-centos-postgresql-95-centos7-latest centos-container-pipeline-service-pre-build-centos-postgresql-96-centos7-latest centos-container-pipeline-service-pre-build-centos-varnish-4-centos7-latest centos-container-pipeline-service-pre-build-centos-varnish-5-centos7-latest centos-container-pipeline-service-pre-build-fabric8-services-fabric8-wit-latest centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-latest centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-testing centos-container-pipeline-service-pre-build-kbsingh-centos7-atomic-latest centos-container-pipeline-service-pre-build-kbsingh-container-prebuild-1 centos-container-pipeline-service-pre-build-service-job centos-container-pipeline-service-pre-build-user-cont-tools-latest ci-pipeline-atomic-host-tests ci-pipeline-duffy-allocate ci-pipeline-duffy-gc ci-pipeline-kernel-f26-ltp ci-pipeline-kernel-trigger ci-pipeline-ostree-boot-sanity ci-pipeline-ostree-compose ci-pipeline-ostree-image-boot-sanity ci-pipeline-ostree-image-compose ci-pipeline-rpmbuild cloudinstance-vagrant-test devtools-api-machine-stacks-f8a-build-master devtools-api-machine-stacks-fabric8-analytics devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-spring-boot devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-thorntail devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-vert.x devtools-chat-build-master devtools-chat-integrations-github-build-master devtools-chat-integrations-gitlab-build-master devtools-chat-integrations-irc-build-master devtools-chat-integrations-rssfeeds-build-master devtools-chat-push-proxy-build-master devtools-che-functional-tests-after-rh-che-build-prod-preview.openshift.io devtools-che-functional-tests-periodical-openshift.io-2 devtools-che-functional-tests-periodical-prod-preview.openshift.io-2a devtools-che-functional-tests-periodical-prod-preview.openshift.io-free-stg devtools-che-functional-tests-prcheck-openshift.io-2 devtools-che-functional-tests-prcheck-prod-preview.openshift.io-2a devtools-che-plugin-registry-build-master devtools-che-starter-build-che-credentials-master devtools-che-starter-prcheck devtools-che-tenant-maintainer-build-master devtools-che-vertx-server-build-master devtools-cvedb devtools-cvedb-build-master devtools-cvejob devtools-cvejob-build-master devtools-cvejob-fabric8-analytics-pydoc devtools-cvejob-fabric8-analytics-pylint devtools-e2e-fabric8-analytics devtools-eclipse-che-build-dockerfiles devtools-eclipse-che-promotion devtools-error-tracking devtools-error-tracking-build-master devtools-f8a-3scale-connect-api-f8a-build-master devtools-f8a-3scale-connect-api-fabric8-analytics devtools-f8a-hpf-insights-f8a-build-master devtools-f8a-hpf-insights-fabric8-analytics devtools-f8a-hpf-insights-fabric8-analytics-pydoc devtools-f8a-hpf-insights-fabric8-analytics-pylint devtools-f8a-master-deploy-e2e-test devtools-f8a-server-backbone-f8a-build-master devtools-f8a-server-backbone-fabric8-analytics devtools-f8a-server-backbone-fabric8-analytics-pydoc devtools-f8a-server-backbone-fabric8-analytics-pylint devtools-fabric8-admin-proxy devtools-fabric8-admin-proxy-build-master devtools-fabric8-analytics-api-gateway-f8a-build-master devtools-fabric8-analytics-api-gateway-fabric8-analytics devtools-fabric8-analytics-api-gateway-fabric8-analytics-pydoc devtools-fabric8-analytics-api-gateway-fabric8-analytics-pylint devtools-fabric8-analytics-auth-fabric8-analytics devtools-fabric8-analytics-auth-fabric8-analytics-pydoc devtools-fabric8-analytics-auth-fabric8-analytics-pylint devtools-fabric8-analytics-common-fabric8-analytics-pydoc devtools-fabric8-analytics-common-fabric8-analytics-pylint devtools-fabric8-analytics-cvedb-s3-dump-docker-f8a-build-master devtools-fabric8-analytics-cvedb-s3-dump-docker-fabric8-analytics devtools-fabric8-analytics-data-model-f8a-build-master devtools-fabric8-analytics-data-model-fabric8-analytics devtools-fabric8-analytics-data-model-fabric8-analytics-pydoc devtools-fabric8-analytics-data-model-fabric8-analytics-pylint devtools-fabric8-analytics-dependency-editor devtools-fabric8-analytics-dependency-editor-npm-publish-build-master devtools-fabric8-analytics-firehose-fetcher-f8a-build-master devtools-fabric8-analytics-firehose-fetcher-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics-pydoc devtools-fabric8-analytics-ingestion-fabric8-analytics-pylint devtools-fabric8-analytics-jenkins-plugin-fabric8-analytics devtools-fabric8-analytics-jobs-f8a-build-master
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Hi All,
Zach: Thanks for including a list here. We keep nightly backups of the jenkins master, and I was going to see about restoring these job before I included a note to the list but it's taking a bit longer to sync over and extract the job configs.
We should have these back in short order (if your service jobs haven't already caught up with some of them).
The Foreman account was one of our legacy accounts (we provisioned this project almost 3 years ago), and included more lax permissions than are necessary. Foreman folks: let's work together soon to get a service job going so you won't need to run this by hand any more, and we can drop permissions here.
Cheers!
-- Brian Stinson CentOS CI Infrastructure Team _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
These are restored to last night's backup.
--Brian
Thanks Brian, I'm glad everything was able to be recovered!
On Tue, Sep 4, 2018 at 6:32 PM Brian Stinson brian@bstinson.com wrote:
On Sep 04 14:52, Brian Stinson wrote:
On Sep 04 13:28, Zach Huntington Meath wrote:
Hello all,
My name is Zach, and I'm a member of the Foreman team. We use
ci.centos.org
to run some of our testing. About 30 minutes ago I was setting up a
job to
maintain the jobs that we use on ci.centos.org and I copied over a
command
that we use on ci.theforeman.org which will update jobs and delete old jobs. We use this with ci.theforeman.org as we maintain all of our
jobs in
the same repository, also not realizing that the foreman user would
have
permissions to delete other jobs. I realized a bit into the job that
it was
not doing this, and instead was deleting other users jobs. I stopped
the
job, but it had already deleted 128 jobs. I sincerely apologize for
this,
it was not intentional, but a big mistake. I'm attaching a list of
all the
deleted jobs so that users can be aware. We will be removing this
command
and not running this again.
Once again I sincerely apologize, for any undue stress or harm this may have caused.
Thanks,
-- Zachary Huntington-Meath Red Hat Engineering (919)-619-4189 irc: zhunting
atomic-cahc-image-cloud-continuous atomic-cahc-image-cloud-smoketested atomic-cahc-installer-continuous atomic-cahc-installer-smoketested atomic-duffy-allocate atomic-duffy-gc atomic-rdgo-centos7 atomic-tree-smoketest-centos7 atomic-treecompose-centos7 atomicapp-test-docker-master atomicapp-test-docker-pr atomicapp-test-kubernetes-master atomicapp-test-kubernetes-pr atomicapp-test-openshift-master atomicapp-test-openshift-pr atomicapp-test-unittest-master atomicapp-test-unittest-pr atomictest-build-git-evtag-master atomictest-duffy-allocate atomictest-duffy-gc bodhi-bodhi bstinson-test-build bstinsonmhk-compose_tester centos-container-pipeline-service-ci-cleanup centos-container-pipeline-service-ci-master centos-container-pipeline-service-ci-pr centos-container-pipeline-service-ci-pr-debug centos-container-pipeline-service-ci-pr-openshift centos-container-pipeline-service-ci-pr-test centos-container-pipeline-service-ci-pr-unittests centos-container-pipeline-service-ci-return-failed-nodes centos-container-pipeline-service-container-index centos-container-pipeline-service-job
centos-container-pipeline-service-pre-build-caddyserver-caddyserver-latest
centos-container-pipeline-service-pre-build-centos-centos7-atomic-latest
centos-container-pipeline-service-pre-build-centos-postgresql-94-centos7-latest
centos-container-pipeline-service-pre-build-centos-postgresql-95-centos7-latest
centos-container-pipeline-service-pre-build-centos-postgresql-96-centos7-latest
centos-container-pipeline-service-pre-build-centos-varnish-4-centos7-latest
centos-container-pipeline-service-pre-build-centos-varnish-5-centos7-latest
centos-container-pipeline-service-pre-build-fabric8-services-fabric8-wit-latest
centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-latest
centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-testing
centos-container-pipeline-service-pre-build-kbsingh-centos7-atomic-latest
centos-container-pipeline-service-pre-build-kbsingh-container-prebuild-1
centos-container-pipeline-service-pre-build-service-job centos-container-pipeline-service-pre-build-user-cont-tools-latest ci-pipeline-atomic-host-tests ci-pipeline-duffy-allocate ci-pipeline-duffy-gc ci-pipeline-kernel-f26-ltp ci-pipeline-kernel-trigger ci-pipeline-ostree-boot-sanity ci-pipeline-ostree-compose ci-pipeline-ostree-image-boot-sanity ci-pipeline-ostree-image-compose ci-pipeline-rpmbuild cloudinstance-vagrant-test devtools-api-machine-stacks-f8a-build-master devtools-api-machine-stacks-fabric8-analytics devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-spring-boot devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-thorntail devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-vert.x devtools-chat-build-master devtools-chat-integrations-github-build-master devtools-chat-integrations-gitlab-build-master devtools-chat-integrations-irc-build-master devtools-chat-integrations-rssfeeds-build-master devtools-chat-push-proxy-build-master
devtools-che-functional-tests-after-rh-che-build-prod-preview.openshift.io
devtools-che-functional-tests-periodical-openshift.io-2 devtools-che-functional-tests-periodical-prod-preview.openshift.io-2a
devtools-che-functional-tests-periodical-prod-preview.openshift.io-free-stg
devtools-che-functional-tests-prcheck-openshift.io-2 devtools-che-functional-tests-prcheck-prod-preview.openshift.io-2a devtools-che-plugin-registry-build-master devtools-che-starter-build-che-credentials-master devtools-che-starter-prcheck devtools-che-tenant-maintainer-build-master devtools-che-vertx-server-build-master devtools-cvedb devtools-cvedb-build-master devtools-cvejob devtools-cvejob-build-master devtools-cvejob-fabric8-analytics-pydoc devtools-cvejob-fabric8-analytics-pylint devtools-e2e-fabric8-analytics devtools-eclipse-che-build-dockerfiles devtools-eclipse-che-promotion devtools-error-tracking devtools-error-tracking-build-master devtools-f8a-3scale-connect-api-f8a-build-master devtools-f8a-3scale-connect-api-fabric8-analytics devtools-f8a-hpf-insights-f8a-build-master devtools-f8a-hpf-insights-fabric8-analytics devtools-f8a-hpf-insights-fabric8-analytics-pydoc devtools-f8a-hpf-insights-fabric8-analytics-pylint devtools-f8a-master-deploy-e2e-test devtools-f8a-server-backbone-f8a-build-master devtools-f8a-server-backbone-fabric8-analytics devtools-f8a-server-backbone-fabric8-analytics-pydoc devtools-f8a-server-backbone-fabric8-analytics-pylint devtools-fabric8-admin-proxy devtools-fabric8-admin-proxy-build-master devtools-fabric8-analytics-api-gateway-f8a-build-master devtools-fabric8-analytics-api-gateway-fabric8-analytics devtools-fabric8-analytics-api-gateway-fabric8-analytics-pydoc devtools-fabric8-analytics-api-gateway-fabric8-analytics-pylint devtools-fabric8-analytics-auth-fabric8-analytics devtools-fabric8-analytics-auth-fabric8-analytics-pydoc devtools-fabric8-analytics-auth-fabric8-analytics-pylint devtools-fabric8-analytics-common-fabric8-analytics-pydoc devtools-fabric8-analytics-common-fabric8-analytics-pylint devtools-fabric8-analytics-cvedb-s3-dump-docker-f8a-build-master devtools-fabric8-analytics-cvedb-s3-dump-docker-fabric8-analytics devtools-fabric8-analytics-data-model-f8a-build-master devtools-fabric8-analytics-data-model-fabric8-analytics devtools-fabric8-analytics-data-model-fabric8-analytics-pydoc devtools-fabric8-analytics-data-model-fabric8-analytics-pylint devtools-fabric8-analytics-dependency-editor devtools-fabric8-analytics-dependency-editor-npm-publish-build-master devtools-fabric8-analytics-firehose-fetcher-f8a-build-master devtools-fabric8-analytics-firehose-fetcher-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics-pydoc devtools-fabric8-analytics-ingestion-fabric8-analytics-pylint devtools-fabric8-analytics-jenkins-plugin-fabric8-analytics devtools-fabric8-analytics-jobs-f8a-build-master
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Hi All,
Zach: Thanks for including a list here. We keep nightly backups of the jenkins master, and I was going to see about restoring these job before I included a note to the list but it's taking a bit longer to sync over and extract the job configs.
We should have these back in short order (if your service jobs haven't already caught up with some of them).
The Foreman account was one of our legacy accounts (we provisioned this project almost 3 years ago), and included more lax permissions than are necessary. Foreman folks: let's work together soon to get a service job going so you won't need to run this by hand any more, and we can drop permissions here.
Cheers!
-- Brian Stinson CentOS CI Infrastructure Team _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
These are restored to last night's backup.
--Brian _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Hi,
following the restore we notice that some jobs are not triggering when an upstream job finishes.
For instance, job "devtools-fabric8-analytics-npm-insights-f8a-build-master" should trigger job "devtools-f8a-master-deploy-e2e-test".
Is something missing from the restore with regards to jobs that are "linked"? Would we need to re-create the jobs to restore this functionallity?
Please advise. Thanks!
-- Regards, Jean-Francois Chevrette Principal Software Engineer | Service Delivery - Hosted Services jfchevrette = [irc, github, twitter]
On Tue, Sep 4, 2018 at 6:32 PM Brian Stinson brian@bstinson.com wrote:
On Sep 04 14:52, Brian Stinson wrote:
On Sep 04 13:28, Zach Huntington Meath wrote:
Hello all,
My name is Zach, and I'm a member of the Foreman team. We use
ci.centos.org
to run some of our testing. About 30 minutes ago I was setting up a
job to
maintain the jobs that we use on ci.centos.org and I copied over a
command
that we use on ci.theforeman.org which will update jobs and delete old jobs. We use this with ci.theforeman.org as we maintain all of our
jobs in
the same repository, also not realizing that the foreman user would
have
permissions to delete other jobs. I realized a bit into the job that
it was
not doing this, and instead was deleting other users jobs. I stopped
the
job, but it had already deleted 128 jobs. I sincerely apologize for
this,
it was not intentional, but a big mistake. I'm attaching a list of
all the
deleted jobs so that users can be aware. We will be removing this
command
and not running this again.
Once again I sincerely apologize, for any undue stress or harm this may have caused.
Thanks,
-- Zachary Huntington-Meath Red Hat Engineering (919)-619-4189 irc: zhunting
atomic-cahc-image-cloud-continuous atomic-cahc-image-cloud-smoketested atomic-cahc-installer-continuous atomic-cahc-installer-smoketested atomic-duffy-allocate atomic-duffy-gc atomic-rdgo-centos7 atomic-tree-smoketest-centos7 atomic-treecompose-centos7 atomicapp-test-docker-master atomicapp-test-docker-pr atomicapp-test-kubernetes-master atomicapp-test-kubernetes-pr atomicapp-test-openshift-master atomicapp-test-openshift-pr atomicapp-test-unittest-master atomicapp-test-unittest-pr atomictest-build-git-evtag-master atomictest-duffy-allocate atomictest-duffy-gc bodhi-bodhi bstinson-test-build bstinsonmhk-compose_tester centos-container-pipeline-service-ci-cleanup centos-container-pipeline-service-ci-master centos-container-pipeline-service-ci-pr centos-container-pipeline-service-ci-pr-debug centos-container-pipeline-service-ci-pr-openshift centos-container-pipeline-service-ci-pr-test centos-container-pipeline-service-ci-pr-unittests centos-container-pipeline-service-ci-return-failed-nodes centos-container-pipeline-service-container-index centos-container-pipeline-service-job
centos-container-pipeline-service-pre-build-caddyserver-caddyserver-latest
centos-container-pipeline-service-pre-build-centos-centos7-atomic-latest
centos-container-pipeline-service-pre-build-centos-postgresql-94-centos7-latest
centos-container-pipeline-service-pre-build-centos-postgresql-95-centos7-latest
centos-container-pipeline-service-pre-build-centos-postgresql-96-centos7-latest
centos-container-pipeline-service-pre-build-centos-varnish-4-centos7-latest
centos-container-pipeline-service-pre-build-centos-varnish-5-centos7-latest
centos-container-pipeline-service-pre-build-fabric8-services-fabric8-wit-latest
centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-latest
centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-testing
centos-container-pipeline-service-pre-build-kbsingh-centos7-atomic-latest
centos-container-pipeline-service-pre-build-kbsingh-container-prebuild-1
centos-container-pipeline-service-pre-build-service-job centos-container-pipeline-service-pre-build-user-cont-tools-latest ci-pipeline-atomic-host-tests ci-pipeline-duffy-allocate ci-pipeline-duffy-gc ci-pipeline-kernel-f26-ltp ci-pipeline-kernel-trigger ci-pipeline-ostree-boot-sanity ci-pipeline-ostree-compose ci-pipeline-ostree-image-boot-sanity ci-pipeline-ostree-image-compose ci-pipeline-rpmbuild cloudinstance-vagrant-test devtools-api-machine-stacks-f8a-build-master devtools-api-machine-stacks-fabric8-analytics devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-spring-boot devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-thorntail devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-vert.x devtools-chat-build-master devtools-chat-integrations-github-build-master devtools-chat-integrations-gitlab-build-master devtools-chat-integrations-irc-build-master devtools-chat-integrations-rssfeeds-build-master devtools-chat-push-proxy-build-master
devtools-che-functional-tests-after-rh-che-build-prod-preview.openshift.io
devtools-che-functional-tests-periodical-openshift.io-2 devtools-che-functional-tests-periodical-prod-preview.openshift.io-2a
devtools-che-functional-tests-periodical-prod-preview.openshift.io-free-stg
devtools-che-functional-tests-prcheck-openshift.io-2 devtools-che-functional-tests-prcheck-prod-preview.openshift.io-2a devtools-che-plugin-registry-build-master devtools-che-starter-build-che-credentials-master devtools-che-starter-prcheck devtools-che-tenant-maintainer-build-master devtools-che-vertx-server-build-master devtools-cvedb devtools-cvedb-build-master devtools-cvejob devtools-cvejob-build-master devtools-cvejob-fabric8-analytics-pydoc devtools-cvejob-fabric8-analytics-pylint devtools-e2e-fabric8-analytics devtools-eclipse-che-build-dockerfiles devtools-eclipse-che-promotion devtools-error-tracking devtools-error-tracking-build-master devtools-f8a-3scale-connect-api-f8a-build-master devtools-f8a-3scale-connect-api-fabric8-analytics devtools-f8a-hpf-insights-f8a-build-master devtools-f8a-hpf-insights-fabric8-analytics devtools-f8a-hpf-insights-fabric8-analytics-pydoc devtools-f8a-hpf-insights-fabric8-analytics-pylint devtools-f8a-master-deploy-e2e-test devtools-f8a-server-backbone-f8a-build-master devtools-f8a-server-backbone-fabric8-analytics devtools-f8a-server-backbone-fabric8-analytics-pydoc devtools-f8a-server-backbone-fabric8-analytics-pylint devtools-fabric8-admin-proxy devtools-fabric8-admin-proxy-build-master devtools-fabric8-analytics-api-gateway-f8a-build-master devtools-fabric8-analytics-api-gateway-fabric8-analytics devtools-fabric8-analytics-api-gateway-fabric8-analytics-pydoc devtools-fabric8-analytics-api-gateway-fabric8-analytics-pylint devtools-fabric8-analytics-auth-fabric8-analytics devtools-fabric8-analytics-auth-fabric8-analytics-pydoc devtools-fabric8-analytics-auth-fabric8-analytics-pylint devtools-fabric8-analytics-common-fabric8-analytics-pydoc devtools-fabric8-analytics-common-fabric8-analytics-pylint devtools-fabric8-analytics-cvedb-s3-dump-docker-f8a-build-master devtools-fabric8-analytics-cvedb-s3-dump-docker-fabric8-analytics devtools-fabric8-analytics-data-model-f8a-build-master devtools-fabric8-analytics-data-model-fabric8-analytics devtools-fabric8-analytics-data-model-fabric8-analytics-pydoc devtools-fabric8-analytics-data-model-fabric8-analytics-pylint devtools-fabric8-analytics-dependency-editor devtools-fabric8-analytics-dependency-editor-npm-publish-build-master devtools-fabric8-analytics-firehose-fetcher-f8a-build-master devtools-fabric8-analytics-firehose-fetcher-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics-pydoc devtools-fabric8-analytics-ingestion-fabric8-analytics-pylint devtools-fabric8-analytics-jenkins-plugin-fabric8-analytics devtools-fabric8-analytics-jobs-f8a-build-master
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Hi All,
Zach: Thanks for including a list here. We keep nightly backups of the jenkins master, and I was going to see about restoring these job before I included a note to the list but it's taking a bit longer to sync over and extract the job configs.
We should have these back in short order (if your service jobs haven't already caught up with some of them).
The Foreman account was one of our legacy accounts (we provisioned this project almost 3 years ago), and included more lax permissions than are necessary. Foreman folks: let's work together soon to get a service job going so you won't need to run this by hand any more, and we can drop permissions here.
Cheers!
-- Brian Stinson CentOS CI Infrastructure Team _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
These are restored to last night's backup.
--Brian _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Looks like after re-running the jjb with a dummy change it updated the jobs and the links were restored.
-- Regards, Jean-Francois Chevrette Principal Software Engineer | Service Delivery - Hosted Services jfchevrette = [irc, github, twitter]
On Tue, Sep 11, 2018 at 9:37 AM Jean-Francois Chevrette jchevret@redhat.com wrote:
Hi,
following the restore we notice that some jobs are not triggering when an upstream job finishes.
For instance, job "devtools-fabric8-analytics-npm-insights-f8a-build-master" should trigger job "devtools-f8a-master-deploy-e2e-test".
Is something missing from the restore with regards to jobs that are "linked"? Would we need to re-create the jobs to restore this functionallity?
Please advise. Thanks!
-- Regards, Jean-Francois Chevrette Principal Software Engineer | Service Delivery - Hosted Services jfchevrette = [irc, github, twitter]
On Tue, Sep 4, 2018 at 6:32 PM Brian Stinson brian@bstinson.com wrote:
On Sep 04 14:52, Brian Stinson wrote:
On Sep 04 13:28, Zach Huntington Meath wrote:
Hello all,
My name is Zach, and I'm a member of the Foreman team. We use
ci.centos.org
to run some of our testing. About 30 minutes ago I was setting up a
job to
maintain the jobs that we use on ci.centos.org and I copied over a
command
that we use on ci.theforeman.org which will update jobs and delete
old
jobs. We use this with ci.theforeman.org as we maintain all of our
jobs in
the same repository, also not realizing that the foreman user would
have
permissions to delete other jobs. I realized a bit into the job that
it was
not doing this, and instead was deleting other users jobs. I stopped
the
job, but it had already deleted 128 jobs. I sincerely apologize for
this,
it was not intentional, but a big mistake. I'm attaching a list of
all the
deleted jobs so that users can be aware. We will be removing this
command
and not running this again.
Once again I sincerely apologize, for any undue stress or harm this
may
have caused.
Thanks,
-- Zachary Huntington-Meath Red Hat Engineering (919)-619-4189 irc: zhunting
atomic-cahc-image-cloud-continuous atomic-cahc-image-cloud-smoketested atomic-cahc-installer-continuous atomic-cahc-installer-smoketested atomic-duffy-allocate atomic-duffy-gc atomic-rdgo-centos7 atomic-tree-smoketest-centos7 atomic-treecompose-centos7 atomicapp-test-docker-master atomicapp-test-docker-pr atomicapp-test-kubernetes-master atomicapp-test-kubernetes-pr atomicapp-test-openshift-master atomicapp-test-openshift-pr atomicapp-test-unittest-master atomicapp-test-unittest-pr atomictest-build-git-evtag-master atomictest-duffy-allocate atomictest-duffy-gc bodhi-bodhi bstinson-test-build bstinsonmhk-compose_tester centos-container-pipeline-service-ci-cleanup centos-container-pipeline-service-ci-master centos-container-pipeline-service-ci-pr centos-container-pipeline-service-ci-pr-debug centos-container-pipeline-service-ci-pr-openshift centos-container-pipeline-service-ci-pr-test centos-container-pipeline-service-ci-pr-unittests centos-container-pipeline-service-ci-return-failed-nodes centos-container-pipeline-service-container-index centos-container-pipeline-service-job
centos-container-pipeline-service-pre-build-caddyserver-caddyserver-latest
centos-container-pipeline-service-pre-build-centos-centos7-atomic-latest
centos-container-pipeline-service-pre-build-centos-postgresql-94-centos7-latest
centos-container-pipeline-service-pre-build-centos-postgresql-95-centos7-latest
centos-container-pipeline-service-pre-build-centos-postgresql-96-centos7-latest
centos-container-pipeline-service-pre-build-centos-varnish-4-centos7-latest
centos-container-pipeline-service-pre-build-centos-varnish-5-centos7-latest
centos-container-pipeline-service-pre-build-fabric8-services-fabric8-wit-latest
centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-latest
centos-container-pipeline-service-pre-build-gluster-storagesig-heketi-testing
centos-container-pipeline-service-pre-build-kbsingh-centos7-atomic-latest
centos-container-pipeline-service-pre-build-kbsingh-container-prebuild-1
centos-container-pipeline-service-pre-build-service-job centos-container-pipeline-service-pre-build-user-cont-tools-latest ci-pipeline-atomic-host-tests ci-pipeline-duffy-allocate ci-pipeline-duffy-gc ci-pipeline-kernel-f26-ltp ci-pipeline-kernel-trigger ci-pipeline-ostree-boot-sanity ci-pipeline-ostree-compose ci-pipeline-ostree-image-boot-sanity ci-pipeline-ostree-image-compose ci-pipeline-rpmbuild cloudinstance-vagrant-test devtools-api-machine-stacks-f8a-build-master devtools-api-machine-stacks-fabric8-analytics
devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-spring-boot
devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-thorntail devtools-bdd-test-booster-openshift.io-us-east-2a-rest-http-vert.x devtools-chat-build-master devtools-chat-integrations-github-build-master devtools-chat-integrations-gitlab-build-master devtools-chat-integrations-irc-build-master devtools-chat-integrations-rssfeeds-build-master devtools-chat-push-proxy-build-master
devtools-che-functional-tests-after-rh-che-build-prod-preview.openshift.io
devtools-che-functional-tests-periodical-openshift.io-2 devtools-che-functional-tests-periodical-prod-preview.openshift.io-2a
devtools-che-functional-tests-periodical-prod-preview.openshift.io-free-stg
devtools-che-functional-tests-prcheck-openshift.io-2 devtools-che-functional-tests-prcheck-prod-preview.openshift.io-2a devtools-che-plugin-registry-build-master devtools-che-starter-build-che-credentials-master devtools-che-starter-prcheck devtools-che-tenant-maintainer-build-master devtools-che-vertx-server-build-master devtools-cvedb devtools-cvedb-build-master devtools-cvejob devtools-cvejob-build-master devtools-cvejob-fabric8-analytics-pydoc devtools-cvejob-fabric8-analytics-pylint devtools-e2e-fabric8-analytics devtools-eclipse-che-build-dockerfiles devtools-eclipse-che-promotion devtools-error-tracking devtools-error-tracking-build-master devtools-f8a-3scale-connect-api-f8a-build-master devtools-f8a-3scale-connect-api-fabric8-analytics devtools-f8a-hpf-insights-f8a-build-master devtools-f8a-hpf-insights-fabric8-analytics devtools-f8a-hpf-insights-fabric8-analytics-pydoc devtools-f8a-hpf-insights-fabric8-analytics-pylint devtools-f8a-master-deploy-e2e-test devtools-f8a-server-backbone-f8a-build-master devtools-f8a-server-backbone-fabric8-analytics devtools-f8a-server-backbone-fabric8-analytics-pydoc devtools-f8a-server-backbone-fabric8-analytics-pylint devtools-fabric8-admin-proxy devtools-fabric8-admin-proxy-build-master devtools-fabric8-analytics-api-gateway-f8a-build-master devtools-fabric8-analytics-api-gateway-fabric8-analytics devtools-fabric8-analytics-api-gateway-fabric8-analytics-pydoc devtools-fabric8-analytics-api-gateway-fabric8-analytics-pylint devtools-fabric8-analytics-auth-fabric8-analytics devtools-fabric8-analytics-auth-fabric8-analytics-pydoc devtools-fabric8-analytics-auth-fabric8-analytics-pylint devtools-fabric8-analytics-common-fabric8-analytics-pydoc devtools-fabric8-analytics-common-fabric8-analytics-pylint devtools-fabric8-analytics-cvedb-s3-dump-docker-f8a-build-master devtools-fabric8-analytics-cvedb-s3-dump-docker-fabric8-analytics devtools-fabric8-analytics-data-model-f8a-build-master devtools-fabric8-analytics-data-model-fabric8-analytics devtools-fabric8-analytics-data-model-fabric8-analytics-pydoc devtools-fabric8-analytics-data-model-fabric8-analytics-pylint devtools-fabric8-analytics-dependency-editor devtools-fabric8-analytics-dependency-editor-npm-publish-build-master devtools-fabric8-analytics-firehose-fetcher-f8a-build-master devtools-fabric8-analytics-firehose-fetcher-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics devtools-fabric8-analytics-ingestion-fabric8-analytics-pydoc devtools-fabric8-analytics-ingestion-fabric8-analytics-pylint devtools-fabric8-analytics-jenkins-plugin-fabric8-analytics devtools-fabric8-analytics-jobs-f8a-build-master
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Hi All,
Zach: Thanks for including a list here. We keep nightly backups of the jenkins master, and I was going to see about restoring these job before I included a note to the list but it's taking a bit longer to sync over and extract the job configs.
We should have these back in short order (if your service jobs haven't already caught up with some of them).
The Foreman account was one of our legacy accounts (we provisioned this project almost 3 years ago), and included more lax permissions than are necessary. Foreman folks: let's work together soon to get a service job going so you won't need to run this by hand any more, and we can drop permissions here.
Cheers!
-- Brian Stinson CentOS CI Infrastructure Team _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
These are restored to last night's backup.
--Brian _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
On Tue, Sep 4, 2018, at 1:28 PM, Zach Huntington Meath wrote:
Hello all,
My name is Zach, and I'm a member of the Foreman team. We use ci.centos.org to run some of our testing. About 30 minutes ago I was setting up a job to maintain the jobs that we use on ci.centos.org and I copied over a command that we use on ci.theforeman.org which will update jobs and delete old jobs. We use this with ci.theforeman.org as we maintain all of our jobs in the same repository, also not realizing that the foreman user would have permissions to delete other jobs.
FWIW, I basically don't think shared Jenkins instances make much sense. It's saner to share base *configurations* and tools - run Jenkins masters as containers in e.g. Kubernetes/OpenShift.
On Sep 05 10:55, Colin Walters wrote:
On Tue, Sep 4, 2018, at 1:28 PM, Zach Huntington Meath wrote:
Hello all,
My name is Zach, and I'm a member of the Foreman team. We use ci.centos.org to run some of our testing. About 30 minutes ago I was setting up a job to maintain the jobs that we use on ci.centos.org and I copied over a command that we use on ci.theforeman.org which will update jobs and delete old jobs. We use this with ci.theforeman.org as we maintain all of our jobs in the same repository, also not realizing that the foreman user would have permissions to delete other jobs.
FWIW, I basically don't think shared Jenkins instances make much sense. It's saner to share base *configurations* and tools - run Jenkins masters as containers in e.g. Kubernetes/OpenShift.
Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Yes! That's the model we're looking toward for the future of CICO (apps.ci.c.o is where we're making that happen).
Converting legacy tenants will take some time.
--Brian
-
Brian Stinson -
Colin Walters -
Jean-Francois Chevrette -
Laurentiu Pancescu -
Zach Huntington Meath