Hi All!
We have been dealing with a memory leak in the kernel for IKEv2 and IPSec
connections relating to a memory leak in xfrm support on both el8 and el7.
The symptom of this issue is that memory will continue allocating in slab
over time making a box oom after too many connections.
As per some external discussions I am sending the patch + bug report on to
this list. It has already been accepted into upstream kernels (4.19
included) and is a pretty straight forward backport. I have tested and
installed this on a few centos8 systems to validate that this does indeed
solve the memory leak issue.
rbz# 1780470
# CPE Weekly: 2020-02-14
Background:
The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS. Our goal is to keep
core servers and services running and maintained, build releases, and
other strategic tasks that need more dedicated time than volunteers
can give.
For better communication, we will be giving weekly reports to the
CentOS and Fedora communities about the general tasks and work being
done. Also for better communication between our groups we have
created #redhat-cpe on Freenode IRC! Please feel free to catch us
there, a mail has landed on both the CentOS and Fedora devel lists
with context here.
## CentOS Updates
### CentOS
* Tuning/adding more mirrorlist servers due to EC2/AWS change from last week
* Updates are really faster now for CentOS users in AWS/EC2 (more
than 100TiB of updated packages served internally with a 98.33% ratio
served from cache, so faster than ever)
* Moving data from CentOs Infra and switching to a new solution for
backup for infra:
* https://github.com/CentOS/ansible-role-centos-backup/
* https://github.com/CentOS/ansible-role-restic
* VDO (https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/ht…)
* Investigating how to incorporate CentOS auth into the new AAA solution
* We'll need proper communication plan to ask everybody to get a
FAS account when we'll migrate
### CentOS Stream
* The CPE team are collaborating with the Packit team internally to
develop a workflow for contributor patches
* We are working towards demoing a contributor patch from submission
to end for Red Hat Summit in April!
## Fedora Updates
### Data Centre Move
By June 15th 2020, dedicated Fedora servers will have moved to a new
data centre in Northern Virginia.
If you want to read more, check out our post on hackmd:
https://hackmd.io/@Ap8CkTlpSfmjb44UGV-kWA/rJsk7A-QL
Here are some key dates and actions we need you to know:
* On 28th Feb, the old OpenStack instance in Fedora Cloud will be retired.
* Please reach out to Kevin before 25th Feb if you need to take action
for your instance if you are an instance owner. Here is the link to
the current status: https://pagure.io/fedora-infrastructure/issue/8614
* On 13th April, the CPE team will begin to ship servers to our new
data centre - this does not affect F32 distribution or release
* From March 1st, the CPE team will begin to build the Minimum Viable
Fedora infrastructure in the new datacenter
* Between May 20th - July 1st, we will be redirecting Fedora services
to run on the 'MVF' offering to facilitate the final part of the move
and allow for re-racking and testing
* By July 1st we hope to have BAU (business as usual) for Fedora infra
### AAA Replacement
This project is replacing our old existing fas (fedora account system)
with a new freeipa based system.
* Check out our blog on the teams progress to date!
* You can also see our jira board for tickets we are working on
* And we have an IRC channel - #fedora-AAA
* We are currently working on the FreeIPA API integration and the
folks at FreeIPA have been really helpful so far to work with
### CI/CD
* The team have been trying to get a local instance of Koji running this week
* They are investigating different algorithms that could help generate
the next release if a package based on its git history.
* Monitoring-gating is being deployed in OpenShift in staging
### Sustaining Team
* The team meets daily @ 1900 UTC in #fedora-admin on IRC
* The priority work is:
** Fedora 32
** Assisting with technical debt to facilitate the colo move
** mbbox upgrading
** CentOS CI OpenShift upgrading
* The team also have a public thread on knowledge sharing,
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapr…
so take a look!
## Docs
* Merged https://pagure.io/fedora-docs/quick-docs/pull-request/175
* Merged https://pagure.io/fedora-docs/quick-docs/pull-request/174
* Merged https://pagure.io/fedora-docs/quick-docs/pull-request/173
* Merged https://pagure.io/fedora-docs/quick-docs/pull-request/167
* Reviewed https://pagure.io/fedora-docs/install-guide/pull-request/40
* Also work underway on CentOS 8.1 docs
### Misc Updates
* The team are also working on creating Fedora infra application map -
stay tuned for the publication soon!
* keys.fedoraproject.org has been turned off
* boot.fedoraproject.org will be retired this week
* Certs have been changed to letsencrypt for fedorahosted/fedoracommunity
* Moved Koji calls to a backend task worker in Bodhi:
https://github.com/fedora-infra/bodhi/issues/3061
* kdreyer's playbook repo has been integrated:
https://pagure.io/koji-vagrant/pull-request/1
* Testing dist-git repo as a submodule has been added:
https://pagure.io/koji-vagrant/pull-request/2
* Ticket in Monitor-Gating has been reviewed:
https://pagure.io/fedora-ci/monitor-gating/pull-request/8
As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.
Have a great weekend!
Aoife
--
Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford
hi,
storage-sig managed to rebuild / crosstag all ceph nautilus deps into
storage8-ceph-nautilus-candidate [1] target thanks to the new el8 buildroot
many thanks to all the people who worked hard to get the el8 buildroot
working; we're hoping to see soon ceph nautilus tested on el8 in quite a
few openstack jobs thanks to the new packages
we can't build the centos-release-ceph-nautilus package to provision the
.repo files for el8 though [2] because the build target we expected to
have for that "core8-extras-common-el8.centos" doesn't exist yet
do people have ideas/plans on how to get the -release package built and
distributed?
thanks!
1.
https://cbs.centos.org/koji/builds?order=nvr&tagID=1891&inherited=1&latest=1
2. https://cbs.centos.org/koji/packageinfo?packageID=7290
--
Giulio Fidente
GPG KEY: 08D733BA
Hello,
I am member of packit team, working on integration packit into centos
infrastructure.
I would like to ask if there is fedmsg bus implemented in centos
infrastructure and if you can provide me with configuration details.
--
Thanks,
Ján Sakáloš
hi devel guys,
I thought I'd ask here directly for it begins to worry me a bit. What's
that? Well.. it's a freshly set up Centos8 box which has no direct way
out but via Squid proxy(Centos7 squid-3.5.20-12.el7_6.1.x86_64) and it
seems that lots of things do not want to work, eg.:
$ podman search centos
ERRO[0000] error searching registry "registry.fedoraproject.org":
couldn't search registry "registry.fedoraproject.org": error pinging
docker registry registry.fedoraproject.org: Get
https://registry.fedoraproject.org/v2/: proxyconnect tcp: tls: first
record does not look like a TLS handshake
ERRO[0000] error searching registry "docker.io": couldn't search
registry "docker.io": error pinging docker registry index.docker.io: Get
https://index.docker.io/v2/: proxyconnect tcp: tls: first record does
not look like a TLS handshake
ERRO[0000] error searching registry "registry.access.redhat.com":
couldn't search registry "registry.access.redhat.com": error pinging
docker registry registry.access.redhat.com: Get
https://registry.access.redhat.com/v2/: proxyconnect tcp: tls: first
record does not look like a TLS handshake
ERRO[0000] error searching registry "registry.centos.org": couldn't
search registry "registry.centos.org": error pinging docker registry
registry.centos.org: Get https://registry.centos.org/v2/: proxyconnect
tcp: tls: first record does not look like a TLS handshake
Another example is R from EPEL, installing any package/library in R also
fails in similar way and at Squid's end I get lots of:
...
1581503634.209 1 10.5.8.17 TAG_NONE/400 4300
%1F%8Dl%E4%C9z%CFD$%ED%87%EF%A9%F4%F7%05%E7%9Cja%E8%23Y%B5%A5%EBb%7BT%8F%B4
- HIER_NONE/- text/html
1581503634.211 1 10.5.8.17 TAG_NONE/400 4315 NONE
error:invalid-request - HIER_NONE/- text/html
1581503634.211 0 10.5.8.17 TAG_NONE/400 4120 &%AFi%BB%1A%AD%03%9C
- HIER_NONE/- text/html
1581503634.211 0 10.5.8.17 TAG_NONE/400 4270
T%88vH5%BAw%EE%FB%1F9%DE%D5%B9%90%C7%05?%F1%D6%22%E3%5B%8F%7F%7C%E6 -
HIER_NONE/- text/html
1581503634.212 0 10.5.8.17 TAG_NONE/400 4300
%85S%80%BAKh%8E%AB+%90%D4%8Ad%F0%B4%EB%C1or%5E%BEY%800+%F8%98%AF%04!%97%F0
- HIER_NONE/- text/html
1581503634.212 0 10.5.8.17 TAG_NONE/400 4192
%DA%E6%9E3%DB%9AP%E0q%A3%89c%BBeO%C2%A5%0F - HIER_NONE/- text/html
1581503634.213 0 10.5.8.17 TAG_NONE/400 4074 %1Ej%8D%17 -
HIER_NONE/- text/html
1581503634.213 0 10.5.8.17 TAG_NONE/400 4564 NONE
error:invalid-request - HIER_NONE/- text/html
1581503663.358 529 10.8.9.208 TCP_TUNNEL/200 4442 CONNECT
v10.events.data.microsoft.com:443 - HIER_DIRECT/52.114.128.10 -
1581503708.562 1 10.5.8.17 TAG_NONE/400 4300
%EF%1E%F9%10:%9E%CE(%85%F4%CD%DEc%809%0EnU%BD%E3%9F@%14%8C%FF!%03%7C?*%B5l
- HIER_NONE/- text/html
1581503708.563 1 10.5.8.17 TAG_NONE/400 4315 NONE
error:invalid-request - HIER_NONE/- text/html
1581503708.564 0 10.5.8.17 TAG_NONE/400 4315
%9D%7D%17.%D0%F4%B2%C9%B6V%8E%B5%BB%10X%AF%F1%E3g%3C%14%90%C2%F7%AF%E6P%19%1D6%98%C1%DB
- HIER_NONE/- text/html
1581503708.564 0 10.5.8.17 TAG_NONE/400 4242
-N%08,%3E.%93%F87l%0F%7F%89G%0E%1C%A0%A7%90%DF%8A+%D9%E4c - HIER_NONE/-
text/html
1581503708.565 1 10.5.8.17 TAG_NONE/400 4315
%D1f%E3%891%EA%86%07%07%B7%EEu%BF%83F%AD%E4%A2%FB7%CE%ACw%1Cf*%E2%FD%BD%9A%5E%07
- HIER_NONE/- text/html
1581503708.565 0 10.5.8.17 TAG_NONE/400 4315 NONE
error:invalid-request - HIER_NONE/- text/html
1581503708.565 0 10.5.8.17 TAG_NONE/400 4280
%A3%13%EE%D9%5CIfKzS%F39x%AB%CE%F8%D0A%D7Y%8A4%C17%FC%9A%B9%98%87%CBz -
HIER_NONE/- text/html
1581503708.566 0 10.5.8.17 TAG_NONE/400 4174
%C1;%A4q%8E%81%E6%CE%E1%DC%81N%1D%F0 - HIER_NONE/- text/html
Everything else seems to work fine, a small group of Centoses 7 use that
Squid just fine, Windows boxes too.
Would you share any thoughts as to what might be going on there?
many thanks, L.
hi guys,
Anything I try to do with dnf and outside what 'centos', eg. adding a
gluster repo or any third-party repo I get:
...
[MIRROR] rdo-release.rpm: Curl error (60): Peer certificate cannot be
authenticated with given CA certificates for
https://rdoproject.org/repos/rdo-release.rpm [SSL certificate problem:
certificate is not yet valid]
[FAILED] rdo-release.rpm: Curl error (60): Peer certificate cannot be
authenticated with given CA certificates for
https://rdoproject.org/repos/rdo-release.rpm [SSL certificate problem:
certificate is not yet valid]
...
Any ideas, suggestions on why centos is not happy that way?
many thanks, L.
Hello,
I am trying to run a Dockerfile with 'npm install' but getting errors.
Version details as follows =
Cent OS version: 8
Docker CE version: 19.03.4
Dockerfile:
-------------------------------------
# Create image based on the official Node 10 image from dockerhub
FROM node:10.16.3
# Create a directory where our app will be placed
RUN mkdir -p /app
# Change directory so that our commands run inside this new directory
WORKDIR /app
# Copy dependency definitions
COPY package*.json /app/
# Install dependecies
RUN npm cache clear --force
RUN npm install -verbose
# Get all the code needed to run the app
COPY . /app/
# Expose the port the app runs in
EXPOSE 4200
# Serve the app
CMD ["npm", "start"]
-------------------------------------
Error:
-------------------------------------
Step 6/9 : RUN npm install -verbose
---> Running in f6cf6cf24712
npm info it worked if it ends with ok
npm verb cli [ '/usr/local/bin/node',
npm verb cli '/usr/local/bin/npm',
npm verb cli 'install',
npm verb cli '-verbose' ]
npm info using npm(a)6.9.0
npm info using node(a)v10.16.3
npm verb npm-session 8c25dbc16da110c3
npm info lifecycle app01-uiapp(a)0.0.0~preinstall: app01-uiapp(a)0.0.0
npm timing stage:loadCurrentTree Completed in 20ms
npm timing stage:loadIdealTree:cloneCurrentTree Completed in 0ms
npm timing stage:loadIdealTree:loadShrinkwrap Completed in 1768ms
npm timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 2534ms
npm timing stage:loadIdealTree Completed in 5336ms
npm timing stage:generateActionsToTake Completed in 1623ms
npm verb correctMkdir /root/.npm/_locks correctMkdir not in flight; initializing
npm verb makeDirectory /root/.npm/_locks creation not in flight; initializing
npm verb lock using /root/.npm/_locks/staging-bd3355550c5a469e.lock
for /app/node_modules/.staging
npm timing action:extract Completed in 315443ms
npm verb unlock done using
/root/.npm/_locks/staging-bd3355550c5a469e.lock for
/app/node_modules/.staging
npm timing stage:rollbackFailedOptional Completed in 3ms
npm timing stage:runTopLevelLifecycles Completed in 323677ms
npm verb type system
npm verb stack FetchError: request to
https://registry.npmjs.org/core-js/-/core-js-3.6.4.tgz failed, reason:
getaddrinfo EAI_AGAIN registry.npmjs.org registry.npmjs.org:443
npm verb stack at ClientRequest.req.on.err
(/usr/local/lib/node_modules/npm/node_modules/node-fetch-npm/src/index.js:68:14)
npm verb stack at ClientRequest.emit (events.js:198:13)
npm verb stack at TLSSocket.socketErrorListener (_http_client.js:392:9)
npm verb stack at TLSSocket.emit (events.js:198:13)
npm verb stack at emitErrorNT (internal/streams/destroy.js:91:8)
npm verb stack at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
npm verb stack at process._tickCallback
(internal/process/next_tick.js:63:19)
npm verb cwd /app
npm verb Linux 4.18.0-80.11.2.el8_0.x86_64
npm verb argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-verbose"
npm verb node v10.16.3
npm verb npm v6.9.0
npm ERR! code EAI_AGAIN
npm ERR! errno EAI_AGAIN
npm ERR! request to
https://registry.npmjs.org/core-js/-/core-js-3.6.4.tgz failed, reason:
getaddrinfo EAI_AGAIN registry.npmjs.org registry.npmjs.org:443
npm verb exit [ 1, true ]
npm timing npm Completed in 624434ms
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2020-02-10T04_26_38_115Z-debug.log
The command '/bin/sh -c npm install -verbose' returned a non-zero code: 1
-------------------------------------
Other details: I can ping/wget the tgz file from base Cent OS 8
machine that means it's not the DNS issue.
Any hints how to fix this issue? Thanks.
Regards,
Amey.
Hi everyone,
Welcome (back!) to the CPE team weekly project update mail!
There was a break on this email blast of two weeks to facilitate team
travel to DevConf & FosDem, but we are back in action again from this
week onwards :)
Background:
The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS. Our goal is to keep
core servers and services running and maintained, build releases, and
other strategic tasks that need more dedicated time than volunteers
can give.
For better communication, we will be giving weekly reports to the
CentOS and Fedora communities about the general tasks and work being
done. Also for better communication between our groups we have
created #redhat-cpe on Freenode IRC! Please feel free to catch us
there, a mail has landed on both the CentOS and Fedora devel lists
with context here.
High Level Project Updates:
CentOS
Due to team members traveling home from DevConf & FOSDEM, we don't
have many CentOS updates this week!
At the CentOS Dojo, we had a talk about DNS in centos.org and
letsencrypt dns challenges.
Some of the team also manned the booth at Fosdem.
Fedora
AAA Replacement
The team have been working on replacing the current FAS solution and
Rick Elrod presented their work to date at DevConf.cz
They are currently having open discussion about account migrations to
make sure there is a plan in place in time
We are also scoping the work involved to have CentOS use this account
system also - more details to follow on this soon!
General Updates
The GitForge ODF document is still open for consideration and comment
until Monday 10th February 2020.
There is some discussion on emails sent to both Fedora & CentOS devel
lists, so please take a look and we look forward to hearing your
feedback!
CI/CD
Release/Changelog
We have worked on the logic to find the next release for a package
based on the build history
https://pagure.io/Fedora-Infra/generate_changelog/blob/master/f/next_build.…
We have a POC of a script that generates a changelog from a git commit history
https://pagure.io/Fedora-Infra/generate_changelog/blob/master/f/generate_ch…
We are looking more into rpm macros vs rpm variable
We are looking at deploying a local koji instance so we can test our
POC work there before looking at pushing it to stg
Monitoring-gating
A script for single-build running in communishift for now
Single-build currently fails in staging:
https://pagure.io/fedora-infrastructure/issue/8611
The multi-build monitoring script has been ported to the new structure
Sustaining
Kick off happening this week
The team are planning to work on
Infra and Releng tickets
Mbbox upgrade
Application Retirements
Elections
Move to communishift is on hold till the GDPR situation is clear
Fedocal
Jlanda wants to try different approach due to permission errors - no
new update https://pagure.io/fedora-infrastructure/issue/8274
No progress on kanban board - looks abandoned
https://teams.fedoraproject.org/project/fedora-calendar/kanban
Nuancier
PR from sebwoj - no update
Benson Muite is now working on OIDC authentication
Did some changes
Trying to figure out, how to test these changes
Misc Updates
Pagure
Improvements around the pagure API tokens were made this week
Single template for the two pages
Include a link to where the token can be renewed in the email
Add the possibility to set your own expiration date (up to 2 years)
A couple of bugs were fixed in the /groups API endpoint that were
preventing the Fedora-Infra group from being added to some projects…
Anyita
Deploy new version of Anitya on production - 0.18.0 is now live
Move Anitya to fedora-infra
The-new-hotness
Continue working on Packit PR for the-new-hotness -
https://github.com/fedora-infra/the-new-hotness/pull/235 (currently on
hold, need some changes in the packit itself
https://github.com/packit-service/packit/issues/689)
Update OpenShift deployment config for the-new-hotness (Don’t trigger
build on config change automatically - this prevent double building on
new release)
Cleaning the-new-hotness issue tracker - 5 issues closed
Fedora-flatpack
Fix in Packit (https://github.com/packit-service/packit/pull/682)
Report few issues for flatpak-module-tools (Fedora flatpak)
https://pagure.io/flatpak-module-tools/issue/7https://pagure.io/flatpak-module-tools/issue/6https://pagure.io/flatpak-module-tools/issue/5
Note:
This document is currently built from individual reports rolled into a
document which we edit and copy into a final document. We are aware
that this causes problems with some email readers, and are working on
a method to make this less problematic.
Comments? Suggestions? Feedback? Let Us Know!
Hi everyone,
Welcome (back!) to the CPE team weekly project update mail!
There was a break on this email blast of two weeks to facilitate team
travel to DevConf & FosDem, but we are back in action again from this
week onwards :)
Background:
The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS. Our goal is to keep
core servers and services running and maintained, build releases, and
other strategic tasks that need more dedicated time than volunteers
can give.
For better communication, we will be giving weekly reports to the
CentOS and Fedora communities about the general tasks and work being
done. Also for better communication between our groups we have
created #redhat-cpe on Freenode IRC! Please feel free to catch us
there, a mail has landed on both the CentOS and Fedora devel lists
with context here.
High Level Project Updates:
Fedora
AAA Replacement
The team have been working on replacing the current FAS solution and
Rick Elrod presented their work to date at DevConf.cz
They are currently having open discussion about account migrations to
make sure there is a plan in place in time
We are also scoping the work involved to have CentOS use this account
system also - more details to follow on this soon!
General Updates
The GitForge ODF document is still open for consideration and comment
until Monday 10th February 2020.
There is some discussion on emails sent to both Fedora & CentOS devel
lists, so please take a look and we look forward to hearing your
feedback!
CI/CD
Release/Changelog
We have worked on the logic to find the next release for a package
based on the build history
https://pagure.io/Fedora-Infra/generate_changelog/blob/master/f/next_build.…
We have a POC of a script that generates a changelog from a git commit history
https://pagure.io/Fedora-Infra/generate_changelog/blob/master/f/generate_ch…
We are looking more into rpm macros vs rpm variable
We are looking at deploying a local koji instance so we can test our
POC work there before looking at pushing it to stg
Monitoring-gating
A script for single-build running in communishift for now
Single-build currently fails in staging:
https://pagure.io/fedora-infrastructure/issue/8611
The multi-build monitoring script has been ported to the new structure
Sustaining
Kick off happening this week
The team are planning to work on
Infra and Releng tickets
Mbbox upgrade
Application Retirements
Elections
Move to communishift is on hold till the GDPR situation is clear
Fedocal
Jlanda wants to try different approach due to permission errors - no
new update https://pagure.io/fedora-infrastructure/issue/8274
No progress on kanban board - looks abandoned
https://teams.fedoraproject.org/project/fedora-calendar/kanban
Nuancier
PR from sebwoj - no update
Benson Muite is now working on OIDC authentication
Did some changes
Trying to figure out, how to test these changes
Misc Updates
Pagure
Improvements around the pagure API tokens were made this week
Single template for the two pages
Include a link to where the token can be renewed in the email
Add the possibility to set your own expiration date (up to 2 years)
A couple of bugs were fixed in the /groups API endpoint that were
preventing the Fedora-Infra group from being added to some projects…
Anyita
Deploy new version of Anitya on production - 0.18.0 is now live
Move Anitya to fedora-infra
The-new-hotness
Continue working on Packit PR for the-new-hotness -
https://github.com/fedora-infra/the-new-hotness/pull/235 (currently on
hold, need some changes in the packit itself
https://github.com/packit-service/packit/issues/689)
Update OpenShift deployment config for the-new-hotness (Don’t trigger
build on config change automatically - this prevent double building on
new release)
Cleaning the-new-hotness issue tracker - 5 issues closed
Fedora-flatpack
Fix in Packit (https://github.com/packit-service/packit/pull/682)
Report few issues for flatpak-module-tools (Fedora flatpak)
https://pagure.io/flatpak-module-tools/issue/7https://pagure.io/flatpak-module-tools/issue/6https://pagure.io/flatpak-module-tools/issue/5
CentOS
Due to team members traveling home from DevConf & FOSDEM, we don't
have many CentOS updates this week!
At the CentOS Dojo, we had a talk about DNS in centos.org and
letsencrypt dns challenges.
Some of the team also manned the booth at Fosdem.
Note:
This document is currently built from individual reports rolled into a
document which we edit and copy into a final document. We are aware
that this causes problems with some email readers, and are working on
a method to make this less problematic.
Comments? Suggestions? Feedback? Let Us Know!
--
Aoife Moloney
Feature Driver
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford