I'm not sure if this is an appropriate place for this question - please gently steer me to a better place if you think there is one.
I have a machine using a kernel package from the CentOS Community Build System (specifically kernel-3.18.21-16.el6 from http://cbs.centos.org/koji/buildinfo?buildID=4757, but my question applies to other CBS kernel versions as well from what I can tell)
The machine is crashing with some regularity on Sunday mornings. Looking at a kdump core dump might help narrow down the cause of the crashes.
But the CBS kernel packages don't appear to include kernel-debuginfo packages, and the kernel.spec file appears to me (a non-expert for sure) to have had the necessary pieces removed, so I don't know how to rebuild from the srpm and get the debuginfo packages that way either.
If someone could help me find or build the kernel-debuginfo package, it will be appreciated.
Thank you,
On 06/07/2016 09:56 AM, Wayne Betts wrote:
I'm not sure if this is an appropriate place for this question - please gently steer me to a better place if you think there is one.
I have a machine using a kernel package from the CentOS Community Build System (specifically kernel-3.18.21-16.el6 from http://cbs.centos.org/koji/buildinfo?buildID=4757, but my question applies to other CBS kernel versions as well from what I can tell)
The machine is crashing with some regularity on Sunday mornings. Looking at a kdump core dump might help narrow down the cause of the crashes.
But the CBS kernel packages don't appear to include kernel-debuginfo packages, and the kernel.spec file appears to me (a non-expert for sure) to have had the necessary pieces removed, so I don't know how to rebuild from the srpm and get the debuginfo packages that way either.
If someone could help me find or build the kernel-debuginfo package, it will be appreciated.
Those kernel packages do not build debuginfo files. That is also not nearly the newest package. There is a 3.18.34-20 package here:
http://cbs.centos.org/koji/buildinfo?buildID=11179
You can see the packages that are built and look at all the build logs and root logs, etc, by going to:
and adding kernel in the search box .. then picking the version you are interested in.
But, as you can see in the above link, there are no debuginfo files produced.
However, we do take updates directly from the kernel.org 3.18 LTS kernel tree, so there are a lot of things that have been fixed (and a lot of security updates) since 3.18.21 .. in fact you can see all of the changelogs starting with :
https://cdn.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.18.22
and using 23, 24. 25, 26, 27, 28, 29, 30, 31, 32, 33, and 34 for the last number to review all the changelogs.
Thanks, Johnny Hughes
On 6/7/2016 11:22 AM, Johnny Hughes wrote:
On 06/07/2016 09:56 AM, Wayne Betts wrote:
I'm not sure if this is an appropriate place for this question - please gently steer me to a better place if you think there is one.
I have a machine using a kernel package from the CentOS Community Build System (specifically kernel-3.18.21-16.el6 from http://cbs.centos.org/koji/buildinfo?buildID=4757, but my question applies to other CBS kernel versions as well from what I can tell)
The machine is crashing with some regularity on Sunday mornings. Looking at a kdump core dump might help narrow down the cause of the crashes.
But the CBS kernel packages don't appear to include kernel-debuginfo packages, and the kernel.spec file appears to me (a non-expert for sure) to have had the necessary pieces removed, so I don't know how to rebuild from the srpm and get the debuginfo packages that way either.
If someone could help me find or build the kernel-debuginfo package, it will be appreciated.
Those kernel packages do not build debuginfo files. That is also not nearly the newest package. There is a 3.18.34-20 package here:
http://cbs.centos.org/koji/buildinfo?buildID=11179
You can see the packages that are built and look at all the build logs and root logs, etc, by going to:
and adding kernel in the search box .. then picking the version you are interested in.
But, as you can see in the above link, there are no debuginfo files produced.
However, we do take updates directly from the kernel.org 3.18 LTS kernel tree, so there are a lot of things that have been fixed (and a lot of security updates) since 3.18.21 .. in fact you can see all of the changelogs starting with :
https://cdn.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.18.22
and using 23, 24. 25, 26, 27, 28, 29, 30, 31, 32, 33, and 34 for the last number to review all the changelogs.
Thanks, Johnny Hughes
Thank you for your reply Johnny. I tried the latest, kernel-3.18.34-20.el6.x86_64, and that backfired - attempting to read from our ceph cluster triggers an immediate crash.
The behaviour of crashing on cephFS access starts at 3.18.30-20 in the CBS kernels.
Meanwhile, 3.18.25-20 crashes early at boot, failing to mount the root filesystem (so I don't know if the Ceph crash happens at this kernel version).
I'm leaving the system live now at 3.18.25-19. Assuming this hasn't introduced some additional problem(s), it will be a while before we know if this has reduced or eliminated the crashes we are experiencing.
-Wayne
On 06/07/2016 03:18 PM, Wayne Betts wrote:
On 6/7/2016 11:22 AM, Johnny Hughes wrote:
On 06/07/2016 09:56 AM, Wayne Betts wrote:
I'm not sure if this is an appropriate place for this question - please gently steer me to a better place if you think there is one.
I have a machine using a kernel package from the CentOS Community Build System (specifically kernel-3.18.21-16.el6 from http://cbs.centos.org/koji/buildinfo?buildID=4757, but my question applies to other CBS kernel versions as well from what I can tell)
The machine is crashing with some regularity on Sunday mornings. Looking at a kdump core dump might help narrow down the cause of the crashes.
But the CBS kernel packages don't appear to include kernel-debuginfo packages, and the kernel.spec file appears to me (a non-expert for sure) to have had the necessary pieces removed, so I don't know how to rebuild from the srpm and get the debuginfo packages that way either.
If someone could help me find or build the kernel-debuginfo package, it will be appreciated.
Those kernel packages do not build debuginfo files. That is also not nearly the newest package. There is a 3.18.34-20 package here:
http://cbs.centos.org/koji/buildinfo?buildID=11179
You can see the packages that are built and look at all the build logs and root logs, etc, by going to:
and adding kernel in the search box .. then picking the version you are interested in.
But, as you can see in the above link, there are no debuginfo files produced.
However, we do take updates directly from the kernel.org 3.18 LTS kernel tree, so there are a lot of things that have been fixed (and a lot of security updates) since 3.18.21 .. in fact you can see all of the changelogs starting with :
https://cdn.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.18.22
and using 23, 24. 25, 26, 27, 28, 29, 30, 31, 32, 33, and 34 for the last number to review all the changelogs.
Thanks, Johnny Hughes
Thank you for your reply Johnny. I tried the latest, kernel-3.18.34-20.el6.x86_64, and that backfired - attempting to read from our ceph cluster triggers an immediate crash.
The behaviour of crashing on cephFS access starts at 3.18.30-20 in the CBS kernels.
Meanwhile, 3.18.25-20 crashes early at boot, failing to mount the root filesystem (so I don't know if the Ceph crash happens at this kernel version).
I'm leaving the system live now at 3.18.25-19. Assuming this hasn't introduced some additional problem(s), it will be a while before we know if this has reduced or eliminated the crashes we are experiencing.
I have not specifically tried to enable ceph in those kernels .. is there anyone on the Storage sig who would take a look at the x86_64 kernek config file and recommend changes to enable ceph:
Here is the config file if someone from CEPH can help:
https://raw.githubusercontent.com/hughesjr/xen-kernel/master/SOURCES/config-...
Thanks, Johnny Hughes
On 06/07/2016 03:18 PM, Wayne Betts wrote:
On 6/7/2016 11:22 AM, Johnny Hughes wrote:
On 06/07/2016 09:56 AM, Wayne Betts wrote:
I'm not sure if this is an appropriate place for this question - please gently steer me to a better place if you think there is one.
I have a machine using a kernel package from the CentOS Community Build System (specifically kernel-3.18.21-16.el6 from http://cbs.centos.org/koji/buildinfo?buildID=4757, but my question applies to other CBS kernel versions as well from what I can tell)
The machine is crashing with some regularity on Sunday mornings. Looking at a kdump core dump might help narrow down the cause of the crashes.
But the CBS kernel packages don't appear to include kernel-debuginfo packages, and the kernel.spec file appears to me (a non-expert for sure) to have had the necessary pieces removed, so I don't know how to rebuild from the srpm and get the debuginfo packages that way either.
If someone could help me find or build the kernel-debuginfo package, it will be appreciated.
Those kernel packages do not build debuginfo files. That is also not nearly the newest package. There is a 3.18.34-20 package here:
http://cbs.centos.org/koji/buildinfo?buildID=11179
You can see the packages that are built and look at all the build logs and root logs, etc, by going to:
and adding kernel in the search box .. then picking the version you are interested in.
But, as you can see in the above link, there are no debuginfo files produced.
However, we do take updates directly from the kernel.org 3.18 LTS kernel tree, so there are a lot of things that have been fixed (and a lot of security updates) since 3.18.21 .. in fact you can see all of the changelogs starting with :
https://cdn.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.18.22
and using 23, 24. 25, 26, 27, 28, 29, 30, 31, 32, 33, and 34 for the last number to review all the changelogs.
Thanks, Johnny Hughes
Thank you for your reply Johnny. I tried the latest, kernel-3.18.34-20.el6.x86_64, and that backfired - attempting to read from our ceph cluster triggers an immediate crash.
The behaviour of crashing on cephFS access starts at 3.18.30-20 in the CBS kernels.
Meanwhile, 3.18.25-20 crashes early at boot, failing to mount the root filesystem (so I don't know if the Ceph crash happens at this kernel version).
I'm leaving the system live now at 3.18.25-19. Assuming this hasn't introduced some additional problem(s), it will be a while before we know if this has reduced or eliminated the crashes we are experiencing.
Also, are you using this with XEN as well or just using this because it is a newer kernel?
On 6/7/2016 6:31 PM, Johnny Hughes wrote:
On 06/07/2016 03:18 PM, Wayne Betts wrote:
On 6/7/2016 11:22 AM, Johnny Hughes wrote:
On 06/07/2016 09:56 AM, Wayne Betts wrote:
I'm not sure if this is an appropriate place for this question - please gently steer me to a better place if you think there is one.
I have a machine using a kernel package from the CentOS Community Build System (specifically kernel-3.18.21-16.el6 from http://cbs.centos.org/koji/buildinfo?buildID=4757, but my question applies to other CBS kernel versions as well from what I can tell)
The machine is crashing with some regularity on Sunday mornings. Looking at a kdump core dump might help narrow down the cause of the crashes.
But the CBS kernel packages don't appear to include kernel-debuginfo packages, and the kernel.spec file appears to me (a non-expert for sure) to have had the necessary pieces removed, so I don't know how to rebuild from the srpm and get the debuginfo packages that way either.
If someone could help me find or build the kernel-debuginfo package, it will be appreciated.
Those kernel packages do not build debuginfo files. That is also not nearly the newest package. There is a 3.18.34-20 package here:
http://cbs.centos.org/koji/buildinfo?buildID=11179
You can see the packages that are built and look at all the build logs and root logs, etc, by going to:
and adding kernel in the search box .. then picking the version you are interested in.
But, as you can see in the above link, there are no debuginfo files produced.
However, we do take updates directly from the kernel.org 3.18 LTS kernel tree, so there are a lot of things that have been fixed (and a lot of security updates) since 3.18.21 .. in fact you can see all of the changelogs starting with :
https://cdn.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.18.22
and using 23, 24. 25, 26, 27, 28, 29, 30, 31, 32, 33, and 34 for the last number to review all the changelogs.
Thanks, Johnny Hughes
Thank you for your reply Johnny. I tried the latest, kernel-3.18.34-20.el6.x86_64, and that backfired - attempting to read from our ceph cluster triggers an immediate crash.
The behaviour of crashing on cephFS access starts at 3.18.30-20 in the CBS kernels.
Meanwhile, 3.18.25-20 crashes early at boot, failing to mount the root filesystem (so I don't know if the Ceph crash happens at this kernel version).
I'm leaving the system live now at 3.18.25-19. Assuming this hasn't introduced some additional problem(s), it will be a while before we know if this has reduced or eliminated the crashes we are experiencing.
Also, are you using this with XEN as well or just using this because it is a newer kernel?
We are not using this with Xen. The Ceph project recommendation is to upgrade Ceph clients from the "stock" 2.6.32 Scientific Linux/RHEL 6 kernels (see the notes at http://docs.ceph.com/docs/master/start/os-recommendations/ ). We used some 3.10 LTS series kernels for a bit, but our Ceph expert subsequently suggested the use of the CBS 3.18 series kernel for our CephFS clients, though I don't know if there was a specific reason other than 18 > 10. (I'd guess he found some anecdotal evidence or a recommendation online from a fellow CephFS user to use the CBS 3.18 kernels instead of the LTS 3.10 kernels for Ceph, but don't really know.)
-Wayne
-
Johnny Hughes -
Wayne Betts