I created a CentOS 8 stream machine on aws, and configured crashkernel in grub
``` $ cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 root=UUID=e52ef623-609b-4202-9b2c-ac7aba5c3bee ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 crashkernel=128M ```
Then I installed and verified kdump is functioning
``` # systemctl status kdump ● kdump.service - Crash recovery kernel arming Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: enabled) Active: active (exited) since Sat 2024-04-13 08:42:55 UTC; 40s ago Process: 808 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS) Main PID: 808 (code=exited, status=0/SUCCESS)
Apr 13 08:42:43 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Install squash loader *** Apr 13 08:42:43 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Stripping files *** Apr 13 08:42:44 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Stripping files done *** Apr 13 08:42:44 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Squashing the files inside the initramfs *** Apr 13 08:42:54 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Squashing the files inside the initramfs done *** Apr 13 08:42:54 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Creating image file '/boot/initramfs-4.18.0-552.el8.x86_64kdump.img' *** Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Creating initramfs image file '/boot/initramfs-4.18.0-552.el8.x86_64kdump.img' done *** Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal kdumpctl[814]: kdump: kexec: loaded kdump kernel Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal kdumpctl[814]: kdump: Starting kdump: [OK] Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal systemd[1]: Started Crash recovery kernel arming. ```
Now I need to mimic kernel crash, so I used sysrq to do that
``` echo c > /proc/sysrq-trigger ```
However, after reboot, I didn't see a dmesg file in /var/crash as in CentOS 7.X
``` # ls /var/crash - nothing, empty folder - ```
The only thing I can see is /var/log/kdump.log
``` + 2024-04-13 08:35:56 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:35:56 /usr/bin/kdumpctl@680: set +x + 2024-04-13 08:42:55 /usr/bin/kdumpctl@675: /sbin/kexec -s -d -p '--command-line=BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0' --initrd=/boot/initramfs-4.18.0-552.el8.x86_64kdump.img /boot/vmlinuz-4.18.0-552.el8.x86_64 Try gzip decompression. Try LZMA decompression. + 2024-04-13 08:42:55 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:42:55 /usr/bin/kdumpctl@680: set +x + 2024-04-13 08:44:31 /usr/bin/kdumpctl@675: /sbin/kexec -s -d -p '--command-line=BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0' --initrd=/boot/initramfs-4.18.0-552.el8.x86_64kdump.img /boot/vmlinuz-4.18.0-552.el8.x86_64 Try gzip decompression. Try LZMA decompression. + 2024-04-13 08:44:31 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:44:31 /usr/bin/kdumpctl@680: set +x ```
I need the dmesg to understand what's causing the crash. How can I do that? P.S I have a kernel module that triggers a crash but I don't know why, and to demonstrated the problem with kdump, I use sysrq so anyone want to help can easily reproduce it.
As far as I remember, kdump won't write anything if the produced file will be bigger than the FS.Go through RHEL8 documentation and setup kdump .
Best Regards, Strahil Nikolov
On Sat, Apr 13, 2024 at 13:16, public1020 via Discussdiscuss@lists.centos.org wrote: I created a CentOS 8 stream machine on aws, and configured crashkernel in grub
``` $ cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 root=UUID=e52ef623-609b-4202-9b2c-ac7aba5c3bee ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 crashkernel=128M ```
Then I installed and verified kdump is functioning
``` # systemctl status kdump ● kdump.service - Crash recovery kernel arming Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: enabled) Active: active (exited) since Sat 2024-04-13 08:42:55 UTC; 40s ago Process: 808 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS) Main PID: 808 (code=exited, status=0/SUCCESS)
Apr 13 08:42:43 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Install squash loader *** Apr 13 08:42:43 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Stripping files *** Apr 13 08:42:44 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Stripping files done *** Apr 13 08:42:44 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Squashing the files inside the initramfs *** Apr 13 08:42:54 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Squashing the files inside the initramfs done *** Apr 13 08:42:54 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Creating image file '/boot/initramfs-4.18.0-552.el8.x86_64kdump.img' *** Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Creating initramfs image file '/boot/initramfs-4.18.0-552.el8.x86_64kdump.img' done *** Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal kdumpctl[814]: kdump: kexec: loaded kdump kernel Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal kdumpctl[814]: kdump: Starting kdump: [OK] Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal systemd[1]: Started Crash recovery kernel arming. ```
Now I need to mimic kernel crash, so I used sysrq to do that
``` echo c > /proc/sysrq-trigger ```
However, after reboot, I didn't see a dmesg file in /var/crash as in CentOS 7.X
``` # ls /var/crash - nothing, empty folder - ```
The only thing I can see is /var/log/kdump.log
``` + 2024-04-13 08:35:56 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:35:56 /usr/bin/kdumpctl@680: set +x + 2024-04-13 08:42:55 /usr/bin/kdumpctl@675: /sbin/kexec -s -d -p '--command-line=BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0' --initrd=/boot/initramfs-4.18.0-552.el8.x86_64kdump.img /boot/vmlinuz-4.18.0-552.el8.x86_64 Try gzip decompression. Try LZMA decompression. + 2024-04-13 08:42:55 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:42:55 /usr/bin/kdumpctl@680: set +x + 2024-04-13 08:44:31 /usr/bin/kdumpctl@675: /sbin/kexec -s -d -p '--command-line=BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0' --initrd=/boot/initramfs-4.18.0-552.el8.x86_64kdump.img /boot/vmlinuz-4.18.0-552.el8.x86_64 Try gzip decompression. Try LZMA decompression. + 2024-04-13 08:44:31 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:44:31 /usr/bin/kdumpctl@680: set +x ```
I need the dmesg to understand what's causing the crash. How can I do that? P.S I have a kernel module that triggers a crash but I don't know why, and to demonstrated the problem with kdump, I use sysrq so anyone want to help can easily reproduce it. _______________________________________________ Discuss mailing list -- discuss@lists.centos.org To unsubscribe send an email to discuss-leave@lists.centos.org
I have a disk large enough to hold the whole memory dump. And the same configuration works fine on CentOS 7.
On Saturday, 13 April 2024 at 6:47 PM, Strahil Nikolov via Discuss discuss@lists.centos.org wrote:
CentOS 8 is with newer stack.I remember there were subtle differences.
This article is a good start: https://access.redhat.com/articles/5332081#ssh
Best Regards, Strahil Nikolov
On Sat, Apr 13, 2024 at 17:46, public1020public1020@proton.me wrote: I have a disk large enough to hold the whole memory dump. And the same configuration works fine on CentOS 7.
On Saturday, 13 April 2024 at 6:47 PM, Strahil Nikolov via Discuss discuss@lists.centos.org wrote:
I checked the article you mentioned, I don't need ssh or nfs, I just want the same behaivor on CentOS 7. It dumps everything to /var/crash/IP-date/{vmlinux,dmesg.txt}
I checked the kdump.conf and I didn't see anything changed, I suspect it might be a problem from makedumpfile
Is there any way to create a debug output from this program or kdump? Maybe it failed somehow. https://www.linux.org/docs/man5/makedumpfile.html
On Saturday, 13 April 2024 at 10:51 PM, Strahil Nikolov via Discuss discuss@lists.centos.org wrote:
-
public1020 -
Strahil Nikolov