I created a CentOS 8 stream machine on aws, and configured crashkernel in grub
``` $ cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 root=UUID=e52ef623-609b-4202-9b2c-ac7aba5c3bee ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 crashkernel=128M ```
Then I installed and verified kdump is functioning
``` # systemctl status kdump ● kdump.service - Crash recovery kernel arming Loaded: loaded (/usr/lib/systemd/system/kdump.service; enabled; vendor preset: enabled) Active: active (exited) since Sat 2024-04-13 08:42:55 UTC; 40s ago Process: 808 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS) Main PID: 808 (code=exited, status=0/SUCCESS)
Apr 13 08:42:43 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Install squash loader *** Apr 13 08:42:43 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Stripping files *** Apr 13 08:42:44 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Stripping files done *** Apr 13 08:42:44 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Squashing the files inside the initramfs *** Apr 13 08:42:54 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Squashing the files inside the initramfs done *** Apr 13 08:42:54 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Creating image file '/boot/initramfs-4.18.0-552.el8.x86_64kdump.img' *** Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal dracut[1101]: *** Creating initramfs image file '/boot/initramfs-4.18.0-552.el8.x86_64kdump.img' done *** Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal kdumpctl[814]: kdump: kexec: loaded kdump kernel Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal kdumpctl[814]: kdump: Starting kdump: [OK] Apr 13 08:42:55 ip-172-31-2-139.ap-east-1.compute.internal systemd[1]: Started Crash recovery kernel arming. ```
Now I need to mimic kernel crash, so I used sysrq to do that
``` echo c > /proc/sysrq-trigger ```
However, after reboot, I didn't see a dmesg file in /var/crash as in CentOS 7.X
``` # ls /var/crash - nothing, empty folder - ```
The only thing I can see is /var/log/kdump.log
``` + 2024-04-13 08:35:56 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:35:56 /usr/bin/kdumpctl@680: set +x + 2024-04-13 08:42:55 /usr/bin/kdumpctl@675: /sbin/kexec -s -d -p '--command-line=BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0' --initrd=/boot/initramfs-4.18.0-552.el8.x86_64kdump.img /boot/vmlinuz-4.18.0-552.el8.x86_64 Try gzip decompression. Try LZMA decompression. + 2024-04-13 08:42:55 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:42:55 /usr/bin/kdumpctl@680: set +x + 2024-04-13 08:44:31 /usr/bin/kdumpctl@675: /sbin/kexec -s -d -p '--command-line=BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-4.18.0-552.el8.x86_64 ro console=ttyS0,115200n8 no_timer_check net.ifnames=0 nvme_core.io_timeout=4294967295 nvme_core.max_retries=10 irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never nokaslr novmcoredd hest_disable disable_cpu_apicid=0' --initrd=/boot/initramfs-4.18.0-552.el8.x86_64kdump.img /boot/vmlinuz-4.18.0-552.el8.x86_64 Try gzip decompression. Try LZMA decompression. + 2024-04-13 08:44:31 /usr/bin/kdumpctl@679: ret=0 + 2024-04-13 08:44:31 /usr/bin/kdumpctl@680: set +x ```
I need the dmesg to understand what's causing the crash. How can I do that? P.S I have a kernel module that triggers a crash but I don't know why, and to demonstrated the problem with kdump, I use sysrq so anyone want to help can easily reproduce it.