IMHO the whole IT industry and community does quite a bad job here making
systems obsolete on a large scale with no real technical requirement.
Technically, it would be possible to implement complete operating systems
and applications in a way which makes them dynamically select and use the
offered capabilities of the underlying hardware. We really should never
forget that.

I don't think anyone has forgotten that, or that they don't know how to support both.  Red Hat's stated reasoning seems to be that they want to reduce the burden of testing multiple code paths when older systems continue to be supported:

https://developers.redhat.com/articles/2024/01/02/exploring-x86-64-v3-red-hat-enterprise-linux-10#

"if RHEL 10 requires the x86-64-v3 baseline, ISVs will be able to rely on it, too. This reduces maintenance cost for some ISVs because they no longer need to maintain (and test) AVX and non-AVX code paths in their manually tuned software."