[CentOS-devel] Balancing the needs around the CentOS platform

On 12/21/20 5:43 AM, Neal Gompa wrote:
> distribution testing of packages as they're updated has been almost
> entirely automated. Rather than spending ridiculous amounts of time
> having*people*  test it


Thank you, Neal.

Most of the technical conferences that I attend, year after year, 
include groups that stress the difficulty of improving anything due to 
the "That's the way we've always done it" barrier.

I think that behind a lot of the fear of using CentOS Stream there's an 
implicit belief that the more time passes between a piece of software 
being built and an end-user deploying that software, the more reliable 
or trustworthy that software will be. I can't think of a mechanism by 
which that could possibly be true for pre-release packages.  Letting 
packages sit, idle, until the next point release doesn't make them more 
reliable.  (And Red Hatters have told us that there are virtually no 
beta testers doing any testing).

On the other hand, it is theoretically possible that bugs will be found 
through real-world testing after release.  At this point, I think it's 
all but explicit that some of the most vocal objectors view all software 
as a "beta" until someone else has used it for some period of time.  For 
those people, the value of the point releases is that they believe they 
can reasonably expect to wait for an arbitrary period after a point 
release to let other users work out the bugs.  Rather than testing 
software themselves, they're relying on the broader community to deploy 
software and report bugs.  To be clear, though, there's no evidence that 
doing so is necessary or effective.  If it were, we'd expect to see a 
flurry of fixes for bugs that were introduced BY the point release after 
each point release.

There's a non-trivial amount of "that's the way we've always done it" in 
many of the arguments against moving to CentOS Stream.  In my opinion, 
the more we do to push back against the idea that humans should be 
involved in the QA process itself, and the more we do to expand our 
trust in automated tests (which we also continue to expand and improve), 
the better we become as a community and an industry.