[CentOS-devel] RHEL moving to issues.redhat.com only long term

Tue Mar 15 11:10:10 UTC 2022
Josh Boyer <jwboyer at redhat.com>

On Mon, Mar 14, 2022 at 11:12 AM Adam Williamson
<adamwill at fedoraproject.org> wrote:
> On Mon, 2022-03-07 at 12:44 -0500, Josh Boyer wrote:
> > Hi Fedora, CentOS, and EPEL Communities!
> >
> > As part of our continued 3 year major Red Hat Enterprise Linux release
> > cadence, RHEL 9 development is starting to wrap up with the spring
> > 2022 release coming soon.  That means planning for the next release
> > will start in earnest in the very near future.  As some of you may
> > know, Red Hat has been using both Bugzilla and Jira via
> > issues.redhat.com for RHEL development for several years.  Our
> > intention is to move to using issues.redhat.com only for the major
> > RHEL releases after RHEL 9.
> >
> > What does this mean for Fedora and CentOS?  This discussion is in part
> > to figure that out.  Based on some very brief analysis, the following
> > should hold:
> >
> > - RHEL customers should continue to file support cases through the Red
> > Hat Customer portal, which will remain consistent regardless of the
> > backend tooling used.
> >
> > - There is no imminent retirement of the Red Hat Bugzilla instance
> > being planned at this time.  RHEL 7, 8, and 9 will continue to use
> > bugzilla in some form and RHEL 9 has a very long lifecycle.
> >
> > - Fedora Linux and EPEL have their own Bugzilla product families and
> > are not directly impacted in their own workflows by the choice to use
> > only issues.redhat.com for RHEL.
> >     - There will be impacts on existing documentation that provide
> > guidance on requesting things from RHEL in various places like EPEL.
> > We will be happy to help adjust these.
> >
> > - CentOS Stream contribution and bug reporting workflows will be
> > adjusted to use issues.redhat.com instead of bugzilla in the relevant
> > places.  This should apply to all versions of CentOS Stream for a
> > unified contributor workflow.  This will happen gradually as we
> > discover the best workflow to use.
> >
> > If there are other impacts that you can think of, please raise them on
> > this thread.  We’d like to ensure we’re covering as much as possible
> > as this rolls out.
> So if I'm understanding this correctly, the ultimate consequence here
> is "Red Hat Bugzilla might go away, or stop being maintained, at
> whatever point it's no longer needed for RHEL 9", right?

I have no idea, to be honest.  There's a lot of assumption in that
statement and it certainly could be an outcome, but I'm not aware of
any plans towards that directly.

> That could obviously have pretty significant consequences for Fedora.
> Bugzilla isn't only an issue tracker for Fedora; we run some
> significant processes through it, notably the Change process, the
> blocker/FE bug process, and the prioritized bug process. In A World
> Without Bugzilla all of those would need adapting (and their
> documentation updating). There's fairly tight integration between Bodhi
> and Bugzilla, which would need to be redesigned. Those are just things
> I can think of off the top of my head. There are also a couple of
> decades worth of internet links to Fedora issues on RH Bugzilla, of
> course.

Those all sound like the things I'm familiar with.

> I guess the two big choices for Fedora if RH said "we're not
> maintaining Bugzilla any more" would be 1) take over maintaining
> Bugzilla or 2) switch to something else. 1) would probably be the path
> of least resistance, I guess.
> This does also kinda lead to a larger question for me, trying to wear
> both Red Hat and Fedora hats at the same time[0]. I wonder if we're
> kind of lacking a...mechanism, for want of a better word, to handle the
> *generic* case here. Let's rewind to Ye Olde Days, when "the Fedora
> project" first started. At that point Fedora and Red Hat shared a lot
> of tooling and infrastructure, and this was useful to both sides in
> many ways; it saves on development costs and it makes it easy for
> people to work in both worlds. With my Red Hat on, I think I'm allowed
> to say that internally we often talk about this being desirable -
> having consistency between how X is done in Fedora and how it's done
> for RHEL - and it obviously has benefits to Fedora too (it means we
> don't have to find the resources to do that same work at Fedora level).

Fedora and RHEL process and tooling has deviated significantly over
the years.  So much so that by the time I joined the RHEL team, it was
already very different.  That was almost 5 years ago, which means the
individual decisions that led to it were even earlier.  I don't really
want to revisit those decisions because I wasn't around and can't
speak to why they were made, but the connection between Fedora and
RHEL via bugzilla is minimal at best.

The commonality that brings the most shared benefit is the activities
of our communities, the quality and rigor they bring into Fedora, and
the sources we share.  Tooling and process are orthogonal to those.
Important, because they certainly lend themselves to aiding that
quality and rigor, but orthogonal.

> However, situations like this make me wonder if we might have an issue
> with keeping shared infra/tooling where it's desirable. It seems like
> this is a decision/conversation that's been happening within RH, about
> what makes sense for RH in terms of RHEL development. AFAIK this is the
> first time it's been formally talked about in a Fedora context, and the
> messaging is "RH has already decided to stop using Bugzilla for RHEL
> after 9". In other words, RH has decided on its own to move away from
> something that is part of the shared RH/Fedora "heritage way of doing
> things".

I don't think this is the first instance of those kinds of decisions.
It is perhaps one of the few instances where we've been up-front about
it well in advance.

> I'm not saying that's wrong, but as I said it does make me wonder
> whether, if both sides do find shared tooling/approaches beneficial, we

I think we do in some cases, and not in others.  I don't find bugzilla
to be one of those.  Even the clone function is questionable.

> might want to approach this kind of potential change differently in
> future. Otherwise it does seem like we could sort of gradually drift
> apart, with no explicit intention to do so, and lose the benefits of
> shared tooling and process. Unless the ultimate outcome of this is

>From my perspective, we minimally share tooling.  Process is common in
concepts, but RHEL process is far more additive and heavy-weight than
anything Fedora does.

> "Fedora adopts issues.redhat.com for bug tracking" - which would be a
> possibility, but doesn't seem like a certainty - the result will be

To be clear, it is not an explicit goal for Fedora to adopt
issues.redhat.com.  That's up to the Fedora project, much as adoption
of gitlab was.

> that we go from having a shared bug tracker, with the benefits of
> shared maintenance and being able to easily clone or reference bugs
> between Fedora and RHEL, to each maintaining our own bug tracker and
> not having those benefits.

Personally, I don't think we're taking advantage of perceived benefits
at all.  They are effectively separate.

> Of course, there would be sensitivities in developing such a process -
> it could look a lot like Red Hat telling Fedora how to do stuff, which
> I think isn't exactly the relationship we want to have. But at the same
> time I'm not sure "Red Hat or Fedora just deciding unilaterally to stop
> using this thing they'd previously both used" is always the best choice
> either.

The Fedora decision is up to Fedora.  RHEL is driven by product
constraints and will evaluate based on those needs.  Where we can
align, we should.  For a recent example of a decision to align more,
look at the CKI project.  Both Fedora and RHEL benefit greatly from
maintaining the kernel in the CKI manner, and this is the closest the
Fedora and RHEL kernels have ever been.