[CentOS] Btrfs going forward, was: Errors on an SSD drive

Changing the subject since this is rather Btrfs specific now.



On Fri, Aug 11, 2017 at 5:41 AM, hw <hw at gc-24.de> wrote:
> Chris Murphy wrote:
>>
>> On Wed, Aug 9, 2017, 11:55 AM Mark Haney <mark.haney at neonova.net> wrote:
>>
>>> To be honest, I'd not try a btrfs volume on a notebook SSD. I did that on
>>> a
>>> couple of systems and it corrupted pretty quickly. I'd stick with
>>> xfs/ext4
>>
>>
>> if you manage to get the drive working again.
>>>
>>>
>>
>> Sounds like a hardware problem. Btrfs is explicitly optimized for SSD, the
>> maintainers worked for FusionIO for several years of its development. If
>> the drive is silently corrupting data, Btrfs will pretty much immediately
>> start complaining where other filesystems will continue. Bad RAM can also
>> result in scary warnings where you don't with other filesytems. And I've
>> been using it in numerous SSDs for years and NVMe for a year with zero
>> problems.
>
>
> That´s one thing I´ve been wondering about:  When using btrfs RAID, do you
> need to somehow monitor the disks to see if one has failed?

Yes.

The block layer has no faulty device handling, i.e. it just reports
whatever problems the device or the controller report. Where md/mdadm
and md/LVM have implemented policies for ejecting (setting a device to
faulty) a block device. Btrfs does not do that, it'll just keep trying
to use a faulty device.

So you have to setup something that monitors for either physical
device errors, or btrfs errors or both, depending on what you want.




>
>> On CentOS though, I'd get newer btrfs-progs RPM from Fedora, and use
>> either
>> an elrepo.org kernel, a Fedora kernel, or build my own latest long-term
>> from kernel.org. There's just too much development that's happened since
>> the tree found in RHEL/CentOS kernels.
>
>
> I can´t go with a more recent kernel version before NVIDIA has updated their
> drivers to no longer need fence.h (or what it was).
>
> And I thought stuff gets backported, especially things as important as file
> systems.

There's 1500 to 3000 line changes to Btrfs code per kernel release.
There's too much to backport most of it. Serious fixes do get
backported by upstream to longterm kernels, but to what degree, you
have to check the upstream changelogs to know about it.

And right now most backports go to only 4.4 and 4.9. And I can't tell
you what kernel-3.10.0-514.10.2.el7.x86_64.rpm translates into, that
requires a secret decoder ring near as I can tell as it's a kernel
made from multiple branches,  and then also a bunch of separate
patches.



>> Also FWIW Red Hat is deprecating Btrfs, in the RHEL 7.4 announcement.
>> Support will be removed probably in RHEL 8. I have no idea how it'll
>> affect
>> CentOS kernels though. It will remain in Fedora kernels.
>
>
> That would suck badly to the point at which I´d have to look for yet another
> distribution.  The only one ramaining is arch.
>
> What do they suggest as a replacement?  The only other FS that comes close
> is
> ZFS, and removing btrfs alltogether would be taking living in the past too
> many
> steps too far.

Red Hat are working on a new user space wrapper and volume format
based on md, device mapper, LVM, and XFS.
http://stratis-storage.github.io/
https://stratis-storage.github.io/StratisSoftwareDesign.pdf

It's an aggressive development schedule and as so much of it is
journaling and CoW based I have no way to assess whether it ends up
with its own set of problems, not dissimilar to Btrfs. We'll just have
to see. But if there are underlying guts in the device-mapper that do
things better/faster/easier than Btrfs, the Btrfs devs have said they
can hook into device-mapper for these things to consolidate code base,
in particular for the multiple device handling. By its own vague time
table it will be years before it has "rough ZFS features" and again
estimating bootloader support, and to what degree other distros pick
up on it, it very well could end up being widely adopted, or it could
be a Red Hat only thing in practice.

Canonical appears to be charging ahead with OpenZFS included by
default out of the box (although not for rootfs yet I guess), and that
has an open ended and possibly long window before legal issues get
tested. But this is by far the most cross platform solution: FreeBSD,
Illumos, Linux, macOS. And ZoL has RHEL/CentOS specific packages.

But I can't tell you for sure what ZoL's faulty device behavior is
either, whether it ejects faulty or flaky devices and when, or if like
Btrfs is just tolerates it.

The elrepo.org folks can still sanely set CONFIG_BTRFS_FS=m, but I
suspect if RHEL unsets that in RHEL 8 kernels, that CentOS will do the
same.



-- 
Chris Murphy