Hi,
Just installed CentOS 7 that serves a home dir automounted over nfs. SELinux is disabled. If I go to the client (oldish version of Fedora) doing su - username works fine and the nfs export is mounted and I can see all files and everything seems well. But trying to actually login to the desktop from the client machine does not work. It starts to login but then just hangs with a black screen. Trying to just launch libreoffice --writer from a terminal as the nfs mounted user also hangs on the spash screen. If I then go to the server and 'systemctl stop firewalld', the desktop instantly logs in fine and libreoffice works from the term.
So firewalld is blocking something that the Fedora desktop needs. What is it? What services do I need to add to firewalld?
Thanks,
Mike
On Fri, 20 Nov 2020 11:05:25 -0500 Michael B Allen wrote:
So firewalld is blocking something that the Fedora desktop needs. What is it? What services do I need to add to firewalls?
https://www.cyberciti.biz/faq/enable-firewalld-logging-for-denied-packets-on...
On Fri, Nov 20, 2020 at 11:19 AM Frank Cox theatre@sasktel.net wrote:
So firewalld is blocking something that the Fedora desktop needs. What is it? What services do I need to add to firewalls?
https://www.cyberciti.biz/faq/enable-firewalld-logging-for-denied-packets-on...
Hi Frank,
Thanks for that tip. Here's what I get:
Nov 20 12:03:15 goose kernel: FINAL_REJECT: IN=enp4s0 OUT= MAC=c8:1snip8:00 SRC=192.168.1.46 DST=192.168.1.14 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=48746 DF PROTO=TCP SPT=760 DPT=41285 WINDOW=29200 RES=0x00 SYN URGP=0 Nov 20 12:03:18 goose kernel: FINAL_REJECT: IN=enp4s0 OUT= MAC=c8:1snip8:00 SRC=192.168.1.46 DST=192.168.1.14 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=55190 DF PROTO=TCP SPT=760 DPT=41285 WINDOW=29200 RES=0x00 SYN URGP=0 Nov 20 12:03:21 goose kernel: FINAL_REJECT: IN=enp4s0 OUT= MAC=c8:1snip8:00 SRC=192.168.1.46 DST=192.168.1.14 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=31389 DF PROTO=TCP SPT=760 DPT=41285 WINDOW=29200 RES=0x00 SYN URGP=0 Nov 20 12:03:24 goose kernel: FINAL_REJECT: IN=enp4s0 OUT= MAC=c8:1snip8:00 SRC=192.168.1.46 DST=192.168.1.14 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=21119 DF PROTO=TCP SPT=760 DPT=41285 WINDOW=29200 RES=0x00 SYN URGP=0 Nov 20 12:03:26 goose kernel: FINAL_REJECT: IN=enp4s0 OUT= MAC=c8:1snip8:00 SRC=192.168.1.46 DST=192.168.1.14 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=63694 DF PROTO=TCP SPT=760 DPT=41285 WINDOW=29200 RES=0x00 SYN URGP=0
So TCP src 760 to 41285. What's that?
Mike
On Fri, 20 Nov 2020 12:07:40 -0500 Michael B Allen wrote:
So TCP src 760 to 41285. What's that?
Apparently "that" is what you need to allow in order for your desktop to work.
What it is actually doing, I'm not sure. Google tells me that port 760 has something to do with Kerberos registration.
On Fri, Nov 20, 2020 at 12:18 PM Frank Cox theatre@sasktel.net wrote:
On Fri, 20 Nov 2020 12:07:40 -0500 Michael B Allen wrote:
So TCP src 760 to 41285. What's that?
Apparently "that" is what you need to allow in order for your desktop to work.
What it is actually doing, I'm not sure. Google tells me that port 760 has something to do with Kerberos registration.
Apparently I don't know how to do "that" because this:
# iptables -A INPUT -p tcp --sport 760 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
still doesn't allow the traffic through (not that I would want to allow an --sport rule anyway but I'd just like to confirm that this traffic is indeed responsible). What am I doing wrong here? I've also tried simpler rules without conntrack or cstate but it's still not getting through.
Incidentally I added kerberos and kadmin firewalld services without effect either.
Mike
On Fri, Nov 20, 2020 at 2:06 PM Michael B Allen ioplex@gmail.com wrote:
Apparently I don't know how to do "that" because this:
# iptables -A INPUT -p tcp --sport 760 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
still doesn't allow the traffic through (not that I would want to allow an --sport rule anyway but I'd just like to confirm that this traffic is indeed responsible). What am I doing wrong here? I've also tried simpler rules without conntrack or cstate but it's still not getting through.
Incidentally I added kerberos and kadmin firewalld services without effect either.
Well I've managed to resolve the issue but I'm not entirely satisfied with the solution. Apparently firewalld and iptables are at least partially mutually exclusive such that changes to iptable have no effect. If I add a Source Port rule using the Firewalld GUI to allow source port 760, it resolves the issue. But it seems pretty dubious to allow traffic from any particular source port. The service using port 760 is krbupdate but there isn't a lot of information about it on the net. It doesn't look like destination ports are a range because they have changed from 41285 and 46167. There must be something on the CentOS 7 side broadcasting info about what ports to use. What a PITA. I can't log into a desktop with an nfs home dir without punching a reverse hole in my firewall? That shouldn't be. 99% of people will just drop the pants on their machine.
Mike
On 11/20/20 11:31 AM, Michael B Allen wrote:
I can't log into a desktop with an nfs home dir without punching a reverse hole in my firewall? That shouldn't be.
I'm pretty sure your client is using NFSv3, and the ports you need opened are for RPC, and they *are* dynamic (so the next time these systems reboot, you'd probably need to open different ports, or the correct range of ports).
The easiest solution would be to use NFSv4, and not to adjust your firewall at all.
On 11/20/20 2:31 PM, Michael B Allen wrote:
On Fri, Nov 20, 2020 at 2:06 PM Michael B Allen ioplex@gmail.com wrote:
Apparently I don't know how to do "that" because this:
# iptables -A INPUT -p tcp --sport 760 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
still doesn't allow the traffic through (not that I would want to allow an --sport rule anyway but I'd just like to confirm that this traffic is indeed responsible). What am I doing wrong here? I've also tried simpler rules without conntrack or cstate but it's still not getting through.
Incidentally I added kerberos and kadmin firewalld services without effect either.
Well I've managed to resolve the issue but I'm not entirely satisfied with the solution. Apparently firewalld and iptables are at least partially mutually exclusive such that changes to iptable have no effect. If I add a Source Port rule using the Firewalld GUI to allow source port 760, it resolves the issue. But it seems pretty dubious to allow traffic from any particular source port. The service using port 760 is krbupdate but there isn't a lot of information about it on the net. It doesn't look like destination ports are a range because they have changed from 41285 and 46167. There must be something on the CentOS 7 side broadcasting info about what ports to use. What a PITA. I can't log into a desktop with an nfs home dir without punching a reverse hole in my firewall? That shouldn't be. 99% of people will just drop the pants on their machine.
Mike
You didn't state what version of NFS you're using. We're still on nfsv3. What you're describing looks like an issue with locked.
Curious: Try giving the login ~10 minutes to see if something 'gives up.'
On the nfs server: rpcinfo -p
Look at nlockmgr ports & protocols. My hunch is your dst ports reported are listed.
On CentOS 7 & 8, I lock down ports on my clients and server using /etc/nfs.conf (c8) or /etc/sysconfig/nfs (c7). I used random high numbers, pick your own to taste:
$ egrep -v '^($|#)' /etc/nfs.conf [general] [exportfs] [gssd] use-gss-proxy=1 [lockd] port = 43090 udp-port = 43090 [mountd] port = 43091 [nfsdcltrack] [nfsd] [statd] port = 43092 [sm-notify]
On the server and clients, I allow those corresponding ports.
I believe on centos 7 I used /etc/modprobe.d/lockd.conf to use something like:
options lockd nlm_udpport=43094 nlm_tcpport=43094
and
# cat /etc/sysconfig/nfs LOCKD_TCPPORT=43090 LOCKD_UDPPORT=43090 MOUNTD_PORT=43091 STATD_PORT=43092 RQUOTAD_PORT=43093
Hope that helps!
On Fri, Nov 20, 2020 at 2:52 PM Chris Schanzle chris.schanzle@nist.gov wrote:
On 11/20/20 2:31 PM, Michael B Allen wrote:
On Fri, Nov 20, 2020 at 2:06 PM Michael B Allen ioplex@gmail.com wrote:
Apparently I don't know how to do "that" because this:
# iptables -A INPUT -p tcp --sport 760 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
still doesn't allow the traffic through (not that I would want to allow an --sport rule anyway but I'd just like to confirm that this traffic is indeed responsible). What am I doing wrong here? I've also tried simpler rules without conntrack or cstate but it's still not getting through.
Incidentally I added kerberos and kadmin firewalld services without effect either.
Well I've managed to resolve the issue but I'm not entirely satisfied with the solution. Apparently firewalld and iptables are at least partially mutually exclusive such that changes to iptable have no effect. If I add a Source Port rule using the Firewalld GUI to allow source port 760, it resolves the issue. But it seems pretty dubious to allow traffic from any particular source port. The service using port 760 is krbupdate but there isn't a lot of information about it on the net. It doesn't look like destination ports are a range because they have changed from 41285 and 46167. There must be something on the CentOS 7 side broadcasting info about what ports to use. What a PITA. I can't log into a desktop with an nfs home dir without punching a reverse hole in my firewall? That shouldn't be. 99% of people will just drop the pants on their machine.
Mike
You didn't state what version of NFS you're using. We're still on nfsv3. What you're describing looks like an issue with locked.
Thanks for the inputs but my problem has nothing to do with NFS.
Mike
On 11/20/20 1:26 PM, Michael B Allen wrote:
Thanks for the inputs but my problem has nothing to do with NFS.
Do you think that because you saw "krbupdate" in /etc/services?
The problem you've described is definitely an NFSv3 problem. The connections causing the client to hang are portmap connections. They're dynamic, and don't necessarily conform to /etc/services.
The lesson to learn, here, is that /etc/services maps names to numbers, but it does NOT map numbers to names. Port numbers aren't reserved simply because there is a mapping to them in /etc/services.
On Fri, Nov 20, 2020 at 6:37 PM Gordon Messmer gordon.messmer@gmail.com wrote:
On 11/20/20 1:26 PM, Michael B Allen wrote:
Thanks for the inputs but my problem has nothing to do with NFS.
Do you think that because you saw "krbupdate" in /etc/services?
The problem you've described is definitely an NFSv3 problem. The connections causing the client to hang are portmap connections. They're dynamic, and don't necessarily conform to /etc/services.
The lesson to learn, here, is that /etc/services maps names to numbers, but it does NOT map numbers to names. Port numbers aren't reserved simply because there is a mapping to them in /etc/services.
Hi Gordon,
You're right! My mistake. I removed the Source Port rule and did the following instead:
# firewall-cmd --add-service=nfs3 --permanent # firewall-cmd --reload
This fixed the hanging issue (and probably other stuff I haven't run into yet).
So even though NFS worked fine just doing the usual file related ops in a terminal, apparently my client is old enough that it's still doing NFSv3 whereas CentOS 7 has moved on to NFSv4 and that incompatibility was responsible for the desktop / libreoffice hanging issue.
Much thanks.
Mike
On Nov 20, 2020, at 14:31, Michael B Allen ioplex@gmail.com wrote:
Well I've managed to resolve the issue but I'm not entirely satisfied with the solution. Apparently firewalld and iptables are at least partially mutually exclusive such that changes to iptable have no effect.
That’s not strictly true, at least with firewalld and iptables. You added the iptables rule with -A (append). The firewalld rules add jump rules to the input table and your rule simply was never reached, because traffic was blocked in one of the earlier rules. This would be the case in any complex iptables config too. Had you really wanted to test something with iptables, use -I (insert) which puts it at the front of the rules. Obviously, the best thing to do is to use firewalld tools with firewalld.
-- Jonathan Billings billings@negate.org
On Sun, Nov 22, 2020 at 7:34 AM Jonathan Billings billings@negate.org wrote:
On Nov 20, 2020, at 14:31, Michael B Allen ioplex@gmail.com wrote:
Well I've managed to resolve the issue but I'm not entirely satisfied with the solution. Apparently firewalld and iptables are at least partially mutually exclusive such that changes to iptable have no effect.
That’s not strictly true, at least with firewalld and iptables. You added the iptables rule with -A (append). The firewalld rules add jump rules to the input table and your rule simply was never reached, because traffic was blocked in one of the earlier rules. This would be the case in any complex iptables config too. Had you really wanted to test something with iptables, use -I (insert) which puts it at the front of the rules. Obviously, the best thing to do is to use firewalld tools with firewalld.
Ah, very interesting. Despite using linux for as long as I have I don't recall ever realizing that. Very good to know.
Thanks, Mike
-
Chris Schanzle -
Frank Cox -
Gordon Messmer -
Jonathan Billings -
Michael B Allen