I have a tftpd server S running on centos 7 and managed by systemd
It is not respoding to A server which is sending the tftp read request RRQ.
I do see the RRQ packets coming from A to S, but S never responds back from a different port Y to A
So this part is working fine
https://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol#/media/File:Tft...
But I do not see any attempts to even send a data packet back in my packet capture running on S
So this event is not occuring, as if my tftpd server is dead. I have the firewalld turned off on S to eliminate the possibility that firewalld blocking those packets from reeaching to tftpd daemon. I also turned off selinux to eliminate any permission issue.
https://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol#/media/File:Tft...
I do have tftpd running and managed by systemd
$ systemctl status -l tftp ● tftp.service - Tftp Server Loaded: loaded (/etc/systemd/system/tftp.service; indirect; vendor preset: disabled) Active: active (running) since Wed 2018-03-28 18:57:42 UTC; 1min 44s ago Docs: man:in.tftpd Main PID: 1685 (in.tftpd) Memory: 136.0K CGroup: /system.slice/tftp.service └─1685 /usr/sbin/in.tftpd --verbose --verbosity 10 --secure /tftpboot --port-range 4069:4169
Mar 28 18:57:42 S.example.net systemd[1]: Started Tftp Server. Mar 28 18:57:42 S.example.net systemd[1]: Starting Tftp Server...
Any help is appreciated!
On Wed, Mar 28, 2018 at 3:16 PM Asif Iqbal vadud3@gmail.com wrote:
It is not respoding to A server which is sending the tftp read request RRQ.
I do see the RRQ packets coming from A to S, but S never responds back from a different port Y to A
So this part is working fine
https://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol#/media/File:Tft...
But I do not see any attempts to even send a data packet back in my packet capture running on S
Are A and S on different IP subnets? Does S have a second IP on the SAME subnet as A? Any ASA or other firewalls between the two? If so this is expected behavior.
On Wed, Mar 28, 2018 at 6:25 PM, Steven Tardy sjt5atra@gmail.com wrote:
On Wed, Mar 28, 2018 at 3:16 PM Asif Iqbal vadud3@gmail.com wrote:
It is not respoding to A server which is sending the tftp read request
RRQ.
I do see the RRQ packets coming from A to S, but S never responds back
from
a different port Y to A
So this part is working fine
Protocol#/media/File:Tftp-rrq.svg
But I do not see any attempts to even send a data packet back in my
packet
capture running on S
Are A and S on different IP subnets?
Yes
Does S have a second IP on the SAME subnet as A?
No
Any ASA or other firewalls between the two?
Firewall is set to any any between the two. Also internal firewall is down Firewall is not seeing any return pkts
If so this is expected behavior.
I was hoping S will at least try to reply to the RRQ pkt with a DATA pkt I do not see S is even bothering to try.
A(x) ---- RRQ ---> S(69) and then I am expecting this S(y) --- DAT 1 --> A(x)
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On Wed, Mar 28, 2018 at 9:15 PM, Asif Iqbal vadud3@gmail.com wrote:
On Wed, Mar 28, 2018 at 6:25 PM, Steven Tardy sjt5atra@gmail.com wrote:
On Wed, Mar 28, 2018 at 3:16 PM Asif Iqbal vadud3@gmail.com wrote:
It is not respoding to A server which is sending the tftp read request
RRQ.
I do see the RRQ packets coming from A to S, but S never responds back
from
a different port Y to A
So this part is working fine
https://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol
#/media/File:Tftp-rrq.svg
But I do not see any attempts to even send a data packet back in my
packet
capture running on S
Are A and S on different IP subnets?
Yes
Does S have a second IP on the SAME subnet as A?
No
Any ASA or other firewalls between the two?
Firewall is set to any any between the two. Also internal firewall is down Firewall is not seeing any return pkts
If so this is expected behavior.
I was hoping S will at least try to reply to the RRQ pkt with a DATA pkt I do not see S is even bothering to try.
A(x) ---- RRQ ---> S(69) and then I am expecting this S(y) --- DAT 1 --> A(x)
BTW, If I reverse the role and have S try to send a tftp read request, A reply back right away and I do the see the file.
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?
A STATEFUL firewall with “ip any any” can and will still block asymmetric communications due to the firewall keeping track of state (hence tha name stateful firewall).
Tcpdump on your servers /other/ NICs and you’ll see the tftp traffic leaving your server on some other NIC (probably on with the default route).
The upstream firewall will then block the tftp response if it never saw the tftp request (due to asymmetry).
On Thu, Mar 29, 2018 at 7:21 AM, Steven Tardy sjt5atra@gmail.com wrote:
A STATEFUL firewall with “ip any any” can and will still block asymmetric communications due to the firewall keeping track of state (hence tha name stateful firewall).
Tcpdump on your servers /other/ NICs and you’ll see the tftp traffic leaving your server on some other NIC (probably on with the default route).
A (192.168.1.10) S (192.168.1.20)
I do not see tftp traffic is leaving from S
A:~$ tftp (to) 192.168.1.20 tftp> get file Transfer timed out.
As you can see no pkt is leaving. If it were leaving S, but A were not receiving then I would think firewall is dropping it.
[ S ~]$ sudo tcpdump -A -nniany host 192.168.1.10 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
16:40:08.390939 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,J1@.>..n./...oAt...E..#...file.netascii................... 16:40:13.391133 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,N.@.>..../...oAt...E..#...file.netascii................... 16:40:18.391220 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,QK@.>..T./...oAt...E..#...file.netascii................... 16:40:23.391373 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,T^@.>..@./...oAt...E..#...file.netascii................... 16:40:28.391469 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,X.@.>..../...oAt...E..#...file.netascii...................
The upstream firewall will then block the tftp response if it never saw the tftp request (due to asymmetry). _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On Thu, Mar 29, 2018 at 12:48 PM, Asif Iqbal vadud3@gmail.com wrote:
On Thu, Mar 29, 2018 at 7:21 AM, Steven Tardy sjt5atra@gmail.com wrote:
A STATEFUL firewall with “ip any any” can and will still block asymmetric communications due to the firewall keeping track of state (hence tha name stateful firewall).
Tcpdump on your servers /other/ NICs and you’ll see the tftp traffic leaving your server on some other NIC (probably on with the default route).
A (192.168.1.10) S (192.168.1.20)
I do not see tftp traffic is leaving from S
A:~$ tftp (to) 192.168.1.20 tftp> get file Transfer timed out.
As you can see no pkt is leaving. If it were leaving S, but A were not receiving then I would think firewall is dropping it.
[ S ~]$ sudo tcpdump -A -nniany host 192.168.1.10 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
16:40:08.390939 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,J1@.>..n./...oAt...E..#...file.netascii................... 16:40:13.391133 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,N.@.>..../...oAt...E..#...file.netascii................... 16:40:18.391220 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,QK@.>..T./...oAt...E..#...file.netascii................... 16:40:23.391373 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,T^@.>..@./...oAt...E..#...file.netascii................... 16:40:28.391469 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,X.@.>..../...oAt...E..#...file.netascii...................
I still like some help on this
The upstream firewall will then block the tftp response if it never saw the tftp request (due to asymmetry). _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
have you checked that tftp is added to hosts.allow. syslog may be reporting libwrap errors, libwrap is trcpwrappers regards peter
On 11 April 2018 16:57:04 "Asif Iqbal" vadud3@gmail.com wrote:
On Thu, Mar 29, 2018 at 12:48 PM, Asif Iqbal vadud3@gmail.com wrote:
On Thu, Mar 29, 2018 at 7:21 AM, Steven Tardy sjt5atra@gmail.com wrote:
A STATEFUL firewall with “ip any any” can and will still block asymmetric communications due to the firewall keeping track of state (hence tha name stateful firewall).
Tcpdump on your servers /other/ NICs and you’ll see the tftp traffic leaving your server on some other NIC (probably on with the default route).
A (192.168.1.10) S (192.168.1.20)
I do not see tftp traffic is leaving from S
A:~$ tftp (to) 192.168.1.20 tftp> get file Transfer timed out.
As you can see no pkt is leaving. If it were leaving S, but A were not receiving then I would think firewall is dropping it.
[ S ~]$ sudo tcpdump -A -nniany host 192.168.1.10 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
16:40:08.390939 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,J1@.>..n./...oAt...E..#...file.netascii................... 16:40:13.391133 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,N.@.>..../...oAt...E..#...file.netascii................... 16:40:18.391220 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,QK@.>..T./...oAt...E..#...file.netascii................... 16:40:23.391373 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,T^@.>..@./...oAt...E..#...file.netascii................... 16:40:28.391469 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,X.@.>..../...oAt...E..#...file.netascii...................
I still like some help on this
The upstream firewall will then block the tftp response if it never saw the tftp request (due to asymmetry). _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Sent with AquaMail for Android https://www.mobisystems.com/aqua-mail
On Thu, Apr 12, 2018 at 2:25 AM, peter.winterflood < peter.winterflood@ossi.co.uk> wrote:
have you checked that tftp is added to hosts.allow. syslog may be reporting libwrap errors, libwrap is trcpwrappers regards peter
yes hosts.allow is wide open and I did test with tcpdmatch and it says granted
On 11 April 2018 16:57:04 "Asif Iqbal" vadud3@gmail.com wrote:
On Thu, Mar 29, 2018 at 12:48 PM, Asif Iqbal vadud3@gmail.com wrote:
On Thu, Mar 29, 2018 at 7:21 AM, Steven Tardy sjt5atra@gmail.com
wrote:
A STATEFUL firewall with “ip any any” can and will still block
asymmetric
communications due to the firewall keeping track of state (hence tha
name
stateful firewall).
Tcpdump on your servers /other/ NICs and you’ll see the tftp traffic leaving your server on some other NIC (probably on with the default route).
A (192.168.1.10) S (192.168.1.20)
I do not see tftp traffic is leaving from S
A:~$ tftp (to) 192.168.1.20 tftp> get file Transfer timed out.
As you can see no pkt is leaving. If it were leaving S, but A were not receiving then I would think firewall is dropping it.
[ S ~]$ sudo tcpdump -A -nniany host 192.168.1.10 tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size
262144
bytes
16:40:08.390939 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,J1@.>..n./...oAt...E..#...file.netascii................... 16:40:13.391133 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,N.@.>..../...oAt...E..#...file.netascii................... 16:40:18.391220 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,QK@.>..T./...oAt...E..#...file.netascii................... 16:40:23.391373 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,T^@.>..@./...oAt...E..#...file.netascii................... 16:40:28.391469 IP 192.168.1.10.35553 > 192.168.1.20.69: 16 RRQ "file" netascii E..,X.@.>..../...oAt...E..#...file.netascii...................
I still like some help on this
The upstream firewall will then block the tftp response if it never saw the tftp request (due to asymmetry). _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
-- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Sent with AquaMail for Android https://www.mobisystems.com/aqua-mail
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On Thu, Mar 29, 2018 at 12:48:15PM -0400, Asif Iqbal wrote:
I do not see tftp traffic is leaving from S
A:~$ tftp (to) 192.168.1.20 tftp> get file Transfer timed out.
As you can see no pkt is leaving. If it were leaving S, but A were not receiving then I would think firewall is dropping it.
[ S ~]$ sudo tcpdump -A -nniany host 192.168.1.10 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
Most likely the firewall on the system running your tftp client is blocking the traffic from the tftp server. The easiest way to test would be to put in a rule that allows all packets from the server (or to at least log them so you can see what's happening). The firewall issue is most likely *not* with the tftp server.
Reading back through prior emails. . . TFTP client requests packets *are* making it to the TFTP server. So it seems like something on the TFTP server itself.
Like previously mentioned server side firewall/iptables/tcp-wrappers/selinux are all possible culprits.
Hmmm just thought of something else, what are the file permissions of the file you are requesting? Try `chmod a+r filename`?
On Thu, Apr 12, 2018 at 9:26 AM, Steven Tardy sjt5atra@gmail.com wrote:
Reading back through prior emails. . . TFTP client requests packets *are* making it to the TFTP server. So it seems like something on the TFTP server itself.
Right. I am not sure how to debug that
Like previously mentioned server side firewall/iptables/tcp-wrappers/selinux are all possible culprits.
I tested with firewalld turned off and selinux all permissive. I also did not see any denied in audit log related to this when selinux was enforced
Hmmm just thought of something else, what are the file permissions of the file you are requesting? Try `chmod a+r filename`?
Yes it is readable.
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On Wed, 2018-04-18 at 20:52 -0400, Asif Iqbal wrote:
On Thu, Apr 12, 2018 at 9:26 AM, Steven Tardy sjt5atra@gmail.com wrote:
Reading back through prior emails. . . TFTP client requests packets *are* making it to the TFTP server. So it seems like something on the TFTP server itself.
Right. I am not sure how to debug that
Just reading back through the thread, I'm still not sure, but does the server have multiple ethernet interfaces? If so, can you turn off the others temporarily?
Is it possible that IPv6 is getting in the way?
If you do
lsof -i :69
what do you get?
Like previously mentioned server side firewall/iptables/tcp-wrappers/selinux are all possible culprits.
I tested with firewalld turned off and selinux all permissive. I also did not see any denied in audit log related to this when selinux was enforced
Hmmm just thought of something else, what are the file permissions of the file you are requesting? Try `chmod a+r filename`?
Yes it is readable.
What about all the directories above the file - are they readable and searchable?
P.
On Wed, Apr 18, 2018 at 08:52:32PM -0400, Asif Iqbal wrote:
I tested with firewalld turned off and selinux all permissive. I also did not see any denied in audit log related to this when selinux was enforced
Have you checked the *client* firewall? TFTP responses to client requests are blocked by the default firewall, due to the nature of the TFTP protocol.
Early in this thread you mentioned these are on different network subnets. . .
Just thought about a similar issue. . . sysctl -a | grep rp_filter
If a packet comes in to Linux and the path BACK to the remote IP is NOT out that same interface (asymmetric routing) the Linux kernel will drop the packet. “rp_filter” controls how Linux behaves regarding this.
Please provide real `ifconfig` and `route -rn` and `tcpdump port 69` output to properly diagnose. . .
-
Asif Iqbal -
Jonathan Billings -
Pete Biggs -
peter.winterflood -
Steven Tardy