How do you know when a Linux system has been compromised?
Every day I watch our systems with all the typical tools, ps, top, who, I watch firewall / IPS logs, I have logwatch setup and mailing daily summaries to me and I dive deeper into logs if something looks suspicious.
What am I missing or not looking at that you security gurus are looking at?
I subscribe to the centos and SANS newsletters, and I try to keep current on all technology with credible sources of articles online and with the Lynda library.
What other sources of information do you use to stay current about the latest threats and technology updates?
I appreciate the feedback.
Chris
Do you run rkhunter?
On 11/14/19 9:40 AM, Christopher Wensink wrote:
How do you know when a Linux system has been compromised?
Every day I watch our systems with all the typical tools, ps, top, who, I watch firewall / IPS logs, I have logwatch setup and mailing daily summaries to me and I dive deeper into logs if something looks suspicious.
What am I missing or not looking at that you security gurus are looking at?
I subscribe to the centos and SANS newsletters, and I try to keep current on all technology with credible sources of articles online and with the Lynda library.
What other sources of information do you use to stay current about the latest threats and technology updates?
I appreciate the feedback.
Chris
I have not, I'll look into that one, thanks!
On 11/14/2019 9:48 AM, SternData wrote:
Do you run rkhunter?
On 11/14/19 9:40 AM, Christopher Wensink wrote:
How do you know when a Linux system has been compromised?
Every day I watch our systems with all the typical tools, ps, top, who, I watch firewall / IPS logs, I have logwatch setup and mailing daily summaries to me and I dive deeper into logs if something looks suspicious.
What am I missing or not looking at that you security gurus are looking at?
I subscribe to the centos and SANS newsletters, and I try to keep current on all technology with credible sources of articles online and with the Lynda library.
What other sources of information do you use to stay current about the latest threats and technology updates?
I appreciate the feedback.
Chris
On 2019-11-14 10:01, Christopher Wensink wrote:
I have not, I'll look into that one, thanks!
On 11/14/2019 9:48 AM, SternData wrote:
Do you run rkhunter?
On 11/14/19 9:40 AM, Christopher Wensink wrote:
How do you know when a Linux system has been compromised?
I'm sure you have followed the procedure how to install system and services so everything is secure.
If, in a longer run no matter that you have system set up and configured securely and keep updating, if still the system gets compromised, then you need:
1. compromise warming 2. forensic investigation 3. recovery from compromise.
I figure your is about 1. You probably will not get detailed description of actual setup people on this list have. Information about what the defense is is the first step in every attack. The best you may get are the advises of what to look for.
One of the things you can set up is [host based, maybe] system integrity checking system (or intrusion detection system). That only makes sense on freshly installed system in known good state. There were a variety of these: tripwire (which went commercial), eics, ... If you search for linux intrusion detection system you should find what you need.
I hope, this helps.
Valeri
Every day I watch our systems with all the typical tools, ps, top, who, I watch firewall / IPS logs, I have logwatch setup and mailing daily summaries to me and I dive deeper into logs if something looks suspicious.
What am I missing or not looking at that you security gurus are looking at?
I subscribe to the centos and SANS newsletters, and I try to keep current on all technology with credible sources of articles online and with the Lynda library.
What other sources of information do you use to stay current about the latest threats and technology updates?
I appreciate the feedback.
Chris
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 14/11/2019 16:57, Valeri Galtsev wrote:
On 2019-11-14 10:01, Christopher Wensink wrote:
I have not, I'll look into that one, thanks!
On 11/14/2019 9:48 AM, SternData wrote:
Do you run rkhunter?
On 11/14/19 9:40 AM, Christopher Wensink wrote:
How do you know when a Linux system has been compromised?
I'm sure you have followed the procedure how to install system and services so everything is secure.
If, in a longer run no matter that you have system set up and configured securely and keep updating, if still the system gets compromised, then you need:
- compromise warming
- forensic investigation
- recovery from compromise.
I figure your is about 1. You probably will not get detailed description of actual setup people on this list have. Information about what the defense is is the first step in every attack. The best you may get are the advises of what to look for.
One of the things you can set up is [host based, maybe] system integrity checking system (or intrusion detection system). That only makes sense on freshly installed system in known good state. There were a variety of these: tripwire (which went commercial), eics, ... If you search for linux intrusion detection system you should find what you need.
I hope, this helps.
Valeri
I would add Trusted Path Execution (TPE) to any sysdamin's toolbox who cares about security. It's easy to install from elrepo.org (kmod-tpe). I wrote an overview (below) so won't repeat myself here, but I would strongly encourage people to try it out:
http://lists.elrepo.org/pipermail/elrepo/2017-June/003620.html
This is one where there's probably no limit to what you could do. We have a high-security environment and are using Aide and OSSEC.
Aide has been good at reporting file system changes and is very granular, the dilemma is what to monitor and what to ignore (keep from being inundated with reports of innocuous changes at the risk of missing something). However, it is not daemon-based so changes between runs which are undone go unnoticed. Also, somehow you need to protect the executable and configuration file so that an attacker can't replace the executable or read the configuration and find a way around it. The executable could be placed on mounted read-only media, last time I checked Netac and Kanguru still made USB sticks with write-protect switches. Our best effort for protecting configuration is to deliver the configuration file just-in-time and delete it after the scheduled run, not a great solution, anybody have a better idea?
OSSEC is daemon-based and centrally-managed. It is a HIDS rather than just a FIMS as Aide is. Its log monitoring has surfaced operational issues in addition to security ones (Postfix got in an odd state and had to be restarted for example). Unfortunately, false positives are common, especially if you use the "detect new files" feature. They admit that dealing with software updates is problematic.
I've used auditd to trace down what ended up being a funny situation, Aide detected that /etc/hosts.deny would change timestamp but nothing else, turns out OSSEC has an active response feature to block attacks which involves updating that file to block a host for 10 minutes.
You could also look into inotify options and Samhain is another HIDS (I'd love to hear about anyone's experience with it). A free variant of tripwire may still exist but is probably unsupported and Aide is a clone of it.
I noticed that rootkit detection has also been mentioned in another reply. ________________________________ From: CentOS centos-bounces@centos.org on behalf of Christopher Wensink cwensink@five-star-plastics.com Sent: Thursday, November 14, 2019 9:40 AM To: CentOS mailing list centos@centos.org Subject: [EXTERNAL] [CentOS] how to know when a system is compromised
How do you know when a Linux system has been compromised?
Every day I watch our systems with all the typical tools, ps, top, who, I watch firewall / IPS logs, I have logwatch setup and mailing daily summaries to me and I dive deeper into logs if something looks suspicious.
What am I missing or not looking at that you security gurus are looking at?
I subscribe to the centos and SANS newsletters, and I try to keep current on all technology with credible sources of articles online and with the Lynda library.
What other sources of information do you use to stay current about the latest threats and technology updates?
I appreciate the feedback.
Chris
_______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Harriscomputer
Leroy Tennison Network Information/Cyber Security Specialist E: leroy@datavoiceint.com
[cid:Data-Voice-International-LOGO_aa3d1c6e-5cfb-451f-ba2c-af8059e69609.PNG]
2220 Bush Dr McKinney, Texas 75070 www.datavoiceint.comhttp://www..com
This message has been sent on behalf of a company that is part of the Harris Operating Group of Constellation Software Inc.
If you prefer not to be contacted by Harris Operating Group please notify ushttp://subscribe.harriscomputer.com/.
This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message.
Once upon a time, Leroy Tennison leroy@datavoiceint.com said:
The executable could be placed on mounted read-only media
That's not as secure as you think. Linux bind mounts can mount a file over another file (plus there's overlay filesystems), so it's possible to replace a binary even on a read-only device.
<sigh> Thanks - I'll keep that in mind... ________________________________ From: CentOS centos-bounces@centos.org on behalf of Chris Adams linux@cmadams.net Sent: Thursday, November 14, 2019 10:57 AM To: centos@centos.org centos@centos.org Subject: [EXTERNAL] Re: [CentOS] how to know when a system is compromised
Once upon a time, Leroy Tennison leroy@datavoiceint.com said:
The executable could be placed on mounted read-only media
That's not as secure as you think. Linux bind mounts can mount a file over another file (plus there's overlay filesystems), so it's possible to replace a binary even on a read-only device.
-- Chris Adams linux@cmadams.net _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Harriscomputer
Leroy Tennison Network Information/Cyber Security Specialist E: leroy@datavoiceint.com
[cid:Data-Voice-International-LOGO_aa3d1c6e-5cfb-451f-ba2c-af8059e69609.PNG]
2220 Bush Dr McKinney, Texas 75070 www.datavoiceint.comhttp://www..com
This message has been sent on behalf of a company that is part of the Harris Operating Group of Constellation Software Inc.
If you prefer not to be contacted by Harris Operating Group please notify ushttp://subscribe.harriscomputer.com/.
This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message.
Actually, a defense here is to umount the path then remount it as a part of running the Aide script. There may be an end-run to this as well- security is a never-ending battle.
________________________________ From: CentOS centos-bounces@centos.org on behalf of Leroy Tennison leroy@datavoiceint.com Sent: Thursday, November 14, 2019 1:20 PM To: CentOS mailing list centos@centos.org Subject: Re: [CentOS] how to know when a system is compromised
<sigh> Thanks - I'll keep that in mind...
Harriscomputer
Leroy Tennison Network Information/Cyber Security Specialist E: leroy@datavoiceint.com
[cid:Data-Voice-International-LOGO_aa3d1c6e-5cfb-451f-ba2c-af8059e69609.PNG]
2220 Bush Dr McKinney, Texas 75070 www.datavoiceint.comhttp://www..com
This message has been sent on behalf of a company that is part of the Harris Operating Group of Constellation Software Inc.
If you prefer not to be contacted by Harris Operating Group please notify ushttp://subscribe.harriscomputer.com/.
This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message.
________________________________ From: CentOS centos-bounces@centos.org on behalf of Chris Adams linux@cmadams.net Sent: Thursday, November 14, 2019 10:57 AM To: centos@centos.org centos@centos.org Subject: [EXTERNAL] Re: [CentOS] how to know when a system is compromised
Once upon a time, Leroy Tennison leroy@datavoiceint.com said:
The executable could be placed on mounted read-only media
That's not as secure as you think. Linux bind mounts can mount a file over another file (plus there's overlay filesystems), so it's possible to replace a binary even on a read-only device.
-- Chris Adams linux@cmadams.net _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Harriscomputer
Leroy Tennison Network Information/Cyber Security Specialist E: leroy@datavoiceint.com
[cid:Data-Voice-International-LOGO_aa3d1c6e-5cfb-451f-ba2c-af8059e69609.PNG]
2220 Bush Dr McKinney, Texas 75070 https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fwww.datavoiceint.com&...http://www..com
This message has been sent on behalf of a company that is part of the Harris Operating Group of Constellation Software Inc.
If you prefer not to be contacted by Harris Operating Group please notify ushttps://linkprotect.cudasvc.com/url?a=http%3a%2f%2fsubscribe.harriscomputer.com%2f&c=E,1,bJ-3jUtOeY3WPfKHckYn-Ynl3cYkeINegX0H-YsrIDlgsWb1g8GzM6JCS3rmWWxVwOPgOf_AMxvsKjsW_iVVobRWFKpTzsvz4Bfhlu5s&typo=1.
This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged or confidential or otherwise legally exempt from disclosure. If you are not the named addressee, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message.
_______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
-
Chris Adams -
Christopher Wensink -
Leroy Tennison -
Phil Perry -
SternData -
Valeri Galtsev