Everyone,
This morning I received a notice from PayPal that one of our sites got hacked and was spoofing a PayPal web site.
When I checked the the site, I was surprised to find they were correct. About 5 days a go we had a server that got hacked and somehow the file paypal.com.tar got uploaded to our server and then stored in a a subdirectory of /var/www/.
I had previously started a mysqld server and planned on using it for web authorizations. I had not been able to work on it, but left it in place. I looked like the hacker downloaded his paypal spoof files into a subdirectory of /var/www/phpmyadmin.
I am running 5.3 with all current updates.
I do not have telnet or ftp active on this server, and have password authentication of sshd turned off.
I have tried to obtain dialog with PayPal about this but they have not responded to my queries. If any of you have had some experience with this I would be interested in knowing how this may have happened. I have shutdown the mysqld server as well as removed access in httpd.conf of the /var/www/phpmyadmin directory in order to shutdown the spoofing site.
If any of you have a leg up on this I would appreciate your help.
Greg Ennis P.S. I found the following entry in my error_log of /var/log/httpd/ :
[Sun Aug 16 04:26:19 2009] [info] Server built: Jul 14 2009 06:02:39 --00:21:14-- http://code.go.ro/paypal.com.tar Resolving code.go.ro... 81.196.20.134 Connecting to code.go.ro|81.196.20.134|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 645120 (630K) [application/x-tar] Saving to: `paypal.com.tar'
0K .......... .......... .......... .......... .......... 7% 70.0K 8s 50K .......... .......... .......... .......... .......... 15% 265K 5s 100K .......... .......... .......... .......... .......... 23% 284K 3s 150K .......... .......... .......... .......... .......... 31% 1.81M 2s 200K .......... .......... .......... .......... .......... 39% 1.79M 2s 250K .......... .......... .......... .......... .......... 47% 323K 1s 300K .......... .......... .......... .......... .......... 55% 1.80M 1s 350K .......... .......... .......... .......... .......... 63% 1.76M 1s 400K .......... .......... .......... .......... .......... 71% 431K 1s 450K .......... .......... .......... .......... .......... 79% 1.77M 0s 500K .......... .......... .......... .......... .......... 87% 1.75M 0s 550K .......... .......... .......... .......... .......... 95% 1.82M 0s 600K .......... .......... .......... 100% 1.87M=1.6s 00:21:16 (405 KB/s) - `paypal.com.tar' saved [645120/645120]
sh: line 0: cd: pma: Not a directory
gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error exit delayed from previous errors
On Fri, Aug 21, 2009 at 04:08:43PM -0500, Gregory P. Ennis wrote:
Everyone,
This morning I received a notice from PayPal that one of our sites got hacked and was spoofing a PayPal web site.
When I checked the the site, I was surprised to find they were correct. About 5 days a go we had a server that got hacked and somehow the file paypal.com.tar got uploaded to our server and then stored in a a subdirectory of /var/www/.
I had previously started a mysqld server and planned on using it for web authorizations. I had not been able to work on it, but left it in place. I looked like the hacker downloaded his paypal spoof files into a subdirectory of /var/www/phpmyadmin.
I am running 5.3 with all current updates.
I do not have telnet or ftp active on this server, and have password authentication of sshd turned off.
I have tried to obtain dialog with PayPal about this but they have not responded to my queries. If any of you have had some experience with this I would be interested in knowing how this may have happened. I have shutdown the mysqld server as well as removed access in httpd.conf of the /var/www/phpmyadmin directory in order to shutdown the spoofing site.
If any of you have a leg up on this I would appreciate your help.
Some advice (assuming the culprit here is phpMyAdmin):
- Keep phpMyAdmin up to date. Best way to do this is to use a package from a well known repository like EPEL that keeps the package at the latest version for you. - Run with SELinux Enforcing - Protect phpMyAdmin with Basic HTTP authentication instead of relying only on phpMyAdmin's authentication which does nothing to prevent the exploitation of many URL-based vulnerabilities.
Ray
On Fri, Aug 21, 2009 at 5:17 PM, Ray Van Dolsonrayvd@bludgeon.org wrote:
- Keep phpMyAdmin up to date. Best way to do this is to use a package from a well known repository like EPEL that keeps the package at the latest version for you.
I've not beaten EPEL up too much on things like this, but here is one instance where it counts. EPEL relies on its packagers to keep things current, and in a startling number of cases, they do not. Case in point is the wiki software, moin. Moin is at something like 1.8.x or 1.9.x now, and has several posted security issues, which have been fixed in the most recent versions. EPEL however is still shipping 1.5.9 -> http://download.fedora.redhat.com/pub/epel/5/i386/repoview/moin.html
Just because it's from a well known 3rd party repository doesn't mean it's fully patched. While your advice to use known repositories is good, please don't let it fool you into a false sense of security.
On Fri, Aug 21, 2009 at 05:34:27PM -0400, Jim Perrin wrote:
On Fri, Aug 21, 2009 at 5:17 PM, Ray Van Dolsonrayvd@bludgeon.org wrote:
- Keep phpMyAdmin up to date. Best way to do this is to use a package from a well known repository like EPEL that keeps the package at the latest version for you.
I've not beaten EPEL up too much on things like this, but here is one instance where it counts. EPEL relies on its packagers to keep things current, and in a startling number of cases, they do not. Case in point is the wiki software, moin. Moin is at something like 1.8.x or 1.9.x now, and has several posted security issues, which have been fixed in the most recent versions. EPEL however is still shipping 1.5.9 -> http://download.fedora.redhat.com/pub/epel/5/i386/repoview/moin.html
Just because it's from a well known 3rd party repository doesn't mean it's fully patched. While your advice to use known repositories is good, please don't let it fool you into a false sense of security.
The upgrade from Moin 1.5.x to newer versions is not something that can be automated (as I understand it). Thus the decision was to leave Moin as is and likely provide a newer moin18 or moin19 package (whatever the latest is) in the interim and at some point obsolete the older version. (Hopefully I didn't get that wrong)
Moin is a special case. For the most part EPEL maintainers do a good job of keeping things as up to date as they can.
Of course, as Jim pointed out, with any repository maintained by volunteers (this includes rpmforge, CentOS-extras, etc), you're at the whim of the packager. Tread with adequate caution! This is why Sysadmins should have some skills of their own to identify packages that might require a little extra TLC or to keep an eye on the appropriate security mailing lists.
We all have a number of tools in our toolchests. :)
For the most part, however, I'm going to prefer a package from an active repository like EPEL or rpmforge over handbuilding something like phpMyAdmin every time there's a new release.
To each their own though...
Ray
On Aug 21, 2009, at 4:17 PM, Ray Van Dolson wrote:
- Keep phpMyAdmin up to date. Best way to do this is to use a package from a well known repository like EPEL that keeps the package at the latest version for you.
- Run with SELinux Enforcing
- Protect phpMyAdmin with Basic HTTP authentication instead of relying only on phpMyAdmin's authentication which does nothing to prevent the exploitation of many URL-based vulnerabilities.
What he said, plus change the URL to something other than the default / phpmyadmin/
--Chris
Chris Boyd wrote:
On Aug 21, 2009, at 4:17 PM, Ray Van Dolson wrote:
- Keep phpMyAdmin up to date. Best way to do this is to use a package from a well known repository like EPEL that keeps the package at the latest version for you.
- Run with SELinux Enforcing
- Protect phpMyAdmin with Basic HTTP authentication instead of relying only on phpMyAdmin's authentication which does nothing to prevent the exploitation of many URL-based vulnerabilities.
What he said, plus change the URL to something other than the default / phpmyadmin/
and, heh, don't post any sort of log analyzer output on any publically accessible pages, or your hidden URLs will likely show up and get googled.
Am 21.08.2009 um 23:08 schrieb Gregory P. Ennis:
I have tried to obtain dialog with PayPal about this but they have not responded to my queries.
Big surprise. They're like ebay (well, they *are* ebay...). Only boilerplate responses. Or nothing. In their defense, they must get a lot of spam.
[....]
Interesting. What version of phpmyadmin was that?
Rainer
Am 21.08.2009 um 23:08 schrieb Gregory P. Ennis:
I have tried to obtain dialog with PayPal about this but they have not responded to my queries.
Big surprise. They're like ebay (well, they *are* ebay...). Only boilerplate responses. Or nothing. In their defense, they must get a lot of spam.
[....]
Interesting. What version of phpmyadmin was that?
Rainer ------------------------------ Rainer,
The phpmyadmin version is : RELEASE-DATE-2.11.7.1
I had not had the opportunity to really use it yet, but sounds like phpmyadmin has some real weaknesses.
Greg
Gregory P. Ennis wrote:
P.S. I found the following entry in my error_log of /var/log/httpd/ :
[Sun Aug 16 04:26:19 2009] [info] Server built: Jul 14 2009 06:02:39 --00:21:14-- http://code.go.ro/paypal.com.tar Resolving code.go.ro... 81.196.20.134 Connecting to code.go.ro|81.196.20.134|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 645120 (630K) [application/x-tar] Saving to: `paypal.com.tar'
....
looks like they spoofed something on your server, probably some kinda sloppy php, into running wget. I'd take a look at the access_log around the same timestamp to see if there any hints as to how they did this.
On Fri, 21 Aug 2009, Gregory P. Ennis wrote:
place. I looked like the hacker downloaded his paypal spoof files into a subdirectory of /var/www/phpmyadmin
I am running 5.3 with all current updates.
and third party software as well.
We do not ship phpmyadmin, and clearly and repeatedly caution against it in the IRC channel -- its CVE history is appalling, and people are just not willing to remove it, or limit it to just a specific IP (not that I expect its ACL model to work either)
-- Russ herrold
Am 21.08.2009 um 23:24 schrieb R P Herrold:
On Fri, 21 Aug 2009, Gregory P. Ennis wrote:
place. I looked like the hacker downloaded his paypal spoof files into a subdirectory of /var/www/phpmyadmin
I am running 5.3 with all current updates.
and third party software as well.
We do not ship phpmyadmin, and clearly and repeatedly caution against it in the IRC channel -- its CVE history is appalling, and people are just not willing to remove it, or limit it to just a specific IP (not that I expect its ACL model to work either)
Is there an alternative? I do think that it's the Internet Explorer of OSS. The General Public loves it, the admins hate it - but use it nevertheless.... Because there's no alternative.
Rainer
On Fri, Aug 21, 2009 at 11:29:17PM +0200, Rainer Duffner wrote:
Am 21.08.2009 um 23:24 schrieb R P Herrold:
On Fri, 21 Aug 2009, Gregory P. Ennis wrote:
place. I looked like the hacker downloaded his paypal spoof files into a subdirectory of /var/www/phpmyadmin
I am running 5.3 with all current updates.
and third party software as well.
We do not ship phpmyadmin, and clearly and repeatedly caution against it in the IRC channel -- its CVE history is appalling, and people are just not willing to remove it, or limit it to just a specific IP (not that I expect its ACL model to work either)
Is there an alternative? I do think that it's the Internet Explorer of OSS. The General Public loves it, the admins hate it - but use it nevertheless.... Because there's no alternative.
Nope, but you can take steps to prevent (or make it more difficult) for people that shouldn't be accessing it from accessing it.
Apache allow from, etc... basic authentication, make sure you're using HTTPS and selinux.
Ray
On Fri, Aug 21, 2009 at 5:31 PM, Ray Van Dolsonrayvd@bludgeon.org wrote:
Nope, but you can take steps to prevent (or make it more difficult) for people that shouldn't be accessing it from accessing it.
Apache allow from, etc... basic authentication, make sure you're using HTTPS and selinux.
Along these lines (following up here, though it's mostly to the OP), you may also want to look at your php.ini for some hardening as well. The default php.ini ships with allow_url_fopen enabled, which tells php to treat remote files like they're local. In some cases this is needed, but I really consider it a huge security hole, and if disabling doesn't break your website, I would suggest you do so.
On Fri, Aug 21, 2009 at 5:31 PM, Ray Van Dolsonrayvd@bludgeon.org wrote:
Nope, but you can take steps to prevent (or make it more difficult) for people that shouldn't be accessing it from accessing it.
Apache allow from, etc... basic authentication, make sure you're using HTTPS and selinux.
Along these lines (following up here, though it's mostly to the OP), you may also want to look at your php.ini for some hardening as well. The default php.ini ships with allow_url_fopen enabled, which tells php to treat remote files like they're local. In some cases this is needed, but I really consider it a huge security hole, and if disabling doesn't break your website, I would suggest you do so.
----------------
Jim,
Great suggestion. Thank you!!!!!
On Aug 21, 2009, at 5:47 PM, "Gregory P. Ennis" PoMec@PoMec.Net wrote:
On Fri, Aug 21, 2009 at 5:31 PM, Ray Van Dolsonrayvd@bludgeon.org wrote:
Nope, but you can take steps to prevent (or make it more difficult) for people that shouldn't be accessing it from accessing it.
Apache allow from, etc... basic authentication, make sure you're using HTTPS and selinux.
Along these lines (following up here, though it's mostly to the OP), you may also want to look at your php.ini for some hardening as well. The default php.ini ships with allow_url_fopen enabled, which tells php to treat remote files like they're local. In some cases this is needed, but I really consider it a huge security hole, and if disabling doesn't break your website, I would suggest you do so.
Jim,
Great suggestion. Thank you!!!!!
You weren't the only one who had phpmyadmin used to exploit their server.
There was a thread not too long back of another who's server was hacked through some phpmyadmin script injection exploit.
For everyone who reads this:
Do Not run phpmyadmin on a forward facing server!
It is for behind the firewall only! And even then to restricted users over SSL protected by password.
-Ross
On Fri, 21 Aug 2009, Rainer Duffner wrote:
Is there an alternative?
mysql at the command line works fine here
Because there's no alternative.
There may be no GUI alternative but ignorance needs to be solved -- either with a setup wizard such as mediawiki (php GUI), or bugzilla (perl checking script), rather than handing out a loaded weapon with an un-proof'd breech.
I wrote a RFQ, part of which is after my sig, earlier this week. Note item D that was in it. If a package is reviewed by me, and is not doing all this, at a minimum, I'll not be considering running it exteriorly
-- Russ herrold
Deliverables:
A. php based well formed HTML, at W3C 4.0 or later with all sub scripts in a single directory, except the credentials file, which shall shall be php 'include()'-ed from a non-remotely accesible directory (eg, /var/www/credentials/memail/config.inc )
B. all input shall be validated, not to exceed a configurable length (ie support say up to 140 char field values but not longer), and to an allowed character set of: a-zA-Z0-9.,-_+@\SP
all email addresses downcased to lower -- perhaps obviously, , and \SP are prohibited in email addresses LHS, and domains shall exist and be unexpired per an (optional if cached as live) whois test on the RHS at verification time
C. with a min 16 char hex hash session key (of a maximum configurable life) only externally visible, passed through a post or get, and any needed interior state solved in the database via that session key
D. MySQL-root account database setup script (with a versioned schema noted in a single row table.field: schema.version), and application level use of a separate [MySQL]userid [account] setup in a config file (with a php config file wizard, usually rm'd, mv'd or changed to perms 000, when in production, to emit a scrape and paste one), for access across a network to the backend store; the code shall confirm its absence or those these perms on the wizard script before initiating any connection to the database
E. delivered and packaged as a SRPM which builds non-root, and installs (at least) on a stock archive [base] and [updates] C5 or later, currently updated
Am 21.08.2009 um 23:58 schrieb R P Herrold:
On Fri, 21 Aug 2009, Rainer Duffner wrote:
Is there an alternative?
mysql at the command line works fine here
So our non-geek customers need not apply ;-)
Because there's no alternative.
There may be no GUI alternative but ignorance needs to be solved -- either with a setup wizard such as mediawiki (php GUI), or bugzilla (perl checking script), rather than handing out a loaded weapon with an un-proof'd breech.
If it was easy to write a phpmyadmin replacement, somebody surely would have done so by now. It's probably another of those 80/20 cases....
Rainer
On Sat, 22 Aug 2009, Rainer Duffner wrote:
So our non-geek customers need not apply ;-)
not at all, and I provided both two good examples, and some useful rules for rough auditing
handing out a loaded weapon with an un-proof'd breech.
If it was easy to write a phpmyadmin replacement, somebody surely would have done so by now.
It's probably another of those 80/20 cases....
If a customer is not willing to pay for doing an engagement right, I'll probably not quote on it
-- Russ herrold
Am 22.08.2009 um 00:37 schrieb Les Mikesell:
Rainer Duffner wrote:
Is there an alternative?
mysql at the command line works fine here
So our non-geek customers need not apply ;-)
Isn't there something in openoffice that hooks to databases these days?
There might be - but do you think telling non-IT users/customers to install OOo will really fly? It's like being told to install Word, when you use Linux ;-)
Rainer
Les Mikesell wrote:
Isn't there something in openoffice that hooks to databases these days?
OOo "Data", its a report-n-forms app, somewhat analogous to Microsoft Access. Natively it uses Derby, I think , but it can connect to any database you have a JDBC driver for and execute SQL commands.
Not really an admin tool.
Am Freitag, den 21.08.2009, 23:29 +0200 schrieb Rainer Duffner:
Am 21.08.2009 um 23:24 schrieb R P Herrold:
On Fri, 21 Aug 2009, Gregory P. Ennis wrote:
place. I looked like the hacker downloaded his paypal spoof files into a subdirectory of /var/www/phpmyadmin
I am running 5.3 with all current updates.
and third party software as well.
We do not ship phpmyadmin, and clearly and repeatedly caution against it in the IRC channel -- its CVE history is appalling, and people are just not willing to remove it, or limit it to just a specific IP (not that I expect its ACL model to work either)
Is there an alternative? I do think that it's the Internet Explorer of OSS. The General Public loves it, the admins hate it - but use it nevertheless.... Because there's no alternative.
mysql gui-tools (http://dev.mysql.com/downloads/gui-tools/5.0.html) openoffice base
financial.com AG
Munich head office/Hauptsitz München: Maria-Probst-Str. 19 | 80939 München | Germany Frankfurt branch office/Niederlassung Frankfurt: Messeturm | Friedrich-Ebert-Anlage 49 | 60327 Frankfurt | Germany Management board/Vorstand: Dr. Steffen Boehnert (CEO/Vorsitzender) | Dr. Alexis Eisenhofer | Dr. Yann Samson | Matthias Wiederwach Supervisory board/Aufsichtsrat: Dr. Dr. Ernst zur Linden (chairman/Vorsitzender) Register court/Handelsregister: Munich – HRB 128 972 | Sales tax ID number/St.Nr.: DE205 370 553
Am 22.08.2009 um 10:26 schrieb Christoph Maser:
Am Freitag, den 21.08.2009, 23:29 +0200 schrieb Rainer Duffner:
Because there's no alternative.
mysql gui-tools (http://dev.mysql.com/downloads/gui-tools/5.0.html) openoffice base
Fat client - FAIL ;-) *Some* of our customers do use fat-clients for access to mysql. But there's no way we can force all of them to use some fat-client. Some probably don't have the right to install stuff on the computer they use to access phpmyadmin now, you know.
We *have* to provide a web-client.
Rainer
Rainer Duffner wrote:
Am 22.08.2009 um 10:26 schrieb Christoph Maser:
Am Freitag, den 21.08.2009, 23:29 +0200 schrieb Rainer Duffner:
Because there's no alternative.
mysql gui-tools (http://dev.mysql.com/downloads/gui-tools/5.0.html) openoffice base
Fat client - FAIL ;-) *Some* of our customers do use fat-clients for access to mysql. But there's no way we can force all of them to use some fat-client. Some probably don't have the right to install stuff on the computer they use to access phpmyadmin now, you know.
We *have* to provide a web-client.
There are some recent versions of mysql, php, phpMyAdmin, etc. maintained in the 'remi' repository. I'm not sure how secure they are but I run them on an internal machine so I can use the 'ocsinventory-ng' server packaged there, and haven't seen any obvious problems. Currently the phpMyAdmin there is 3.2.1, vs. epel's 2.11.9.5. http://blog.famillecollet.com/pages/Config-en
Everyone,
Thanks for all your suggestions. Sure appreciate your advice!!!!
Greg
-
Chris Boyd -
Christoph Maser -
Gregory P. Ennis -
Jim Perrin -
John R Pierce -
Les Mikesell -
R P Herrold -
Rainer Duffner -
Ray Van Dolson -
Ross Walker