Linux - SecurityThis forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Oh, geez... What'd I SAY here? My most abject and sincere apologies to all the very nice ayrabs, commies and bastards in the world. That was unquestionably the MOST UN-politically-correct statement I've made in years (felt good too)!! I actually LIKE ayrabs... and a few of my oldest friends are probably commies and I couldn't even BEGIN to count the illegitimate bastards who are my friends (including ME, in fact). It's the hackers and petty thieves who have nothing better to do than steal and damage the work of others that I TRULY hate. F&$# You, You Mo Fos!!!!
Bottom line... lighten up guys. This isn't the end of the damn world. It's just one stinkin server among the legions of compromised computers worldwide that has been infected with two stupid viruses. Hell I've personally deloused customer PCs infected with dozens of similar beasties... and ain't nobody ever sued nobody over it yet.
My customers are all small struggling businesses who are for the most part thrilled to have someone with my experience who's willing to work for them for what amounts to peanuts. Ain't none of them gonna sue me 'cause I hain't got no diņero for them to take. Kapeesh (or capite, capishe or capiche as you prefer...)?
Shit happens. We wipe our butts and get up to fight another day. C'est la vie!
No problema, seņors! Uno momento, por favore... esta el tiempo para mi siesta... Ahore!
(Note to ayrab-commie-spanglish-speaking-hacker-bastards: Mi esapaņol esta mui mala! My Italian and French suck osos muertos too.)
If you're gonna be unPC, at least aim your guns in the direction of the right target. Most hackers these days are operating out of Russia, China, and eastern Europe. Brush up on mangling your Mandarin and Russian...
If you're gonna be unPC, at least aim your guns in the direction of the right target. Most hackers these days are operating out of Russia, China, and eastern Europe. Brush up on mangling your Mandarin and Russian...
I'm acutely aware of where the world's evolving cyber threats come from, Jim. That's precisely why I aimed my guns in another direction. The guys in Uzbekistan are well aware of it too. The scariest part is it's 'volunteers' from those same places who are now selflessly maintaining some of our favorite defensive weapons. Did someone mumble, 'conspiracy'? Not ME...
Which reminds me...now that non-latin characters will be allowed in URLs, Fail2Ban (and Squid, Squidguard, and possibly DNS) will have to be updated to handle the bad guys under those URLs...
The intentional spelling of certain ethnicities in abnormal ways can quite easily be interpreted as derogatory, which is not something we want to see happen here. This is a technical forum, so let's keep things technical. As for the countries, there will of course be some which are more prone to being mentioned when talking about cyber attacks (due to, for example, statistical data), but talk about conspiracies and such doesn't belong here. I'd also like to remind everyone that posts should be in English, as is made clear in the LQ Rules. I'd appreciate everyone's cooperation in getting this thread back on topic. Thanks.
The intentional spelling of certain ethnicities in abnormal ways can quite easily be interpreted as derogatory, which is not something we want to see happen here. This is a technical forum, so let's keep things technical. As for the countries, there will of course be some which are more prone to being mentioned when talking about cyber attacks (due to, for example, statistical data), but talk about conspiracies and such doesn't belong here. I'd also like to remind everyone that posts should be in English, as is made clear in the LQ Rules. I'd appreciate everyone's cooperation in getting this thread back on topic. Thanks.
Sorry, win32sux. Didn't realize I'd broken any rules. I had no clue you were an English only site. God knows I wasn't intentionally being bigoted or insulting. I was just trying to lighten things up a bit. It got so lead-heavy here yesterday even natural flatulence fell straight to the ground as pellets. But I understand now. Sarcasm and humor aren't allowed here... no matter what. Better that we all drop dead of strokes or hypertension.
I get it. Just the facts. No spitting, mud balls, fist fights, kicking, screaming or hair-pulling. And especially never utter the forbidden "c-word". No foreign words or phrases - and absolutely no Spanish, French, Italian, Yiddish, Russian or Chinese. We can't risk offending anyone. Gallows humor not allowed.
My sincere apologies to all you hackers too. I'm sure you're all very nice gentlemen or ladies deep down.
Now, back to the topic. My current plan is to take my server down, move all my clients to another temporary location. Then rebuild and fully harden the server before moving them back again. Does anyone strenuously object to that plan? Have I overlooked any obvious pot holes?
Can anyone recommend sites that offer reliable prefab server building scripts OTHER than the tutorials at LinuxHowto.com and YoLinux.com? I found Falco's Tutorials at LinuxHowTo to be excellent. But by the time I figured out and resolved things like apachie configuration, install, setup and integration, postfix installation, configuration and setup, plus openssh, spamassassin, dovecot, squirrelmail and mailman integration along with virtual mailboxes, DNS server selection, integration, installation and testing, php, perl and mysql installation, setup, integration and testing, etc, etc, etc. And handled the reinstall and testing of 20 web sites. It literally took weeks for me to build, configure, test and minimally harden this server last year and roll it out.
Needless to say, I'd like to find ways to vastly shorten that path while not taking any foolish or fatal shortcuts. That's especially true where server hardening, mail setup and spam filtering are concerned.
Wizened suggestions from veterans with scars on their backs would be greatly appreciated! Threats of violence, lawsuits and global anthrax releases will all be completely ignored.
Which reminds me...now that non-latin characters will be allowed in URLs, Fail2Ban (and Squid, Squidguard, and possibly DNS) will have to be updated to handle the bad guys under those URLs...
The good news is volunteers from Eastern Europe, Russia and the far east are already maintaining some of those tools. That means it should be easy for them to fix them. Other than that, I'm not touching that particular live transformer with a 75' rubber pole.
Speaking of server rebuilds and homeland security, here's good news from our federal government published 3 hours ago. Maybe I can convince Joe Lieberman to come help rebuild my server or at least send me a few bucks to underwrite the costs.
As I said above, I was wrong when I said the ls -v command was in a bootlog. I had run a debian script (dpkg-reconfigure) and captured the output to check it. I noticed the ls error in the log and questioned it. That's how the rootkit was discovered.
I realize the server will need to be rebuilt; but before I start that, I'd like to try to figure out how long this compromise has been present and determine if there is any way at all to lobotomize it or keep it from doing any further harm. If it has been there for months (I suspect it may have been), then rather than take 25 client sites down for weeks while I rebuild, I might conclude it's best to leave the compromised system running and keep client sites running while I work on building a completely new server or building a completely new drive on THIS server.
I know I may be whistling in the dark here. But most of my clients are small businesses and their www server needs are not complex - a few html pages, a web form or two and some streaming music or videos. Only two of them involve significant databases and both of those sites are mine. But despite their size, my small business customers really do NEED their sites to be up and accessible on the web. So, my goal here would be to avoid putting them completely out of the web business while I tackle this rebuild.
Thanks a lot for your comments and any feedback you can offer.
I use my site for mostly personal reasons. I have a backup system, not web accessible with 1.7 Tb raid 10 for backups.
All my other systems have a single drive but I'm going to change that soon. Pondering the rout to go on that.
Now, my two reasons actually are two in one.
1. Drives are cheap now days.
2. You get to keep the infected drive for those late nights that you want to dig around and find the way it happened. Could be good information for you, and yes all the rest of us as well.
Sorry, win32sux. Didn't realize I'd broken any rules. I had no clue you were an English only site. God knows I wasn't intentionally being bigoted or insulting. I was just trying to lighten things up a bit.
This is an English-language only site; I'm not sure what the rule on strings of non-ASCII characters is, but best to avoid them, unless they are directly germane to the issue at hand.
Quote:
I get it. Just the facts. No spitting, mud balls, fist fights, kicking, screaming or hair-pulling. And especially never utter the forbidden "c-word". No foreign words or phrases - and absolutely no Spanish, French, Italian, Yiddish, Russian or Chinese. We can't risk offending anyone. Gallows humor not allowed.
For my taste, this thread has gotten a bit random (maybe, scatter-gun would be a better phrase). That isn't to outlaw humour and there is a fine line between humour directed to a purpose and just offending random people because you can. After all, if you think of the way that the media usually portrays people who have anything to do with computers, we wouldn't want to repeat that kind of stereotyping here, whether it is stereotyping of techies or of any particular race (well, unless it is really, really, really funny and in an appropriate forum, like general, where there wouldn't be a risk of distracting from an important thread).
Quote:
Now, back to the topic. My current plan is to take my server down, move all my clients to another temporary location. Then rebuild and fully harden the server before moving them back again. Does anyone strenuously object to that plan? Have I overlooked any obvious pot holes?
...reliable prefab server building scripts OTHER than the tutorials at LinuxHowto.com and YoLinux.com? I found Falco's Tutorials at LinuxHowTo to be excellent...Needless to say, I'd like to find ways to vastly shorten that path while not taking any foolish or fatal shortcuts. That's especially true where server hardening, mail setup and spam filtering are concerned.
There has not been any report of the evidence of the 'how...' part of the 'how did the bad guys do this' question; if you do get such evidence please be sure to post it.
My belief is that someone did something really, really stupid with this server - probably the previous administrators, but that is just a wild guess on my part - like, say, having a root password of root and allowing root logins (which would be an extraordinarily stupid thing to do...but something bad happened somehow, probably twice). It would be excellent to know that whatever it was isn't going to happen again.
There was an excellent tutorial recommended back on page one of this thread (http://www.cyberciti.biz/tips/linux-security.html way back in post #8) and I would also recommend the bastille set of hardening scripts (but with the usual proviso that you cannot just get a fire 'n forget solution to security...it should be one of a series of measures and an ongoing process of patching and monitoring).
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.