Stupid Linux Tricks: Basic Server Hardening (Debian Lenny)
A list of basic security precautions for web-facing Debian Linux servers
Sunday, 2010-03-28 | AlmostEffortless, Careerism, On the Internet, Programming
Due to recent events involving some computers I administer, I've become very interested in security. Basically, I've dodged enough bullets thanks to little more than beginner's luck and I figure that it's about time to take responsibility for the safety and security of my computers.
Since I was interested in hardening up two Debian boxes running Lenny, I started off by taking a look at the Securing Debian Manual, this very helpful page on LQ and the results of a tiger audit.
Do this to generate a tiger audit of your server:
# aptitude install tiger # tiger -E
Additionally, the package version of tiger comes with some nice default settings for the main executable and for tigercron, which, as you might imagine, runs some minor scans on a pre-defined schedule.
At any rate, once I had my audit and had picked up a few bright ideas from the SDM, I made a number of changes to all of my web-facing production machines. What follows are some things that you might want to consider doing on your Debian Lenny servers:
Users and Permissions
- Password Audit: first, I decided to get to know my user accounts a little better. This meant running john (formerly "john the ripper", a password cracker that reads hashed passwords and tries to decipher them) against my /etc/shadow to see who was using dictionary-based passwords and who was using other types of insecure passwords:This took a while to run--a little over a day, but I had it niced pretty high--but of the 10 user accounts it cracked, it was good to know which ones were using hilariously insecure passwords and which ones probably weren't going to cracked by your garden variety brute force password cracker.
# aptitude install john # john /etc/shadow
NB: if you run john against your /etc/shadow and realize that you've got a problem child on your hands, there's always chroot. Here is a really good how-to on chroot-jailing a user.
- The Prunening: odds are, if you've been living on a system for more than a few months, you've accumulated some users (either from software that you've installed and then removed or by meeting user/developer needs, etc.) that aren't doing anything. One of the basic tenets of server security is having the smallest amount of users with the least amount of access to the smallest number of programs possible.
In some environments, you've simply got to have a bunch of users in your /etc/passwd. In most situations, however, it makes good sense to just hit all the derelicts with a userdel and only having to worry about angry users not having enough access (rather than having to worry about unauthorized users having too much access).
Keeping Script Kiddies Under Control
In my (limited) experience, the most trouble you're going to run into from script kiddies are anonymous, unfocused attacks that attempt to gain access to your machine via either a.) the /tmp directory, b.) DoS-based exploints or c.) application attacks like SQL injections, XSS or directory traversal attacks.Since guarding against application attacks is something that programmers are supposed to be handling, I decided to focus on stopping /tmp abuse and trying to stymie DoS attacks.
- Mounting /tmp with noexec: in this age of VPSes and shared hosting, it's more often the case than not that you won't get to decide how your machine is partitioned. If, like me, you live on Slicehost and you're running Debian, your partition scheme looks like this: What this means is that you've got your /tmp directory on your / partition. Which partition is, by necessity, allows files to be executable.
lana:/# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 19G 9.0G 8.9G 51% / tmpfs 256M 0 256M 0% /lib/init/rw udev 256M 20K 256M 1% /dev
And that, as I have learned (the hard way), is bad news. What this means is that, you've got a directory on your / partition--the partition where all your apps and data probably live--that is writable/readable by every Li, Ivan and Harry from Taipei to Yaktusk. And if one of the thousands of bots who knocks on your door every month knows how to write to /tmp and you don't catch him right away, it's pretty much game over for your TLD or your IP Address: one day, you'll wake up and find that your info is on every spam list on the Internet because your server has been under remote control via IRC for the last three weeks and now your full time job is trying to get your info off of those lists while planning a full OS re-install.
Ideally, you would be able /tmp its own partition and mount that partition with noexec. And while it would be optimal, it sometimes isn't an option if you're a part of the VPS set.
And if you can't control your installation or maybe you just can't take down a production server, what you can do is warn your users/developers that you're about to cause a little temporary chaos (get it? temporary chaos?), move your existing /tmp to some place else and create a small filesystem that you can mount noexec to use as /tmp. On Debian Lenny, that would look approximately like this:Et voila! You've got a 1GB "drive" that's mounted noexec at /tmp that's ready to roll out. Any attempts to execute anything on that partition will result in a bad interpreter error. Check it:gonzo:/# mv /tmp /old_tmp # dd if=/dev/zero of=/.tmpfs bs=1024 count=1000000 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 29.1302 s, 35.2 MB/s gonzo:/# mkfs.ext3 -j .tmpfs [...] gonzo:/# mount -o loop,noexec,nosuid,rw /.tmpfs /tmp/ gonzo:/# mv /old_tmp/* /tmp/.
All you've got to do now is add that mount info to your fstab and you're ready to start sleeping at night again:gonzo:/# ls -l tmp/ total 20 -rwxr-xr-x 1 root root 37 2009-05-15 14:54 executeMe.py drwx------ 2 root root 16384 2009-05-15 14:51 lost+found gonzo:/# test/executeMe.py bash: test/executeMe.py: /usr/bin/env: bad interpreter: Permission denied
NB: don't forget that /tmp wants to have the sticky bit turned on (i.e. be chmoded to 1777). Also don't forget to make /var/tmp a symlink that points to /tmp./.tmpfs /tmp ext3 loop,nosuid,noexec,rw 0 0
Also: props to Vincent Danen's post on /tmp at TechRepublic for the idea. - mod_evasive to Prevent DoS: after a recent DoS experience, I decided to install Apache mod_evasive to reduce the risk of getting flat-lined/broad-sided by DoS/DDoS attacks:The beauty of using packaged software is that that's really all you have to do: apt will copy the files, create the symbolic links and restart apache for you. Nice.
# aptitude install libapache2-mod-evasive
- sysctl real-time kernel Modifications: Additionally, you might also want to use a slightly obscure command called sysctl (which modifies kernel perameters while the kernel is running, so consider yourself warned) to take a precaution against a DoS tactic called "syn flooding":This is a sort of controversial measure--apparently it defies some RFC docs for TCP/IP--but setting tcp_syncookies to False has yet to have affected any of my computers.
# sysctl -w net.ipv4.tcp_syncookies=1
There are a number of additional security features you can activate with this command; Google it and prepared to be awed by some of the features of your OS you can control in real-time with sysctl.
For the Tin-foil Hat Crowd
What follows are non-specific countermeasures and settings that, while obscure and probably unlikely to save you from becoming an unwitting member of some Russian bot master's herd, might help you feel more secure in the knowledge that even if someone does get non-root shell access, he's probably not going to be able to do too much damage.- Add /usr/bin/mesg n to root's .bashrc File: executing mesg n on log in, prevents an admittedly rare exploit through which other users can execute arbitrary code as root by sending messages to his terminal.
- Modify /etc/inittab to Prevent Non-root Users from Rebooting the System with ctrl+alt+del: In the stock /etc/inititab on Debian Lenny, you've got this line: Which is hardly optimal: this lets pretty much anyone who figures out a way to execute programs in /etc/sbin reboot the system. I changed it to:
# What to do when CTRL-ALT-DEL is pressed. ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
ca:12345:ctrlaltdel:/bin/false
- SMTPD Settings: If you run postfix, you should probably check up on your relay settings and update your external blacklist providers if you haven't done it in a while. Your mail server is the world's first line of defense against everything from phishing/spear-phishing to headline-making super worms:
That smtpd_help_required line might not seem like anything special, but I have a script that parses /var/log/mail.log output and, when you do get a spammer that responds to the
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination, reject_unauth_pipelining, reject_invalid_hostname, reject_rbl_client sbl-xbl.spamhaus.org, reject_rbl_client cbl.abuseat.org, reject_rbl_client bl.spamcop.net, reject_rbl_client zen.spamhaus.org smtpd_helo_required = yes disable_vrfy_command = yes smtpd_data_restrictions = reject_unauth_pipelining, permitrequest, a lot of times he'll come back with his actual domain.helo
It's for research/study/personal amusement reasons, basically.
While some people might say that the above is overkill--that it's just not worth the time and effort to audit and harden at this level--but I'd say that this level of focus on security isn't so much "overkill" as it is "a pretty good start."
Because my thinking is that if you've got the root password, you're probably already the anxious type. And at the very least, being a little bit OCD about security on your all-important, mission-critical application servers might help you feel a little less anxious. Which is definitely worth the effort.
