I like using debian for that sort of thing because I'm familiar with it, it's easy to upgrade as newer releases come out (and they don't come out very often), and it's usually small *enough* for me. Use a netinst iso - those are under 200MB, I think, and the installer will let you choose not to install most everything.
You can also install debian directly into a chroot, so you don't need any iso, using debootstrap or cdebootstrap, but that's mostly for if you already have a running debian system.
If you want/need even smaller than that, there's "damn small linux", which I haven't used but have heard of. linky:
http://www.damnsmalllinux.org/
I've done a fairly small (~300MB installed) centos install into a chroot, too. I can dig up the details of that if you're interested. I usually prefer debian because I trust Apt more than Yum.