This won't help much for programs that you've already installed, but in the future when you install from source, try using
checkinstall. It replaces the 'make install' step and creates a Redhat *.rpm (or Slackware *.tgz or Debian *.deb) package, so you can easily and completely uninstall it with rpm (or pkgtool or dpkg).
To remove existing programs, you need to go to the source tree and type 'make uninstall'. If you've already deleted it, you can try recreating it from the original tarball, re-installing, and then running 'make uninstall'. However, keep in mind that some Makefiles don't have uninstall procedures, and of those that do, some are less than ideal.
Enjoy!
--- Cerbere