There is a file called /etc/security/limits.conf
This file is use to control a how much memory, CPU time, processes and other resources a user can use. See the man page for limits.conf
Use top to monitor high CPU and memory usage. Investigate the process before killing it. A user could be using a program that uses lots of CPU usage and not be aware of it.
Optional: Install rkhunter or chkrootkit and clamav. The first two are rootkit checkers and clamav is a anti-virus scanner.
Clamav can integrated in mail servers and other services.