Nope, you don't need to delete the old kernel. If it works the way mine did, next time you boot, there should be another entry in your boot menu called 2.6.12-22 or something like that. If you want to edit lilo to make the new kernel the default, click on configure your computer, after you enter your root password and it starts, go to the bottom of the list.
http://pcworld.com/reviews/graphics/...drakconf_b.jpg
Once in boot, click on the advanced button and then you can set the default.
Hope that helps.