Just curious, what firmware (and maybe what method) are you using?
I found that the Prism54.org firmware (.arm file) had some odd behavior (would never connect with WEP). The Netgear supplied .arm file is working like a champ.
What driver (.o or .ko) module is referenced in /etc/modprobe.conf?
See if any of this info at my site helps:
http://www.sonnik.com/archives/000030.html (if you need the other arm file, you can get it here...)
For some reason, I don't think it's IRQ's or anything...
What kind of encryption are you using?
Are you using iwconfig at the command line?