Have you ever downloaded&installed anything from the internet?
I'm sure you have. Can you say 100% certainly that there hasn't been anything harmful in that code you gave full privileges to do whatever it wants?
The problem is that there's no point giving some simple install unneeded privileges but currently you have to.
You can do installs without risking anything important but not without third party applications.
It's not that UAC is annoying, it's the lack of security even though it's continuously praised here.
You seem to be confused about what UAC is there to protect you against. UAC is not
designed to stop you from downloading and installing malware that horks up your machine. That's the job of Windows Defender and anti-virus software (and the warnings from the browser, to some extent).
UAC is there as a mitigation of attacks against everyday applications, like Outlook / AIM / Firefox / whatever. The goal of UAC is that if such an application is hijacked by any kind of remote code execution exploit, the damage the attacker can do is constrained based on the privilege level of the application.
In the case of IE, UAC means that IE can't even read from or write to the disk outside of specific locations. So it really can't do anything
to hurt your system if someone takes it over. That's why every attack against IE in Vista so far has been a non-issue. Other apps that run with "normal" privileges (Firefox, Outlook, whatever) can still do damage to your personal files if hijacked, but at least they can't affect other users on the system or damage the system itself.
It's pretty frustrating when people like you advise others to disable UAC when you don't even understand how it works, or what it's for.