Have you tried switching to detailed view? Could it be the thumbnails loading causing a hang?
Also, I remember a bug in Windows with corupted wave files that cause serious slowdowns. Do you have any .wav files on the drive that might be corrupted?
This does seem like an issue with some of the files being corrupted because you say the drive is healthy.\
Could it be your Anti Virus software's real time scanning option? Sometimes they slow stuff down quite a bit. Try turning off real time scanning and see if that helps.
Good luck on solving the problem.