I would strongly recommend a better aftermarket cooler if you are thinking of serious overclocking, I mean beyond, lets say, 4.0 GHz
And besides, Air coolers are very cheap these days and good ones too, from 10 USD onwards
Mights be a bit expensive but has some very positive reviews.
If you want cheaper, then its no problem, you aren't gonna lose anything
And this one is probably the best of the lot
But as you said, if you want to keep it under 4.0 GHz, then there is no need for another cooler. Stock one will do just fine