Honestly, yes, you would be better off. The stock manifolds really aren't all that restrictive. The bigger problem is heat control and the thick cast iron manifolds retain far more heat than the thin walls of aftermarket headers. Of course the heat shields help as well. With headers, you lose the insulating properties of the cast iron AND the heat shields. You also open yourself up to potential leaks from the probable fitment issues associated with cheap headers. If you have one of those "pretty" hot air intakes (the cheap and popular ones that are a shiny tube with a cone filter open to the engine bay), you'll lose even more power from the engine sucking in the now even hotter air.
What it boils down to is go with long tubes or stay stock. Long tubes provide quite a noticeable bump in power and can be had for $300-$400 with a Y-pipe, so they're a pretty decent bang-for-the-buck bolt-on. They'll still breed more heat in the engine bay than the stock manifolds, but at least you'll get something useful out of them other than just looks if you have an air intake that is fed fresh outside air (such as the stock setup). Investing in a good ceramic coating will help with the heat retention. My personal plan of attack is to get Speed Engineering headers and Y-pipe and have the headers coated by Swaintech. Yes, I'd be spending more on the coating than the headers, but it all would still be cheaper than the high-end/high-price headers that aren't coated and they'd flow just as well.
