Agents - worth the hassel?

Just wondering what every one's thoughts are on finding an agent? I have sent out a few submissions to various agents over the years, but have had little success. In fact, only one agent actually bothering writing anything personal back and that was years ago. This is in complete contrast to publishers, who I have found to be more then accessible. It just gets me wondering, is it really such a big deal to have an agent? I know there are a number of big publishing houses that won't even accept your m/s without it going through an agent, and obviously there is the bonus of them negotiating a better deal for you, but if you don't have one are you less likely to be a successful writer? I don't know.
Despite my doubts, I won't stop searching. Just the idea of being able to drop the phrase 'my agent' into conversation gets me excited (is that just insanely vain?). Also, the submitting and publicity side of writing definitely imposes on the amount of time I actually have to write and it would be great to know that there was someone else out there doing it for me.
Mx