does your contact have you on speakerphone? That would do it, and it's annoying. I usually hang up on people if I hear my own voice coming back. They soon get the message.
I thinks it's the network because when it happens the other person didn't have me on speaker, it is annoying. But it never happened with my 3GS only my 4S, so maybe it has something to do with the design of the phone as well as the network.