They mentioned something with the iPad that makes me think the 4" will happen. If you recall, they mentioned it will automatically upscale the older apps to the newer resolution until apps are updated and as newer apps come out. If that is correct, than that would very well take care of the scaling issue on the iPhone 5 if it jumps in size so there wouldn't have to be an immediate re release of older apps. At least that is my speculation.
While I do like the size of the phone now, I wouldn't cry in the least if it went to 4". 4" and LTE will be a necessity to hold up against Android, I believe. I always found 4" on my old Inc 2 to be a perfect size. Not too big, not too small and still easily pocketable.