Interesting observation. FWIW, the Apple forum has an over 200 page thread on this matter - many irate people. One poster has made the rather remarkable observation (needs to be verified) that the only people not having issues have the white 4s - although this does not explain the issue with IOS5 on other non 4s phones.
https://discussions.apple.com/thread/3391947?start=3060&tstart=0
O.K., white phone theory is not true as people with them are now reporting in with the same issue.