Those are Apple supplied applications and are automatically protected from deletion. The only suggestion I can give is make a folder that stores the apps you don't use very often and move it to your last screen.
Yup. If you are jailbroken, you can hide them. They can be hidden from your SpringBoard as if they were deleted, while being able to unhide them at any time. I always hide the Stocks.app and Newsstand folder.
Tho if you are not jailbroken, then all you can do is what SkullOne said.