This is probably because you have Contacts toggled on in more the one email account.
Check all email accounts you have on your phone..sure Contacts is toggled off.....on every account except for iCloud.
Now that no email account except for iCloud has Contacts toggled off...
Delete the iCloud account.
Reboot phone.
Add the iCloud account again...make sure that you keep Contacts toggled on.
That should fix duplicate/missing contacts.
This "usually" happens because more then one email account is trying to download contacts at the same time.