Welcome to iPhone Forums, cjj!
When your iPhone is connected to iTunes, go to the Summary tab, scroll to Options there, make sure "Manually manage music and videos" is checkmarked. Then go back to the voice memos, select all and delete them. Sync your device with iTunes once more. They should be deleted.
Hope that helps.