You can turn syncing back on and change your sync filters to remove those contacts from Outlook.
If you're using Outlook 2010, click File, CRM, Synchronize, Outlook filters
Find the contact sync filter, for example, My Outlook Contacts, and deactivate it.
Next time CRM syncs with Outlook, the contacts should be removed.