You could use the old Exchange client, when accessing a mailbox using this doesn't recognise the Outlook special folder namess, so treats these just like regular folders that you can delete.
Make sure you don't delete the Calendar full of appointments!