Does your ISP offer Classes of Service?
If not, the only thing worth doing at your end would be to apply a bandwidth restriction on your outgoing traffic on your router, prioritising RTP traffic.
Your incoming traffic can't be helped from your end though, and seeing as most traffic overall will be incoming, not outgoing (web-browsing) it's the incoming you most need to protect.