I have noticed on some of the cheaper access points and routers that this sometimes happens. I get worse clarity on the wifi here at work than I do without it. We have several people connected to it and I think it is not giving me the QOS that the phone needs. We have another one at the other end of the building that works fine. One is just a cheap Netgear AP and the other is a higher end Netgear. The clear one has much better throughput in general.
Try connecting to another Wifi and see if it does it. If not get another router.