You stated this issue is during peak hours. Since it is during peak hours, this could be an issue with choking or bottlenecking on your network.
I would recommend reading this article to learn more about this:
Sling Media Moderation Team
Assuming your Slingbox is setup correctly, the stream will be transmitted direct from the Slingbox to your viewing location.
You need to make sure either UPnP is turned on and working correctly at the location of your router, or if you aren't using UPnP then the TCP port your Slingbox uses must be forwarded manually. This will allow the best type of connection for the Slingbox. Without this being setup it's possible it is using the 'Relay' or 'SNATT' method to stream, the former will bounce the stream via some server somewhere ('maybe amazon cloud or something).
The most valid test you should try when this happens is a speed test from the viewing location to the same city as where your Slingbox is located. Use global speedtest dot net where you can choose a server.
If this also shows a slowdown then it could be a routing issue and using a proxy server may help.
If it doesn't show a slowdown then ISP throttling could be a possibility. I am getting something similar these days during the evenings - speed test shows a fast connection between London (Slingbox location) and Istanbul (viewing location), typically 4.5mbps to 5mbps which is essentially as fast as my download here can go. However, the Slingbox won't stream at more than 2-2.5mbps during these times. Come the next day, speed test reports identical results but the Slingbox can stream at the full 4.5-5mbps again. I haven't worked it out yet, I've tried various VPNs, proxies etc to try and evade any throttling but I'm beginning to think it is something else, though I can't put my finger on it yet.
Hope this helps.