I ran some tests and believe I know the problem but there is no way for me fix it. The DCT6412 requires the each digit in the channel number to be entered with in about 2 seconds of the last one. I changed the DCT6412 setup to not change the channel until it gets all three digits. Unfortunitely, with my custom remote after it sees the first digit if it does not see another digit within this 2 second window it just clears the entry. You can't change the channel unless you get 3 digits input with no more than 2 seconds between the digits. If I change the DCT6412 setup back to not require the 3 digits then it just takes whatever is input after the 2 second timeout. In this cast I always get the single digit input and the box switches channels 3 times for a 3 digit input. I can enter the digits as fast as I can on the custom remote but they still don't get sent to the DCT6412 fast enough to recognize a multi-digit channel number.
I can't see any setting on the DCT6412 to make the inter digit timer longer.
The DCT6412 remote supplied by Slingbox must somehow know to deliver the digits fast enough to allow 2 or 3 digit channel selection. The custom remote does not. I tried all the advanced options on the Slingbox setup for the remote but I can't see any difference. This seems to be either a bug or not be supported.