Stange Jitter issue polycom phones...

I have posted this in the polychrome section as well, I am not sure if this is a polycom problem or a trixbox issue. Either way any advice is much appreciated.

I am having a strange issue with my polycom phone, 501, and 601 phones. Incomming calls will have a lot of jitter on them if we use the handset, or the headset, if we put the caller on speaker phone then the jitter goes away. The jitter is only on my end, the caller on the other end can hear me just fine. Thanks in advance for you help.

I am using the latest firmware available from the package manager in trixbox. also running asterisk 1.4. Let me know if you need any other info to help trouble shoot this.

Thanks in advance.