Are you utilizing ADFS? If so, is your ADFS server added to the Internal Sites? Actually, you can probably just add *.yourdomain.com to the list and that should cover it (assuming they are both in the same domain.)
Also, what rollup are you on? I know there was an issue with the security tokens expiring that was addressed in a recent rollup (10 or 11 I believe). You can also increase the token timeout value. Take a look at this article: http://social.technet.microsoft.com/wiki/contents/articles/7681.setting-the-adfs-timeout-for-crm-2011-internet-facing-deployments-ifd.aspx