Blog Post: CRM for Outlook Client Synchronization Blog Series Part 2: What happens when you track an item in Outlook

Local Database Storage

During the tracking of the object, the add-in also writes information to the OutlookSyncCache.sdf client-side database file, which is located in the user’s roaming appdata folder (C:\Users\<user>\AppData\Roaming\Microsoft\MSCRM\Client).  The OutlookSyncCache contains several tables including the OutlookSyncTable and the idMappingTable.

OutlookSyncTable

The OutlookSyncTable can be thought of as a queue that holds the change requests between synchronization cycles for any changes made on the client to synchronized records. Then, when the sync process occurs, this table is updated to contain the record data for changes that occurred in CRM in between the synchronization cycle. The columns found in the OutlookSyncTable include:

• Id– The Unique identifier and Primary Key for each row in the table.
• MapID– Foreign key identifier to the idMappingTable, discussed below.
• OutlookTimestamp– This time/date value reflects when the item was last changed in Outlook.
• OutlookChange– This integer value reflects the type of change action that occurred in Outlook.
• CrmTimestamp– This time/date value reflects when the item was last changed in CRM. This column is only populated during the synchronization process.
• CrmChange - This integer value reflects the type of change action that occurred in CRM. 1 is create/update, 2 is delete. This column is also only populated during the synchronization process.
• Priority – This integer columndetermines if the synchronization of the object should occur immediately or during the next manual or automatic sync.

In the circumstance of performing a manual track, a row is first inserted into the OutlookSyncTable with a Priority of 1 for the CRM record create request, which is processed immediately by the add-in. This is in contrast to how updates to existing tracked items are handled, which will have a different priority. These items are instead processed during the next synchronization cycle. The full behavior of the background synchronization process will be discussed in a future blog.

idMappingTable

The idMappingTable acts as a pointer table for the OutlookSyncTable to correlate the unique Outlook entry ID to the unique CRM ID during the synchronization process. In other words, this table can be thought of as an index which tells the add-in where it needs to look in the user’s mailbox to find the actual Outlook object that is linked to a corresponding CRM record.  Each row in this table is not fully populated until the CRMID is returned from the initial create request.

The columns contained in the idMappingTable are as follows:

• ID– This is the primary key for each row in the table. This key is related back as the foreign key (MapId) in the OutlookSyncTable.
• CrmId– This relates to the GUID of the record in CRM.
• EntryId– This is the Outlook Entry ID. As discussed in the user-defined fields section below, the EntryID is the unique identifier for each mail object in the user’s mailbox.
• Type– This defines the entity type code for the tracked object. For example, contacts will have a type code of 2, while appointments will have a type code of 4201. 
• IsDeletedLocally– This bit column helps the client and server keep track of which tracked items the user has deleted locally to prevent them from resynchronizing to the client. This field also plays a role in the SubscriptionManuallyTrackedObject table on the server, which will be discussed later.
• IsLinked– This bit column determines whether or not the item is still linked to a CRM record. This value goes from True to False when the item is untracked in Outlook.
• Visited– This bit column shows whether or not the user has visited the item in Outlook.
• VerifiedTime–The add-in runs a periodic job to determine if any tracked objects in Outlook have been deleted. If the add-in detects that an object is now missing in Outlook, a delete insert is created in the OutlookSyncTable, which is carried out in the next synchronization. This column indicates the last time the Outlook client verified the existence of the record.

After the create response is received, the add-in then insert a row into the idMappingTable to correlate the Outlook item’s EntryId to the CRM record id.

Server Side Processing and Storage

As discussed above, the track process begins with an SDK Create request from the client. The SDK Create message can be captured in a server side platform trace, and will appear with the following message for a contact:

MessageProcessor start processing message:'Create' for entity:'contact'

When the server receives this request, the platform will perform a series of validations before the request is fulfilled and a response is generated. This includes:

• Checking the user’s security privilege to ensure that the user has the appropriate permissions to create the requested object type.
• Running duplicate detection, if applicable for the entity type. By default, Contacts have a duplicate detection rule that fires on E-mail Address.
• For appointments, the platform may need to invoke the scheduling engine and related functions for availability and rescheduling, if needed.
• Creation of the requested record, which would be performed as an insert on the appropriate table. For example, for Contacts, an insert would occur on the ContactBase table. This insert will also include the SubscriptionId of the user who had tracked the object. The SubscriptionId prevents the object from being deleted in the user’s Outlook mailbox during synchronization when the following actions occur:
  • The corresponding record is deleted in CRM
  • The record falls outside of the user’s synchronization filter (ex: The owner of the record changes in CRM).
• Inserting a row into the user’s SyncEntry table for the newly created record. The role of the SyncEntry table is discussed in a future Synchronization blog.
• Sending the create response back to the client, containing the CRM record’s GUID.

User-Defined Fields

When an Outlook object is manually tracked, several user-defined fields are created in the Outlook object to complete the track process. The fields that are introduced may include:

• crmEntryID– This value corresponds to the Outlook EntryID of the object. The EntryID is the unique identifier for each mail object in the user’s mailbox.  This value is also used in the idMappingTable to relate the CRMID to the appropriate Outlook EntryID.
• Crmid– This value corresponds to the created record’s unique GUID in CRM.
• crmLinkState– This value indicates the synchronization status of the item. 0 is untracked, 1 is pending track, and 2 is tracked.
• Crmorgid– This value relates to the organization id in which the record persists.
• crmownerId– This value relates to the user who owns the record in CRM, who by default is the user who tracked the record. This field does not appear in Appointments.
• Crmpartyinfo– This value appears in Appointment records to indicate all of the related party members associated with the appointment. This is maintained in the form of xml. 
• Crmxml– This field stores the OriginalXML of the synchronized record. The OriginalXML can be thought of as a “snapshot” of the record’s last synchronized state. When a tracked object is modified and then synchronized, this field is updated. This property also plays a role in the handling of merges between items altered in both Outlook and CRM between sync cycles, which will be explained in a future article.

These custom fields are used by the add-in for the following tasks, among others:

• To provide instructions to the CRM add-in ribbon interface for the “Show in CRM” button.
• How to handle merges between changes that occur in Outlook and CRM between sync cycles using the crmxml column.
• Allows the add-in to perform a search on MAPI folders if the idMappingTable (OutlookSyncCache) is removed. This is accomplished by querying for the CRMID of the associated items being synchronized to the client.  A scenario where this might occur may be when a user reconfigures the Outlook client on a new computer using the same mailbox store and sets the new computer as the Primary Synchronizing client. This functionality prevents duplicate Outlook items from being created in the user’s mailbox. 
• Allows the add-in to remember if a previously tracked item still exists in CRM when the user attempts to re-promote the item based on the CRMID property. This is illustrated in the screenshot below:

It is important to note that manual modification of these fields do not necessarily impact the record in CRM. For example, even though it would be possible to set the crmLinkState property to 0, this would not “untrack” the record and cause the item to be deleted in CRM. Instead, the CRM add-in would no longer view the record as "tracked" from within Outlook, even though synchronization would still occur with the item. Likewise, if the crmid property is changed to an invalid value, the “Show in CRM” button would no longer work correctly, as the add-in uses the value of this property to construct the URL back to the related CRM record.

Should these fields be altered, they can be corrected during the synchronization process if the same item has changed on the server. If the item has not been changed on the server, these fields will remain in a “broken” state and may not allow the add-in to perform necessary functions on the item, especially if the CRMID field has been changed. The reason for this behavior is that the add-in will not retrieve the corrected state of the record from the server if no changes have been made to the object in the application since the last synchronized time. As a result, manual manipulation of these fields can have unintended consequences and is not recommended.

For example, one issue that we see in support from time to time is when these fields are modified or dropped by 3^rd party applications, such as smartphones that synchronize with the user’s mailbox. If these fields are modified or deleted, the mail server may create a duplicate (untracked) item in Outlook, or may alter the properties of the existing object in such a way that the item’s ribbon contextual buttons do not work as expected.

Often the User Defined fields can be used to help troubleshoot the synchronization status of an item. To view the user-defined fields, enable the Developer feature in Outlook:

Note: These instructions are for Outlook 2010

1. Click File, Options
2. Click Customize Ribbon
3. Under the Customize the Ribbon area, place a checkmark in “Developer”
4. Click Ok

Afterwards, the user defined fields for a tracked item can be viewed by completing these steps:

1. Open the Outlook item
2. Click on the Developer tab
3. Click Design This Form
4. Click on the All Fields tab