Reading the SharePoint change log from CSOM

I am quite often asked to design or review SharePoint-based solutions. A very common business requirement in these solutions is a business process that is based on a list. Of course, this is how SharePoint workflows are designed and this is a natural approach for many businesses. As you can imagine, there are instances where workflow is not used. (The reasons for not using workflow are outside the scope of this article.) A common replacement for workflow is event receivers, but using them creates a very brittle dependency between SharePoint and your process and has the risk of server performance degradation, data loss, or both. Some organizations will read the entire list and compare it to a known state. (I call this the "rewrite SharePoint search approach and I do not recommend it.)

One possible alternative approach is the SharePoint change log. This approach will not work for all scenarios, but in a recent customer solution it is very helpful. This solution was using a SharePoint list to make requests. The request process uses only a few columns, and once a request is processed, the SharePoint list is forgotten. (Long-time programmers like me call this a fire-and-forget process. Start the process and then forget about it.) As I set out to code the proof-of-concept, I noticed a distinct lack of examples. And thus a blog post idea was born! And, since we are in a cloud-first world, I will use the Client-side Object Model (CSOM) for my example.

SharePoint Change Log object model

Accessing the SharePoint change log is accomplished using the GetChanges() method on the primary objects:

As you would imagine, the methods return a collection of objects that are descendants of the object on which the call is made. For example, calling GetChanges on a list will return changes made to the list and to items (and folders and files) in that list, but will not return changes made to other lists in the same web or to the site collection-based galleries.

Each of these methods has a parameter of type ChangeQuery, and this type provides insight into the capabilities of the Change Log that far surpasses the MSDN reference.

ChangeQuery properties

The properties of the ChangeQuery object can be separated into two general categories – change actions and objects changed. (Abstracting the properties to a couple of Enumerations would be a nice exercise for some ambitious reader.)

Change Action

  • Add
  • DeleteObject
  • GroupMembershipAdd
  • GroupMembershipDelete
  • Move
  • Rename
  • Restore
  • RoleAssignmentAdd
  • RoleAssignmentDelete
  • RoleDefinitionAdd
  • RoleDefinitionDelete
  • RoleDefinitionUpdate
  • SystemUpdate
  • Update

Objects Changed

  • Alert
  • ContentType
  • Field
  • File
  • Folder
  • Group
  • Item
  • List
  • Navigation
  • SecurityPolicy
  • Site
  • User
  • View
  • Web
    (Note that not all actions apply to all object types.)

Change class and its inheritance hierarchy

As mentioned previously, the GetChanges methods return a ChangeCollection. The items in the collection all inherit from the Change class. This inheritance hierarchy is crucial to understanding the items returned by the query.

System.Object
   Microsoft.SharePoint.Client.ClientObject
     Microsoft.SharePoint.Client.Change
     Microsoft.SharePoint.Client.ChangeAlert
     Microsoft.SharePoint.Client.ChangeContentType
     Microsoft.SharePoint.Client.ChangeField
     Microsoft.SharePoint.Client.ChangeFile
     Microsoft.SharePoint.Client.ChangeFolder
     Microsoft.SharePoint.Client.ChangeGroup
     Microsoft.SharePoint.Client.ChangeItem
     Microsoft.SharePoint.Client.ChangeList
     Microsoft.SharePoint.Client.ChangeSite
     Microsoft.SharePoint.Client.ChangeUser
     Microsoft.SharePoint.Client.ChangeView
     Microsoft.SharePoint.Client.ChangeWeb

With the understanding of the returned objects, we can process the changes by casting the change to the appropriate type, which then makes available properties that further identify the specific item that was changed.

Processing all the changes in a site collection

The following code snippet will get all the changes to a site collection:

ClientContext clientContext = new ClientContext(siteUrl);
clientContext.Credentials = new SharePointOnlineCredentials(
                                        "user@tenant.domain", 
                                        ConvertToSecureString("SuperSecurePassword"));

var site = clientContext.Site;
clientContext.Load(site);
ChangeQuery siteCQ = new ChangeQuery(true, true);
var siteChanges = site.GetChanges(siteCQ);
clientContext.Load(siteChanges);
clientContext.ExecuteQuery();

foreach (Change change in siteChanges)
{
    Console.WriteLine("{0}, {1}", change.ChangeType, change.TypedObject);
}

When run against my test site, the following types and objects are returned:

Getting changes for a specific action and/or object type

Using the ChangeQuery properties, a subset of the actions and types can be queried:

 //ChangeQuery siteCQ = new ChangeQuery(true, true);
 ChangeQuery siteCQ = new ChangeQuery(false, false);
 siteCQ.Item = true;
 siteCQ.Add = true;
 siteCQ.SystemUpdate = true;
 siteCQ.DeleteObject = true;

Processing the changed items

To process the items, simply test the object type and cast as appropriate:

foreach (Change change in changes)
{
  if (change is Microsoft.SharePoint.Client.ChangeItem)
  {
    ChangeItem ci = change as ChangeItem;
    changeType = ci.ChangeType.ToString();
    itemId = ci.ItemId.ToString();
    Console.WriteLine("{0}: {1}", itemId, changeType);
  }
}

Armed with the object type, and its identifier, we can make the necessary call to get the object. The list item example would be:

ListItem li = list.GetItemById(ci.ItemId);
clientContext.Load(li);
clientContext.ExecuteQuery();

However, there is a significant is we need to account for: there is no guaranteed that the item referenced in the change log currently exists. While you may be processing an Add change, that change could have been followed by a delete. This is precisely why the change log is not appropriate for all scenarios. So, be sure to handle exceptions for missing items. (For list items, issue a CamlQuery for the ItemId and test for 0 records returned.)

Getting Changes since last run

One additional processing item – if you poll the change log on a schedule, you will not want to re-read all the changes from the last run. The ChangeQuery has a few properties that indicated which changes are desired: ChangeTokenStart and ChangeTokenEnd. Your processing loop should store the ChangeToken of the current Change object, and then persist the last token. During subsequent runs of the program, read the last-processed token from storage and set it as the ChangeTokenStart property of the ChangeQuery. (The ChangeTokenEnd property would usually be left blank.)

ChangeQuery siteCQ = new ChangeQuery(false, false);
siteCQ.ChangeTokenStart = lastProcessedToken;
siteCQ.Item = true;
siteCQ.Add = true;
siteCQ.SystemUpdate = true;
siteCQ.DeleteObject = true;

Summary

Processing the change log in sharepoint in not too different than processing any other sharepoint object: Create a query, execute the query, and process the objects. While the change log is not appropriate for all scenarios, when used it can dramatically reduce the load on the SharePoint farm and substantially reduce code execution time.

The code snippets from this article a part of a demonstration console app, available on GitHub.