Sending and receiving SPListItem references via JSON

One of the big changes in SharePoint 2013 is the focus on client side processing. Over the next few years I suspect SharePoint developers will be spending a lot of time in the wacky world of JavaScript, JQuery and JSON. Bearing that in mind, there’s one problem that’s likely to come up time and time again: You’re laying down some funky UX that interacts with server side code via a custom web service. Your server side code naturally makes use of SPListItem objects and you need to handle references to these across the web service boundary.

For example, you have a custom web part that provides edit functionality for a list item, before the edits are applied additional processing must take place in your server side code. Your client side code therefore calls a custom web service that performs the steps required to apply the edits.

The server side business object

So let’s say we have a server-side business object that looks like this (of course it’d be more complicated than this in the real world):

public class BusinessObject
{
    public SPListItem Item { get; set; }

    public string AdditionalProp { get; set; }

    public int AnotherProp { get; set; }
}

We can see that this object is effectively a wrapper for an SPListItem. This is a common scenario since practically all data in SharePoint is an SPListItem. So what happens if we try to send this as a response from a REST endpoint? As is often the case we promptly drown in the quicksand that is ULS. The problem: SPListItem is not serializable.

Possible Solutions

There are a few possible solutions. Probably the most obvious is don’t send the business object as the response. Instead define a different business object that uses only serializable properties and send that instead. That would work – it’d mean writing code to convert between the two types and ensuring that changes were applied to both types – but it’d work.

Another possibility is to rage against the machine – viva la punk – make SPListItem serializable.

Here’s how it’s done

Serializing every property in an SPListItem would be madness. All we really need is enough info to enable us to recreate the object on the server side when a call is made back into the webservice. For all practical purposes all we need is the GUID of the SPSite, the GUID of the SPWeb, the GUID of the SPList, the GUID of the SPListItem and the SPFileSystemObject type of the SPListItem.

While we could pollute the interface of our business object with this data, that would just be plain nasty. Instead, we can have the serialization process take care of it for us by building a JavaScriptConverter.

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Web.Script.Serialization;
using Microsoft.SharePoint;

namespace Acme.Widget
{
    internal class SPListItemScriptConverter : JavaScriptConverter
    {
        public override IEnumerable<Type> SupportedTypes
        {
            get { return new ReadOnlyCollection<Type>(new List<Type>(new[] {typeof (SPListItem)})); }
        }

        public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer)
        {
            if (dictionary == null) throw new ArgumentNullException("dictionary");

            if (type == typeof (SPListItem))
            {
                using (var site = new SPSite(new Guid(dictionary[@"SiteId"].ToString())))
                {
                    SPWeb web = site.OpenWeb(new Guid(dictionary[@"WebId"].ToString()));
                    SPList list = web.Lists[new Guid(dictionary[@"ListId"].ToString())];
                    SPFileSystemObjectType objectType;

                    if(Enum.TryParse(dictionary[@"Type"].ToString(),out objectType))
                    {
                        if (objectType == SPFileSystemObjectType.Folder)
                        {
                            return list.Folders[new Guid(dictionary[@"ListItemId"].ToString())];
                        }
                    }
                    return list.Items[new Guid(dictionary[@"ListItemId"].ToString())];
                }
            }

            return null;
        }

        public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)
        {
            var listItem = obj as SPListItem;

            if (listItem != null)
            {
                var result = new Dictionary<string, object>
                    {
                        {@"SiteId", listItem.Web.Site.ID},
                        {@"WebId", listItem.Web.ID},
                        {@"Type", listItem.FileSystemObjectType},
                        {@"ListId", listItem.ParentList.ID},
                        {@"ListItemId", listItem.UniqueId}
                    };

                return result;
            }
            return new Dictionary<string, object>();
        }
    }
}

We can then make use of the convertor when serializing/deserializing a business object in our web service code like this:

public Stream GetBusinessObject(string someProperty)
{
    //Get the object from somewhere
    var theObj = new BusinessObject();

    if (WebOperationContext.Current != null)
        WebOperationContext.Current.OutgoingResponse.ContentType = "application/json; charset=utf-8";

    var serializer = new JavaScriptSerializer();
    serializer.RegisterConverters(new JavaScriptConverter[] {new SPListItemScriptConverter() });
    string json = serializer.Serialize(theObj);

    return new MemoryStream(Encoding.UTF8.GetBytes(json));
}

public Void SetBusinessObject(Stream jsonData)
{
    var jsonText = new StreamReader(jsonData).ReadToEnd();
    var serializer = new JavaScriptSerializer();
    serializer.RegisterConverters(new JavaScriptConverter[] { new SPListItemScriptConverter() });
    var theObj = serializer.Deserialize<BusinessObject>(jsonText);

    //Do something useful with the object complete with valid SPListItem reference
}

Posted in JavaScript, SharePoint, SharePoint 2013, WCF | Leave a comment

Capturing ULS data via TypeMock

As I’ve probably mentioned before, I use TypeMock for unit testing of my SharePoint server side code. Most of the time I’m not interested in what my components will write to ULS when operating normally so I’ll simply fake my ULS component using TypeMock.

I have a standard ULS wrapper that I use that looks like this:

using System;
using Microsoft.SharePoint.Administration;

namespace Acme.Widget
{
    /// <summary>
    /// Wrapper class for writing trace messages to the ULS
    /// </summary>
    internal class Uls
    {
        public static void WriteException(UlsCategory category, Exception ex)
        {
            Write(TraceSeverity.Unexpected, EventSeverity.Error, category, ex.ToString(), null);
        }

        public static void WriteError(UlsCategory category, string message, params object[] parameters)
        {
            Write(TraceSeverity.Monitorable, EventSeverity.Warning, category, message, parameters);
        }

        public static void WriteHigh(UlsCategory category, string message, params object[] parameters)
        {
            Write(TraceSeverity.High, EventSeverity.Information, category, message, parameters);
        }

        public static void WriteMedium(UlsCategory category, string message, params object[] parameters)
        {
            Write(TraceSeverity.Medium, EventSeverity.None, category, message, parameters);
        }

        public static void WriteLow(UlsCategory category, string message, params object[] parameters)
        {
            Write(TraceSeverity.Verbose, EventSeverity.None, category, message, parameters);
        }

        protected static void Write(TraceSeverity trcSeverity, EventSeverity evtSeverity, UlsCategory category,
                                  string message, object[] parameters)
        {
            UlsDiagnosticsManager svc = UlsDiagnosticsManager.Local;
            if (svc != null)
            {
                SPDiagnosticsCategory cat = svc[category];

                try
                {
                    if (evtSeverity != EventSeverity.None)
                    {
                        svc.WriteEvent(1, cat, evtSeverity, message, parameters);
                    }
                }
                catch (Exception ex)
                {
                    svc.WriteTrace(1,svc[UlsCategory.Administration],TraceSeverity.Medium,"Unable to write to event log {0}",ex);
                }

                if (trcSeverity != TraceSeverity.None)
                {
                    svc.WriteTrace(1, cat, trcSeverity, message, parameters);
                }
            }
        }
    }
}

So in my unit tests I’ll fake all calls to this using:

Isolate.Fake.StaticMethods(typeof(Uls));

Pretty simple eh?

Anyways, rather than simply ignoring this potentially useful information, I’ve recently started redirecting ULS calls to the console using this:

Isolate.NonPublic.WhenCalled(typeof(Uls), "Write").DoInstead(callContext =>
    {
        var trcSeverity = (TraceSeverity)callContext.Parameters[0];
        var evtSeverity = (EventSeverity)callContext.Parameters[1];
        var category = (UlsCategory)callContext.Parameters[2];
        var message = (string)callContext.Parameters[3];
        var parameters = (object[])callContext.Parameters[4];

        Write(trcSeverity, evtSeverity, category, message, parameters);
    });

This simple bit of TypeMock trickery allows me to redirect my ULS calls to a local static implementation such as:

private static void Write(TraceSeverity trcSeverity, EventSeverity evtSeverity, UlsCategory category,
                        string message, object[] parameters)
{
    if (parameters == null) parameters=new object[0];

    Console.WriteLine(@"Category:{0}, Message:{1}",category,string.Format(message,parameters));
}

I can now view my any ULS logs generated by my code by clicking on the Output link

image

image

Posted in Development, SharePoint, SharePoint 2013, Tip, TypeMock | Tagged | Leave a comment

TypeScript in a SharePoint Farm Solution

One of the good things about the holiday season is that I’ve got plenty of free time to catch up with new developments. I suppose one of the drawbacks of being a SharePoint Developer is that I tend to focus on all things SharePoint while ignoring potentially useful developments elsewhere. One such development is TypeScript. I stumbled upon this video on channel 9 and it really got me thinking about how useful the tool is for SharePoint development, especially in light of the changes in SharePoint 2013.

Much as I’m sure I’ll get slated for this statement: JavaScript is the future of user experience development. Sure it’s been around for a while and is creaking under the weight of the burdens placed upon it but there really is no other alternative that ticks all of the boxes. Evolutionary pressure will enhance the capabilities of JavaScript and undoubtedly address some of it’s shortcomings, however, while the interminable commercial wrangling continues at ECMA, TypeScript plugs a few significant holes and makes the language more suitable for ‘serious’ development.

In this post I’m going to look at how TypeScript can be used for SharePoint development. There are a few posts that tackle how TypeScript can be used for App development but I’m interested in building a farm solution that uses TypeScript to improve the quality of scripts that are used by my web parts.

Setup

The TypeScript compiler can be installed using the TypeScript for Visual Studio 2012 add-in. You can download it from here:http://www.microsoft.com/en-us/download/details.aspx?id=34790. For this article I’m using v 0.8.1.1.

Since TypeScript works by compiling TypeScript files into JavaScript files, we need to tell Visual Studio to run the TypeScript compiler. Although Installing the Visual Studio add-in gives us the ability to add TypeScript items to a project it doesn’t modify the project file to include the required compilation.

Steps to enable this are:

  1. Unload the project
  2. Edit the project file.
  3. At the end of the .proj file add the following xml:
<PropertyGroup>
  <TypeScriptSourceMap> --sourcemap</TypeScriptSourceMap>
</PropertyGroup>
<Target Name="BeforeBuild">
  <Message Text="Compiling TypeScript files" />
  <Message Text="Executing tsc$(TypeScriptSourceMap) @(TypeScriptCompile ->'&quot;%(fullpath)&quot;', ' ')" />
  <Exec Command="tsc$(TypeScriptSourceMap) @(TypeScriptCompile ->'&quot;%(fullpath)&quot;', ' ')" />
</Target>

Save the changes and reload the project and we’re good to go.

Deploying TypeScript output to SharePoint

When I’m working on a SharePoint solution I like to keep any scripts that will be deployed to the Layouts folder in a single element rather than having them scattered around in web part elements etc. To show how TypeScript can be deployed I’ll demonstrate this approach although the process is pretty much the same if your scripts are contained in other elements.

1. Create an Empty Element – rename it to Scripts

2. Delete the Elements.xml file – we don’t need it when deploying to the file system

image

3. Add a new TypeScript file (You can find the TypeScript template by typing Ctrl-E  then typing ‘TypeScript’ and hitting enter). I’ve called my file chaholl.sample.ts

image

4. Once the new file has been added to the Scripts element we need to change the deployment settings for the compiled script file. Expand the chaholl.sample.ts node and select the chaholl.sample.js item.

5. Using the Properties pane set the Deployment Type to TemplateFile and expend the Deployment Location section to allow you to change the Path property to layouts\chaholl\Scripts\

image

The compiled JavaScript will now be deployed to the layouts folder when the wsp is installed.

Debugging

TypeScript makes use of SourceMap v3 to provide a mechanism for us to debug our compiled script using the original TypeScript code rather than the compiled JavaScript. To make this work we first need to compile our project. Hit Shift-F6 to rebuild.

If we now examine the chaholl,sample.js file we can see that the last line reads:

//@ sourceMappingURL=chaholl.sample.js.map

This is a reference to a mapping file that contains information necessary to map the compiled JavaScript back to our TypeScript file. (You can find out more about the SourceMap standard here). If we want to debug from SharePoint we need this mapping file to be deployed to the server as well. From the PROJECT menu select Show All Files. Notice that chaholl.sample.js.map appears in the Scripts element. To ensure that this file is deployed right-click on it and select ‘Include in Project’ and then use the Properties pane as before to deploy the file to the layouts folder,

image

We can now rebuild and deploy the package and our TypeScript generated files will be deployed and ready for debugging.

References:

http://www.typescriptlang.org/

http://www.ecma-international.org/publications/standards/Ecma-402.htm

http://channel9.msdn.com/Shows/Going+Deep/Anders-Hejlsberg-and-Lars-Bak-TypeScript-JavaScript-and-Dart

http://www.microsoft.com/en-us/download/details.aspx?id=34790

https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k/edit?pli=1

Posted in JavaScript, SharePoint 2013, TypeScript, Visual Studio 2012 | 7 Comments

SharePoint 2010: Search via REST

With the release of SharePoint 2010 we got a whole load of useful REST services.  For example, we can read data from any list or library using ListData.svc and we can do some clever stuff with Excel Services using ExcelRest.aspx. These features make it easy to enhance the user experience by using tools such as JQuery and ajax. However, one thing that’s missing, in my opinion, is a REST service for Search. In this post, I’m going to go through the steps to build a basic proof-of-concept service. The techniques used will be useful for building and deploying other types of REST service to the SharePoint 2010 platform.

Note: Although the isn’t a REST API for search, there is an RSS facility that achieves many of the same goals. It can be accessed at _layouts/srchrss.aspx and accepts the same ‘k’ parameter as other search pages.

Adding a WCF endpoint to SharePoint

I covered this some time ago in this post. However, rather than take all these steps manually, I highly recommend installing the CKSDev extension for Visual Studio. The rest of this post will assume that you have it installed.

1. Create a new Empty SharePoint Project. Set it to ‘Deploy as a farm solution’

imageimage

2. Add a new WCF Service named ‘SearchService’ (With no space)

image

3. A new WCF SPI will be added to the project containing three files: ISearchService.cs, SearchService.svc and SearchService.svc.cs . If we were building a normal WCF service we could jump straight in and start adding our methods but for a REST service we need to make a few changes first. In the SearchService.svc file, change:

<%@ ServiceHost Language="C#" Debug="true"
    Service="RESTDemo.SearchService, $SharePoint.Project.AssemblyFullName$"
    CodeBehind="SearchService.svc.cs"
    Factory="Microsoft.SharePoint.Client.Services.MultipleBaseAddressBasicHttpBindingServiceHostFactory, Microsoft.SharePoint.Client.ServerRuntime, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>

to:

<%@ ServiceHost Language="C#" Debug="true"
    Service="RESTDemo.SearchService, $SharePoint.Project.AssemblyFullName$"
    CodeBehind="SearchService.svc.cs"
    Factory="Microsoft.SharePoint.Client.Services.MultipleBaseAddressWebServiceHostFactory, Microsoft.SharePoint.Client.ServerRuntime, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>

There are a few service host factories available for use with SharePoint, MultipleBaseAddressBasicHttpBindingServiceHostFactory will create an endpoint that uses basicHttpBinding whereas MultipleBaseAddressWebServiceHostFactory will create an endpoint that uses webHttpBinding. For a REST service we need a webHttpBinding.

4. Since there is no way to generate WSDL type information for a REST service, we need to disable this functionality. This can be done by deleting the [BasicHttpBindingServiceMetadataExchangeEndpoint] attribute from the SearchService class in SearchService.svc.cs

5. The next thing we need to do is to signify that our methods are callable via HTTP get. We can do this by adding the WebGet attribute to each method in the ISearchService interface.

using System.ServiceModel;
using System.ServiceModel.Web;

namespace RESTDemo
{
    [ServiceContract]
    public interface ISearchService
    {
        [OperationContract]
        [WebGet]
        string HelloWorld();
    }
}

With this done, we can deploy the solution and check that everything works as it should. Accessing the service at: http://<Your-site-url>/_vti_bin/RESTDemo/SearchService.svc/HelloWorld

Will show the result:

image

6. Now the we have the basics in place, the next step is to add the implementation for our search service. In ISearchService, add a new method:

[OperationContract]
[WebGet]
string Search(string query,string scope);

7. Add the implementation to SearchService.svc.cs. (You’ll need to reference, Microsoft.Office.Server.dll, Microsoft.Office.Server.Search.dll and System.Web.dll):

using System.Globalization;
using System.IO;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using System.Web.Configuration;
using Microsoft.Office.Server.Search.Administration;
using Microsoft.Office.Server.Search.Query;
using Microsoft.SharePoint;

namespace RESTDemo
{
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Required)]
    public class SearchService : ISearchService
    {
        #region ISearchService Members

        public string HelloWorld()
        {
            return "Hello World from WCF and SharePoint 2010";
        }

        public Stream Search(string query, string scope)
        {
            const int maxResultCount = 10;
            var ms = new MemoryStream();
            using (KeywordQuery oKqry = GetKeywordQuery(query, maxResultCount, scope))
            {
                ResultTableCollection oResults = oKqry.Execute();
                oResults[ResultType.RelevantResults].Table.WriteXml(ms);
            }

            if (WebOperationContext.Current != null) WebOperationContext.Current.OutgoingResponse.ContentType = "text/xml";
            ms.Seek(0, SeekOrigin.Begin);
            return ms;
        }

        #endregion

        private static KeywordQuery GetKeywordQuery(string queryString, int maxResultCount, string currentScope)
        {
            var query = new KeywordQuery(
                SPServiceContext.Current.GetDefaultProxy(typeof (SearchServiceApplicationProxy)) as
                SearchServiceApplicationProxy)
                            {
                                RowLimit = maxResultCount,
                                KeywordInclusion = KeywordInclusion.AllKeywords,
                                ResultTypes = ResultType.RelevantResults,
                                QueryText = queryString,
                                Culture = CultureInfo.CurrentCulture,
                                AuthenticationType = SPSecurity.AuthenticationMode !=
                                                     AuthenticationMode.Windows
                                                         ? QueryAuthenticationType.PluggableAuthenticatedQuery
                                                         : QueryAuthenticationType.NtAuthenticatedQuery
                            };

            if (!string.IsNullOrEmpty(currentScope))
            {
                query.HiddenConstraints = "scope:"" + currentScope + """;
            }

            return query;
        }
    }
}

We can now build and redeploy and the job’s almost done. We can execute search queries using:http://<Your-site-url>/_vti_bin/RESTDemo/SearchService.svc/Search?query=internet and we’ll see results like:

image

To add a bit of polish, we can change the URL format so that we can execute queries like …/Search/<Query>?scope=whatever

In ISearchService, change the UriTemplate property on the WebGet attribute to: [WebGet(UriTemplate = "search/{query}?scope={scope}")]. Redeploy. Job done! We can now see results using URL like:

http://<Your-Site_url>/_vti_bin/RESTDemo/SearchService.svc/Search/internet

Posted in Development, Enterprise Search, HowTo, JQuery, REST, SharePoint, SharePoint 2010, WCF | Tagged , , , , , , , | 3 Comments