Radi Atanassov

SharePoint MCM, MVP, MCT and owner of OneBit Software

Recompiling your own version of the SharePoint code to suit your needs

Ever been pissed off at the SharePoint code? Banging your head against the wall, trying to figure out why something is not working for you, you're looking at it with a reflection tool and just want to see what would happen if you change a single line of code?

You don't have the SharePoint source code, but you still want to fiddle with it and compile your own build? You might have never thought that is possible, but here in this post I will show you how!

This is by far extremely awesome and some seriously fun shit, so enjoy!

(Disclaimer: don't do this, seriously. Just learn from it. I am sure that reverse engineering is written to be forbidden somewhere. And don't do this anywhere near a production server.)

STEP 1) Get the Reflexil plugin, either in Reflector or Telerik JustDecompile. It is called "Assembly Editor" and is available as a JustDecompile Plugin. Ever since Red Gate basterdised Reflector I prefer Telerik JustDecompile, it is free and built in Sofia, where I live and where OneBit Software is located.

Once you get all the components in there, load your tool and the assembly that you want to fiddle with.

In my example I modify Microsoft.SharePoint.ApplicationPages.Administration.dll, to make the CA UI show that I have a custom build, but you could easily do this with Microsoft.SharePoint.dll or any other assembly.

STEP 2) Browse through the assembly and find the code you want to modify. If you are tweaking a property, you need to specifically select the getter or setter. The UI will look like below, with many tabs in the Reflexil UI:

Right-click and either choose to edit the instruction, or "Replace all with code" for the fun stuff.

STEP 3) You will end up in an interface looking like the one below. On the left you will see generated code, on the right you will see instruction information. In my example, I replaced return default(string); with return "Radi's Custom SP Build!";

Make sure the Compiler version is the one relevant for the assembly, in my case v3.5, and hit Compile. You will get standard compiler errors if you have screwed something up. Otherwise if successful, the Instructions on the right will change as per your modification. That stuff is IL, you might already be familiar.

STEP 4) Go ahead and save your modified assembly. It will prompt you to save it with "Patched" in the filename:

STEP 5) After you save it, Reflexil/Telerik JustDecompiler will tell you some very important stuff about the Strong Name of the assembly. This step is key to get this to work and there's deep theory about assembly signing coming to play.

Assemblies are usually signed with a key so that they don't get tampered with in the way I'm doing it. Unless you have the key you can't modify it and resign it. This is something that Visual Studio does when it builds an assembly.

You can either remove the Strong Name, or register the assembly for verification skipping. This is something you could do with the sn.exe tool, or just through the interface below:

"Register it for verification sipping" requires that "sn.exe" is in the path variable so the tool can call it. It will basically execute:

sn -Vr yourAssembly.dll

I had to add "C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools\x64" to my Environment PATH variable and restart JustDecompiler. You might want to do this before you edit assemblies so you don't have to do it again. This is what you will see if sn.exe is not found:

 

STEP 6) What you should see next is the patched assembly, but the most important part is the PublicKeyToken, it should not be null:

 

STEP 7) The last step, rename the "Patched" DLL to its original name, then do an IISRESET.

 

And here she is, the most beautiful SharePoint release ever:

Please comment if you like this post!

Day 1 highlights for developers at SPC14

 

The hype here at #SPC14 is absolutely huge. I am extremely excited about so many new things that are announced. I'm going to start keeping track of everything I find interesting and post it, so keep an eye out.

For developers:

 

Office Web Widgets (Preview) - http://www.nuget.org/packages/Microsoft.Office.WebWidgets.Experimental/ Nice components for provider hosted apps. The People Picker is something to check out.

 

Office SDK for Android - https://github.com/OfficeDev - this looks like a repository that the MS Office team will publish dev related stuff. So far the Android SDK is there, but maybe that’s it.

 

SharePoint Workflow Apps - I still need to find a link for this one. Keep an eye for an update.

 

Office 365 Device and website Apps - Same as above. Keep an eye for an update.

 

 

SharePoint 2013 and SharePoint Online solution pack for branding and site provisioning http://www.microsoft.com/en-us/download/details.aspx?id=42030 - Nice, putting this in my reading todo list.

 

Office App Model samples - http://officeams.codeplex.com/ (Blogged by Vesa here: http://blogs.msdn.com/b/vesku/archive/2014/03/03/announcing-office-app-model-samples-codeplex-project.aspx) - Really nice stuff, I saw Steve Walker hit the "publish" button in front of me. That was epic…. I can't wait to find time to check these out

 

There's some other new stuff like O365 Discovery Services & Azure AD OAuth. I still have to get my head around that one.

 

Deploying and debugging SharePoint Apps – Sideloading the easy way

If you try to deploy an App from Visual Studio to anything other than a developer site you will get this error:

“Error occurred in deployment step 'Install app for SharePoint': Sideloading of apps is not enabled on this site.”

This is due to a feature that is not enabled on any other site template apart from the Developer Site. This error will also occur if you are deploying to a host named site collection that is not at the root of the web application. You will also get this when you are deploying to a site in Office 365 that is not a Developer Site.

You might be developing a solution and you may want to deploy it to a site that is not a Developer Site.

To enable this for on-premises environments, run this PowerShell script:


Enable-SPFeature -Identity AE3A1339-61F5-4f8f-81A7-ABD2DA956A7D–url <siteurl>

To enable this on Office 365 sites you should use the client-side object model. Here is an example:

   1: $siteCollectionUrl = ""
   2: $username = ""
   3: $password = ""
   4: [Microsoft.SharePoint.Client.ClientContext]$context = New-Object Microsoft.SharePoint.Client.ClientContext($siteCollectionUrl)
   5: [Microsoft.SharePoint.Client.SharePointOnlineCredentials]$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $password)
   6: $context.Credentials = $credentials
   7: $siteCollection = $cc.Site;
   8: $sideLoadingGuid = new-object System.Guid "AE3A1339-61F5-4f8f-81A7-ABD2DA956A7D"
   9: $siteCollection.Features.Add($sideLoadingGuid, $true, [Microsoft.SharePoint.Client.FeatureDefinitionScope]::None);
  10: $context.ExecuteQuery();

The only thing that this feature does is “register” itself as an activated feature on the site. You can check the event receiver, it does nothing but checks.

Here is a piece of the Microsoft SharePoint code that actually checks if sideloading is enabled:

   1:internalstaticbool IsAppSideloadingEnabled(SPSite site)
   2: {
   3:if (site == null)
   4:     {
   5:thrownew ArgumentNullException("site");
   6:     }
   7:return SPDeveloperData.IsDeveloperSite(site) || site.Features[FeatureIds.EnableAppSideLoading] != null;
   8: }

This is performed in the API that VS uses to deploy Apps.

Hope this helps!

Architecting your SharePoint application–things you shouldn’t miss out on

So a few people recommended I post my slides or content from my European SharePoint Conference session. I cover a list of considerations that makes a good reference for people undertaking the design of custom applications on SharePoint.

Usually in projects you would have people responsible for the design of the infrastructure and then a development team would dig into the technical design of the SharePoint application. They will try to answer how different components will be used to satisfy the requirements. Solution Architects explore various options for meeting each requirement and all these options and choices intertwine into a proposed design, maybe a model or a proof of concept, and hopefully a document. Projects that miss this communication are either chaotic, or extremely agile.

When doing architecture there is usually more than one possible way to achieve the same thing – the “right” one will depend on the situation and each of the things I’m pointing out here could be equally right or wrong. This is especially true for SharePoint. That is why “awareness” and the knowledge of SharePoint are one of the key requirements for an architect to be any good.

This list is not the ultimate list – it is just my summary of what always lands on my table. It is also not the base for a definitive Technical Design document (it’s a start!). Some items below didn’t get mentioned in my talk as time was limiting what I could include.

Front-end Planning

Page Model – How you plan to store you (ASPX) pages

  • Application Pages (_layouts)
  • ASPX files served from a Document Library
  • ASPX files in folders
  • Publishing Infrastructure – page layouts
  • Consider Web Part Pages vs. Wiki Pages

Form Strategy – Consider how you will capture data and what controls/interfaces you will use

  • Understand how SharePoint forms work and consider using SharePoint’s API’s
  • SharePoint InputControls are great but may be difficult to use and may have limitations
  • Consider exchanging data between forms. Plan Session and ViewState requirements
  • SharePoint Scenario Framework
  • Consider validation requirements and the UX on the validaiton
  • Also consider Silverlight and InfoPath forms as alternatives for capturing data

Client-Side Scripting – plan out any requirements for client-side JavaScript

  • If using validation, plan out your client-side scripting and the use of any frameworks
  • Will you use the Dialog Framework or any other popup/dialog framework/toolkit?
  • Consider the use of jQuery, Modernizr, Knockout

Page Components – define what controls, web parts or other components you will use to create the actual interfaces

  • Web Parts vs. User Controls
  • SharePoint Rendering Templates and the Form UI (_controltemplates)
  • Consider the styling/branding of your custom interfaces – you don’t want your developers to be designers (unless they really are)
  • Iframe – various solutions use Iframes to display external content or components hosted elsewhere (plan out authentication)
  • Consume HTML asynchronously – I have seen solutions that grab HTML from an ASHX or other services
  • InfoPath – always think about the User Experience when you deal with InfoPath

Resource Files – evaluate how you will use resource files and what components will require localisation

  • Code-behind resource files
  • ASPX resource files (14\Resources, AppGlobalResources)
  • Feature Resources
  • Localised Web Templates

Back-end Planning

SharePoint Data Model – define the storage of data

Consider:

  • How are you going to store data?
  • How will it scale?
  • How are you going to access it? (SP OM, External Data (BCS), Web Service calls)
  • How are you going to “replicate” it? (backup, archive, move/copy, log)
  • How are you going to store applicaiton settings (SP list, P&P Settings)

Visual Studio Solution Structure – How will you structure code and artefacts

  • Number of WSPs/Number of Features
  • Separation of code
  • Separation of SharePoint items
  • Namespaces and naming conventions
  • Source Control strategy

Design Patterns/Anti-Patterns – make your code maintainable and nice if it makes sense to do so

  • SharePoint Service Locator (P&P)
  • Façade/Adapter
  • Consider/plan your data access layer

Security Model – don’t think about it in production

AuthN:

  • Kerberos/NTLM (consider the requirement for Kerberos)
  • Claims (consider the effort to pull it off and any side-effects)
  • Plan the use of service accounts and access for/to external systems

AuthZ:

  • Use SP groups or AD groups?
  • Nested groups
  • Do you really need item-level security?

Exception Handling and Logging – define how to display/capture errors and how you log them

  • Consider how you will display errors to the user (don’t do lblMessage.Text = ex.ToString(); )
  • Define what needs to be logged and how
  • SharePoint Logger (P&P) gives you a good API for Diagnostic categories and areas

Deployment

Solution Deployment Frameworks – define how your developers will deploy

  • MSBuild
  • NAnt
  • 100% PowerShell
  • CKS-Dev addin  for Visual Studio 2010

Other Knobs and Dials – there are many other things that affect deployment

  • WSP lifecycle
  • Feature activation
  • Activate on Default
  • Deployment configuration
  • WSP additional DLL's
  • Force on Activate
  • Safe Controls
  • Web.config modifications
  • Web Part deployment
  • List re-creation
  • Feature Upgrade

Site Templates – reusable functionality that admins or end users could provision

  • Site Definitions
  • Web Templates

Continuous Integration and Testing

  • Plan and define the CI/Build process
  • Always consider how you will upgrade the solution
  • Define the unit testing and mocking requirements (Pex & Moles, TypeMock)
  • Funcitonal UI Testing (Selenium, Coded UI, Telerik Test Suite)
  • Always sync with the test team and what they do/how they will test your solution

To close off the talk I finished off with a few tips:

  • Build applications in such a way that makes you feel proud of what you have built. Doctors feel good when they help people, architects feel good when their creation is built, lawyers are happy when they get paid – there is no reason why SharePoint developers and architects shouldn’t feel good about what they do if they do it well.
  • Motivate your team to do good work – As an architect you are most likely a role model. Do your best to motivate your team members to be champions and get good stuff out there. Reward them for good efforts.
  • Change your job if your boss/architect/team leader/project manager is pressuring you to do crap work with no process around it, no scope, no clarity, no design, etc. Your SharePoint career is way to short for you to be doing crap work in crap teams. Only you could make that change.
  • Apply development practices and architecture to SharePoint solutions – many say that there is no real development in SharePoint. It’s true that there is a lot of “other stuff” in SharePoint projects, but for the development part – make it count.

Here is my slide deck: SharePoint Solution Architecture – Radi Atanassov

During my talk I showed bits of a solution I use to POC various SharePoint components. I use it to explain and demonstrate things to students, forums and colleagues. I call it Community.SharePoint and as soon as it has a few other key components I will post it on CodePlex.

Here is the version I used at the European SharePoint Conference: Community.SharePoint-EUSPC

Hope this helps someone!

My Session at the #EuropeanSP Conference

I’m at the Estrel hotel in Berlin on the tutorial day of European SharePoint Conference. Most people will be arriving today – you can definitely feel the SharePoint vibe in the air. The Estrel Convention is absolutely HUGE, walking around this place is a calorie-burning experience!

I’m engaged in a few activites:

  • Wednesday 19th October 15:00 – Ask the Expert Session – IT Decision Makers
  • Thursday 20th October 11:15 – Ask the Expert Session – Developers
  • Thursday 20th October 14:00 – Advanced Solution Architecture & Development – my talk for this conference.

Please come and say hello!

Custom WCF Services and setting Reader Quotas in SharePoint 2010

I’ve been working quite a bit with custom WCF services in SharePoint 2010 and have found there’s quite a bit to it. The part that worries me is that there is a ton of information out there that is not always best for enterprise scenarios, like setting the site in IIS to anonymous just to get service calls working.

Anyway… I was trying to figure out why the reader quota settings weren’t getting applied to our services. We had the following debug code:

  1. SPWebService contentService = SPWebService.ContentService;
  1.  
  2. contentService.ClientRequestServiceSettings.MaxReceivedMessageSize = -1;
  3.  
  1. SPWcfServiceSettings wcfServiceSettings = new SPWcfServiceSettings();
  1. wcfServiceSettings.ReaderQuotasMaxStringContentLength = Int32.MaxValue;
  2. wcfServiceSettings.ReaderQuotasMaxArrayLength = Int32.MaxValue;
  3. wcfServiceSettings.ReaderQuotasMaxBytesPerRead = Int32.MaxValue;
  4. wcfServiceSettings.MaxReceivedMessageSize = Int32.MaxValue;
  5. wcfServiceSettings.MaxBufferSize = Int32.MaxValue;
  6. wcfServiceSettings.ReaderQuotasMaxDepth = Int32.MaxValue;
  7. wcfServiceSettings.ReaderQuotasMaxNameTableCharCount = Int32.MaxValue;
  8. wcfServiceSettings.ReceiveTimeout = TimeSpan.MaxValue;
  9.  
  10. contentService.WcfServiceSettings["MyService.svc"] = wcfServiceSettings;
  11.  
  12. contentService.Update(true);

This was placed in a feature scoped at the web application level, and the WSP was deploying to /ISAPI/CustomWcf/MyService.svc”. We we’re using SharePoint’s MultipleBaseAddressBasicHttpBindingServiceHostFactory factory, and if you are you don’t need to set anonymous in IIS.  This factory applies your security settings as they are on the web application.

So why weren’t our WCF Service settings getting applied? If you check out this MSDN article, it doesn’t actually tell you what I figured out:http://msdn.microsoft.com/en-us/library/ff599489.aspx

The problem is that “MyService.svc” should actually be lower case: “myservice.svc”. It sounds silly, but that is what worked for me. The /CustomWcf/ folder part should be excluded.

If anyone is doing the same, here are a few tips on how such deployment strategy could be improved:

  • The feature should really be at a farm level if it is modifying settings on the Content Service. There is one per farm.
  • Feature Deactivation code should call contentService.WcfServiceSettings.Remove("myservice.svc");
  • You SHOULD NOT just use Int32.MaxValue, but actually figure out your maximums and what exact properties you should apply them on. In my example I’m obviously trying to get a sample to work.
  • Make sure your deactivation doesn’t break any other solution if you are resetting with this line: contentService.ClientRequestServiceSettings.MaxReceivedMessageSize = 0;

Hope this helps!

“Activate on Default” confusion and features scoped at Web Application level

When creating SharePoint features in Visual Studio 2010, one of the settings that defaults to True is “Activate on Default”.

image

There is a lot of confusion as to what this setting actually does:

  • It ONLY applies to features scoped at Farm and Web Application levels. You can still modify it for other features, but it doesn’t do anything.
  • Any feature (Farm or WebApplication) with that setting set to True will automatically activate when you deploy the WSP solution, no matter which way you deploy it (Install-SPSolution, stsasm.exe, Central Administration)

This setting is not related to the deployment configuration settings in Visual Studio 2010:

clip_image001

These features will still activate, even if VS’s deployment configuration is set to “No Activation”.

Where can this be an inconvenience?

When you create features that deploy Timer Jobs at the Web Application level, you really want to have “Activate on Default” set to False. Otherwise, your feature will be activated on ALL web applications. I did some tests and found out that even if your WSP Solution is not global and is deployed to a specific Web Application, your feature will still get activated. Dangerous, you really want your Timer Jobs to be running where they are meant to run, i.e. don’t deploy Timer Jobs to Central Administration unless you really need to.

If you are ever trying to find out why your Timer Jobs are “attached” to all web applications, this might be why.

Fun with HTTP Handlers, Security Validations, FormDigest, AllowUnsafeUpdates, jQuery, AJAX and POST parameters in SharePoint

Ever seen this error message?

System.Exception: Microsoft.SharePoint.SPException: The security validation for this page is invalid. Click Back in your Web browser, refresh the page, and try your operation again.

It is usually related to a missing SharePoint FormDigest control, or updates to the DB on an HTTP GET request. You might hear people saying you should set AllowUnsafeUpdates to true, but in the case of a POST request that is not the best thing you could do. The best resource that you could ever read on the topic is written a while back (in 2008!) by a good friend of mine and ex colleague - Hristo Pavlov. These two posts are your best starting point if you want to understand what these items are for and how they achieve their purpose.

What You Need To Know About AllowUnsafeUpdates (Part 1)

What You Need To Know About AllowUnsafeUpdates (Part 2)

Generally speaking, AllowUnsafeUpdates = true on POST shouldn’t be required at all. I was working with my team on an HTTP Handler living in the SharePoint Layouts folder and it was failing with the security validation exception. In a typical ASPX web page you would include the SharePoint FormDigest control and SharePoint will handle it from there onwards:

<SharePoint:FormDigest runat="server"/>

 

You will notice the output of this control is a hidden <input> like this one:

 

<input name="__REQUESTDIGEST" id="__REQUESTDIGEST" type="hidden" value="0xDA527A96…A23,22 Apr 2011 14:17:06 -0000"/>

 

SharePoint will use this control (in particular the parameter __REQUESTDIGEST) and validate the “FormDigest”. You can explicitly call the SPUtility.ValidateFormDigest() helper method achieves the same. (See Hristo’s blog posts for more info on how it works). It basically takes the __REQUESTDIGEST value and validates it on the request object.

 

But in an HTTP Handler you don’t have the <SharePoint:FormDigest /> control as there is no ASPX. Developers can get the handler working by setting AllowUnsafeUpdates on the SPWeb object, but this should be avoided when it could (see Hristo’s post on why it is not good). If you are making a POST request, pass in the __REQUESTDIGEST and make sure you call the SPUtility.ValidateFormDigest() method before you do any DB updates.

 

If you want to call your handler asynchronously with AJAX, lets say with jQuery, this adds another level of complexity. You have to pass in the __REQUESTDIGEST parameter for SPUtility.ValidateFormDigest() to succeed. I personally found documentation on the $.ajax jQuery method quite poor, but here is a JavaScript example on how to use it and pass the __REQUESTDIGEST <input/> value:

function UploadFileAsync() {

    var listId = $("input[id$='hdnListID']").val();

 

    $.ajax({

        type: "POST",

        url: "/_layouts/Handlers/FileUpload.ashx?ListID=" + listId,

        contentType: "application/x-www-form-urlencoded",

        data: "__REQUESTDIGEST=" + $("#__REQUESTDIGEST").val(),

        timeout: 30000,

        success: function (response) {

            alert(response);

        },

        error: function (x, t, m) {

            if (t === "timeout") {

                alert("got timeout");

            }

            else {

                alert(t);

            }

        }

    });

}

A few things are important and worth mentioning. In the “data” parameter I get the value of __REQUESTDIGEST and pass it in the POST request. (NOTE: you may want to improve the $(“#__RE..”) selector to get only input/hidden elements and be better performing). This will allow SPUtility.ValidateFormDigest() to pass successfully. If ever in doubt, open the request with Fiddler and validate the contents, you should see something like this:

clip_image001

The other important point is the contentType parameter. For me this did not work when set to “text/plain; charset=utf8”. I didn’t have enough time to figure out why, but “application/x-www-form-urlencoded” succeeded successfully.

Hope this helps!

Custom Upload Controls and the Maximum File Upload Size

If you’re creating a custom page or control that uploads files from a users PC (or somewhere else) you may get this error if the file is too big:

System.Web.HttpException: Maximum request length exceeded.

This is an ASP.NET error before the request is being handled by SharePoint (you can follow the stack trace to understand it a bit more). The challenge developers are facing is handling the exception and producing a nice & customized error message to users. You can easily overcome the problem by modifying the web.config of the Web Application, but that is not good practice and you have to worry about reproducing the modification for scalability and backup/restore purposes.

In a typical custom solution there is a nice solution to this problem. If you use Application Pages (those ASPX files that live in {SharePoint Root}\Templates\Layouts) you can place a web.config in your solution folder within the Layouts folder. Your web.config will go in:

14\TEMPLATE\LAYOUTS\CustomSolutionFolder\web.config

Here’s what you need in the web.config:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>

<configuration>

  <location path="CustomUploadPage.aspx">

    <system.web>

      <httpRuntime maxRequestLength="2097151" />

    </system.web>

  </location>

</configuration>

web.config’s are very flexible and the location element lets you apply your change exactly on the file that you need. This will work even if you are using an ASCX control located in the _CONTROLTEMPLATES folder. It is also a great approach as it can be packed into WSP’s and be part of an enterprise solution.

The number 2097151 is 2GB (boundary), the maximum SharePoint supported file size. It may be good to reduce this depending on your example.

The next point worth mentioning here is that the above setting alone won’t let you upload files over 50MB unless you configure the Web Application to allow it.

clip_image001

If the Web Application setting is below your file size you will get this exception:

Microsoft.SharePoint.SPException: The specified file is larger than the maximum supported file size.
NOTE: it has an ErrorCode of 2147024872

The good news is that you can handle the above exception with a simple Try…Catch, and that is all you need to facilitate a friendly, customized error message.

Hope this helps!