Creating pre-release NuGet packages in VSTS

Visual Studio Team Services has great features for generating NuGet packages and publishing them to a private NuGet feed as part of the build definition. These packages can then be used by your other solutions and will be restored on the build agent when building these other solutions.

If you are generating NuGet packages which have dependencies on pre-release NuGet packages, you need to mark your package as pre-release by adding a suffix to the version number such as '-beta1'.

In the NuGet Packager settings in VSTS you can turn off automatic package versioning and add a NuGet argument of "-version 1.0.0-beta1". This will work for the first build, but on subsequent builds the package will fail to publish because a package with the same version number already exists.

If your build number is in a suitable format you can just add "-beta1" to the end of it and then in the NuGet Packager build step set automatic package versioning to 'Use an environment variable' and the environment variable to 'BUILD_BUILDNUMBER'.

If you do not want the build number to be the same as the package version (you may not want the build to be tagged as beta), you can use a variation on the steps here. 

1. Add a PowerShell Script step before the NuGet Packer step
2. Set the Type to Inline Script
3. Set the Inline Script to something like:
   $PackageVersion = "$env:BUILD_BUILDNUMBER-beta1"
   Write-Host ("##vso[task.setvariable variable=PackageVersion;]$PackageVersion")
4. As before, set the NuGet Packager to use an environment variable (in this case PackageVersion)

Regardless of your approach, you will probably also need a delete step as described here to delete old packages from the build server before creating new ones.

1. Add a Delete Files step to the start of your build definition
2. Set the Source Folder to $(Build.Repository.LocalPath)
3. Set the Contents to *.nupkg

AngularJS Dialogs - Confirm Dialogs

Building on my previous post about creating alert dialogs with AngularJS, here we will look at creating confirmation dialogs we can use to get confirmation from the user before completing a destructive action such as deleting something.

In the directive we add a new scope variable 'confirmPositive' with the '&' binding to allow our controller to assign a function to it. When the OK button is pressed, this function will be called. In the template we add another button - the cancel button will retain the alert behaviour and just hide the dialog whereas the OK button will call a function which hides the dialog and then calls the confirmPositive function.


In the controller we just need to add the function which will carry out our destructive action when the user clicks OK.


In our HTML we just need to add an attribute so that the function in our controller is passed to our directive.


A complete example can be found here.

AngularJS Dialogs - Alert Dialogs

Moving from working with just jQuery to AngularJS can take some getting used to. Separating the front end from the logic of the controller throws up some interesting problems such as displaying alert and confirmation dialogs.

Here's one way to trigger a custom alert dialog from a controller.

To achieve this we create an AngularJS directive which, when activated, will cover the screen with a full-screen translucent element with a child element which will be our pop-up dialog.


We have a simple template with the full-screen background and a dialog with title, content and an OK button. When the button is clicked, visible is set to false. A link function initialises the visible variable to false and we set up some variables so that the title and content can be set and a variable mapped to the visible variable.

In our controller we can create a function to show our alert. In this example, when a button is clicked, we will set our alert message and show the alert:


The HTML including our alert directive looks like this:


A complete example can be found here.

WCF Known Types

WCF allows you to write web service methods that take or return objects of multiple types using the KnownType attribute. This is done by defining a base class and inheriting from this class for other classes that you want to use as parameters. The base type is decorated with the KnownType attribute for each inherited type and you use the base type as the parameter on your web service method.

Here's a quick example of a base class, derived class and using the KnownType attribute on the base class. It also includes a service contract with a web service method that accepts our base class.

When implementing the web service method, we can then check the type of the class we're being passed and react accordingly.

You can call this web service from your web front-end (here I'm using AngularJS):

$http.post(urlBase + '/Items/' + id, {'req': data});

To tell WCF which type you are passing to the web service method, you need to add a property to the object in the following format (that's two underscores):

__type: DerivedClass:#MyNameSpace

After a lot of experimentation, I could only see the object in the web service method as the base type. I finally got it working when I discovered that the __type property has to be the first property when the object is serialized to JSON. The following solves the problem by creating a new object in JavaScript with the __type property and then adds the properties from the original object:

  var d = {__type: 'DerivedClass:#MyNameSpace'};

  for (var i in data) {

      d[i] = data[i];

  }

  $http.post(urlBase + '/Items/' + id, {'req': d});

Microsoft Web Platform Day

This week I spent a day with Microsoft at their Web Platform Day to learn about what's new with ASP.NET. Here are some notes I took back to my team of things we should look into implementing and some things that don't really fit into our platform but are interesting to know about.

Task Runners

Task Runners can be used to….run tasks! This is usually done as part of the build process for tasks such as minifying JavaScript and CSS. There are a few task runners out there such as Grunt, Gulp and Broccoli(?!) but Microsoft were recommending Gulp. These are integrated a bit better in more recent versions of Visual Studio and ASP.NET, but can be used in any web development project.

A few of the task plugins:

gulp-minify-css: minifies your CSS files removing comments, whitespace and new lines to reduce file size and make the site load quicker
gulp-uglify: minifies JavaScript and can also do stuff like shortening names of functions and variables in the minified version of your JavaScript to reduce file size
gulp-autoprefixer: some browsers implement their own CSS features which are prefixed with –webkit, -moz or –ms. This makes sure you have prefixes for all browsers if you use one of them
gulp-concat: bundles files into a single file (e.g. multiple CSS files into one, or multiple JavaScript files into one) to reduce number of requests made by the browser and improve performance
gulp-imagemin: Optimizes .png and .jpg images for the web

Tasks can be chained together in Gulp so that you could minify several CSS files and then concatenate into one. You can also set Gulp to watch for changes to files and run automatically when you make changes.

Performance & Best Practises

There are a few check-lists, site scanners and plugins that will check sites for performance and best practises. Site authentication often prevents site scanning websites from accessing sites but you can still check your work with a plugin such as YSlow. This will give a report with a rating for a page and a list of recommendations. Many of the recommendations are covered by the task runners mentioned above (minifying and concatenating). Another checklist worth reading through is http://webdevchecklist.com/.

Image sprites are a single image file containing multiple icons and CSS to pick out individual icons from the file. This reduces the number of HTTP requests made by the page and improves performance compared to requesting each image individually. Going even further, it is recommended that icon fonts are used, where appropriate, so that the icons can easily be scaled and their colour can be changed.

There are web.config changes that can be implemented for caching settings and compression of files but then ‘cache busting’ strategies should be investigated so that clients get the latest versions of files when caching is being used.

Entity Framework

Entity Framework is a Microsoft technology for working with databases and simplifies going between C# classes and database tables. It is recommended that you take a code first approach – write your classes, set up your database connection string and the tables can be automagically created in the database. You can create classes from an existing database but you can get into problems if you want to make database changes and then import again.

ASP.NET Web API

Used to create RESTful web services and integrates well with Entity Framework. A Visual Studio feature (‘scaffolding’) can automatically create web service methods for CRUD operations from your Entity Framework models. You can also easily add OData support to add filtering and sorting to you web services via a query string (e.g. “‘?$name eq ‘bob’”).

ASP.NET Smart Tags / Tag Helpers

These are ASP.NET tags used in your HTML which are rendered on the server. One example give was loading scripts based on the environment. This lets you deliver full JavaScript and CSS files in the development environment but minified versions in production.

Another example was fall-back href’s when loading scripts. You can refer to CDN versions of popular scripts such as jQuery to improve performance, but provide a fall-back URL to a version on your server if the CDN is not available or the file is no longer on the CDN.

Web Performance and Load Test

A new project template in Visual Studio 2015 allows you to simulate web site load and analyse results. The requests are generated from Azure so presumably there is a cost associated with it and I'm not sure how it would work with site authentication.

ASP.NET Core

Microsoft have released an open source version of ASP.NET which runs on Windows, Linux and OSX. This can handle as much as 8 times the traffic of an ASP.NET 4.6 site on the same server. This is largely achieved by allowing you to configure which pipeline features are used when handling requests. 

You can also ship your site bundled with the framework which avoids version dependency problems on the server.

Despite all of this goodness, it isn't finished yet and doesn't have a lot of the features of ASP.NET 4.6.

Azure

Some interesting Azure features were demonstrated.

Platform as a Service / Infrastructure as a Service:

With Infrastructure as a service, you get a virtual machine and look after the software, patches etc. With platform as a service, that is all handled for you and you just upload your code/content.

Deployment Slots:

You can set up deployment slots in Azure for different environments (dev, test, live etc.). You can also use slots for A/B testing or staged roll-out. You can easily switch slots around when you want to move from test to live.

Continuous Deployment:

Deployment slots can be linked to a source repository such as a GitHub branch and automatically update if code is committed to the repository.

Auto-Scale:

Azure can be set up to automatically scale to more servers (within set limits) when demand exceeds a specified threshold. It can also automatically scale down if demand drops below a set threshold. This lets you maintain availability without spending more than you need.

Azure Storage:

A cheap way to host static content with good performance.

Azure Cloud Explorer:

You can now debug code running on Azure from Visual Studio running locally.

Apps

There is a web standard being developed for a manifest file for web sites. Microsoft have a tool called ManifoldJS that uses this to generate app store apps for Windows, iOS, Android and Chrome from a website.

Visual Studio Code and GitHub for Windows

If you have GitHub for Windows / GitHub Desktop installed and install Visual Studio Code, you may receive the error 'it looks like git is not installed on your system' when clicking on the Git icon. This is most likely because the path to the git executable git.exe isn't in your system path.

Locate the git executable

The first step is finding the path to folder that contains git.exe.

GitHub for Windows installs to C:\Users\<User>\AppData\Local\GitHub which can be shortened to %USERPROFILE%\AppData\Local\GitHub. Within this folder is a sub-folder starting PortableGit.... such as PortableGit_c7e0cbde92ba565cb218a521411d0e854079a28c. 

You can use a shortened name for this folder such as portab~1. To find this shortened name, navigate to the GitHub folder in the command prompt and run:

for /d %I in (*) do @echo %~sI

You might have to hunt around for the git.exe file in this folder. At one point I found it in a bin sub-folder but this appears to have changed recently to mingw32\bin (GitHub for Windows version 3.0.11).

My final path ended up being:
%USERPROFILE%\AppData\Local\GitHub\portab~2\mingw32\bin

Add to the system path

1. Go to System Properties by opening the run dialog (Windows key + R) and opening 'control sysdm.cpl'
2. Change to the Advanced tab and click on Environment Variables
3. In the System variables section, click on Path and click Edit
4. At the end of the value text add a semicolon and the path to the git executable
5. Click OK, OK and OK to close System Properties

Restart Visual Studio Code and Git integration should now work.

Chrome ends support for NAPI (SharePoint 2013 plugin)

Netscape Plugin Application Programming Interface (NPAPI) is a cross-platform plugin architecture. SharePoint 2013 makes use of this API for a plugin for Chrome and Firefox that provides functionality such as allowing documents to be opened in the local application (such as editing Word documents in Word).

Google has slowly been phasing out support for NPAPI plugins in Chrome, but as is so often the case, you don’t notice these things until they are switched off.

The current version of Chrome (42) disables NPAPI plugins by default and in the future they will not be supported at all.

For now you can enable NPAPI by going to chrome://flags/#enable-npapi in Chrome and clicking enable. After restarting the browser, the plugin should work again but this is not a long term solution as this option will eventually be removed. It would appear that unless Microsoft can provide a plugin that doesn’t depend on NPAPI, Chrome won’t be able to provide the best experience for SharePoint 2013.

SharePoint 2013 - Getting the last modified date of solutions

I'm slowly starting to use more and more powershell since moving from SharePoint 2007 to 2013. When deploying and updating multiple solutions in a SharePoint farm (especially with multiple developers working in the same environment), it can be useful to know when a solution was last updated. I put together the following little script to output the name and last changed date of all solutions in the farm to a text file:

Start-Transcript -path lastmod.txt -append

Get-SPSolution | ForEach-Object { Write-Output ($_.Name + " - " + $_.LastOperationEndTime) }

Stop-Transcript

CSS3 Responsive Menu Layout

I'm putting together a home automation / remote control system to run on my Raspberry Pi and allow me to control various devices around the house. I wanted to create a web interface that would work equally well on a laptop and phone which led me to put together a responsive web layout using CSS only which displays the navigation differently depending on the screen size.

Responsive Web Design is an approach to web development whereby a single dynamic layout provides an optimum experience regardless of the screen size or resolution it is being displayed on.

The viewport meta tag instructs mobile browsers to report a screen resolution proportional to the physical screen size. There is an excellent article on its use here.

<meta name="viewport" content="initial-scale=1.0;">

CSS3 media queries allow us to apply different CSS rules to elements depending on the screen size:

  /* mobile */
  @media screen and (max-width:540px)
  {
    .navmenu {
      text-align: center;
      margin: 0;
    }
  }
  
  /* desktop */
  @media screen and (min-width:540px)
  {
    nav {
      min-height: 200px;
      display: inline-block;
    }
  }

This allowed me to create the layout you can see here which displays a menu on the left hand side of the screen if the browser width is greater than 540 pixels and across the top of the screen for widths less than 540 pixels.

Desktop:

Mobile:

Yet Another Raspberry Pi Remote Controlled Socket

There are a few blog posts about using remote controlled electrical switches with the Raspberry Pi but I thought I'd share my experiences and code.

I bought this set of three sockets and a remote - http://www.clasohlson.com/uk/Remote-Control-Switch-3-pack/18-2035 and a cheap set of a 433Mhz transmitter and receiver from eBay. The manual refers to the sockets as EMW200R.

I came across this link which was trying to achieve the same thing and suggested attaching the receiver to the sound card on a PC and recording the signal sent by the remote control using audio software such as Audacity to identity pulses being sent and their length. Originally my recordings seemed to show audio waves instead of on/off pulses until further reading mentioned that the receiver must be connected to a line in socket instead of a microphone input. After making this change I could clearly see the pattern of hi/low pulses and measure their length.

This similar link mentions that commands are made up of the socket group, socket number, command and sync. Each of these is made up of a series of high and low pulses. I found that my sockets used the same series of pulses for each of these with the exception of a different sync command and the pulse lengths being slightly different.

I put together some python code to control the sockets by passing command line arguments of the pin number (1 - 26), the socket group (A - D), the socket number (1 - 4) and the command (On or Off). Because it is using the GPIO port, the code needs to be run as root (using sudo).

To get the code to run using RASPBMC I needed to install the following:

sudo apt-get install python-dev

sudo apt-get install python-pip

sudo pip install rpi.gpio

The code is as follows:



I then wanted to set up a cron job to turn a lamp on at 5PM every day and off at 11PM. To do this I needed to enable cron in the RASPBMC settings (not required for other distributions) and then edit the root cron table using:

sudo crontab -e -u root

I added the following lines to turn a socket on daily at 5PM and off at 11PM:

0 17 * * * /usr/bin/python /home/pi/EMW200R.py 11 A 1 On

0 23 * * * /usr/bin/python /home/pi/EMW200R.py 11 A 1 Off

The one where TMG Link Translation breaks Event Validation

I had a simple ASP.NET page with the following in the markup:

   <asp:DropDownList ID="ddl" runat="server"></asp:DropDownList>

   <asp:Button ID="btn" runat="server" OnClick="btn_Click" />

And something like this in the codebehind Page_Load:

   ddl.Items.Add("https://my.domain.com/");

When clicking the button to submit the form an error occurred which after some digging turned out to be:

Invalid postback or callback argument.  Event validation is enabled using <pages enableEventValidation="true"/> in configuration or <%@ Page EnableEventValidation="true" %> in a page.  For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them.  If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation.

After much head scratching I noticed that the value I was adding (https://my.domain.com) was not what was being displayed in the browser (https://MY.domain.com). Event validation for the page was throwing the error because the value I was submitting by clicking on the button was not the value that was put into the drop down list in the codebehind.

The problem was that Link Translation was enabled in Microsoft Forefront TMG. This automatically rewrites web content so that links with internal names are change to the defined public name before they are presented to the client. In this case the public name was set to https://MY.domain.com. Since the internal and public names are the same in this case, disabling Link Translation in TMG fixed the issue.

Although event validation protects against the user submitted altered data, it would still be a better idea to have them submit an ID and look up the corresponding URL when the form is submitted.

SharePoint 2013 - Querying Property Bags with Search

Property bags in SharePoint are a great way to store simple data with list items or webs. There are a few blog posts about making property bag entries searchable like this one but I couldn't find anything about being able to query them. This means that if you have a web with a property bag key and value set, the web would be returned when you search for that value (along with anywhere else it appears) but you can't search for webs where your key is set to your value. To allow this you need to set up a managed property associated to your property bag key.

1. Set a property bag key/value pair using powershell as in the link above or using the object model like:
   myWeb.AllProperties[“myspecialkey”] = “myawesomevalue";
2. Set it to be indexed and then update the web:
   myWeb.IndexedPropertyKeys.Add(“myspecialkey”);
   myWeb.Update();
3. Carry out a full crawl by going to Search Administration in Central Administration, selecting Content Sources and selecting Start Full Crawl from the drop down menu of your content source.
4. Once this has completed, go to Search Schema in Search Administration and select Crawled Properties. Search for your key (“myspecialkey”) and make sure it was indexed. If it was, you will be able to search for your value ("myawesomevalue") from your SharePoint site but this will return any results that match (e.g. web titles, list items, document metadata), you won’t be able to search for only results where your key is set to a value.
5. To enable searching for results where your key is set to a value you need to add a managed property.
1. In Search Administration go to Search Schema and click New Managed Property
2. Enter a property name for the new managed property (the same name as your key is easiest)
3. In “Mappings to crawled properties” click “Add a Mapping” and search and add your property bag key
4. Make sure Queryable is selected. This is what lets you get only results where your key is equal to a value. If you want your key and value to be returned in the search results (when querying using one of the APIs), make sure Retrievable is enabled
5. Select any other characteristics that you require and click OK
6. After another full crawl you can search for key=value or key:value from a site and you should only get back results where your key is equal to your value. If you add this key to any other webs, they will be indexed when the next crawl runs without having to make any further changes in Search Administration.

Raspbmc as a wireless bridge

I have XBMC installed on a Raspberry Pi using Raspbmc to stream content to my TV over WiFi but I wanted to make use of the Pi’s Ethernet port to give internet access to a Sky+ set top box which only has an Ethernet port.

I started with a Raspberry Pi with Raspbmc installed over WiFi using an Edimax EW-7811UN USB WiFi adapter. My home network uses the 192.168.0.0/24 subnet with my WiFi router using 192.168.0.1 and the Raspberry Pi using a static IP of 192.168.0.51.

Setting up the bridge was based heavily on the following guide which is for Raspian:
http://qcktech.blogspot.dk/2012/08/raspberry-pi-as-router.html

The following post was also helpful in starting things at boot which is an area that Raspbmc differs from the above guide:
http://forum.stmlabs.com/showthread.php?tid=3552&pid=33455#pid33455

Raspbmc doesn’t use /etc/network/interfaces so you need to edit the settings.xml file:

sudo nano /home/pi/.xbmc/userdata/addon_data/script.raspbmc.settings/settings.xml

The WiFi settings were already configured from the install but you need to setup the wired interface. Give it a static IP in the “nm.address” entry (I used 192.168.1.1), set the gateway to the same IP, set an appropriate netmask and make sure DHCP is disabled for the wired interface and finally. I set the DNS to one of Google’s (8.8.8.8) but I’m not sure if this is needed.

My final settings.xml file looked like this:


If you want your Pi to give out IP addresses to devices connected to the wired port using DHCP, you can install isc-dhcp-server using the instructions in the link mentioned earlier, but I decided to use a fixed IP for simplicity.

I found that with both the USB WiFi and wired connections being used, my Pi wasn't reaching the internet any more. I fixed this by creating a script to set the default gateway for the Wifi adapter when the wired adapter comes up:

sudo nano /etc/network/if-up.d/wlan0defgateway

and adding the following:

#!/bin/sh 

if [ "$IFACE" = "eth0" ]; then  

  route add default gw 192.168.0.1 dev wlan0 

fi

then set permissions using:

sudo chmod 755 /etc/network/if-up.d/wlan0defgateway

 Next you need to enable IP forwarding by editing /etc/sysctl.conf using:

sudo nano /etc/sysctl.conf

Uncomment the line “net.ipv4.ip_forward=1” and save the file.

Finally setup iptables using:

sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

Then save the iptables rules. I couldn’t save straight to /etc so I saved it in ~ and then moved it to /etc using:

sudo iptables-save > iptables.up.rules

sudo mv iptables.up.rules /etc

Create a script to restore the rules using:

sudo nano /etc/init/iptables.conf

And paste in:

# Restore iptables rules on boot 

start on (started dbus and started mountall) 

stop on (xbmc-do-stop or runlevel [!2345])  

exec iptables-restore < /etc/iptables.up.rules

Now restart the Pi and plug an Ethernet cable into the Pi and the other end into something else (I used a laptop for testing). Then set a static IP in the same range on the something else such as:

IP: 192.168.1.2

Netmask: 255.255.255.0 

Gateway: 192.168.1.1

DNS: 8.8.8.8 

Update:
I've noticed that the Pi doesn't keep its IP address recently (I'm not sure if a change in a recent release of RaspBMC caused this). I was able to fix it by editing the NetworkManager config file for the wired interface:

sudo nano "/etc/NetworkManager/system-connections/Wired connection 1"

And setting the IP address:

[ipv4]
method=manual
dns=8.8.8.8;
addresses1=192.168.1.1;24;192.168.1.1;

I then restarted NetworkManager with:

sudo service network-manager restart 

 

jQuery Sliding Pages

I wanted a series of HTML pages to act like a wizard where they would slide in and out. The show and hide methods of jQuery provide a nice animation using slide but the key was to have each content div be a fixed size with absolute positioning. This places each content div behind the previous one allowing them to appear to slide to the next. 

Here's the code:

SharePoint 2013 Site Level Ribbon Tabs

There are plenty of examples of adding a custom SharePoint ribbon tab to a list but very few examples of adding a custom ribbon tab to a site. 

If you want to add a custom ribbon tab to a site it requires custom code for the tab to be displayed (SPRibbon.MakeTabAvailable). Enabling buttons on the tab requires some JavaScript which reports that the buttons are enabled when queried.

I came across this example to add a contextual tab when a webpart is selected but I wanted the tab to always be visible. I've adapted the example for a standard (not contextual) tab which is displayed on a site when a webpart is added to the page. I've also changed the tab XML so that it is deployed as a feature rather than being declared in code.

1. Open Visual Studio 2012 and create a new SharePoint 2013 Visual Web Part project.
2. Name it something like CustomTabWebPart and select Farm Solution when prompted. 
3. From Solution Explorer, right click on the project (CustomTabWebPart) and add a new item.
4. Select Empty Element from the SharePoint templates and name it something like CustomTab.
5. Paste the following into Elements.xml:
6. From Solution Explorer, right click on the project (CustomTabWebPart) and add a new item.
7. Select Module from the SharePoint templates and name it something like CustomTabJavascript.
8. Delete the elements.xml and sample.txt files created for the module in Solution Explorer.
9. Right click on the module (CustomTabJavascript) in Solution Explorer and add a new item.
10. From the Web templates select JavaScript file and call it something like CustomTab.js.
11. Past the following into the JavaScript file:
12. Right click on the JavaScript (CustomTab.js) file in Solution Explorer, change the Deployment Type to RootFile and the Deployment Location path to Template\Layouts.
13. Add a reference for the project to Microsoft.Web.CommandUI.
14. Open the visual webpart’s .ascx.cs file and add the following using statements:
15. Add the following to the same file:
16. Add the following to the Page_Load method in the same file:
17. Deploy the solution and add the webpart to a page. You should now see the My Tab tab on the page with a Hello World button which displays an alert when clicked.

If you want to make the button a bit more useful and launch an application page, you can do so by changing the alert in the JavaScript file to something like SP.UI.ModalDialog.showModalDialog. See the MSDN page for more details.

Its worth noting that Internet Explorer has a habit of caching JavaScript files so you might need to delete the file from your temporary internet files if changes aren't taking effect.

SharePoint 2013 - Unexpected response from server. The status code of response is '0'. The status text of response is ''.

I was receiving this error when attempting to use the Content Search WebPart in SharePoint 2013. After pulling plenty of hair out I opened the developer tools in internet explorer (F12) and started a network capture. This showed that it was trying to reach /_vti_bin/client.svc/ProcessQuery which was getting redirected to /_login which was getting redirected to /_windows and finally back to _/vti_bin/client.svc/ProcessQuery. This repeated several times before aborting.

I then tried to reach _vti_bin/Client.svc/lists in the browser which should have returned a list of lists from the web services, but it showed the same in the network capture.

It occurred to me that the web services (client.svc) probably didn’t think I was authenticated and so were maybe redirecting me to login. The login page presumably knew I was already logged in and was redirecting me back again.

In the end I was able to fix this by enabling Anonymous Authentication for the site in IIS Manager. I need to look into the implications of having anonymous authentication enabled, but at least it showed that the problem was due to security settings.

HTML5 Storage

The HTML5 specification introduces some options for storing data on the client side. This can be either through session storage which retains the data while the browser is open or local storage which retains the data after the browser is closed.

It is worth noting that HTML5 storage data is stored unencrypted and is accessible to anything from the same domain. While I was able to play with HTML5 storage in Chrome using a local file, this didn't work with Internet Explorer 9 and it appears that it will only work with IE9 when being pulled from a web server.

You can check if HTML5 storage is supported by the client's browser with the following:

Simple data can be stored and retrieved in the following way:

Complex data types can be stored and retrieved by converting them to a string before storing using JSON.stringify() and then converting them back using JSON.parse():

I put together a quick demo using HTML5 local storage to create a phone book that supports adding, editing and deleting. It can be found here.

Parsing PDF Files with C# - Part 1

I've been working on some code in C# to parse PDF files and check for security restrictions such as the existence of user and owner passwords and settings such as printing disabled and thought I should share some notes. 


PDF File Format
The PDF file format is extremely flexible which is great in some aspects but when writing a parser that supports such flexibility, it can become a challenge. For example newline sequences can vary depending on the OS or software used to create the PDF file. Different versions of the PDF specification do things in different ways and the contents of files may be encrypted or encoded in different ways making writing a complete parser a challenging prospect.


Apart from a couple of elements, PDF files are largely made up of objects. These objects can described where to find other objects in the file, text or images that appear on the page, fonts and descriptions of which objects appear on which page.


PDF files have two passwords. The owner password is required to change the document and the user password is require to open the document. If a file has restrictions such as No Printing but doesn't prompt you to enter a password when you open it in Adobe Reader, you will find that the document is encrypted using a default user password. You can find more information about this in the specification document.


Using C#
I'll include more details of how I approached implementing a PDF parser in C# in a future post but here are some general thoughts about it.


It should go without saying, but write as generic code as possible. Most elements of PDF files are Objects each of which usually has a Dictionary and a content Stream.


I ended up using FileStream to read files. I initially used a StreamReader but came across several problems. When reading a PDF file, you will need to jump around the file to various points and I found that StreamReader buffers its reads so StreamReader.BaseStream.Position and StreamReader.BaseStream.Seek may refer to one location in the file but the StreamReader read methods may read data from a different point in the file.


It is common for object streams (the content of an object) to be compressed using FlateDecode compression. This can be decompressed using the System.IO.Compression.DeflateStream class, although I found that I needed to skip the first two bytes when deflating object streams.


Checking if the file has a user password requires a combination of creating MD5 hashes and encrypting using RC4. An MD5 class is provided by .NET which  works well and I was able to use an RC4 encryption/decryption class I had previously written for encryption purposes.


Resources 
The best resource for information about the PDF file format is the specification document which is available on the Adobe site. 


Adobe also have a very useful forum where you will most likely find that someone has already had the same problem as you.

SPGridView Part 2 - Custom Filters

The SharePoint 2007 SPGridView control allows easy filtering by setting the AllowFiltering property to true, but it generates the list of possible filter options by selecting all of the distinct values for the column when the column header is clicked. In a DB table I'm using with an SPGridView I store integer values of 0 to represent unset, 1 for low, 2 for medium and 3 for high and so the available filter options are 0, 1, 2 or 3.

To allow filtering by the associated text value, I check for a callback in my WebPart's override of CreateChildControls and if the column header in question has been clicked, bind the SPGridView control to a temporary data table which has a column with the same name as the one clicked containing all of the required values:

if (Page.IsCallback && !string.IsNullOrEmpty(Page.Request.Form["__CALLBACKPARAM"]))
                {
                    string[] param = Page.Request.Form["__CALLBACKPARAM"].Split(';');
                    if (param[1] == myColumn)
                    {
                        DataTable dt = new DataTable();
                        dt.Columns.Add(param[1]);
                        dt.Rows.Add("Unset");
                        dt.Rows.Add("Low");
                        dt.Rows.Add("Medium");
                        dt.Rows.Add("High");

                        m_SpGridView.DataSourceID = null;
                        m_SpGridView.DataSource = dt;
                        m_SpGridView.DataBind();
                    }
                } 

Also in CreateChildControls, if it isn't a callback to get the available filter options, I check for an actual filter call or a clear filter call and flip the text value back to the associated id if required (storing the filter in the ViewState) using a call to the following:

protected virtual void CheckFilter()
        {
            // If there is a call back with an event argument and event target and the target is our gridview...
            if (Context.Request.Form["__EVENTARGUMENT"] != null && Context.Request.Form["__EVENTTARGET"] != null &&
                Context.Request.Form["__EVENTTARGET"].EndsWith(m_SpGridView.ID))
            {
                string search = "__SPGridView__;__Filter__;";
                if (Context.Request.Form["__EVENTARGUMENT"].Equals("__SPGridView__;__Filter__;__ClearFilter__"))
                {
                    ViewState.Remove("FilterExpression");
                }
                else if (Context.Request.Form["__EVENTARGUMENT"].StartsWith(search))
                {
                    string[] newFilter = Context.Request.Form["__EVENTARGUMENT"].Replace(search, "").Split(';');

                    // If this is a custom filter field, switch the selected value for the actual value
                    if (newFilter[0] == myColumn)
                    {
                        if (newFilter[1] == "Unset")
                            newFilter[1] = "0";
                        else if (newFilter[1] == "Low")
                            newFilter[1] = "1";
                        else if (newFilter[1] == "Medium")
                            newFilter[1] = "2";
                        else if (newFilter[1] == "High")
                            newFilter[1] = "3";
                    }

                    // Set the filter in the viewstate
                    ViewState["FilterExpression"] = string.Format("{0}='{1}'", newFilter[0], newFilter[1]);
                }
            }
        }

Finally, override OnPreRender and set the FilterExpression of the SqlDataSource control associated with the SPGridView to the query stored in the ViewState:

protected override void OnPreRender(EventArgs e)
        {
            m_SqlDataSource.FilterExpression = string.Empty;
            if (ViewState["FilterExpression"] != null)
                m_SqlDataSource.FilterExpression = ViewState["FilterExpression"].ToString();

            base.OnPreRender(e);
        }

SPGridView Part 1 - Multiple SPGridViews

There appears to be a bug clearing filters when using multiple instances of the SharePoint 2007 GridView (SPGridView) control on the same page. In my case I had two very similar custom WebParts on the same page using SPGridView controls to display data pulled from a database.

If filtering is enabled on the controls, when the user clears the filter on the second SPGridView on the screen, the filter on the first SPGridView is cleared instead.

After some searching, I found a nice solution to this problem here.

I copied the suggested solution into a class that inherits from SPGridView and used that in my WebParts to resolve the problem:

public class MySPGridView : SPGridView
    {
        // Override the OnPreRender to databind and then fix each headerrow control
        protected override void OnPreRender(EventArgs e)
        {
            DataBind();
            if (this.HeaderRow != null)
            {
                foreach (WebControl control in this.HeaderRow.Controls)
                {
                    UpdateTemplateClientID(control);
                }
            }
            base.OnPreRender(e);
        }

        // Fix the ClientOnClickPreMenuOpen property of a menu control
        private void UpdateTemplateClientID(Control control)
        {
            if (control is Microsoft.SharePoint.WebControls.Menu)
            {
                Microsoft.SharePoint.WebControls.Menu menuControl = control as Microsoft.SharePoint.WebControls.Menu;
                string jsFunctionCall = menuControl.ClientOnClickPreMenuOpen;
                menuControl.ClientOnClickPreMenuOpen = jsFunctionCall.Replace("%TEMPLATECLIENTID%", this.ClientID + "_SPGridViewFilterMenuTemplate");
            }
            else if (control.HasControls())
            {
                foreach (WebControl c in control.Controls)
                    UpdateTemplateClientID(c);
            }
        }
    }