Scrummerfall

Scrummerfall. n. The practice of combining Scrum and Waterfall so as to ensure failure at a much faster rate than you had with Waterfall alone. – Brad Wilson.

I like this definition 🙂

Advertisements

RX Extensions & Threading

Like a lot of people, I’ve been playing with Rx recently, the Reactive Extensions for .Net. To find out a bit more about what these are have a look here. One of the things I found interesting was the use of Rx as a fluent interface for creating sets of asynchronous tasks and then subscribing to the collective result with both a result handler and error handler.

I’ve worked up a little example here that shows some code performing some fairly standard asynchronous tasks. When called, my code needs to call a web service, query a database and fetch a value from some cache. It then needs to combine the results. Since these processes each take different and often varying amounts of time and I want to minimise the executing time of my code I’m going to do them in parallel.

I have shown 4 different methods of achieving this, using my own threads, the thread pool’s threads by way of BeginInvoke(), the Tasks namespace new in .NET 4.0 and Rx.

namespace RXThreadingExample
{
	using System;
	using System.Collections.Generic;
	using System.Diagnostics;
	using System.Linq;
	using System.Threading;
	using System.Threading.Tasks;

	public class Program
	{
		public static void Main()
		{
			const bool throwException = false;

			ClassicAsync(throwException);
			WaitHandles(throwException);
			Tasks(throwException);
			RXExtensions(throwException);

			Console.ReadKey(true);
		}

		private static void ClassicAsync(bool throwException)
		{
			var sw = Stopwatch.StartNew();
			var wsResult = 0;
			string dbResult = null;
			string cacheResult = null;

			var callWebService = new Thread(() => wsResult = CallWebService());
			var queryDB = new Thread(() => dbResult = QueryDB(throwException, "Async"));
			var fetchCacheItem = new Thread(() => cacheResult = FetchCacheItem());

			try
			{
				callWebService.Start();
				queryDB.Start();
				fetchCacheItem.Start();

				callWebService.Join();
				queryDB.Join();
				fetchCacheItem.Join();

				Console.WriteLine(dbResult, wsResult, cacheResult, sw.ElapsedMilliseconds);
			}
			catch (Exception ex)
			{
				Console.WriteLine("Exception : {0}", ex);
			}
		}

		private static void WaitHandles(bool throwException)
		{
			var sw = Stopwatch.StartNew();

			var waitHandles = new List<WaitHandle>();
			Func<int> callWebService = CallWebService;
			var wsAsyncResult = callWebService.BeginInvoke(null, null);			
			waitHandles.Add(wsAsyncResult.AsyncWaitHandle);

			Func<bool, string, string> queryDB = QueryDB;
			var dbAsyncResult = queryDB.BeginInvoke(throwException, "WaitHandles", null, null);
			waitHandles.Add(dbAsyncResult.AsyncWaitHandle);

			Func<string> queryLocalCache = FetchCacheItem;
			var cacheAsyncResult = queryLocalCache.BeginInvoke(null, null);
			waitHandles.Add(cacheAsyncResult.AsyncWaitHandle);

			try
			{
				WaitHandle.WaitAll(waitHandles.ToArray());

				var wsResult = callWebService.EndInvoke(wsAsyncResult);
				var dbResult = queryDB.EndInvoke(dbAsyncResult);
				var cacheResult = queryLocalCache.EndInvoke(cacheAsyncResult);

				Console.WriteLine(dbResult, wsResult, cacheResult, sw.ElapsedMilliseconds);
			}
			catch(Exception ex)
			{
				Console.WriteLine("Exception : {0}", ex);
			}
		}

		private static void Tasks(bool throwException)
		{
			var sw = Stopwatch.StartNew();
			var wsResult = 0;
			string dbResult = null;
			string cacheResult = null;

			var tasks = new List<Task>
				{
					new Task(() => wsResult = CallWebService()),
					new Task(() => dbResult = QueryDB(throwException, "Tasks")),
					new Task(() => cacheResult = FetchCacheItem())
				};

			try
			{
				tasks.ForEach(t => t.Start());
				Task.WaitAll(tasks.ToArray());
				Console.WriteLine(dbResult, wsResult, cacheResult, sw.ElapsedMilliseconds);
			}
			catch (Exception ex)
			{
				Console.WriteLine("Exception : {0}", ex);
			}

		}

		private static void RXExtensions(bool throwException)
		{
			var sw = Stopwatch.StartNew();
			Observable.Join(
				Observable.ToAsync<int>(CallWebService)()
					.And(Observable.ToAsync<bool, string, string>(QueryDB)(throwException, "RX"))
					.And(Observable.ToAsync<string>(FetchCacheItem)())
					.Then((wsResult, dbResult, cacheValue) =>
						new { WebServiceResult = wsResult, DatabaseResult = dbResult, CacheValue = cacheValue })
				).Subscribe(
					o => Console.WriteLine(o.DatabaseResult, o.WebServiceResult, o.CacheValue, sw.ElapsedMilliseconds),
					e => Console.WriteLine("Exception: {0}", e));
		}

		private static int CallWebService()
		{
			Thread.Sleep(500);
			return new Random().Next(1,33);
		}

		private static string QueryDB(bool throwException, string name)
		{
			Thread.Sleep(1500);
			if (throwException)
			{
				throw new Exception("You asked for it !");
			}
			return name + " can rescue {0} Chilean miners in {2} ms. {1}";
		}

		private static string FetchCacheItem()
		{
			return new[]{"Awesome!", "Cool!", "Amazing!", "Jinkies!"}[new Random().Next(0,4)];
		}
	}
}

I have to say, I like the syntactic sugar of Rx. In the tests I have run, however, using Task consistently produces the fastest results. More to follow, I think.

Rx : Reactive Extensions for .Net and JavaScript.

Rx is a set of framework extensions to allow developers to easily do asynchronous programming via familiar interfaces. If you are familiar with LINQ and IEnumerable<T> and IEnumerator<T> you will instantly be able to grasp and use the new interfaces IObservable<T> and IObserver<T>. The enumerating interfaces allow you to PULL sequences of T from an interface in your code. The observing interfaces allow you to have sequences of T PUSHED to handlers in your code in response to asynchronous LINQ queries. Rx is implemented over the top of PFx (Parallel Framework extensions) so all the threading and concurrency is handled for you including synchronization contexts which is especially useful if you are programming UI. It is also particularly significant that the Rx extension methods include a FromEvent() method that allows you to attach handlers to a steam of events in a very fluid way.

If you want to know more watch the video on the bottom of this page where Rx author Erik Meijer explains it in 14 minutes.

If you want to see it in action have a look at this Channel9 video with Wes Dyer where he implements drag & drop from scratch using Rx in 6 minutes.

Also, checkout the Rx Team Blog for downloads, updates and samples.

Finally, back to Channel9 to look at RxJS for Reactive extensions for Java Script so the UI I mentioned above now includes web UI too. As Jeffrey Van Gogh says in the RxJS video, web programming is about “asynchronous stuff” but java script is an imperative language. It’s really hard to do good asynchronous stuff in java script. RxJS changes that it a big way.

Code Quality

The Rule of 8 : High Performance Scalable Systems

Wille Faler has an interesting article on his blog where he highlights 8 core principles for building high performance, scalable systems. It’s an extension to the list of Best Practices for Speeding Up Your Web Site, but Wille is focussing on general principles for any scalable system rather than just websites.

In short, we have :

  • Offload the database – Avoid hitting the database, and avoid opening transactions or connections unless you absolutely need to use them.
  • What a difference a cache makes – For read heavy applications caching is the easiest way offload the database.
  • Cache as coarse-grained objects as possible – Coarse-grained objects save CPU and time by requiring fewer reads to assemble objects.
  • Don’t store transient state permanently – Is it really necessary to store your transient data in the database?
  • Location, Location – put things close to where they are supposed to be delivered.
  • Constrain concurrent access to limited resource – it’s quicker to let a single thread do work and finish rather than flooding finite resources with 200 client threads.
  • Staged, asynchronous processing – separate a process using asynchronicity into separate steps mediated by queues and executed by a limited number of workers in each step.
  • Minimize network chatter – Avoid remote communication if you can as it’s slower and less reliable than local computation.

How not to scale

Speaking of scale there’s an interesting article here by Bart Smaalders of Sun Microsystems about performance anti-patterns. He has identified a number of these anti-patterns :

  • Fixing Performance at the End of the Project
  • Measuring and Comparing the Wrong Things
  • Algorithmic Antipathy
  • Reusing Software
  • Iterating Because That’s What Computers Do Well
  • Premature Optimization
  • Focusing on What You Can See Rather Than on the Problem
  • Software Layering
  • Excessive Numbers of Threads
  • Asymmetric Hardware Utilization
  • Not Optimizing for the Common Case
  • Needless Swapping of Cache Lines Between CPUs

Although it does appear to have been written from the perspective of “performance over all” and some sections (e.g. the one about layering) ignore that fact that the history of software development has been a gradual trade-off between performance and other equally important things like flexibility, maintainability, portability, modularity, productivity etc, it’s a short, well written article that makes interesting reading.

What not to code

The NSA, with contributions from, among others, :

  • CERT
  • Symantec
  • MITRE
  • Aspect Security
  • Secunia
  • iDefense Labs at VeriSign
  • Microsoft
  • Oracle Corporation
  • Red Hat Inc.
  • Fortify Software
  • Veracode
  • Breach Security
  • Security Division of EMC Corporation
  • Apple Product Security
  • Department of Homeland Security (DHS) National Cyber Security Division

have published a list of the 25 most dangerous programming errors that lead to security bugs and that enable cyber espionage and cyber crime.

Shockingly, most of these errors are not well understood by programmers; their avoidance is not widely taught by computer science programs; and their presence is frequently not tested by organizations developing software for sale.

These Top 25 Errors are divided in three categories:

  • Category: Insecure Interaction Between Components (9 errors)
  • Category: Risky Resource Management (9 errors)
  • Category: Porous Defenses (7 errors)

The list will be constantly updated and augmented with resources to help combat/prevent these errors.

Full article here : http://www.sans.org/top25errors/

Full list here : http://cwe.mitre.org/top25/

Extending the WCF Stack

In my last post I created a custom serializer that could be injected into the WCF plumbing between the dispatcher and the service class that allowed us to debug errors that occurred during serialization. This operation behaviour could be applied to a method signature in the service contract :
 
[ServiceContract()]
public interface IMyService
{
	[OperationContract]
	[DebuggableSerializer]
	Thing GetThing(Flavour flavour); 
	
	[OperationContract]
	[DebuggableSerializer]
	Thing[] GetOneOfEach();
}
or to the implementation of that method in the service class :
 
[DebuggableSerializer]
public Thing[] GetOneOfEach()
{
	List<Thing> things = new List<Thing>();
	foreach(Flavour f in Enum.GetValues(typeof(Flavour)))
	{
		things.Add(this.GetThing(f));
	}
	return things.ToArray();
}
However, what if I wanted to apply this behaviour as a ServiceBehavior, ContractBehavior or EndpointBehavior rather than an OperationBehavior i.e. if I wanted all the operations in my service implementation or interface to use it and I don’t want to have to go through and add the attribute to every method ? How can I use my [DebuggableSerializer] attribute to decorate a service class, interface or even better, apply the behaviour to the service or endpoint using a config file so I don’t have to change my code at all ?
 
So, in this post I am going to extend my [DebuggableSerializer] attribute so it can be used on a service class or interface and I am going to extend the System.ServiceModel.Configuration.BehaviorExtensionElement to allow me to control and apply that behaviour from my config file to either a service or a specific endpoint.
 
My attribute in the previous example looked like this :
 
public class DebuggableSerializer : Attribute, IOperationBehavior
but now we want to use it as a ServiceBehavior, ContractBehavior and an EndpointBehavior as well as an OperationBehavior, so we’ll implement the IServiceBehavior, IContractBehavior and IEndpointBeahvior  interfaces :
 
public class DebuggableSerializer : Attribute, IOperationBehavior, 
	IServiceBehavior, IEndpointBehavior, IContractBehavior
IServiceBehavior has three methods for which we need to add implementations and IContractBehavior and IEndpointBehavior have 4 :
 
#region IServiceBehavior Members 

public void AddBindingParameters(ServiceDescription serviceDescription,
	ServiceHostBase serviceHostBase, 
	System.Collections.ObjectModel.Collection<ServiceEndpoint> endpoints,
	BindingParameterCollection bindingParameters)
{
	ReplaceDataContractSerializerOperationBehavior(serviceDescription);
}

public void ApplyDispatchBehavior(ServiceDescription serviceDescription,
	ServiceHostBase serviceHostBase)
{
	ReplaceDataContractSerializerOperationBehavior(serviceDescription);
} 

public void Validate(ServiceDescription serviceDescription,
	ServiceHostBase serviceHostBase)
{
} 

#endregion

#region IEndpointBehavior Members 

public void AddBindingParameters(ServiceEndpoint endpoint,
	BindingParameterCollection bindingParameters)
{
} 

public void ApplyClientBehavior(ServiceEndpoint endpoint,
ClientRuntime clientRuntime)
{
	ReplaceDataContractSerializerOperationBehavior(endpoint);
} 

public void ApplyDispatchBehavior(ServiceEndpoint endpoint,
	EndpointDispatcher endpointDispatcher)
{
	ReplaceDataContractSerializerOperationBehavior(endpoint);
} 

public void Validate(ServiceEndpoint endpoint)
{
}

#endregion

#region IContractBehavior Members 

public void AddBindingParameters(ContractDescription contractDescription,
	ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
{ 
} 

public void ApplyClientBehavior(ContractDescription contractDescription,
	ServiceEndpoint endpoint, ClientRuntime clientRuntime)
{
	ReplaceDataContractSerializerOperationBehavior(contractDescription);
} 

public void ApplyDispatchBehavior(ContractDescription contractDescription,
	ServiceEndpoint endpoint, DispatchRuntime dispatchRuntime)
{
	ReplaceDataContractSerializerOperationBehavior(contractDescription);
} 

public void Validate(ContractDescription contractDescription,
	ServiceEndpoint endpoint)
{ 
} 

#endregion
We also need to provide 3 overloads of the ReplaceDataContractSerializerOperationBehavior method, one that takes ServiceDescription, one that takes ServiceEndpoint and one that takes ContractDescription instead of OperationDescription. The method that takes ServiceDescription will call the method that takes ServiceEndpoint for each Endpoint and the method that takes ServiceEndpoint will call the method that takes ContractDescription and the method that takes ContractDescription will call the method that takes OperationDescription for each OperationDescription in its contract.
 
private static void ReplaceDataContractSerializerOperationBehavior(
	ServiceDescription description)
{
	foreach (ServiceEndpoint endpoint in description.Endpoints)
	{
		ReplaceDataContractSerializerOperationBehavior(endpoint);
	}
}

private static void ReplaceDataContractSerializerOperationBehavior(
	ContractDescription description)
{
	foreach (OperationDescription desc in description.Operations)
	{
		ReplaceDataContractSerializerOperationBehavior(desc);
	}
} 

private static void ReplaceDataContractSerializerOperationBehavior(
	ServiceEndpoint endpoint)
{
	// ignore mex
	if (endpoint.Contract.ContractType == typeof(IMetadataExchange))
	{
		return;
	}
	ReplaceDataContractSerializerOperationBehavior(endpoint.Contract);
}
So now we can take our DebuggableSerializer attribute and place it on the service class or interface :
 
[DebuggableSerializer]
public class MyService : IMyService
{ ... }

[DebuggableSerializer]
public interface IMyService
{ ... }
We now have an attribute that can be used to replace the serializer on a service contract or an operation contract or a service operation or on a whole service class or an endpoint of a service. The final step is to be able to add this behaviour to a service or an endpoint from within our configuration rather than in code. For this we need to create a BehaviorExtensionElement that will create an instance of our DebuggableSerializer behaviour. This is very simple :
 
public class UseDebuggableSerializer : BehaviorExtensionElement
{ 
	public override Type BehaviorType
	{
		get { return typeof(DebuggableSerializer); }
	}
	
	protected override object CreateBehavior()
	{
		return new DebuggableSerializer();
	}
}
Next, in our config file, we need to define this behaviour extension :
 
<system.serviceModel>
	<extensions>
		<behaviorExtensions>
			<add name="UseDebuggableSerializer"
				type="WCFLibrary.UseDebuggableSerializer, WCFLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
		</behaviorExtensions>
	</extensions>
	<!-- elided -->
</system.serviceModel>
To add this behaviour to a service we add the extension to the service’s assigned behaviour :
 
<system.serviceModel>
	<!-- elided -->
	<behaviors>
		<serviceBehaviors>
			<behavior name="MetadataExchange">
				<serviceMetadata httpGetEnabled="true" />
				<UseDebuggableSerializer />
			</behavior>
		</serviceBehaviors>
	</behaviors>
	<!-- elided -->
</system.serviceModel>
To add it to an endpoint we define an endpoint behaviour and add the extension to that :
 
<system.serviceModel>
	<!-- elided -->
	<behaviors>
		<endpointBehaviors>
			<behavior name="DebuggableSerializer">
				<UseDebuggableSerializer />
			</behavior>
		</endpointBehaviors>
	</behaviors>
	<!-- elided -->
	<service behaviorConfiguration="MetadataExchange" 
		name="WCFLibrary.MyService">
	<endpoint
		address="net.tcp://localhost:7000/MyService"
		behaviorConfiguration="DebuggableSerializer"
		binding="netTcpBinding"
		name="TCP_7000_MyService"
		contract="WCFLibrary.IMyService" />
	</service>
	<!-- elided -->
</system.serviceModel>

Debugging the WCF Stack

As if starting developing with WCF wasn’t a hard enough paradigm shift what with getting used to sharing schema instead of type and being cruelly abstracted away from all our lovely plumbing, the abstraction itself it can throw some interesting curve-balls. It’s all very well taking the plumbing and abstracting it away but what happens when errors occur down there…out of our reach ? The errors may still be in our code but if they are being hit by the DataContractSerializer for instance it’s hard to handle them or even catch them, for that matter. To illustrate the problem and provide a possible solution, here’s an example.
 
First we’ll create a nice simple service library.
 
using System;
using System.Collections.Generic;
using System.Text;
using System.ServiceModel;
using System.Runtime.Serialization; 

namespace WCFLibrary
{
	[ServiceContract()]
	public interface IMyService
	{
		[OperationContract]
		Thing GetThing(Flavour flavour); 
		
		[OperationContract]
		Thing[] GetOneOfEach();
	} 
	
	public class MyService : IMyService
	{ 
		#region IMyService Members 
		public Thing GetThing(Flavour flavour)
		{
			switch (flavour)
			{
				case Flavour.Orange:
					return new Thing("Ollie", Flavour.Orange);
				case Flavour.Raspberry:
					return new Thing("Roger", Flavour.Raspberry);
				case Flavour.Strawberry:
					return new Thing("Sally", Flavour.Strawberry);
				case Flavour.Banana:
					return new Thing("Bernice", Flavour.Banana);
				default:
					throw new FaultException("No such flavour");
			}
		} 

		public Thing[] GetOneOfEach()
		{
			List<Thing> things = new List<Thing>();
			foreach(Flavour f in Enum.GetValues(typeof(Flavour)))
			{	
				things.Add(this.GetThing(f));
			}
			return things.ToArray();
		} 
		#endregion
	}
	
	public enum Flavour
	{
		Strawberry,
		Raspberry,
		Orange,
		Banana
	} 

	[DataContract]
	public class Thing
	{
		string name;
		Flavour flavour; 
		public Thing(string name, Flavour flavour)
		{
			this.Name = name;
			this.Flavour = flavour;
		}
		
		[DataMember]
		public string Name
		{
			get { return name; }
			set { name = value; }
		} 

		[DataMember]
		public Flavour Flavour
		{
			get { return flavour; }
			set { flavour = value; }
		}
	}
}
Then a host application.
 
using System;
using System.Collections.Generic;
using System.Text; 
using System.ServiceModel;

namespace WCFHost
{
	class Program
	{
		static void Main(string[] args)
		{ 
			using (ServiceHost host = new ServiceHost(
				typeof(WCFLibrary.MyService),	
				new Uri("http://localhost:7001/MyService")))
			{
				host.Closed += new EventHandler(host_Closed);
				host.Open();
				Console.WriteLine(
					"{0} is running at {1}.\nPress any key to close and exit...",
					host.Description.ServiceType.FullName,
					host.BaseAddresses[0].ToString());
				Console.ReadKey(true);
			}	 
		}
		
		static void host_Closed(object sender, EventArgs e)
		{
			Console.WriteLine("{0} closed.",
				((ServiceHost)sender).Description.ServiceType.FullName);
		}
	}
}
With the following config :
 
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
	<system.serviceModel>
		<behaviors>
			<serviceBehaviors>
				<behavior name="MetadataExchange">
					<serviceMetadata httpGetEnabled="true" />
				</behavior>
			</serviceBehaviors>
		</behaviors>
		<services>
			<service behaviorConfiguration="MetadataExchange" 
				name="WCFLibrary.MyService">
			<endpoint address="net.tcp://localhost:7000/MyService" 
				binding="netTcpBinding"
				bindingConfiguration="" name="TCP_7000_MyService"
				contract="WCFLibrary.IMyService" />
			</service>
		</services>
	</system.serviceModel>
</configuration>
And finally having generated a client proxy we can call it like this :
 
using System;
using System.Collections.Generic;
using System.Text; 
using WCFClient.MyService; 
namespace WCFClient
{
	class Program
	{
		static void Main(string[] args)
		{
			using (MyServiceClient proxy = new MyServiceClient())
			{
				foreach (Thing thing in proxy.GetOneOfEach())
				{
					Console.WriteLine("{0} {1}", 
						thing.Name, 
						thing.Flavour.ToString());
				}
			}
			Console.ReadKey(true);
		}
	}
}
Now, we’ll introduce an error in the service that will not be encountered until the Data Contract class is serialized, i.e. it will occur out of scope of the actual service method call. We are going to change this line in our GetThing service method :
 
case Flavour.Banana:
	return new Thing("Bernice", Flavour.Banana);
to this :
 
case Flavour.Banana:
	return new Thing("Bernice", (Flavour)9);
So we are sneakily casting the integer value 9 into the integer-based enum Flavour even though it’s not a valid value of Flavour. I have actually seen a live version of this error where some unexpected integer values got into a database column so the line actually looked more like this :
 
return new Thing("Bernice", (Flavour)reader.GetInt32(3));
When the service is run and called by the client no error is seen in the service even when debugging and the error appears on the client as a timeout. So, imagine this error is intermittent because it only occurs in a very few out of thousands of database records. Some clients are complaining of timeout errors even though the service isn’t busy and there are no errors on the server side. We can’t debug this and there doesn’t seem to be anywhere we can put a try-catch block. What we really need to be able to do is debug into the DataContractSerializer but we can’t. One option is to enable tracing in the service diagnostics. This can be done either using the WCF Configuration Editor :
 
configeditor
 
or by editing the service config :
 
<system.diagnostics>
	<sources>
		<source name="System.ServiceModel" 
			switchValue="Verbose,ActivityTracing"
			propagateActivity="true">
			<listeners>
				<add type="System.Diagnostics.DefaultTraceListener" 
					name="Default">
					<filter type="" />
				</add>
				<add name="ServiceModelTraceListener">
					<filter type="" />
				</add>
			</listeners>
		</source>
	</sources>
	<sharedListeners>
		<add initializeData="c:\temp\app_tracelog.svclog"
			type="System.Diagnostics.XmlWriterTraceListener, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
			name="ServiceModelTraceListener" 
			traceOutputOptions="LogicalOperationStack, DateTime, Timestamp, ProcessId, ThreadId, Callstack">
			<filter type="" />
		</add>
	</sharedListeners>
</system.diagnostics>
If we run the service again and make the same call we can view the resulting trace log in the Service Trace Viewer :
 
traceviewer
 
There we can clearly see the exception that occurred. The great thing about the tracing and the Trace Viewer is that if we run tracing on all the consumers and services for a particular activity and enable activity propagation we can get the trace logs from all participants and the Trace Viewer will stitch them together for us so we can see the whole thing end to end and step through.
 
This approach is great in many instances but sometimes when debugging it’s a bit "after the fact" and actually what you’d like is to be be able to set a break point or have the debugger catch it in the act so you can get in and look at the stack.
 
To help with this we really need our own serializer but the serializers (DataContractSerializer and NetDataContractSerializer) are sealed. However, XmlObjectSerializer that they both inherit from is not and it is this type that is used so maybe be can hook in there somehow. I can create a class that inherits from XmlObjectSerializer but I don’t want to write my own serializer…way too much work. However, I can use this class as a wrapper for the actual serializer and delegate to it. This effectively injects my debuggable code into the hard-to-debug WCF plumbing. Here’s my DebuggableDataContractSerializer :
 
public class DebuggableDataContractSerializer : XmlObjectSerializer
{
	private XmlObjectSerializer _Serializer;
	
	public DebuggableDataContractSerializer(XmlObjectSerializer serializerToUse)
	{
		if (serializerToUse == null)
		{
			throw new ArgumentException("Argument cannot be null.", "serializerToUse");
		}
		this._Serializer = serializerToUse;
	}
	
	public override bool Equals(object obj)
	{
		return this._Serializer.Equals(obj);
	}
		
	public override bool IsStartObject(XmlDictionaryReader reader)
	{
		return this._Serializer.IsStartObject(reader);
	}
	
	public override object ReadObject(XmlDictionaryReader reader)
	{
		return this._Serializer.ReadObject(reader);
	}
	
	public override object ReadObject(XmlReader reader)
	{
		return this._Serializer.ReadObject(reader);
	} 
	
	public override object ReadObject(XmlReader reader, bool verifyObjectName)
	{
		return this._Serializer.ReadObject(reader, verifyObjectName);
	}
	
	public override string ToString()
	{
		return this._Serializer.ToString();
	}
	
	public override int GetHashCode()
	{
		return this._Serializer.GetHashCode();
	}
	
	public override bool IsStartObject(XmlReader reader)
	{
		return this._Serializer.IsStartObject(reader);
	}
	
	public override object ReadObject(System.IO.Stream stream)
	{
		return this._Serializer.ReadObject(stream);
	}
	
	public override object ReadObject(XmlDictionaryReader reader, bool verifyObjectName)
	{
		return this._Serializer.ReadObject(reader, verifyObjectName);
	} 

	public override void WriteEndObject(XmlDictionaryWriter writer)
	{
		this._Serializer.WriteEndObject(writer);
	} 
	
	public override void WriteEndObject(XmlWriter writer)
	{
		this._Serializer.WriteEndObject(writer);
	} 
	
	public override void WriteObject(System.IO.Stream stream, object graph)
	{
		this._Serializer.WriteObject(stream, graph);
	} 
	
	public override void WriteObject(XmlDictionaryWriter writer, object graph)
	{
		this._Serializer.WriteObject(writer, graph);
	} 
	
	public override void WriteObject(XmlWriter writer, object graph)
	{
		this._Serializer.WriteObject(writer, graph);
	} 
	
	public override void WriteObjectContent(XmlDictionaryWriter writer, object graph)
	{
		this._Serializer.WriteObjectContent(writer, graph);
	} 
	
	public override void WriteObjectContent(XmlWriter writer, object graph)
	{
		this._Serializer.WriteObjectContent(writer, graph);
	} 
	
	public override void WriteStartObject(XmlDictionaryWriter writer, object graph)
	{
		this._Serializer.WriteStartObject(writer, graph);
	} 
	
	public override void WriteStartObject(XmlWriter writer, object graph)
	{
		this._Serializer.WriteStartObject(writer, graph);
	}
}
Now I need to get my serializer swapped out with the real one. I remembered seeing a blog post by Aaron Skonnard way back in April ’06 in which he showed how to substitute the NetDataContractSerializer for the DataContractSerializer so let’s try that here :
 
public class DebuggableSerializer : Attribute, IOperationBehavior
{ 
	public void AddBindingParameters(OperationDescription description,
		BindingParameterCollection parameters)
	{
	}
	
	public void ApplyClientBehavior(OperationDescription description,
		ClientOperation proxy)
	{
		ReplaceDataContractSerializerOperationBehavior(description);
	} 
	
	public void ApplyDispatchBehavior(OperationDescription description,
		DispatchOperation dispatch)
	{
		ReplaceDataContractSerializerOperationBehavior(description);
	} 
	
	public void Validate(OperationDescription description)
	{
	} 
	
	private static void ReplaceDataContractSerializerOperationBehavior(
		OperationDescription description)
	{
		DataContractSerializerOperationBehavior dcsOperationBehavior =
			description.Behaviors.Find<DataContractSerializerOperationBehavior>();
		if (dcsOperationBehavior != null)
		{
			description.Behaviors.Remove(dcsOperationBehavior);
			description.Behaviors.Add(
				new DebuggableDataContractSerializerOperationBehavior(description));
		}
	}
} 
	
public class DebuggableDataContractSerializerOperationBehavior
		: DataContractSerializerOperationBehavior
{
	
	public DebuggableDataContractSerializerOperationBehavior(
		OperationDescription operationDescription) : base(operationDescription)
	{ 
	} 
	
	public override XmlObjectSerializer CreateSerializer(Type type, string name,
	string ns, IList<Type> knownTypes)
	{
		DataContractSerializer dcs = new DataContractSerializer(type, name,
			ns, knownTypes);
		DebuggableDataContractSerializer mdcs = new DebuggableDataContractSerializer(dcs);
		return mdcs;
	} 
	
	public override XmlObjectSerializer CreateSerializer(Type type,
		XmlDictionaryString name,
		XmlDictionaryString ns, IList<Type> knownTypes)
	{
		DataContractSerializer dcs = new DataContractSerializer(type, name, ns, knownTypes);
		DebuggableDataContractSerializer mdcs = new DebuggableDataContractSerializer(dcs);
		return mdcs;
	}
}
Now we add our new custom attribute to our service method and we are good to go.
 
[ServiceContract()]
public interface IMyService
{
	[OperationContract]
	[DebuggableSerializer]
	Thing GetThing(Flavour flavour); 

	[OperationContract]
	[DebuggableSerializer]
	Thing[] GetOneOfEach();
}
 
debugging
 
I have since written an article on how to extend this concept so that it can be used as a ServiceBehavior, ContractBehavior or EndpointBehavior. See Extending the WCF Stack.

Quick ‘n’ Dirty Redirects in DotNetNuke

I have recently been involved in some SEO work on a website written in DotNetNuke. SEO (Search Engine Optimization) is the arcane and mystical art of making your website simultaneously appear interesting/relevant to both users and and search engine bots, most especially the GoogleBot. One of the keys to this it not to have duplicate content on your website, that is to say, pages that appear to be the same that are available via different URLs. If you do this, the GoogleBot will think you are trying to trick it and it devalues the relevance of that content. As a result, if you move content from one URL to another you need to 301 (permanently redirect) the old URL to the new URL.
 
A common way to do this is to buy or write a DNN module that redirects and install it on your site on the pages that should be redirected. This is a pain though because it means that you have to keep those pages in your portal even though you don’t need them any more and each request to them has to load the page and the module before it is redirected.
 
The fact is that DNN is constantly internally redirecting anyway using a technique called URL rewriting. The "real" address of most pages in a DNN portal is /Default.aspx?tabid=1234 where 1234 is a unique identifier for a "page" of content in DNNs database. This rewriting is done by an HttpModule whose configuration is stored in an xml file in the root of the portal (/SiteURLs.config). An example of a URLRewriter rule can be seen here :
 
<RewriterRule>
	<LookFor>.*/MyNiceURL.aspx</LookFor>
	<SendTo>~/Default.aspx?tabid=1234</SendTo>
</RewriterRule>
It occurred to me that it would be easier just to hijack this mechanism and with a very small tweak add permanent (301) and temporary (302) redirects to its features. A new rule would look something like this :
 
<RewriterRule Type="Permanent">
	<LookFor>.*/MyNiceURL.aspx</LookFor>
	<SendTo>~/Default.aspx?tabid=1234</SendTo>
</RewriterRule>
The Type attribute could have values of Permanet, Temporary or Rewrite (the default) or be absent. We then need to make a couple of simple changes to the URLRewriter module which, thanks to open source, we can.
Someone emailed me about this so I thought I’d add a quick clarification. The URLRewriter can only rewrite the path segment of the URL. So if you give it a rule that looks like this :
 
<RewriterRule>
	<LookFor>.*/MyNiceURL.aspx</LookFor>
	<SendTo>http://www.mydomain.com/Default.aspx?tabid=1234</SendTo>
</RewriterRule>
it will instead perform a Response.Redirect. This sends a 302 to the client, not a 301, so without this modification we still have no way of sending a 301 and we can only send a 302 if we include the whole url, i.e. we have limited control over how the rewriter behaves.

First we add an enum to represent our rule type in RewriterRule.vb:
 
Public Enum RewriterRuleType
        Rewrite = 0
        Permanent = 1
        Temporary = 2
End Enum
Then we add the new public property and private field to the RewriterRule class :
 
Private _ruleType As RewriterRuleType = RewriterRuleType.Rewrite

Public Property RuleType() As RewriterRuleType

    Get
        Return _ruleType
    End Get

    Set(ByVal Value As RewriterRuleType)
        _ruleType = Value
    End Set

End Property
We also need to modify the GetConfig method of the RewriteConfiguration class so it knows about our new XML attribute :
 
...

For Each nav As XPathNavigator In doc.CreateNavigator.Select("RewriterConfig/Rules/RewriterRule")

    Dim rule As New RewriterRule()

    rule.LookFor = nav.SelectSingleNode("LookFor").Value

    rule.SendTo = nav.SelectSingleNode("SendTo").Value

    ' >>> begin new code

    Try

        rule.RuleType = CType(System.Enum.Parse(GetType(RewriterRuleType), nav.GetAttribute("Type", ""), False), RewriterRuleType)

    Catch ex As Exception

        ' Do nothing, either there was no attribute or it contained an invalid value
        ' NB : yes this is a dirty hack but this is a blog post not production code

    End Try

    ' >>> end new code

    Config.Rules.Add(rule)

Next

...
Next we need to modify the RewriteURL method of the URLRewriteModule class so it can take different actions based on the newly added Rule Type :
 
(NB : the ruleType variable has been declared at the top of the method and is assigned to as soon as a matching rule is found.)
 
...

If

 intMatch <> -1 Then

    ' >>> begin new code 

    Select Case (ruleType)

        Case Config.RewriterRuleType.Permanent

            RewriterUtils.PermanentlyRedirectURL(app.Context, sendTo)

        Case Config.RewriterRuleType.Rewrite

            ' >>> begin old code 

            If rules(intMatch).SendTo.StartsWith("~") Then

                ' rewrite the URL for internal processing

                RewriterUtils.RewriteUrl(app.Context, sendTo)

            Else

                ' it is not possible to rewrite the domain portion of the URL so redirect to the new URL

                Response.Redirect(sendTo, True)

            End If

            ' >>> end old code 



        Case Config.RewriterRuleType.Temporary

            RewriterUtils.TemporarilyRedirectURL(app.Context, sendTo)

    End Select

    ' >>> end new code 

End If

...
 
Finally, we add these new actions to the RewriterUtils class :
 
Friend

 Shared Sub TemporarilyRedirectURL(ByVal context As HttpContext, ByVal sendToUrl As String)

    context.Response.Clear()

    context.Response.Status = "302 Found"

    context.Response.AddHeader("Location", sendToUrl)

    context.Response.End()

End Sub

 

Friend Shared Sub PermanentlyRedirectURL(ByVal context As HttpContext, ByVal sendToUrl As String)

    context.Response.Clear()

    context.Response.Status = "301 Moved Permanently"

    context.Response.AddHeader("Location", sendToUrl)

    context.Response.End()

End Sub
 
We can then compile the DNN source and replace the DotNetNuke.HttpModules.UrlRewrite.dll in our DNN installation with the newly compiled one.

%d bloggers like this: