RX Extensions & Threading

Like a lot of people, I’ve been playing with Rx recently, the Reactive Extensions for .Net. To find out a bit more about what these are have a look here. One of the things I found interesting was the use of Rx as a fluent interface for creating sets of asynchronous tasks and then subscribing to the collective result with both a result handler and error handler.

I’ve worked up a little example here that shows some code performing some fairly standard asynchronous tasks. When called, my code needs to call a web service, query a database and fetch a value from some cache. It then needs to combine the results. Since these processes each take different and often varying amounts of time and I want to minimise the executing time of my code I’m going to do them in parallel.

I have shown 4 different methods of achieving this, using my own threads, the thread pool’s threads by way of BeginInvoke(), the Tasks namespace new in .NET 4.0 and Rx.

namespace RXThreadingExample
{
	using System;
	using System.Collections.Generic;
	using System.Diagnostics;
	using System.Linq;
	using System.Threading;
	using System.Threading.Tasks;

	public class Program
	{
		public static void Main()
		{
			const bool throwException = false;

			ClassicAsync(throwException);
			WaitHandles(throwException);
			Tasks(throwException);
			RXExtensions(throwException);

			Console.ReadKey(true);
		}

		private static void ClassicAsync(bool throwException)
		{
			var sw = Stopwatch.StartNew();
			var wsResult = 0;
			string dbResult = null;
			string cacheResult = null;

			var callWebService = new Thread(() => wsResult = CallWebService());
			var queryDB = new Thread(() => dbResult = QueryDB(throwException, "Async"));
			var fetchCacheItem = new Thread(() => cacheResult = FetchCacheItem());

			try
			{
				callWebService.Start();
				queryDB.Start();
				fetchCacheItem.Start();

				callWebService.Join();
				queryDB.Join();
				fetchCacheItem.Join();

				Console.WriteLine(dbResult, wsResult, cacheResult, sw.ElapsedMilliseconds);
			}
			catch (Exception ex)
			{
				Console.WriteLine("Exception : {0}", ex);
			}
		}

		private static void WaitHandles(bool throwException)
		{
			var sw = Stopwatch.StartNew();

			var waitHandles = new List<WaitHandle>();
			Func<int> callWebService = CallWebService;
			var wsAsyncResult = callWebService.BeginInvoke(null, null);			
			waitHandles.Add(wsAsyncResult.AsyncWaitHandle);

			Func<bool, string, string> queryDB = QueryDB;
			var dbAsyncResult = queryDB.BeginInvoke(throwException, "WaitHandles", null, null);
			waitHandles.Add(dbAsyncResult.AsyncWaitHandle);

			Func<string> queryLocalCache = FetchCacheItem;
			var cacheAsyncResult = queryLocalCache.BeginInvoke(null, null);
			waitHandles.Add(cacheAsyncResult.AsyncWaitHandle);

			try
			{
				WaitHandle.WaitAll(waitHandles.ToArray());

				var wsResult = callWebService.EndInvoke(wsAsyncResult);
				var dbResult = queryDB.EndInvoke(dbAsyncResult);
				var cacheResult = queryLocalCache.EndInvoke(cacheAsyncResult);

				Console.WriteLine(dbResult, wsResult, cacheResult, sw.ElapsedMilliseconds);
			}
			catch(Exception ex)
			{
				Console.WriteLine("Exception : {0}", ex);
			}
		}

		private static void Tasks(bool throwException)
		{
			var sw = Stopwatch.StartNew();
			var wsResult = 0;
			string dbResult = null;
			string cacheResult = null;

			var tasks = new List<Task>
				{
					new Task(() => wsResult = CallWebService()),
					new Task(() => dbResult = QueryDB(throwException, "Tasks")),
					new Task(() => cacheResult = FetchCacheItem())
				};

			try
			{
				tasks.ForEach(t => t.Start());
				Task.WaitAll(tasks.ToArray());
				Console.WriteLine(dbResult, wsResult, cacheResult, sw.ElapsedMilliseconds);
			}
			catch (Exception ex)
			{
				Console.WriteLine("Exception : {0}", ex);
			}

		}

		private static void RXExtensions(bool throwException)
		{
			var sw = Stopwatch.StartNew();
			Observable.Join(
				Observable.ToAsync<int>(CallWebService)()
					.And(Observable.ToAsync<bool, string, string>(QueryDB)(throwException, "RX"))
					.And(Observable.ToAsync<string>(FetchCacheItem)())
					.Then((wsResult, dbResult, cacheValue) =>
						new { WebServiceResult = wsResult, DatabaseResult = dbResult, CacheValue = cacheValue })
				).Subscribe(
					o => Console.WriteLine(o.DatabaseResult, o.WebServiceResult, o.CacheValue, sw.ElapsedMilliseconds),
					e => Console.WriteLine("Exception: {0}", e));
		}

		private static int CallWebService()
		{
			Thread.Sleep(500);
			return new Random().Next(1,33);
		}

		private static string QueryDB(bool throwException, string name)
		{
			Thread.Sleep(1500);
			if (throwException)
			{
				throw new Exception("You asked for it !");
			}
			return name + " can rescue {0} Chilean miners in {2} ms. {1}";
		}

		private static string FetchCacheItem()
		{
			return new[]{"Awesome!", "Cool!", "Amazing!", "Jinkies!"}[new Random().Next(0,4)];
		}
	}
}

I have to say, I like the syntactic sugar of Rx. In the tests I have run, however, using Task consistently produces the fastest results. More to follow, I think.

Advertisements

Writing Style

I fed my last three blog posts into the “I write like…” analyzer at http://iwl.me. I got 2 David Foster Wallace’s and a Stephen King. Not a bad haul, I think 🙂

Animal Magnetism

We’ve recently acquired new neighbours who seem lovely. They have 3 cats who also seem lovely. Our cat does not agree. He is getting on a bit now and is finding the whole territorial warfare lark a bit tough going in the face of such insurmountable odds. The neighbouring cats have noticed and have taken to popping into our house through his cat flap to borrow a bit of cat food from time to time. Mostly at night. This has moved the turf war from our neighbourhood’s interlocking gardens and driveways into our house with all the associated yowling and marking of territory. Not good.

As a result I spent my Bank Holiday Saturday morning upgrading our cat flap to one of those posh ones that only opens for our cat…that’s the theory anyway. The flap in question is a Staywell 400 Series Magnetic 4 Way Locking Deluxe Cat Flap. Homebase did not mention the even posher 500 series which is an infrared operated version. If they had I might have got that instead, given how much I like to be on the bleeding edge technologically, and saved myself a bit of bother.

catflap The cat flap is operated by a small magnet worn on the cat’s collar. So actually it would appear that the 400 Series will grant admission to any cat with similar magnetic accessories. As the cat brings it’s head within pushing distance of the flap the magnet triggers the lock and the flap will open. Brilliant.

Our cat was not impressed. Within minutes of having his new magnetic medallion fitted cathe ran downstairs, upended his food dish in a fit of pique and ran out of the house. The ungrateful bugger returned later in the day after I had swept up all the IAMS and promptly upended the whole thing again. From upstairs I could hear him rattling the empty dish around the hallway and shouting. I cleared up the foodquake debris again.

During the night he tipped all the food out of his dish again. Over breakfast on Sunday we were debating the cause of his protests. Did he not like the new flap ? Did he not like the magnet on his collar ? Did he resent our interference in his complicated relationship with the neighbourhood cats ? We persevered.

The vicious cycle of IAMS-sweeping, re-filling and dish-tipping continued through Sunday and Monday. On Monday night I was woken up at about 1 AM by a particularly violent bought of dish rattling. Trying to get cat biscuit out from between your toes in the wee hours must concentrate the mind wonderfully because the answer came to me in a flash as I was refilling the dish. The cat dish. The biscuit dish. The nice shiny dish. The STAINLESS STEEL F**CKING DISH! There had never been any act of protest at all. Every time the cat had leaned in to get a bite of food the dish had attached itself to his magnet so that when he lifted his head between mouthfuls the dish would tip up and he’d go clanging about about the place trailing biscuit everywhere until he managed to shake it off.

Cat forgiven. Plastic dish purchased. Still considering upgrade to Series 500. INFRARED, Man ! Now that is cool for cats 🙂

Back in the game

sc1I’ve recently had a bit of a games bonanza on the XBox 360. I have been playing Splinter Cell Conviction and Alan Wake. This combination is slightly freaky because in Splinter Cell you are sneaking around shooting out the lights and trying to stay in in the dark while in Alan Wake you are running around turning on lights and  trying to stay lit up like a Christmas tree at all times.

Splinter Cell Conviction is the latest in a line of 5 Splinter Cell games from Ubisoft. It’s a notable game series because in my opinion it’s gone from strength to strength and they’ve never had a dud. Even the Tomb Raider series only managed to get to Tomb Raider 2 before producing a complete lemon in the form of Tomb Raider 3. The plot lines and characters have remained contiguous and consistently engaging. It is also the only stealth-based action adventure I’ve played that manages to sustain it’s stealth element so consistently throughout the game without it ever getting tired or stale.sc2

The graphics in the latest episode are some of the best I’ve ever seen. They are rich and smooth and glossy and detailed. The playing environments are large, rich and detailed and the whole feel of the game is immensely satisfying. There are a number of game modes in addition to the main narrative that come under the title “Deniable Ops”. These arcade modes include a “Hunter” elimination mode, a survival mode and an additional “Infiltration” mode that can be unlocked via download.

aw1 Alan Wake was announced 5 years ago…yes, 5 ! The build-up and hype since including the release of a series of short films before the game has been hard to live up to. The format of the game is “survival horror” similar to Resident Evil and the game does evoke some of the same sensations of dread and apprehension. It has a great story. It’s like a mish-mash of Twin Peaks and The Twilight Zone. It is formatted into episodes which give it a TV-like quality. There are some nice touches like episodes of a TV series very similar to The Twilight Zone that you can watch on TVs you find around the game and being able to tune into the local radio station.

The overall game play is OK and the environments are rich and detailed. However it is let down by poor graphics and clunky animation. Opening doors for instance is performed by bumping into them. Even in cut scenes the animation is clunky and when you see detail like hands picking up objects or putting them down the gestures are vague and unconvincing. Yes, the work they’ve done with light and shadow is impressive but I’m guessing that was added at the end and the animation and graphics are the stuff from 5 years ago when the game was first announced.

These *are* the droids I am looking for

droidAndroid is, hands down, the best mobile OS I’ve ever used and HTCs implementations of Android are, hands down, the best Android phones I’ve ever used. I have had a lot of phones. With the exception of WebOS and iPhone I think I’ve had some form of every major phone type, from the earliest analogue Nokias and Motorolas through successions of Windows Mobile, Blackberry, Palm Treo and Symbian devices.

So far I’ve had 3 Android phones starting with the HTC Hero, then the Motorola Milestone/Droid and now the HTC Desire. At the time the Hero was the best phone I’d ever had. That title has now been taken by the Desire. With the exception of the Palm Treo 600, which was truly revolutionary, pretty much every Smartphone I’ve had up to the Hero has been a crushing disappointment. The Smartphone is a platform that has taken a LOOOOOOONG time to get right.

Android on it’s own isn’t quite enough though as evidenced by the Motorola Milestone which is appalling. I mean REALLY bad. If this is the device that is supposed to save Motorola’s Consumer Mobile division then I’m getting my funeral suit out now. It stinks. It’s ugly. It looks like the front and the back of the device were designed by two people who never met or met once and fell out so badly they couldn’t even agree on what size the device should be. It’s clunky. It reboots itself at random sometimes getting stuck in a viscous cycle of continuous reboots. The HTC devices on the other hand are beautiful, sleek and ergonomic. hero_lineup1

Gorgeous HTC hardware isn’t enough on it’s own either though as evidenced by the HTC HD2 which is probably the most beautiful piece of mobile hardware I’ve ever had. Yes, Windows Mobile can ruin even hardware this good. Even with the deepest, most thorough-going reskinning WinMo has ever seen you can’t hide the crap below the surface forever and soon or later it leaks out all over the user.

The Desire (and the Hero before it) are the perfect union of magnificent hardware, solid OS and a great skin. HTC Sense has its detractors but I love it. Here are the things that have absolutely clinched it for me.

desireEmail & Calendar – I can seamlessly sync my email, contacts and calendar with my company Microsoft Exchange server. I can do the same with desktop Outlook. I can also directly pull in my various POP3 and IMAP email accounts. All this JUST WORKS. The push works, the notifications work, sending and receiving meeting invitations works, the signatures, the quotas, the attachments the peak and off-peak sync schedule ALL JUST WORK. My calendar items from Exchange, desktop, the device itself and Google calendar can all appear, neatly colour-coded, in one calendar view.

Contacts & Social Networking – the people view is brilliant. My Exchange contacts, my phone book contacts, my Facebook friends, my Twitter and Flickr friends all woven together 95% automagically. I can also merge Facebook, twitter and Flickr into a single “Friend Stream” and drop that on one of my many customizable home screens.

Browsing – here we have the best mobile browsing experience ever. The rendering is crisp and lightening fast. Zooming using a combination of double-taps or pinch/pull gestures is a breeze. Flash video is supported. Even a busy, media rich site like Sky Sports or the BBC will render quickly and smoothly in full screen and be completely usable. There is also an excellent newsreader application with a full-screen, home screen widget.

Music – The music player looks nice both in full screen and widget mode. The album art animations when using gestures to switch songs is smooth and sleek. The sound quality is excellent. As with many devices the maximum volume is not very high so you will need good, noise-cancelling headphones for use on the train. The app itself allows you to easily create, edit and organise playlists. The ace in the pack is Amazons MP3 app for Android which lets you quickly, find, buy and download music by the song or album on the go. Purchased songs arrive quickly and seamlessly in the player with full album art.

Photos and Video – The camera on the Desire is 5 MP with LED flash and takes good photos and video. Media is organised is easy to browse albums which a wealth of tools for organising and sharing.

Maps and Navigation – Google Maps is a triumph on Android and they have recently enabled turn-by-turn voice navigation for the UK. There is also an excellent, free, open-source navigation app in the Android Market called Nav4All.

Applications – The Android Market is a complete treasure trove with over 35,000 apps and is growing fast (i.e. it went from 16,000 to 30,000 in only 3 months). I have yet to find a requirement for which there is not an app and over 60% of them are free. The Apple App store may have over 140,000 apps but given that 57 of those alone are just for making fart noises I don’t think that represents a huge gap in actual utility and feature coverage.

Here are some of my top tips for great Android apps (in no particular order) :

  • Catch (formerly known as 3Banana Notes) : 3Banana let you take, store, tag, filter, search and organise notes like a notepad or post-it notes. You can include html, links, QR Codes, images, video, photos, geo-tags amongst other things. 3Banana also syncs your notes into the cloud so you can view them from any device or PC. You can also share notes via Twitter and Facebook. It also has a desktop widget so you can pin notes to you home screen as reminders.
  • Foursquare : if you are a Foursquare user you will love this. A very nice, full-featured client for the popular social networking service.
  • Layar : one of the first Augmented Reality (AR) browsers and still the best. Layar uses a combination of your camera, compass, accelerometer and feeds of data to superimpose layers of data over reality via the screen of your phone. You point your camera at the world around you and magically it is augmented with additional data, graphics, photos, 3d models and messages.
  • Amazon MP3 – Buy songs and albums in just a few simple clicks and have them downloaded straight into the media player on your phone in moments complete with artist details and album art. Average price for an album seems to be about £6.49.
  • Google Maps and Goggles : Google maps with turn-by-turn voice navigation is brilliant and very simple to use. Goggles is part Augmented Reality browser part visual search engine. Simply take a photo of anything around you and Google will provide you with a set of search results. For example, take a photo of a movie posted and Google will tell you the film, who’s in it, what the reviews were, where it’s on nearest you, directions to the cinema and provide links to book tickets to the next showing on line. Also works with books, bar codes, QR codes and an ever growing plethora of things. Amazing !
  • AndroMote : Is a media remote. It can connect via Wi-Fi to any media server on your network and stream music and video to your phone. You can also view other media files such as photos.. It can also connect to media renderers on your network and remote control their streaming functions. I have tested it with Orb and TVersity  and it works fine.
  • RecordIt : Great app that allows you to remote connect to your Sky+ or Sky+ HD box and schedule recordings.
  • beebPlayer : BBC iPlayer for Android. Watch live TV, browse and watch iPlayer archives, listen to BBC Radio. NB : this app has now been discontinued. Checkout myPlayer instead or, if you have Android 2.2, try the BBC iPlayer Website directly in your browser.
  • Ethereal Dialpad : A very cool synthesizer app that allows you to use the screen of the phone as a control surface to play sounds. Also provides nice touch feedback using visualisations.
  • Remote RDP : Have full remote desktop access to your PC via Remote Desktop Protocol. Very cool.
  • Uloops : A very cool sequencer and mixer that allows you to create and mix multi-instrument tracks on your phone. Upload them to the ULoops community portal. Doanload, rate and remix other peoples tracks.
  • Ringdriod : Open and play any MP3 or music file. Select any segment of the song and save that as your ring tone.
  • Seesmic for Twitter : One of the best Twitter clients for Android.
  • Shazam : The classic Shazam service for Android. Hear a song playing that you can’t identify ? Open up Shazam, let it listen for a few seconds and it will tell you the song.
  • PicSay : Quickly edit photos to add speech bubbles, fun effects, cartoon hats, facial hair and other accessories.
  • Nav4All : A free, open-source satellite navigation system.
  • Nimbuzz : a single Client for multiple IM services such as Windows Live, MSN, Yahoo, Skype, Facebook, Google Talk. Orkut, AIM, MySpace, ICQ, Gadu Gadu and Hyves.
  • NetCounter : Keep track of your mobile and wi-fi data usage by hour, day, week and month. Show usage stats and graphs and add custom counters.
  • Google Listen : A great PodCatcher from Google Labs for keeping up with all your podcasts. Features auto-sync and download and a managed listen queue that keeps track of where you’ve got to on your busy listening schedule.
  • DropBox – 2GB of cloud storage accessible from your phone for free.
  • Epicurious – very slick recipe app. Rich, high-quality content nicely presented.
  • Share Your Board – brilliant app that allows you to take a photo of a whiteboard. It then performs edge detection, de-skewing plus colour and contrast correction so you end up with a crisp, clear image of your whiteboard.
  • Swype – one of the best applications for Android. It is an alternative to the built-in on-screen keyboard that you use by swiping or tracing out a word from key to key. This is how on-screen keyboards were MEANT to be. The speed at which you can input and the accuracy with which it guesses what you are swyping means that it beats every other mobile keyboard (inluding hardware ones) by a country mile. It’s in closed beta at the moment but check out these short demo videos to see how revolutionary it is.
  • Square/SquareUp – this is only available in the US at the moment but it is a combination of software and a credit card reader that plugs into the audio jack on the phone and enables anyone with a back account to take mobile payments. A real game changer. Let’s hope we see it in the UK soon.

Apps to watch out for :

  • Kindle – Amazon will soon be launching their Kindle software for Android. NOTE : this has now been released.
  • Sling Player – an Andoird version of the client software for the SlingBox appliance. NOTE : this has now been released but costs $30. Dream on, Sling !

Events in C# 4.0

Rx : Reactive Extensions for .Net and JavaScript.

Rx is a set of framework extensions to allow developers to easily do asynchronous programming via familiar interfaces. If you are familiar with LINQ and IEnumerable<T> and IEnumerator<T> you will instantly be able to grasp and use the new interfaces IObservable<T> and IObserver<T>. The enumerating interfaces allow you to PULL sequences of T from an interface in your code. The observing interfaces allow you to have sequences of T PUSHED to handlers in your code in response to asynchronous LINQ queries. Rx is implemented over the top of PFx (Parallel Framework extensions) so all the threading and concurrency is handled for you including synchronization contexts which is especially useful if you are programming UI. It is also particularly significant that the Rx extension methods include a FromEvent() method that allows you to attach handlers to a steam of events in a very fluid way.

If you want to know more watch the video on the bottom of this page where Rx author Erik Meijer explains it in 14 minutes.

If you want to see it in action have a look at this Channel9 video with Wes Dyer where he implements drag & drop from scratch using Rx in 6 minutes.

Also, checkout the Rx Team Blog for downloads, updates and samples.

Finally, back to Channel9 to look at RxJS for Reactive extensions for Java Script so the UI I mentioned above now includes web UI too. As Jeffrey Van Gogh says in the RxJS video, web programming is about “asynchronous stuff” but java script is an imperative language. It’s really hard to do good asynchronous stuff in java script. RxJS changes that it a big way.

Building from the ground up

I recently carpet-bombed my main work laptop and built it from scratch. Not something I do very often but the time had come. Here’s what went back into the box (the box being my trusty Dell D830) :

OS : Windows 7 Ultimate 64-bit

Productivity Stuff :

Development Stuff :

System Management Stuff :

Utils & Misc :

Observations :

  1. Wow ! That’s a lot of stuff. Now I remember why I don’t do that very often.
  2. Development stuff still outweighs everything else by far.
  3. Why do I still have an optical drive on this laptop? During all of the above I never once used a CD or DVD.
  4. When I started developing I used TextPad and a compiler and I hated IDEs. I still occasionally use TextPad but now I have two versions of Visual Studio with a zillion plug-ins. My daughter once asked me “What’s that game you play where all the words pop-up and change colour.” It took me a while to work out she meant Visual Studio 🙂
  5. I really only installed Snippet Compiler for old time’s sake as development seems to have stopped. I use LINQPad for that stuff now.
  6. Outlook 2010 is amazing.
  7. Visio 2010 drives me nuts because I have LIVED in previous versions of Visio way too long and now everything has moved. It’s like my house has been burgled by Feng Shui experts with OCD. I’m learning to live with it though 🙂
  8. When the hell is LiveMesh going to get out of Beta ?
  9. My beloved Paint Shop Pro disappeared up its own arse around the time it was acquired by Corel (coincidence ?) and since then I have stubbornly stuck to version 6 which is next to useless on Windows 7. Paint.Net is a worthy and FREE replacement. Nice one guys.
  10. I want to love SketchFlow, but I also want to beat it’s stupid head in with a brick.
  11. Reflector is still way cool and it now has a “PRO” version for £125 – OUCH ! Guess how many new cool features the free version will be getting from now on 😉
  12. Loving LINQPad. It has more or less replaced Snippet Compiler, SQL Management Studio and a bunch of other little utilities on my machine. Take the LINQPad Challenge today.
  13. Nearly half my development stuff is now for mobile. Scary 😮
  14. Previous problems with Security Essentials have not manifested themselves. Hmmm….

What price security ?

For the past couple of weeks I’ve been trying out Microsoft Security Essentials. This is Microsoft’s new, free anti-virus, anti-spyware and anti-malware tool. For a long time I’ve been using either NOD32 by Eset or Sophos as my AV of choice but I figured I’d give this a go.
 
My primary requirement of any tool like this is that I should not even know it’s there. I will, at a push, concede to having a single, small icon in the system tray but there had better not been any popups or nags or hellos or splash screens or any of that crap unless there’s an actual, real live virus emergency. And, it had better not interfere with the normal running of my machine or my productivity. I don’t ask much, see ? Both NOD32 and Sophos are excellent in all these respects. Completely reliable when you need them and virtually invisible when you don’t.
 
The machine I ran Microsoft Security Essentials on was my 2 year old work-horse laptop, i.e. the machine I use most of the time, most of the day. It’s a Dell Latitude D830 with a 2Ghz Core 2 Duo CPU, 4GB of RAM, a 200GB 7200 RPM disk and a wide screen 1680 x 1050 TFT driven by a 256MB NVidia Quadro NVS 140M graphics card. I am running a fairly clean build of Windows 7 which performs like lightening and is an absolute joy to use.  It is, in short, a pretty good laptop.
 
Microsoft Security Essentials behaved impecably to begin with. It was quick and easy to download, install and configure and it ran silently in the background without seeming to get in the way. But then the trouble started. Specifically, I noticed a problem with audio playback. It was most apparent with video and especially flash. What I was getting was frequent, irregular buzzes, blips and judders in the audio. This was happening in flash, video players and even in media players when I was just playing MP3s. In Windows Media Player I could actually produce the problem by resizing or moving windows around on the desktop while audio was playing.
 
I tried various configuration changes, killing as many other processes on the system as I could, reinstalling audio drivers etc but all to no avail. Needless to say Microsoft Security Essentials has now been uninstalled and the problem vanished instantly. Obviously this may be a very isolated case related to my specific setup but I’d be ineterested to know if anyone else has similar problems.

SQL Azure

Finally getting round to having a play with SQL Azure. You can’t administer it with the usual UI tools like SQL Server Management Studio unless you want to use this slightly kludgy workaround, only SQLCMD 😦 Fortunately,  Julien Hanssens and Martin Balliauw have created SQL Azure Manager which you can install and run instantly using Click-Once. Nice 🙂
%d bloggers like this: