Adding a Dynamic Robots.txt file to an ASP.NET MVC site

Robots.txt is required to allow search engines to properly index your site, and more importantly not index it. If you have a public-facing staging or preliminary site that you don’t want to show up in Google results, you need to make sure that it returns the correct robots.txt with the

Disallow: /

line to prevent indexing. However, manually adding robots.txt files to staging and production environments as a manual process can be improved with the process below – the same code can serve up a locked down robots.txt in staging or internal URLs, and allow indexing in production.

First, add a route that responds to /robots.txt in either Global.asax.cs or RouteConfig.cs before your Default routes:

routes.MapRoute(
"Robots.txt",
"robots.txt",
    new
    {
        controller = "Robots",
        action = "RobotsText"
    }
);

You’ll also need to make sure that runAllManagedModulesForAllRequests is true in web.config as normally text files bypass the ASP.NET pipeline:

<system.webServer>
    <modules runAllManagedModulesForAllRequests="true"></modules>
    ...
    </handlers>
  </system.webServer>

The create a new controller called “RobotsController” with a single action “RobotsText”. All requests to /robots.txt will go here:

    public class RobotsController : Controller
    {

        public FileContentResult RobotsText()
        {
            var contentBuilder = new StringBuilder();
            contentBuilder.AppendLine("User-agent: *");

            // change this to however you want to detect a production URL
            var isProductionUrl = Request.Url != null && !Request.Url.ToString().ToLowerInvariant().Contains("elasticbeanstalk");

            if (isProductionUrl)
            {
                contentBuilder.AppendLine("Disallow: /elmah.axd");
                contentBuilder.AppendLine("Disallow: /admin");
                contentBuilder.AppendLine("Disallow: /Admin");
                contentBuilder.AppendLine("Sitemap: http://www.mysite.com/sitemap.xml");
            }
            else
            {
                contentBuilder.AppendLine("Disallow: /");
            }

            return File(Encoding.UTF8.GetBytes(contentBuilder.ToString()), "text/plain");

        }
    }

You can try a number of ways of detecting a production environment, from the naïve URL checking above to environment variables in your application container.


Surface Windows RT DOES NOT support Cisco IPSec VPNs

A promising slide from TechEd 2012 promises VPN support for Cisco VPNs:

And from here:

The TechEd 2012 session “Windows 8: Windows RT Devices for Business” (http://channel9.msdn.com/ Events/TechEd/NorthAmerica/2012/WCL202) stated at the 26 min mark that Microsoft incorporate the Cisco VPN client into Windows RT out of the box.

It doesn’t work for most Cisco VPN setups where the IT department has decided to use PSK authentication. Windows never natively supported the IPSec Identifier, or “Group Name” feature in Cisco VPNs, which I argue would be the vast majority of real use cases.

See? No Group Name option. Compare this to OS X:

OS X supports Cisco IPSec natively and has a field for Group Name. iOS also does.

On Windows RT, you cannot install the Cisco AnyConnect client. You shouldn’t have to do this anyway – Windows should natively support this, as Apple has chosen to do. Some people are going to get pretty annoyed when they realize they can’t use Windows RT devices to remote in to corporate VPNs, although some would argue that corporate VPNs should be using certificates and not Pre Shared Keys for IPSec.

iPads can connect to Cisco VPNs natively – sort it out Microsoft.


Remap CapsLock key to Control key on Surface Windows RT

The CapsLock key is rubbish. Remapping it to Control is easy on OS X and the Happy Hacker keyboard I use on my desktops doesn’t even have one. On standard Windows you can run a key remapping program but WinRT is locked down (for now). Here is an easy way to map Caps Lock to Control.

First, hit the Windows key and type “Powershell”. Right click on the Powershell icon. You will see some options appear at the bottom of the screen – click “Run as Administrator. Agree to the UAC elevation prompt.

The elevated Powershell prompt will appear. You need to enter the following command and hit Enter:

Set-ItemProperty -path "HKLM:\SYSTEM\CurrentControlSet\Control\Keyboard Layout" -name "Scancode Map" -Value ([byte[]](0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x02,0x00,0x00,0x00,0x1d,0x00,0x3a,0x00,0x00,0x00,0x00,0x00))

This sets up some keyboard scancode remapping in the Registry, which is what keyboard remapping programs would do if they had access. Restart the machine (swipe from right, Settings Charm, power, Restart) to make the change take effect.

Good riddance Caps Lock!


AdBlock alternative on Windows RT’s IE10

On the Surface’s Windows RT, you cannot install alternative browsers. This means no AdBlock.

There is a workaround to this, using IE10’s “Tracking Protection”. First, fire up desktop IE10, click the little gear in the top right and select Safety -> Tracking Protection. Click on “Get a Tracking Protection List online…”.

You should see a list of tracking protection lists on Microsoft’s site. You will want to click Add next to the Easylist lists. These might not cover all ads – feel free to add a few more.

With these lists added to IE10, both the Desktop and Metro versions will filter out the ads. This method is not quite as good as AdBlock which removes the ads from the screen entirely, but is better than nothing.

See before and after screenshots of The Register in Metro IE10 below.


Why I won’t be returning my Microsoft Surface RT

I’ve picked up a Microsoft Surface RT from the “temporary” New York Times Square Microsoft Store. While the purchasing experience was a bit of a mess (untrained sales staff – I ended up having to type my own details into their POS terminal as I’m not quite sure the guy knew how to type), I managed to make off with a properly boxed (I skipped getting the staff to unbox it and walk me through the “Out of Box Experience”) 32GB Surface, Type Cover and VGA adapter.

The device

This is an amazing piece of hardware. I’ve dabbled in and out of portable computing ever since the original Windows Tablets in the form of the HP TC1100. The TC1100 was ahead of its time – a full version of Windows in a tablet with a decent attached keyboard that went together to make a case. However, the hardware was pretty bulky – fans and a display too far away from the glass, making touch, even with the included stylus, inaccurate.

This really looks like something out of a Bond film. The tapered edges and kickstand both apparently go off at 22 degrees (I haven’t measured), looking deliberate – as if designed by actual designers. I would not be surprised if some ex-Apple guys worked on this device. The battery is said to go 10 hours of mp4 video – more than enough for a long haul flight, and I can personally attest to the quick charging of the Surface up to around 50% in about an hour. The camera is at an angle to compensate for the slant when using the kickstand. Portrait mode looks slightly ridiculous, but I can’t imagine using it in portrait when just sitting the device landscape in your lap works so well.

The Type Cover is like nothing you’ve seen before – forget the comedy Bluetooth “keyboard covers” for iPad users desperate to do a bit of actual work on their devices – this is genuinely a pleasure to use. The small multitouch trackpad is great, but strangely defaults to inverted scrolling like OS X Mountain Lion. My only gripe is that they still have the Caps Lock key – I will have to delve into the Registry to switch this to a Control key. Yes, I did say Registry…

The software

Windows RT is installed by default and is the most fascinating part of the device. Many pundits have long called for Microsoft to abandon backwards compatibility and build a new operating system – if they did, it would look something like this. Windows RT is Windows 8, but without the backwards compatibility legacy. The full Windows desktop is here – the Command Line, PowerShell, Windows Explorer, Paint, Notepad, VPN connections, BitLocker – the works. The only difference is that WinRT only allows Microsoft-signed applications to run on the desktop. Visual Studio 2012 can build ARM applications but this is blocked by Microsoft – bypass the block with a registry setting and the compiled binaries are blocked by Windows RT. I suspect this is because the device is subsidized and they hope to recoup revenue from the Windows Store apps, but it might just be a ploy to force developers to build Metro-style applications. The thicker, heavier, noisier Surface Pro is due soon and will run the full x86 Windows 8 Pro, bringing legacy support with it.

If the Surface RT is meant to be used as a main or only machine, this is going to cause problems. In the Microsoft Store, non-technical members of the general public were already asking if they “could use their iPhone with it” – yes, they can, but only to charge and download photos. iTunes will not install. Firefox will not install. Even some Windows Store Metro apps will not work – notably Google’s suite. If you or your company uses Google Apps, steer clear from Windows RT until Microsoft or Google budge and allow Google Drive syncing. There could be serious backlash due to Windows RT – if enough customers kick off a storm, Microsoft might remove the signature check when running ARM binaries, allowing third-parties to treat Windows RT like a first class citizen.

In reality though, a Surface RT is not going to be your only machine. It absolutely shines as a Remote Desktop thin client – you can have Visual Studio running on your 12-core beast machine in the office and just remote in to use it. Flash is built into the browser, allowing stuff like Adobe Connect to work without a problem – the web browser version being much better than the iOS version, which still hasn’t been updated to support retina or iPhone 5 long displays. One thing you can’t do is share your screen with Adobe Connect. Full desktop-class Web Apps such as Google Docs work fine in Metro IE10 and without any browser chrome, could be mistaken for a “native” application.

Games from the Windows Store run quite badly on Surface. Jetpack Joyride has a pretty bad frame rate when lots of sprites are on the screen. Hopefully there is room for improvement in drivers, but don’t expect iPad 4 quality graphics. You cannot see pixels when sitting back and typing thanks to ClearType, but the resolution is nowhere near the iPad Retina display.

You might have a few favorite Windows applications that won’t run on Windows RT. I was worried that I would not be able to live without Windows Live Writer, but it turns out that Word has had a great blogging tool since version 2007, and it is even better with 2013.

Office 2013 RT

The first thing you should do with Surface is run Windows Update. This updates the Preview version of Office RT to the final version. Microsoft plan to deliver incremental functional updates, giving another reason to get a Microsoft Surface and not another Windows RT ARM device. Other manufacturers could cause delays to updates like Windows Phone 7 has been plagued with – carriers and device makers literally stopping you from using the latest version of the software running on your device (if MS do a Windows Phone 8, you should get it instead of any third party offering).

Office 2013 starts up very quickly. Saving to SkyDrive or the local drives just works. This really has full compatibility with Office files. The only glaring omission is Outlook – however, Exchange accounts are fully supported in the Mail application and Gmail works great in IE10.

Why I’m keeping it

I can actually do something productive with this device, unlike the iPad which has been reduced to a wife-pleasing YouTube machine. It has a great built in RDP client for when you need to use software that does not run on WinRT. It has a USB port which works with actual mice, USB drives and even my Kensington PowerPoint remote. If the space runs out, I can just insert a Micro SD card and expand the storage capacity. The Surface fills a useful void between an iPad and a MacBook Air – something that you can just throw in your bag without a laptop bag cover but can still get some work done on. Word, PowerPoint and Excel come free with Surface RT – these separately would be $30 or so on an iPad as iWork but here you get the real deal. I really am sick of lugging around proper laptops when I could just remote into a beefy machine under my desk. With LTE tethering in Japan, this is becoming a reality.

Only buy a Surface RT if you know what you are buying and can live without full legacy application support. As a thin client for real world web applications, it might make some people very happy indeed.

 


Using a Nokia Lumia 800 Windows Phone on Softbank Japan

The only official  Windows Phone in Japan is au’s Toshiba IS12T. Not wanting to change plan, I picked up a SIM unlocked Nokia Lumia 800 and put the Softbank micro SIM card from my iPhone 4 inside. If you want a more general guide to setting up Windows Phone for previous iPhone users and what apps should be installed, read Scott Hanselman’s post. Remember to disable iMessage if you are moving from an iPhone.

IMG_8140[1]

About the Nokia Network Setup application

DO NOT use the Nokia Network Setup application as it will not make your phone work with Softbank in any way. It tries to set up WAP-era metered dialup connection which Windows Phone 7 does not support any more. If this ever worked, it would cause massive phone bills and general misery. Steer clear.

What does and doesn’t work

Calls OK
SMS messaging between carriers OK
SMS messaging to Softbank and other iPhone users OK
MMS via @softbank.ne.jp NG
MMS via @i.softbank.jp OK
Data OK
Visual Voicemail NG

Setting up data

The Softbank iPhone unlimited data plan works with other phones if you put the “secret” access point details in. The access point name is “smile.world”, username is “dna1trop” – you will need to Google for the password. Insert this access point information under Settings > Mobile Network > Edit APN and you will have data working, including HDSPA which will show up with an “H” icon in the status bar.

IMG_8136[1]

Messaging – dealing with the lack of MMS

MMS is supported by Softbank, but only on phone models they sell. Windows Phone 7 has MMS support but because the User Agent is unrecognized by Softbank, they block it at their MMS proxy server. Android users can easily change their MMS User Agent to match a phone that Softbank sell, but the Lumia 800 cannot do this as there is no Interop “root” access available yet. Nokia almost allowed the setting of MMS User Agent in the built-in Diagnostics application but for some unfathomable reason locked the “MMSPage.xaml” screen out – .NET Reflector even shows the registry key that the application would have edited:

image

Hopefully Nokia releases an updated version of the Diagnostics application where we can set the MMS UserAgent or an Interop unlock for the Lumia 800 is found.

Last year Japan’s networks turned on SMS, meaning you can send text messages between carriers just like the rest of the world. Messages are free between users on your network but about 3 yen to other networks. Japanese text is supported and Windows Phone will even show some basic emoticons out of the box. When somebody sends you an MMS to your phone number or keitai email address, you get an SMS reading “Get media content now”. You’ll be able to see who sent the message but won’t be able to read the contents or directly reply. With Windows Live and Facebook chat built into the phone, this is only a slight annoyance – it is in your best interests to wean your contacts off your Softbank-locked @softbank.ne.jp keitai mail address anyway as this will make it painless to change provider in the future once Japan’s carrier market has loosened up a bit.

Somebody tries to send you an MMS and this happens

You can still use the @i.softbank.jp email address that you got with your iPhone as it is a standard IMAP account. Just add it as an extra email in Windows Phone and it will auto-detect the settings. You can email other keitai email users as it counts as a “keitai” email address so will not be immediately blocked by spam filters. When somebody sends an email to this address, you get a “flash” SMS. This system was used on the iPhone before it got MMS support.


Free laptop wifi on the Shinkansen (or any “mobilepoint” AP)

Update November 2014: Softbank has changed their WEP keys. This will no longer work. Sorry folks.

I finally figured out how to get wifi access using a laptop on the Shinkansen (Nozomi N700 only I think) or any Softbank “mobilepoint” access point. iPhone and iPad users will already have free “Softbank Wifi” access – this is normally locked to only an iPhone or iPad. Softbank only check your user agent.

  • Select the access point “mobilepoint”
  • Enter the WEP key: 696177616b
  • On Lion, a wifi access window will appear showing the mobilepoint login screen. Close this if it appears.
  • Open Safari
  • On the Develop menu (this might have to be turned on in Preferences) select the correct user agent:
    • For iPad Wifi (which gets you “2 years free wifi”) or iPad Wifi/3G users, select “Safari iOS 4.3.3 – iPad”
    • For iPhone users, select “Safari iOS 4.3.3 – iPhone”
  • Try and access a page in Safari and it should redirect you to the mobilepoint login page.
  • Enter your Softbank username and password. For iPhone or iPad 3G users, this is the “[email protected]” email address and password for the IMAP account you never use. For iPad Wifi users, use the “[email protected]” account with the password written on the letter you got bundled with your wifi-only iPad.
  • Since their portal now thinks you are actually logging in on the correct device, you have access. You can now stop using Safari and use Chrome/Firefox/Mail.app/whatever.
Alternatively you could fire up Skype and pay money per minute for Boingo wireless over Skype Wifi access after entering the mobilepoint WEP key. No thanks.

Fun with jQuery flot charting

I’ve got the Microsoft Money style cashflow chart going with the flot library and some massaging of ASP.NET MVC 3 to output JSON.

image

The chart correctly predicts future cashflow based on your set bills and future transactions. Next up is multiple accounts and currencies at once.


Working on an online MS Money replacement

I am finally fed up of having to install Microsoft Money and set up syncing of the data file, so I’ve decided to try and build a web based version. Mint.com does not support non-US markets and apparently does not even support adding your own transactions that have not been downloaded from a bank. “Private Money” is the codename and it looks a bit like this at the moment:

Capture

The stack:

  • ASP.NET MVC 3 as the application framework
  • Entity Framework 4.1 for ORM
  • NInject for a modular design with dependency injection
  • MigratorDotNet for database versioning with some tweaks to get it to work over different modules
  • jQuery and jQuery UI for some nice buttons, animations and AJAX loading. MVC 3 has great support for unobtrusive javascript with jQuery which is a dream to use.

I am planning on replicating the parts of MS Money that I use the most, mainly the transaction logging, reports and cashflow chart (which should be fun to make on a canvas element). Visually I am going for the Windows Live/Metro motif.


Thoughts on ASP.NET MVC3 Update and Entity Framework 4.1

Microsoft have now released an update to ASP.NET MVC3 imaginatively called “ASP.NET MVC3 Update”. This update upgrades NuGet, bundles Entity Framework 4.1 with all new templates, adds HTML5 template support and splits the jQuery libraries off into NuGet packages so they can be upgraded individually.

The bundling with new MVC projects is a sign that Microsoft has selected Entity Framework as the “chosen” ORM layer from now on. Linq2SQL is not flexible enough when dealing with pre-existing database schemas and third party solutions such NHibernate are difficult to get pointy-haired boss approval. EF 4.1 adds “code first” support where you define a very simple CLR object and it will automatically create the database tables for you. At first your eyes will light up thinking “finally! Rails’ ActiveRecord for .NET! Blessed by Microsoft!” until you find several missing pieces:

  • No migration support. The only option that EF provides when the “code first” objects behind your schema change (for instance, you add a new property) is to drop the entire database and start again. Rails has had this for years.
  • No code first support on real databases. That’s right, code first only works on SQL Compact file-based databases. This is fine for development but you can forget about just dropping the files onto a clients webserver and have it create your database for you.

There are a couple of ways around the limitations. I am having great success with a combination of MigratorDotNet and Entity Framework to provide an ActiveRecord style schema migration path. Hopefully I can write about this soon.