Most of the time I love ASP.NET. But sometimes all of the goofy issues with IIS really make me think I hate ASP.NET. The saddest part of this love-to-hate saga is there should be an easy way to fix the root cause of why I hate IIS. The root cause is that it simply changes too much between each different version (or even update) of Windows.
Read on for the story of my recent tiff with ASP.Net.
A New Development Server
I decided to upgrade my dev server from a virtual machine running Windows Server 2008 R2, to a VM running Windows Server 2012 R2. All in all, the upgrade was pretty painless and since I’ve already been using Windows Server 2012 R2 elsewhere anyhow, the upgrade felt like a total win. There was some troubleshooting as I moved copies of MVC 3, MVC 4, and MVC 5 apps from my old server (running IIS 6.5) to a new host (running IIS 8.5), but nothing too particularly horrible. Or, maybe more accurately: nothing much more than the hour or so of troubleshooting that I expected.
Unfortunately, after I got my dev server all set up and running as I wanted it, I forgot to image the VM itself (to capture OS state and low-level configuration details). About a week later, the RAID controller in my VMware host overheated and took out this VM (and two other test machines) with it. I of course had backups for all of my data in place – but the box and its underlying configuration were toast.
A Tale of Troubleshooting
Happily, as it hadn’t been that long since I’d just spun up a new Windows Server 2012 R2 instance as a dev server, it wasn’t that hard to do it all over again. In fact, setting up a new box with IIS, the .NET Framework, and SQL Server – as well as restoring all backups and such, took less than 30 minutes of active effort.
That’s when the fun began. While one of my dev sites started up with no issues, another site I’ve been actively working on of late kept redirecting me to /account/login every-time I’d try to hit the root of the site. For a few minutes I ended up trying to figure out if what I was seeing might not have been some dev changes recently made to Startup.Auth.cs in my MVC 5 app, except I absolutely couldn’t reproduce this same problem on my workstation when working against IIS Express. It was only happening on IIS 8.5 on my dev server – when it hadn’t been happening on IIS 8.5 a week before.
Troubleshooting: The Sequel
With a bit more poking around and troubleshooting, I was able to determine that I could easily get to a defined route – like /home/index without any issues out on my new server – but simply trying to go to the root of the site and let routing detect and route me to the ‘default route’ would not work at all in IIS 8.5. No problems locally on IIS Express – and not a problem previously.
For a few minutes I worried that routing had somehow become busted on this machine – or at least, that my default route simply wasn’t being matched. With Phil Haack’s excellent route debugger installed I could see that the routes I was manually specifying as part of the URL were working fine – but I still couldn’t access the root of my site without being redirected to /account/login – and being told that the resource didn’t exist (the site I’m working on isn’t quite yet to the auth-stage).
So, I finally disabled Startup.Auth.cs – or removed authentication from my app, only to find that while things worked just fine (again) on IIS Express locally, I was now – quite happily – getting an HTTP 401 out on my dev server. Which, in turn, means that the OWIN plumbing in my app (on my dev server) was catching that 401 and (correctly) trying to redirect me and let me authenticate.
The stupid thing, though, is that I was throwing HTTP 401s in the first place. A bit of googling later, and I finally found out why. Somehow and somewhere in this second setup of my dev server, IIS had switched Authentication away from using the Application Pool Identity (which I thought was the best practice after all these years) to using a specific user – IUSR – instead. Then, because I had granted permissions to the IIS APPPOOL\<sitename> user on the folders for my sites (instead of IUSR), the site I was having problems with was getting HTTP 401s on the root of the site (but NOT on explicitly matched paths) – whereas on my previous dev server running Windows Server 2012 R2 IIS had somehow come configured out of the box to use the Application Pool Identity for anonymous access.
My hunch is that I probably used Windows Server 2012 R2 for my ‘first’ dev server upgrade, and likely grabbed a .iso for Windows Server 2012 R2 Update 1 for the second – and that one of the ‘benefits’ of Update 1 is that it completely rewrote the authentication plumbing. And that, in turn, is why I patently hate IIS. It’s constantly changing security schemes between versions, and updates. Moreover, unless you eat, sleep, and drink IIS and wait with baited breath on every blog post from the IIS team, I’m not sure how you’d know that the (damned) security model had been changed on you – short of banging your head around until you find out that your seeming ‘app related’ problems are security related and due to an underlying ‘update’.
What Could Have Made This Easier
Just as it’s a dumb idea to write (or at least send) emails when you’re angry, me writing this post so ‘fresh’ from frustration might not be the best idea. That said, I spent about an hour troubleshooting this idiotic problem – when, again, it only took me about 30 minutes of active effort to stand up an entirely new server from scratch. That and this particular problem was a huge pain – it manifested in some patently crazy ways. But the mere fact that I’m kind of used to having to spend time troubleshooting some aspect of IIS whenever I stand up a new site is I why I truly hate it.
My hatred aside, one thing that would help me hate IIS less and which would have helped immensely in this case would have been something like an IIS “diff” tool. Or something that would let you serialize EVERY detail about a target IIS site down to text so that I could have done that and then run something like WinMerge against the definition for my IIS 8.5 site and compared it against my IIS Express Site (where everything was working). In this case – and in gobs of cases in the past – I expect I would have been able to see fairly quickly that the security models were patently different. Without something like this, dealing with IIS when it comes to migrating sites or moving them to new hosts is all just extremely painful guesswork. I hope that someday the IIS Team (or some enterprising soul) will create something like this – because, until then, I’m sure I’m going to continue having days where I hate IIS.
