Craig Box's journeys, stories and notes...

Archive for the ‘Technical’ Category

OpenID and MindTouch

Wednesday, September 15th, 2010

The below is all outdated - links updated to for historical interest.

When I was working at Coreworx in Canada I introduced an internal knowledge base in the form of MindTouch (then DekiWiki).  I got involved in the community around the project, and ended up meeting the developers behind it in San Diego.

I evaluated it for a project at Symbian, and in order to try and make it suit us better I wrote a module for using OpenID.

Today, MindTouch have published a couple of posts I wrote on the subject, which I am happy to share with you here:

In other news, over the course of the next few weeks, I'll be running a series on the migration of Symbian's LAMP/LTMJ (unfortunately Linux/Tomcat/MySQL/Java doesn't have a vowel in it) hosting servers to Amazon EC2. Stay tuned!

How do I change the DHCP subnet for NAT on VMware Fusion 3.0?

Tuesday, November 10th, 2009

There are a couple of helpful blog posts (Nilesh Kapadia and Max Newell deserve a shout-out here) which help you with changing the DHCP settings given to your NAT or host networks on VMware Fusion. However, it all changes in 3.0.

The file you now need to edit is /Library/Application Support/VMware Fusion/networking. In there, you will find these lines:


I believe the third octet (the 93 part) is selected randomly when you install; in any case, I wanted to give out addresses on, so I changed the configuration like so:


and restarted the network interfaces:

sudo "/Library/Application Support/VMware Fusion/" --restart

Now, make a note of the MAC address of your virtual network adapter in your guest OS, and you can assign an entry in the dhcpd.conf file (/Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf).  Make sure you do it outside of the area that is marked "this will be overwritten"!

host developer-vm {
    hardware ethernet 00:0c:29:cb:dd:72;

and another service restart.

A more pragmatic, but less common, question about Office Open XML

Sunday, January 25th, 2009

Geek question.

Assume people collaborate using Microsoft Office documents around (probably by e-mail, but that's another discussion).  

The "97-2003" formats are closed, binary formats, but well readable by applications such as, and now documented by their author (albeit under a specification that the GPL people can't yet get behind).    The 2007 format is more interoperable in the sense it's XML and (mostly) documented and went through an infamous "standardization" process, but is still not widely accepted. I would rather see ODF succeed — potentially with patent-free extensions from Microsoft to allow it to support all the features of Microsoft Office — as the 'standard XML document format'.  

Now we've got all that aside, I prefer .doc to .docx, and it's nothing to do with any of that - it's purely pragmatic, with respect to users of previous versions of Office.

Office 2003 and 2007, side by side.

I believe that in the team I work in, maybe 2/3 are on Office 2003 and the other 1/3 are on Office 2007.  Our customers, being that we deal with very large companies, are overwhelmingly all still on Office 2003.  (Personally, I'd rather send a PDF to a customer than a DOC, but that's not a decision I can make company-wide.)

I have the converter pack installed, which makes my Office 2003 installation compatible with Office 2007 documents (though it does prompt me and say that some features may be lost in translation).  I can't assume that everyone does, however, so OOXML files inconvenience anyone who gets sent these documents that does not have the converter installed.  The argument could also be made that PDF files inconvenience people without a PDF reader: everyone just downloads one, so what?

By way of opinions placed in the comments, should I be encouraging colleagues who send me documents in Office 2007 format, to enable saving by default in Office 2003 format?

Two hits

Friday, January 16th, 2009

The question: "why do hash functions use prime numbers?"  I knew I'd seen someone suggest it wasn't necessary in researching this topic last week.  (They cited this, if you care.)

The Google search: "prime hash".

Result #1: Why do hash functions use prime numbers?

Result #2: Prime Hanger Steak with Crispy Hash Browns and Pierre Poivre Sauce

Mmmm, prime hanger steak with crispy hash browns and pierre poivre sauce!

Mmmm, prime hanger steak with crispy hash browns and pierre poivre sauce!

I think it was lunchtime at that point.

Also, did you know the reproductive cycle of cicadas is 17 years?

Google Chrome

Wednesday, September 3rd, 2008

Looks great.  No AdBlock.  Back to Firefox it is.  Call me back when there's an extension API.

More on Windows service permissions

Wednesday, March 12th, 2008

A while ago I wrote about the permission required to query the status of a Windows service.

Today, I started looking for a way to set this permission from the .NET framework, without using sc or subinacl. Registry and file system security is easy; services, not as much. After what seemed like an eternity of searching I came across the answer - an article on 'stackenbloggen'1 on setting ACLs on Windows services. There is magic in System.Security.AccessControl in .NET 2.0 which lets you control security on pretty much any Windows object, and here are some classes to expose it.

Words can be very specific in the English language, and even more so in code. For example, a method called "SetAccessControl" does not do what it would do if it was called "AddAccessControlEntry". So, if you create an ACE that contains just the item you want to add, and run SetAccessControl, you're screwed.

Never mind, I wrote about how to fix this:

C:\Craig\>sc sdset Service1 D:(A;;CCLS...)
[SC] OpenService FAILED 5:Access is denied.


Maybe I'll get a better message out of subinacl?

C:\Craig\>subinacl /service Service1 /sacl=D:(A;;CCLS...)
Service1 - OpenService Error : 5 Access is denied.

Double oops.

New error message though, worth searching for:

OpenService Error 5 Access is Denied - Google results

It's very irritating to only find your own post when searching for something! Lots of old-school #wlug'ers used to get this all the time with the WLUG wiki.

A bit more Googling, and the answer is found in Microsoft's knowledge base: how to reset a DACL on a Windows service. The crux of the matter is you need to run a command prompt as LocalSystem - type something like at 15:37 cmd.exe /interactive, where 15:37 is the time one minute from now. (Don't make it bigger. Waiting 30 seconds is tedious enough!)



C:\WINDOWS\system32>sc sdset Service1 D:(A;;CCLS...)
[SC] SetServiceObjectSecurity SUCCESS

Double booyah!

I'll be calling GetAccessControl and adding my own ACE to its result from now on, I think.

  1. I love the name 'stackenbloggen', but a disclaimer that says "[..] my own personal opinions and do not represent my employer's view in any way" is meaningless if you don't say who you are on your blog! 

Debugging w3wp.exe and IIS application pools

Saturday, December 29th, 2007

Problem: after hitting "Login" on an ASP.NET web application on IIS6, w3wp.exe races to high CPU usage, and about 70MB of memory. CPU usage falls off, but memory usage increases a few KB every second. No indication is made that the application has crashed or hung, no output.

The first utility I installed to check this out was IISTracer. Ignore the horrible web site - this is a nice little utility which lets you see what requests IIS is currently serving. In this instance the default.aspx page calls a dashboard.aspx, which was timing out indefinitely without returning a response to the client.

There are a number of utilities you can use to debug IIS faults. David Wang's guide on how to understand and diagnose an application pool crash gives some insight into how it should be done - attach a debugger and wait for the fault, then look at the stack trace. His suggestions include IISState, an IIS resource kit utility, which will give you an output of the current state of an IIS process; and DebugDiag, a much nicer and newer interactive debugger for IIS.

I spent a couple of days playing with DebugDiag, and unfortunately, due to the nature of my problem, I couldn't get anything useful. It seems that the application wasn't ever actually crashing or hanging - states that DebugDiag should be able to deal with easily. IISState, on the other hand, suggested I was regularly getting first chance exceptions thrown. A first chance exception means an exception has been raised - if the application handles it, then nothing more is mentioned, else it becomes a second chance exception, and the debugger or system exception handler is called). So, more digging was required.

WinDbg, the Windows DebuggerEnter the Windows debug tools, and specifically WinDbg. This is what the professionals use, and as such, it's ultimately powerful and incredibly scary. It's updated regularly, except for the interface, which is nicest described as "circa-1990".

There are a number of blogs that mention running WinDbg, but none that actually put together all the pieces on how to debug this kind of problem. So here's my story for you.

First, attach the debugger to your w3wp.exe process. I found it's process ID in Task Manager - you need to add the PID column. At this point, I get a breakpoint hit - you might not, so use Debug/Break or Ctrl-Break to stop the program.

Debugging symbols let you see not just the memory location of a failure, but the name of the function that it is in. They are not normally included with release software as they make it a lot larger, but can be downloaded separately in this case. DebugDiag is friendly and will automatically download appropriate debugging symbols for you from Microsoft's symbol server, and put them in c:\symcache\. WinDbg is less friendly and requires you set this up yourself. I used the same path, as I already had some symbols downloaded - I told WinDbg to look at the local symbols first, and look at the online symbol server if they're not already cached there. Do this like so:

.sympath SRV*c:\symcache*

Now, we need to load the SOS debugger helper, which lets you debug managed code in WinDbg (Nerd note: SOS stands for Son of Strike, which is a Microsoft in-joke.) If you started the process inside the debugger, you can apparently do this:

0:033> .loadby mscorlib sos
Unable to find module 'sos'

I couldn't, so I'll do this:

0:033> .load C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\SOS.dll

If you're running .NET 1.1, you can .load clr10\sos.dll for the version shipped with the debugger.

Now, we instruct it to break whenever there is an exception thrown:

0:033> sxe CLR

And hit Debug/Go, type 'g and hit Enter, or hit F5 to resume the program.

No sooner than I hit 'g', we get results!

0:033> g
(a78.8b4): CLR exception - code e0434f4d (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=0230ec7c ebx=000d9c50 ecx=00000000 edx=00000024 esi=0230ed08 edi=e0434f4d
eip=77e4bee7 esp=0230ec78 ebp=0230eccc iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00200202
77e4bee7 5e              pop     esi

I don't speak assembly and really have no desire to. Thankfully the SOS extension lets us look at the exception, and the call stack (the list of functions that we called to get where we are). Type !pe to print the exception, and !clrstack to see the call stack.

0:012> !pe
Exception object: 048f8a4c
Exception type: System.Data.SqlClient.SqlException
Message: Cannot open database requested in login 'R622P1Sy1'. Login fails.
Login failed for user 'admin'.
InnerException: <none>
StackTrace (generated):
StackTraceString: <none>
HResult: 80131904
0:012> !clrstack
OS Thread Id: 0x8b4 (12)
ESP       EIP
0230ed54 77e4bee7 [HelperMethodFrame: 0230ed54]
0230edf8 6539bf14 System.Data.SqlClient.SqlInternalConnection.OnError(System.Data.SqlClient.SqlException, Boolean)
0230ee0c 652f695d System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(System.Data.SqlClient.TdsParserStateObject)
0230ee3c 652f79e5 System.Data.SqlClient.TdsParser.Run(System.Data.SqlClient.RunBehavior, System.Data.SqlClient.SqlCommand, System.Data.SqlClient.SqlDataReader, System.Data.SqlClient.BulkCopySimpleResultSet, System.Data.SqlClient.TdsParserStateObject)
0230ee90 652e932a System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean)

And there's the problem. A first-class exception is handled trying to connect to a database with incorrect login details, so it continues, probably never freeing the memory from the last connection. This particular exception doesn't get logged and doesn't get displayed on screen (probably because it's being loaded in the background) and causes an infinite timeout. Oops!

Armed with a little knowledge about the debugging tools, it's easy to see what's going on inside your code, even if it's running in IIS.

Parsing quotes in cmd.exe

Monday, December 24th, 2007

The Microsoft world can be horribly inconsistent. Take escaping a quote character:

  • "" in VB and SQL Server
  • \" in C++/C#
  • Unless you use a @ symbol first, in which case you have to use "" as \ won't be interpreted as an escape character

It never seemed at all you could escape characters on a command line. Until, of course, you go to do something like this:

rem xcopy is old, let's not use it any more
rem the backslash on the end of bar tells xcopy
rem that it's a directory, not a file, it's copying to
rem xcopy /e "C:\foo\" "C:\bar\"
robocopy "C:\foo" "C:\bar\" /e /x "Extra Options"

Whereupon the much newer program parses the output as follows:

destination=C:\bar" /e /x "Extra Options"

This is a problem with anything that uses the .NET framework also. Watch your trailing slashes!

C#: How to tell if an IIS metabase property is inherited

Monday, December 24th, 2007

How can you tell, using managed code, if an IIS metabase property is inherited?

Scenario: I am using System.DirectoryServices to manage the IIS configuration (known as the "metabase", and stored in an XML file). I want to remove a property on a metabase node, but only if it is explicitly set here. This allows me to inherit the correct value from up the tree.

First attempt:

if (dirEntry.Properties["AnonymousUserName"].Count > 0) {
    Console.WriteLine("Log: Local value found for AnonymousUserName, removing");

My assumption was that this would remove the entry the first time, if it exists, and ignore it every time afterwards. However, every time it runs, Count is 1 and the property appears set.

It turns out that the Properties method on a DirectoryEntry object shows all the properties, including ones that are set on parent items in the tree.

How can we see only items that are defined on this particular node? This problem shows up all over the Internet with no solution. However, Microsoft's David Wang isn't one to leave a newsgroup question unanswered:

Never tried it in .Net, but with ADSI you can determine this.
Set objNode = GetObject( "IIS://foo/W3SVC" & "/" & "AppPools/AnAppPool" )
Set objNodeAttribute = objNode.GetPropertyAttribObj( "WamUserName" )
If ( objNodeAttribute.IsInherit = true ) Then
End If

I just needed a way to run GetPropertyAttribObj from C#, which was a much easier problem to solve. There were only a couple of relevant hits, the best one being some abandoned Pastebin code in Google's cache.

We now have an answer:

/// <summary>
/// Establish if a metabase property is set explicitly on this node.
/// </summary>
/// <param name="iisNode">Metabase node</param>
/// <param name="propertyName">Property name</param>
/// <returns>True if the property is defined on this node explicitly; false if the property is not defined at all, or is inherited.</returns>
private static bool PropertyDefinedExplicitly(DirectoryEntry iisNode, String propertyName)

    // check 1. this node has a valid key-type, and 2. it contains the required property
    if (iisNode.Properties.Contains("KeyType") &&
    iisNode.Properties["KeyType"].Value.ToString().Length > 0 &&

        // call ADSI GetPropertyAttribObject method directly
        Object propertyAttributes = iisNode.Invoke("GetPropertyAttribObj",
        new String[] { propertyName });
        IISOle.IISPropertyAttribute attributes =

        // Isinherit will be set if the property is inherited
        if (attributes.Isinherit)
            return false; // property was inherited
            return true; // property explicitly defined on this node

        return false; // not a valid IIsObject node, or property not found

We then rewrite the original function like so:

if (PropertyDefinedExplicitly(dirEntry, "AnonymousUserName")) {
    Console.WriteLine("Log: Local value found for AnonymousUserName, removing");

There is a lesson to learn here for Windows programmers: Microsoft support is in the newsgroups. Don't fear Usenet because it's older than you are!

With useful tips like this, I'll win the NZ .NET blog of the year for sure!

Querying the status of a Windows service

Friday, December 7th, 2007

An interesting puzzle presented itself to me at work yesterday. We have some code called from an ASP.NET application, which queries the status of a service. This works fine when you run the application pool as NetworkService, but if you run it as the IWAM_SERVERNAME user that is created for running IIS processes, it would fail.

I wrote a console application with exactly the same code that the ASP.NET application used, and it worked fine.

Interesting aside: you can find the password for the IWAM_SERVERNAME user by using the Metabase Explorer from the IIS 6 Resource Kit. It's under the Server/LM/W3SVC node as WAMUserPass, and you have to enable "View/Secure Data" to see it. Then, use runas /user:iwam_servername cmd.exe, and enter in the password from the metabase.

Why? Well, surely it must just be some kind of permissions issue, not the Winter Madness.

What can't be done with subinacl isn't worth doing, so I pulled it out and had a look at the service. Here's the important part:

/pace =interactive      ACCESS_ALLOWED_ACE_TYPE-0x0
        SERVICE_INTERROGATE-0x80           READ_CONTROL-0x20000               SERVICE_USER_DEFINED_CONTROL-0x0100

(People finding this useful post via search will want to read the full subinacl output.)

Turns out an interactive user can query a server status, but IIS doesn't run interactively. Assigning permission to the IWAM account is the only way to do this:

subinacl /service Service1 /grant=IIS_WPG=s

Problem solved for the web app.

Always one for complete proof, I tested removing interactive access (subinacl /service Service1 /deny=interactive=s). Result as requested - the console app now throws the same exception the web application does.

But wait...

C:\Program Files\Windows Resource Kits\Tools\>subinacl /service Service1 /revoke=interactiveService1
- OpenService Error : 5 Access is denied.

Oops. Big oops. Nothing I can do now seems to be able to let me fix this, because I've lost access to query the status of the server myself.

Some searching led me to an answer: use the sc.exe utility, which lets you control services from the command line. It also lets you set their security descriptors!

C:\Program Files\Windows Resource Kits\Tools\>sc sdshow Service1 

C:\Program Files\Windows Resource Kits\Tools\>sc sdshow Service2

[SC] SetServiceObjectSecurity SUCCESS

C:\Program Files\Windows Resource Kits\Tools\>subinacl /service Service1
+Service Service1

Whew. See how the SDs differ between the two services? Ignore the gobbledy-gook that is SDDL, and just copy an example from another similar service.

I suspect I could have granted access using with sc instead of subinacl, but the IWAM_SERVERNAME account and IIS_WPG group don't have well-known SIDs, so I would have had to find out the SID using another utility anyway.

(Hi Mum. This is what I do all day.)