Craig Box's journeys, stories and notes...


Posts Tagged ‘programming’

OpenID and MindTouch

Wednesday, September 15th, 2010

The below is all outdated - links updated to archive.org for historical interest.

When I was working at Coreworx in Canada I introduced an internal knowledge base in the form of MindTouch (then DekiWiki).  I got involved in the community around the project, and ended up meeting the developers behind it in San Diego.

I evaluated it for a project at Symbian, and in order to try and make it suit us better I wrote a module for using OpenID.

Today, MindTouch have published a couple of posts I wrote on the subject, which I am happy to share with you here:

In other news, over the course of the next few weeks, I'll be running a series on the migration of Symbian's LAMP/LTMJ (unfortunately Linux/Tomcat/MySQL/Java doesn't have a vowel in it) hosting servers to Amazon EC2. Stay tuned!

Two hits

Friday, January 16th, 2009

The question: "why do hash functions use prime numbers?"  I knew I'd seen someone suggest it wasn't necessary in researching this topic last week.  (They cited this, if you care.)

The Google search: "prime hash".

Result #1: Why do hash functions use prime numbers?

Result #2: Prime Hanger Steak with Crispy Hash Browns and Pierre Poivre Sauce

Mmmm, prime hanger steak with crispy hash browns and pierre poivre sauce!

Mmmm, prime hanger steak with crispy hash browns and pierre poivre sauce!

I think it was lunchtime at that point.

Also, did you know the reproductive cycle of cicadas is 17 years?

More on Windows service permissions

Wednesday, March 12th, 2008

A while ago I wrote about the permission required to query the status of a Windows service.

Today, I started looking for a way to set this permission from the .NET framework, without using sc or subinacl. Registry and file system security is easy; services, not as much. After what seemed like an eternity of searching I came across the answer - an article on 'stackenbloggen'1 on setting ACLs on Windows services. There is magic in System.Security.AccessControl in .NET 2.0 which lets you control security on pretty much any Windows object, and here are some classes to expose it.

Words can be very specific in the English language, and even more so in code. For example, a method called "SetAccessControl" does not do what it would do if it was called "AddAccessControlEntry". So, if you create an ACE that contains just the item you want to add, and run SetAccessControl, you're screwed.

Never mind, I wrote about how to fix this:

C:\Craig\>sc sdset Service1 D:(A;;CCLS...)
[SC] OpenService FAILED 5:Access is denied.

Oops.

Maybe I'll get a better message out of subinacl?

C:\Craig\>subinacl /service Service1 /sacl=D:(A;;CCLS...)
Service1 - OpenService Error : 5 Access is denied.

Double oops.

New error message though, worth searching for:

OpenService Error 5 Access is Denied - Google results

It's very irritating to only find your own post when searching for something! Lots of old-school #wlug'ers used to get this all the time with the WLUG wiki.

A bit more Googling, and the answer is found in Microsoft's knowledge base: how to reset a DACL on a Windows service. The crux of the matter is you need to run a command prompt as LocalSystem - type something like at 15:37 cmd.exe /interactive, where 15:37 is the time one minute from now. (Don't make it bigger. Waiting 30 seconds is tedious enough!)

C:\WINDOWS\system32>whoami
NT AUTHORITY\SYSTEM

Booyah!

C:\WINDOWS\system32>sc sdset Service1 D:(A;;CCLS...)
[SC] SetServiceObjectSecurity SUCCESS

Double booyah!

I'll be calling GetAccessControl and adding my own ACE to its result from now on, I think.

  1. I love the name 'stackenbloggen', but a disclaimer that says "[..] my own personal opinions and do not represent my employer's view in any way" is meaningless if you don't say who you are on your blog! 

Debugging w3wp.exe and IIS application pools

Saturday, December 29th, 2007

Problem: after hitting "Login" on an ASP.NET web application on IIS6, w3wp.exe races to high CPU usage, and about 70MB of memory. CPU usage falls off, but memory usage increases a few KB every second. No indication is made that the application has crashed or hung, no output.

The first utility I installed to check this out was IISTracer. Ignore the horrible web site - this is a nice little utility which lets you see what requests IIS is currently serving. In this instance the default.aspx page calls a dashboard.aspx, which was timing out indefinitely without returning a response to the client.

There are a number of utilities you can use to debug IIS faults. David Wang's guide on how to understand and diagnose an application pool crash gives some insight into how it should be done - attach a debugger and wait for the fault, then look at the stack trace. His suggestions include IISState, an IIS resource kit utility, which will give you an output of the current state of an IIS process; and DebugDiag, a much nicer and newer interactive debugger for IIS.

I spent a couple of days playing with DebugDiag, and unfortunately, due to the nature of my problem, I couldn't get anything useful. It seems that the application wasn't ever actually crashing or hanging - states that DebugDiag should be able to deal with easily. IISState, on the other hand, suggested I was regularly getting first chance exceptions thrown. A first chance exception means an exception has been raised - if the application handles it, then nothing more is mentioned, else it becomes a second chance exception, and the debugger or system exception handler is called). So, more digging was required.

WinDbg, the Windows DebuggerEnter the Windows debug tools, and specifically WinDbg. This is what the professionals use, and as such, it's ultimately powerful and incredibly scary. It's updated regularly, except for the interface, which is nicest described as "circa-1990".

There are a number of blogs that mention running WinDbg, but none that actually put together all the pieces on how to debug this kind of problem. So here's my story for you.

First, attach the debugger to your w3wp.exe process. I found it's process ID in Task Manager - you need to add the PID column. At this point, I get a breakpoint hit - you might not, so use Debug/Break or Ctrl-Break to stop the program.

Debugging symbols let you see not just the memory location of a failure, but the name of the function that it is in. They are not normally included with release software as they make it a lot larger, but can be downloaded separately in this case. DebugDiag is friendly and will automatically download appropriate debugging symbols for you from Microsoft's symbol server, and put them in c:\symcache\. WinDbg is less friendly and requires you set this up yourself. I used the same path, as I already had some symbols downloaded - I told WinDbg to look at the local symbols first, and look at the online symbol server if they're not already cached there. Do this like so:

.sympath SRV*c:\symcache*http://msdl.microsoft.com/download/symbols

Now, we need to load the SOS debugger helper, which lets you debug managed code in WinDbg (Nerd note: SOS stands for Son of Strike, which is a Microsoft in-joke.) If you started the process inside the debugger, you can apparently do this:

0:033> .loadby mscorlib sos
Unable to find module 'sos'

I couldn't, so I'll do this:

0:033> .load C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\SOS.dll

If you're running .NET 1.1, you can .load clr10\sos.dll for the version shipped with the debugger.

Now, we instruct it to break whenever there is an exception thrown:

0:033> sxe CLR

And hit Debug/Go, type 'g and hit Enter, or hit F5 to resume the program.

No sooner than I hit 'g', we get results!

0:033> g
(a78.8b4): CLR exception - code e0434f4d (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=0230ec7c ebx=000d9c50 ecx=00000000 edx=00000024 esi=0230ed08 edi=e0434f4d
eip=77e4bee7 esp=0230ec78 ebp=0230eccc iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00200202
kernel32!RaiseException+0x3c:
77e4bee7 5e              pop     esi

I don't speak assembly and really have no desire to. Thankfully the SOS extension lets us look at the exception, and the call stack (the list of functions that we called to get where we are). Type !pe to print the exception, and !clrstack to see the call stack.

0:012> !pe
Exception object: 048f8a4c
Exception type: System.Data.SqlClient.SqlException
Message: Cannot open database requested in login 'R622P1Sy1'. Login fails.
Login failed for user 'admin'.
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131904
0:012> !clrstack
OS Thread Id: 0x8b4 (12)
ESP       EIP
0230ed54 77e4bee7 [HelperMethodFrame: 0230ed54]
0230edf8 6539bf14 System.Data.SqlClient.SqlInternalConnection.OnError(System.Data.SqlClient.SqlException, Boolean)
0230ee0c 652f695d System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(System.Data.SqlClient.TdsParserStateObject)
0230ee3c 652f79e5 System.Data.SqlClient.TdsParser.Run(System.Data.SqlClient.RunBehavior, System.Data.SqlClient.SqlCommand, System.Data.SqlClient.SqlDataReader, System.Data.SqlClient.BulkCopySimpleResultSet, System.Data.SqlClient.TdsParserStateObject)
0230ee90 652e932a System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean)

And there's the problem. A first-class exception is handled trying to connect to a database with incorrect login details, so it continues, probably never freeing the memory from the last connection. This particular exception doesn't get logged and doesn't get displayed on screen (probably because it's being loaded in the background) and causes an infinite timeout. Oops!

Armed with a little knowledge about the debugging tools, it's easy to see what's going on inside your code, even if it's running in IIS.

C#: How to tell if an IIS metabase property is inherited

Monday, December 24th, 2007

How can you tell, using managed code, if an IIS metabase property is inherited?

Scenario: I am using System.DirectoryServices to manage the IIS configuration (known as the "metabase", and stored in an XML file). I want to remove a property on a metabase node, but only if it is explicitly set here. This allows me to inherit the correct value from up the tree.

First attempt:

if (dirEntry.Properties["AnonymousUserName"].Count > 0) {
    dirEntry.Properties["AnonymousUserName"].Clear();
    Console.WriteLine("Log: Local value found for AnonymousUserName, removing");
}

My assumption was that this would remove the entry the first time, if it exists, and ignore it every time afterwards. However, every time it runs, Count is 1 and the property appears set.

It turns out that the Properties method on a DirectoryEntry object shows all the properties, including ones that are set on parent items in the tree.

How can we see only items that are defined on this particular node? This problem shows up all over the Internet with no solution. However, Microsoft's David Wang isn't one to leave a newsgroup question unanswered:

Never tried it in .Net, but with ADSI you can determine this.
Set objNode = GetObject( "IIS://foo/W3SVC" & "/" & "AppPools/AnAppPool" )
Set objNodeAttribute = objNode.GetPropertyAttribObj( "WamUserName" )
If ( objNodeAttribute.IsInherit = true ) Then
End If

I just needed a way to run GetPropertyAttribObj from C#, which was a much easier problem to solve. There were only a couple of relevant hits, the best one being some abandoned Pastebin code in Google's cache.

We now have an answer:

/// <summary>
/// Establish if a metabase property is set explicitly on this node.
/// </summary>
/// <param name="iisNode">Metabase node</param>
/// <param name="propertyName">Property name</param>
/// <returns>True if the property is defined on this node explicitly; false if the property is not defined at all, or is inherited.</returns>
private static bool PropertyDefinedExplicitly(DirectoryEntry iisNode, String propertyName)
{

    // check 1. this node has a valid key-type, and 2. it contains the required property
    if (iisNode.Properties.Contains("KeyType") &&
    iisNode.Properties["KeyType"].Value.ToString().Length > 0 &&
    iisNode.Properties.Contains(propertyName))
    {

        // call ADSI GetPropertyAttribObject method directly
        Object propertyAttributes = iisNode.Invoke("GetPropertyAttribObj",
        new String[] { propertyName });
        IISOle.IISPropertyAttribute attributes =
        (IISOle.IISPropertyAttribute)propertyAttributes;

        // Isinherit will be set if the property is inherited
        if (attributes.Isinherit)
        {
            return false; // property was inherited
        }
        else
        {
            return true; // property explicitly defined on this node
        }

    }
    else
    {
        return false; // not a valid IIsObject node, or property not found
    }
}

We then rewrite the original function like so:

if (PropertyDefinedExplicitly(dirEntry, "AnonymousUserName")) {
    dirEntry.Properties["AnonymousUserName"].Clear();
    Console.WriteLine("Log: Local value found for AnonymousUserName, removing");
}

There is a lesson to learn here for Windows programmers: Microsoft support is in the newsgroups. Don't fear Usenet because it's older than you are!

With useful tips like this, I'll win the NZ .NET blog of the year for sure!

Debugging .NET - "Common Language Runtime Debugging Services Application has generated an exception that could not be handled"

Friday, May 11th, 2007

ProfileTool doesn't work on half the machines I try it on. I get an error like:

Profiletool.exe Common Language Runtime Debugging Services
Application has generated an exception that could not be handled.
Process ID=0x14 (1300), Thread Id=0xe8(232)

Click OK to terminate the application.
Click CANCEL to debug the application.

There's no point searching Google for the process ID and thread number; they're unique to your instance.

What can you do here? You get an error suggesting that 'debugging failed' if you hit Cancel - this is correct, because you don't have a debugger installed!

First step, if you're running your .NET application off the network, stop, move it to your local disk, and try again. The default security policy doesn't let you do this - you can edit it in the Administrative Tools if you need to.

Now, if you still get the error, drop to a command line, and run the following:

ProfileTool 2>error.log

This redirects the standard error output to error.log, which you can then read. Linux/UNIX outputs this to the terminal by default; Windows hides it.

Thanks to Frank Racis, by way of Mohamed Yehia.

The actual error, in this case, was "No mapping between user accounts and security IDs was done", which really means "don't catch System.SystemException when you are throwing System.Exception". And when you're testing new builds, don't sit behind an ISA server which aggressively caches the old build.

ProfileTool beta released

Wednesday, April 25th, 2007

ProfileTool screenshotI don't often write code, but I do find it's a good complement to cricket watching. So, in honour of New Zealand playing Sri Lanka in the World Cup semi-final, I'm proud to announce the release of the first public beta of the IT Partners ProfileTool.

ProfileTool lets you take a Windows 2000 or XP user's profile and assign it to a different user, without having to copy any files or perform any manual procedures.

This is a common task for people who are taking a network of PCs that have not been on a domain, and joining them to one; or if you replace a domain controller without keeping security information, which is sometimes done on sites upgrading from a badly installed SBS 2000 installation to a nicely installed SBS 2003 installation.

It also has useful features for deleting a profile off the disk/from the registry, and changing profiles from roaming to local, replicating the features of the User Profiles capplet.

The source is available under the Mozilla Public License, with the implication of "Do what you like with this code, but if you improve it, I'd like your source to come back to me please".

Targeting .NET 1.1 harder than it seems

Wednesday, March 21st, 2007

I'd like my application to be able to run on .NET 1.1, so I've avoided using all the new niceties of .NET 2.0.

Or so I thought.

First, I experimented with MSBee, which lets you build for the .NET 1.1 environment using the VS 2005 tools. After seeing how many errors my code generated in 1.1, while it compiled cleanly in 2.0, I thought "maybe reimplementing this in VS 2003 might be easier".
And while I've specifically avoided some of the .NET 2.0 things that would have been really handy, like the SecurityIdentifier class, which makes handling SID-account mapping easy, there's things I didn't know would be a problem. You can't set a registry key to be a certain type in 1.1. And you can't use half the features that the designer implements for you automatically. And even if you could, you'd have to hack at it, because they separate code and design with partial classes in 2.0.

So, to summarize, looks like .NET was pretty crap prior to 2005. And that means you need to have XP SP2, because it's not supported on SP1.

(/rant)

Searching and replacing attached templates in Word documents

Monday, March 19th, 2007

If you have a document take 5-15 seconds at "Requesting virus scan..." in Microsoft Word, you will think "Pesky virus scanner!". But while this fault is most commonly Norton AntiVirus, it can also manifest in another circumstance: where you have a document with an attached template on a share you can't access any more.

If you are a Windows tech, you have no excuse for not knowing about Sysinternal's Filemon - when you know exactly what an application is accessing, it lets you solve a multitude of problems, including this one. We have a customer with an application that generates Word files based on a template, in the same location as the program is being run from. Long before our involvement they appear to have had a server named "Server" (imaginitive naming) and ran the program from \\Server\Share. There is no Server any more, so the documents from that period all cause a timeout trying to find a share that doesn't exist.
Thankfully, when their current server was installed, a drive mapping was created, so all the documents from that point on are attached to a template on G:\. Therefore the task becomes to search-and-replace the template on all the old Word documents.

There is no easier way to do this than using Word's VBA, which is unfortunate, as you have to script a load of each document, which means each one takes 5-15 seconds to process. There are third party utilities I've seen that handle Word automation, but you may as well just run a macro, which appears in its full splendour below the fold.

I'd be interested to know if there was a library that let you edit Word documents, so we could do this by another method.

Net result however, documents now open speedy, and customer is happy.

(more…)

RegUnloadKey failing with error 5 (Access denied)

Sunday, March 18th, 2007

Trying to unload a registry hive and finding access is denied?

I was loading in a user hive (NTUSER.DAT in the profile), and then opening the key using RegOpenKey (exposed as Registry.Users.OpenSubKey in .NET) to prove it was successful. However, I'd forgot to close this key, so the handle was staying open and thus you couldn't unload the hive.

Oddly, after some time, the handle would clean itself up, so it seemed more like a time or permissions based issue. Simple fix however.

A working release of ProfileTool isn't far away..