Damian Mehers' Blog Evernote and Wearable devices. All opinions my own.

14Feb/091

Microsoft fixed my bug

Like a lot of people, when I first began developing on a "proper" Operating System (as opposed to the 8 bit computers I started with), sooner or later I convinced myself I'd found a bug in the operating system.

Of course I'd be wrong, and after suffering the embarrassment of discovering that it was my own stupidity, rather than an OS bug, that was causing the errors, I eventually acquired an automatic reflex to never ever believe that there was a bug in the OS.  It had to be a bug in my code.  This was a good thing, since I was always right.

Despite this, it seems that I never fully learned this lesson.  And so it was that last May I opened a bug report on Microsoft Connect, fully expecting to be gently, but firmly rebuffed, with an explanation of where exactly I'd gone wrong.

The issue that I was facing was in detecting whether a connection to my web application was coming from the local machine (running the web application), or from a remote machine.  I was calling the HttpListenerContext.Request.IsLocal method, and it was always returning false, even though the client was definitely local.

A little digging with Reflector showed that the method was implemented using "==" to compare the local address with the remote address, instead of the Equals method.  If the "==" operator had been overloaded then it would have worked.

A couple of days ago, I got a message saying that this was indeed a bug, and that it was fixed in version 4.0 of the .NET framework.  So it looks like I have finally actually discovered an honest-to-goodness bug.

Or perhaps they are just being kind to me.

Filed under: Debugging 1 Comment
9Sep/083

TraceJS V2 R.I.P.

A couple of years ago I posted a small blog entry on a throwaway tool that I had created that uses the Debug API to log all JavaScript that Internet Explorer executes.

A year ago I decided to take a year off from consulting, mainly to spend a lot more time with our kids while they are still young, but also to have some time to explore some product ideas.

The first idea I wanted to explore was whether or not it might be possible to extend the idea I'd had in that older blog post to actually make a commercial product that would not only trace all JavaScript, but would also show call stacks, and show how often functions were executed.

Technically it was tricky because I wanted my custom JavaScript debugger to be loaded in-process into Internet Explorer, since I thought that this might speed it up considerably. The secret turned out to be that I needed to pass a special flag to IDebugProgramProvider2->GetProviderProcessData (the PFLAG_DEBUGGEE flag).

After a couple of months of development I finally got it all working.

You started it from the IE Tools menu:
image

This would create a separate TraceJS dialog: image

Then as you access a page that contains JavaScript the TraceJS Window shows the statements that are being executed:image image

Once you'd captured the JavaScript execution, you could replay the executed JavaScript, either step-by-step, or continually at a pace you define:image

Finally you could  drill-down to see which functions are called the most, and who calls them:image

I had it all working in my initial build environment, which was XP based.  Then I tested in Vista, and I couldn't get it to work  (the GetProviderProcessData failed).  Ouch!

I took a step back, gave the product a long hard look, and decided to stop development.  It was too slow (despite being entirely in-process), I couldn't implement features I needed such as timing (the debugging/tracing added too much overhead), and I couldn't get it working on Vista.

Now that IE8 has similar profiling features I'm happy I stopped when I did.

Of course the biggest mistake I made was to register the domain names for the product before it was completed (tracejs.com, jsprof.com, jsprofiler.com and javascriptprofiler.com are all mine).

If there is one sure way to kill a product idea it is to register domain names before you've executed on the product idea itself :-).

Filed under: Debugging 3 Comments
2Jul/084

.NET Code Injection

In this post I'll show how you can inject your own .NET code into a process that is already running the .NET framework.

I needed to do this when developing for Windows Media Center because there is no official documented way to determine what page is currently being displayed in Media Center.

I wanted to know when the user is showing the Media Center "now playing" page.

Media Center addins run in a completely separate process to the main Media Center process, which is ehshell.exe.  Calls you make in your addin are remoted across to the Media Center ehshell.exe process.

The solution works using:

  • an "Injecter" C# class  which uses CreateRemoteThread to load a Bootstrap DLL into the ehshell.exe process
  • The "Bootstrap" C++ DLL which is injected into the ehshell.exe process uses the CLR Hosting API to attach to the default .NET domain, and then load an instance of the "Injectee" C# class
  • The "Injectee" C# class is loaded by the "Bootstrap" DLL and runs within the ehshell.exe process.  It uses reflection to subscribe to an event within the ehshell.exe process and then sends an interprocess message when the event fires

The "Injecter"

Using Reflector, I looked around at the ehshell.exe assembly, and related assemblies, and eventually discovered that there is a PageChanged event fired by the ServiceBus.UIFramework.PageBasedUCPService class:

image 

What is more, there is a static PageBasedUCPService.DefaultUCP method to return a PageBasedUCPService instance.

There are of course a couple of problems.  The first problem being that these classes are internal to the ehshell assembly,  the second issue being that these are all running in a totally separate process to my addin.

I searched a little, and came across this example showing how you could use the CreateRemoteThread from .NET to inject your own native code into a remote process.

Essentially it opens a handle to the remote process, finds the LoadLibrary system call's address in the remote process, allocates a chunk of memory in the process, and fills the memory with a LoadLibrary call to load a DLL that you specify, and then creates the thread in the remote process, passing the address of the chunk of memory as the starting point for the thread.

I recommend taking a look at the CRT call in the example.  I'll not be duplicating the code here.

To use the CRT call, I first find Media Center:

public static Process FindMediaCenterProcess() {
  // Could be more than one if an extender is running too
  foreach (Process process in Process.GetProcessesByName("ehshell")) {
    if (process.SessionId == Process.GetCurrentProcess().SessionId) {
      return process;
    }
  }
  return null;
}

I can then call the CRT method:

Process mediaCenterProcess = FindMediaCenterProcess();
IntPtr hwnd;
string error;
CRT(mediaCenterProcess, dllPath, out error, out hwnd);

The dllPath is the full path to my C++ DLL, in my case this was

Environment.GetFolderPath(Environment.SpecialFolder.ProgramFiles) +
                                  @"\Atadore\MceFM\InjectBootstrap.dll"
 

The "Boostrap DLL"

The InjectBootstrap.dll is a pretty simple C++ library project.  The DllMain calls a bootstrap function when the process is loaded:

BOOL APIENTRY DllMain( HMODULE hModule,
                       DWORD  ul_reason_for_call,
                       LPVOID lpReserved
                     )
{
    switch (ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
        Bootstrap();
        break;
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        break;
    }
    return TRUE;
}
 
My bootstrap function attaches to the default .NET domain and loads an instance of one of my own classes:
 
#include "stdafx.h"
#include <stdio.h>
#include "objbase.h"
#include "MSCorEE.h"
#import "C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorlib.tlb" raw_interfaces_only
using namespace mscorlib;

void Bootstrap() {
    OutputDebugString(L"MceFM Bootstrap Started");

    CoInitializeEx(0, COINIT_MULTITHREADED );
    ICorRuntimeHost* pICorRuntimeHost = 0;
    HRESULT st = CoCreateInstance(CLSID_CorRuntimeHost, 0, CLSCTX_ALL,
        IID_ICorRuntimeHost, (void**)&pICorRuntimeHost);
    if(!pICorRuntimeHost) return; // Clean up and log errror ...

    HDOMAINENUM hEnum = NULL;
    pICorRuntimeHost->EnumDomains(&hEnum);

    if(!hEnum) return; // Clean up and log errror ...

    IUnknown* pUunk = 0;
    st = pICorRuntimeHost->NextDomain(hEnum, &pUunk);
    if(!pUunk) return; // Clean up and log errror ...

    _AppDomain * pCurDomain = NULL;
    st = pUunk->QueryInterface(__uuidof(_AppDomain), (VOID**)&pCurDomain);
    if(!pCurDomain) return; // Clean up and log errror ...

    _bstr_t assemblyName =
        "Last, Version=1.1.0.0, Culture=neutral, PublicKeyToken=792d614cdf38e9ce";
    _bstr_t typeName = "MceFM.Last.Inject.Injectee";

    _ObjectHandle* pObjectHandle = 0;

    pCurDomain->CreateInstance(assemblyName, typeName, &pObjectHandle);

}
 

The "Injectee"

Finally we come to the Injectee class, which is loaded by the DLL, and runs inside the remote (ehshell.exe) process.  It is in a strongly named assembly which is stored in the GAC, so that the bootstrap call above can find it (note the strong name in the assembly name bstr).

I use a static constructor (which runs as soon as the class is loaded).  In this static constructor I subscribe to the event:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Net;
using System.Reflection;
using System.Reflection.Emit;
using System.Threading;

namespace MceFM.Last.Inject {
  public class Injectee {

    static Injectee() {
      try {
        Assembly ehshell = Assembly.Load("ehshell");

        // Get the PageBasedUCPService type
        Type pageBasedUCPServiceType =
          ehshell.GetType("ServiceBus.UIFramework.PageBasedUCPService");

        // Call static DefaultUCP method to get a PageBasedUCPService instance
        const BindingFlags bindingFlags = BindingFlags.DeclaredOnly |
          BindingFlags.Static | BindingFlags.Public | BindingFlags.GetProperty;
        object[] args = new object[0];
        object defaultUCP = pageBasedUCPServiceType.InvokeMember("DefaultUCP",
          bindingFlags, null, null, args);

        // Get a reference to the PageChanged event
        EventInfo pageChangedEventInfo =
          pageBasedUCPServiceType.GetEvent("PageChanged");
        Type pageChangedDelegate = pageChangedEventInfo.EventHandlerType;

        // Get the MethodInfo for the method to be called when the event fires
        MethodInfo newPageHandlerMethodInfo =
          typeof(Injectee).GetMethod("NewPageHandler");

        // Build an array of types used for new method we shall dynamically emit
        Type pageType = ehshell.GetType("ServiceBus.UIFramework.Page");
        Type[] pageChangedDelegateParameters = new[] { pageType };

        // Need a dynamic method because we can't create a method that has
        // the Page type as a parameter
        DynamicMethod dynamicHandler = new DynamicMethod("", null,
          pageChangedDelegateParameters, typeof(Injectee));
        ILGenerator ilgen = dynamicHandler.GetILGenerator();
        ilgen.Emit(OpCodes.Nop);
        ilgen.Emit(OpCodes.Ldarg_0);
        ilgen.Emit(OpCodes.Call, newPageHandlerMethodInfo);
        ilgen.Emit(OpCodes.Nop);
        ilgen.Emit(OpCodes.Ret);

        if(pageChangedDelegate != null) {
          // Subscribe to the event
          Delegate dEmitted = dynamicHandler.CreateDelegate(pageChangedDelegate);
          pageChangedEventInfo.GetAddMethod().Invoke(defaultUCP,
                                                     new object[] {dEmitted});
        }
      } catch(Exception ex) {
        Trace.TraceError("Unexpected error in Injectee initializer: {0}", ex);
        Trace.TraceError(ex.StackTrace);
      }
    }

    // Queue of new pages that have been navigated to.  Decouples event handler
    // from re-despatching of the event to the addin process that is told
    // of the new event.
    private static readonly Queue<string> newPages = new Queue<string>();

    // Indicates whether background thread that despatches events is running
    private static bool notifierActive;

    // Called when Media Center navigates to a new page
    public static void NewPageHandler(object page) {
      Trace.TraceInformation("New Page: " + page);
      lock(newPages) {
        if(!notifierActive) {
          Thread thread = new Thread(NewPageNotifier) {IsBackground = true};
          thread.Start();
          notifierActive = true;
        }
        newPages.Enqueue(page.GetType().FullName);
        Monitor.Pulse(newPages);
      }
    }

    // Thread that despatches events to addin process telling it that
    // a new page has been navigated to
    private static void NewPageNotifier() {
      while(true) {
        string page;

        lock(newPages) {
          while(newPages.Count == 0) {
            Monitor.Wait(newPages);
          }
          page = newPages.Dequeue();
        }

        try {
          // Use whatever interprocess notification mechanism you wish.
          // WCF could be good.  Here I use a simple web call.  My
          // addin has a web server in it to receive these calls.
          WebClient webClient = new WebClient();
          webClient.QueryString[Server.PAGE_QUERY_STRING] = page;
          webClient.DownloadString(Util.LocalBaseUrl
                                   + Server.MCE_PAGE_CHANGED_ACTION);
        } catch (Exception ex) {
          Trace.TraceError("Error while sending new page notification: {0}", ex);
        }
      }
#pragma warning disable FunctionNeverReturns
    }
#pragma warning restore FunctionNeverReturns
  }
}
 

The Result

In my web server running within my addin (which is notified by the "NewPageNotifier" method above), I keep track of which is the current page, and log information when the page changes:

 
Server notified of page change MediaCenter.Audio.AudioNowPlayingPage
Server notified of page change MediaCenter.Audio.AudioNowPlayingTracklistPage

All this means that if you are a Last.fm user and want to tell Last.fm that you love a track that is currently playing then you can press the right-button on your remote control on the "now playing" page using my MceFM addin.

I use low level system hooks to know when you press the right button, and the above technique to know that you are on the "now playing" page:

image

Word if warning.  If/When Media Center changes its internal structure this technique will completely fall over.  Use sparingly, entirely at your own risk,  to do the seemingly impossible.

12Jun/080

Using mdbg to debug at the MSIL level instead of assembly code.

Windows Media Center (MCE) lets you create addins to provide new functionality.  There are two kinds of addins -- normal addins, and background addins.  Background addins run continuously as soon as MCE starts up, and are limited in the user-interaction they provide.

One thing they are allowed to do is to pop up a dialog message to the user using the AddInHost.Current.MediaCenterEnvironment.Dialog(...) method call.

I was doing this in my background addin, but I was getting an error from MCE saying that I was not allowed to make that call.

Addins run in separate processes to the main MCE process, which is ehshell.exe, and the Dialog call is remoted into the ehshell.exe process.

Using Reflector I could see that the call was eventually failing in MediaCenter.Extensibility.ExtensibilityAutomation.EnforceApplicationPermission(...) which was calling through to Microsoft.MediaCenter.Hosting.Infrastructure.RegisteredApps.IsBackgroundEntryPoint which looks like this:

public static bool IsBackgroundEntryPoint(ExtensibilityEntryPointInfo epi)
{
    return (0 == string.Compare(epi.Category, "Background", true,
CultureInfo.InvariantCulture));
}

Somehow I needed a way to use the debugger to see what the epi.Category value was when the call was failing.

The standard Visual Studio debugger just drops you through to x86 assembly code if it has no debugging symbols.  I needed a debugger which shows .NET IL instead.

This is where mdbg comes in. Mdbg is a command line debugger created by Mike Stall.

Once you have downloaded it and started running it, then you can use the 'a' command to list available processes, and then 'a <pid>' to attach to a specific process.

Once attached, you can set a breakpoint using 'br <module name>!<type>.<method name>", and then 'go' to resume:

image

Once the breakpoint is hit, you can use the 'print' command to display a parameter value:

image

There is also a GUI which can be invoked using 'load gui':

 image