error——Fusion log——Debugging Assembly Loading Failures

原文

---

So...you're seeing a FileNotFoundException, FileLoadException, BadImageFormatException or you suspect an assembly loading failure? Try the steps below to start your debugging process.

First, get the Message property from the exception.

If the exception's InnerException property is set, get the Message property from that.

If the message is generic, also get the hresult by calling System.Runtime.InteropServices.Marshal.GetHRForException(), passing in the Exception object.

If that info is not readily available, you may need to attach a debugger and set it to catch managed exceptions.

Then, get the hresult by retrieving the private _HResult field of the managed Exception object.

Retrieving the Fusion log

If the exception message isn t enough for you to determine what the problem is, try getting the Fusion log. It will describe the binding failure (if this is due to an assembly binding failure, instead of a loading failure after the file is found).

The exception may already include the log. If not, to get it, run fuslogvw.exe. If you don't have fuslogvw.exe already, install the Framework SDK.

• For pre-v2.0: click on "Log Failures." If this is an ASP.NET or .NET Windows service app, select the Custom option and using regedit, set [HKLM\Software\Microsoft\Fusion\LogPath] to point to an existing directory (like c:\mylogs, not c:\). If you need to log all binds, not just failing ones, set [HKLM\Software\Microsoft\Fusion\ForceLog] as a DWORD value to 1.
• For v2: click on "Settings," then choose "Log bind failures to disk" if you only care about the failures or "Log all binds to disk" if you want to see all binding requests.
• To turn on failure logging during a test run instead of by hand, have your script set [HKLM\Software\Microsoft\Fusion\LogFailures] as a DWORD value to 1 using regedit.

Then, click "OK." Now, re-run the process. Next, click "Refresh" in fuslogvw, and a line should appear for each binding failure (or for each bind, if ForceLog is set). Now, double-click on the line in the "Application" column for the interesting bind, and a web page should come up with the Fusion log.

If the file was found, it was loaded from the last path probed in the log (or from the GAC, if there is no probed path shown).

Note: Unless you are explicitly debugging the failure of a resource to load, you will likely want to ignore failures to find assemblies with the ".resources" extension with the culture set to something other than "neutral". Those are expected failures when the ResourceManager is probing for satellite assemblies.

Troubleshooting Fusion logging

Keep in mind that there may not be any entries if there were no binds or if there were no binding failures (when logging failures only). (And don't forget to restart your application's process after changing logging settings.) If there are not as many entries as you expect, you may need to clear your download cache to make room for them. To do that, in Internet Explorer, choose “Tools“, “Internet Options“, “Delete Files”, “OK“, then re-run your app and finally click “Refresh“ again in fuslogvw.

There is only one Fusion log saved per display name/codebase. So, if the same exact reference is requested twice in the same process, it will only show up once in fuslogvw. To see the duplicate logs, you'll need to stop execution (for example, with breakpoints in your debugger) and then get the log at that time.

For FileNotFoundException:

At the bottom of the log will be the paths that Fusion tried probing for this assembly. If this was a load by path (as in Assembly.LoadFrom()), there will be just one path, and your assembly will need to be there to be found. Otherwise, your assembly will need to be on one of the probing paths listed or in the GAC if it's to be found.

You may also get this exception if an unmanaged dependency or internal module of the assembly failed to load. Try running depends.exe on the file to verify that unmanaged dependencies can be loaded. Note that if you re using ASP.NET, the PATH environment variable it's using may differ from the one the command line uses. If all of them could be loaded, try ildasm.exe on the file, double-click on "MANIFEST" and look for ".file" entries. Each of those files will need to be in the same directory as the manifest-containing file.

For BadImageFormatException:

Try running peverify.exe on the file. That will give a more specific description about why it s considered a bad image. Keep in mind that modules built against v2 can not be loaded by a pre-v2 CLR.

For SecurityException:

You need Execute permission for loading any assembly. Also, if a codebase was used to load this file, you would need both FileIOPermission.Read and FileIOPermission.PathDiscovery or else WebPermission to the location (depending on whether this is a local file or a URL). Try caspol.exe to check your managed security settings.

For FileLoadException:

For an "Access is denied" message (for hresult E_ACCESSDENIED, 0x80070005): 
Run tlist -m on the file to see if another process has the file locked and without share-read access. If not, check the ACLs for the file and its dependencies (especially if you're using impersonation).

For a "The located assembly's manifest definition with name [yourAssembly] does not match the assembly reference" message (for hresult FUSION_E_REF_DEF_MISMATCH, 0x80131040): 
The Fusion log will say which part of the assembly reference failed to match what was found. It will be the assembly name, culture, public key (or token) or version (if the found assembly was strongly-named).

For "Unverifiable image [yourAssembly] can not be run" or "Can not run executable [yourAssembly] because it contains relocations" messages (for hresult COR_E_FIXUPSINEXE, 0x80131019): 
That image must be run as the process exe or else be compiled as a dll. This is because MC++ has made optimizations for you in your image, based on the assumption that it will be the process exe. If it's not the process exe, it won t be loaded at the expected location, so the assumed offsets will be incorrect. When the CLR sees such a file loaded as a non-process exe, it will reject the load.

When to Change File/Assembly Versions

First of all, file versions and assembly versions need not coincide with each other.

I recommend that file versions change with each build. But, don’t change assembly versions with each build just so that you can tell the difference between two versions of the same file; use the file version for that.

Deciding when to change assembly versions takes some discussion of the types of builds to consider: shipping and non-shipping.

Non-Shipping Builds

In general, I recommend keeping non-shipping assembly versions the same between shipping builds.

This avoids strongly-named assembly loading problems due to version mismatches.

Some people prefer using publisher policy to redirect new assembly versions for each build. I recommend against that for non-shipping builds, however: it doesn’t avoid all of the loading problems. For example, if a partner x-copies your app, they may not know to install publisher policy. Then, your app will be broken for them, even though it works just fine on your machine.

But, if there are cases where different applications on the same machine need to bind to different versions of your assembly, I recommend giving those builds different assembly versions so that the correct one for each app can be used without having to use LoadFrom/etc.

Shipping Builds

As for whether it’s a good idea to change that version for shipping builds, it depends on how you want the binding to work for end-users. Do you want these builds to be side-by-side or in-place? Are there many changes between the two builds? Are they going to break some customers? Do you care that it breaks them (or do you want to force users to use your important updates)? If yes, you should consider incrementing the assembly version. But, then again, consider that doing that too many times can litter the user’s disk with outdated assemblies.

When You Change Your Assembly Versions 
To change hardcoded versions to the new one, I recommend setting a variable to the version in a header file and replacing the hardcoding in sources with the variable. Then, run a pre-processor during the build to put in the correct version. I recommend changing versions right after shipping, not right before, so that there's more time to catch bugs due to the change.

Assembly Binding Log Viewer (Fuslogvw.exe)

Fusion binding log and fuslogvw.exe

Suzanne blogs again! This time, she is helping you on diagnose MissingMethodException.

Of course, whatever she tries to help you, she can't live without fusion log.

So what exactly is fusion log?

Remember what fusion's role is in assembly binding? Fusion is responsible for finding the actual bits, and handing it over to loader. Because the fusion probing rule is vastly different from normal LoadLibrary rule, fusion includes some logging information to help diagnostic assembly binding failures. For every major event, fusion will log a message. Those events include app.config lookup, policy resolution, GAC lookup, every location fusion probes, and more.

By default, the log is kept in memory, and is included in FileNotFoundException (If you read FileNotFoundException's document, you will find it has a member called “FusionLog“, surprise!). If you ask, fusion will dump the log into an HTML file, and later can be viewed using fuslogvw.exe, assembly binding log viewer shipped as an SDK tool.

As we shipped in 1.0 and 1.1, there are three things affecting how fusion dumps the log. They are all reg keys under HKLM\Software\Microsoft\Fusion.

1. REG_DWORD  LogFailures:
 If LogFailures is set to 1, fusion will dump the log to disk if the assembly binding did not succeed.

2. REG_DWORD ForceLog: 
If ForceLog is set to 1, fusion will dump all logs to disk, regardless the assembly binding succeeded or not.

3. REG_SZ LogPath:
By default fusion dumps the logs to your IE cache. But if LogPath is set, fusion will dump the logs to the path specified. The LogPath has to be an existing folder to make it effective.

Fuslogvw.exe is covered by MSDN. I don't want to copy the document here. But I'll point out some common pitfall when people use this tool. 
1. Fuslogvw reads the log settings once, and never reads the settings again. So if you changed the settings (especially LogPath), you have to close it, and re-run it to pick up the change.
2. There are three radio buttons in fuslogvw.exe. Default coresponds to IE cache. ASP.NET is useless. Custom means the LogPath. If you set LogPath, you have to click the custom button to see the actual logs. If you did not set LogPath, you have to click the default button to see the logs. 
3. If you want to see logs for services running under a different credential than yours, you have to set LogPath, run the services, then run fuslogvw.exe, and select the custom button to see their logs. The reason is by default the logs are stored in their IE cache which you can't see. You have to set the LogPath so everyone will log to the same place.

Fuslogvw.exe gets a little bit improvement in the next version of .Net framework. But then I am not a UI guy. I only spent half a day reading Charles Petzold's “Programming Windows” before I set improve fuslogvw.exe. When you get change, play around with the new fuslogvw.exe, and send me feedback if possible.

The located assembly’s manifest definition with name xxx.dll does not match the assembly reference

This is a very common Assembly.Load failure exception text. In fusion terminology, this is called Ref-Def-Mismatch. This exception is thrown when fusion finds an assembly that does not match what you are looking for.

A very common mismatch is version number. An example is, you asks for version 1.0.0.0, but version 2.0.0.0 is found. (keep in mind that for simply named assemblies, version number is ignored. So we are really talking about strongly named assemblies here.)

The only time this exception will be thrown is when the mismatched assembly is in your appbase. Fusion will probe GAC first. If the assembly is not found in GAC, fusion will probe your appbase. If fusion finds an assembly that matches the simple name, but has mismatch on other part of the assembly name, this exception will be thrown. Say you call Assembly.Load(”MyAsm,Version=1.0.0.0,Culture=Neutral,PublicKeyToken=0123456789abcdef”, and fusion finds MyAsm.dll in your appbase, but its version is 2.0.0.0, fusion will thrown this exception.

An interesting question is why can't fusion just keep probing. Remember fusion will probe a couple of places before it stops. If the first one is mis-matched, maybe later fusion can find another match?

There are several reasons.

1. It is very rare that people will have several assemblies with the same file name in the probing path. So keep probing usually won't help anyone. 
2. Because of 1, you get unnecessary penalty if fusion keeps probing. This is especially bad for http bind.
3. Most of the time this exception is due to deployment error. So throwing early will help catch deployment error.

There are more reasons. But I'll show them later when I get time to show some binding design decision.

Another interesting observation on this is for dependencies of LoadFrom assemblies. For dependencies of LoadFrom assemblies, we will probe the appbase, then the parent assembly's directory.

If a mismatch assembly is found in appbase, it does not mean a matching assembly won't be found in parent assembly's directory.

In v1/v1.1, we treat this case the same as normal probing, and throw on the first mismatch. In Whidbey, we take a DCR to keep probing the parent assembly's directory. (DCR stands for Design Change Request. It usually means change of behavior)

Back to Basics: Using Fusion Log Viewer to Debug Obscure Loader Errors

It's time to remind people about Fusion. Mostly because I don't see as many people using it as should. I mentioned it as long as six (!) years ago and it's still useful. I used it just this week with .NET 4.

Sometimes when an assembly doesn't load, there's still a sense/feeling that "something in the black box has broken." For some folks, the black box is larger and obscures more, than for others. My point is, if you know where to look, there is no box at all.

When it comes to assemblies there's three "times" to know about:

• Binding before it happens - What do you want?
  • ILDASM or Reflector will tell you what your assembly wants (what it was compiled against)
• Binding as it happens - Where does it look?
  • Fusion (the Assembly Binding Log Viewer) will show you all assembly binds if you set the HKLM\Software\Microsoft\Fusion\ForceLog registry value to 1
• Binding after it happens - What did you get?
  • Process Explorer will tell you what DLL (assembly) is loaded in memory and from where it came.

Here's an example of how a tiny bit of digging saved me hours of confusion recently when I hit an unusual edge case. I was doing a build of sample that was showing C# interop-ing with IronPython, but I was using a daily build of .NET 4 and a random build of IronPython.

I made a mistake and had some really old DLLs floating around that I shouldn't have had. My symptom was a FileNotFoundException for the file "System.Dynamic.dll." I KNEW it was in the GAC (Global Assembly Cache) and I could SEE it as a reference DLL in my directory. Can't find the file? Dude, it's right there!

TURNING ON FUSION LOGGING

You probably have a tool to help on your development system already. Type "Fusion" in the Start Menu. The Assembly Binding Log Viewer, or "Fusion Log Viewer" will tell the CLR to load assembling binding/loading activities to a folder, then let you see them.

Be sure to run it as Administrator if you want to change the Settings, otherwise they'll be grayed out.

Alternatively, just set the Registry keys your self. (I just memorized them, as I set them all the time.) Set HKLM\Software\Microsoft\Fusion\ForceLog registry value to 1 and HKLM\Software\Microsoft\Fusion\LogPath registry value to C:\FusionLogs or some path that exists.

Personally, I leave this on all the time on my dev machines (there's a small speed hit) and just clean the folder out every once in a while.

SOLVING MY BINDING PROBLEM

Once I turned on Fusion Logging I could immediately see a failure in my folder:

The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file specified. 

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework\v4.0.xxxxx\clr.dll
Running under executable  C:\Users\Scott\Desktop\TechEd09\FX4\Demo 5 - DLR\Two.IronPythonInterop\bin\Debug\Two.IronPythonInterop.exe
--- A detailed error log follows. 

=== Pre-bind state information ===
LOG: User = HANSELMAN-DEV10\Scott
LOG: DisplayName = System.Dynamic, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
LOG: Appbase = file:///C:/Users/Scott/Desktop/TechEd09/FX4/Demo 5 - DLR/Two.IronPythonInterop/bin/Debug/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = Two.IronPythonInterop.exe
Calling assembly : IronPython, Version=2.6.0.1, Culture=neutral, PublicKeyToken=31bf3856ad364e35.
===
LOG: This bind starts in default load context.
LOG: No application configuration file found.
LOG: Using host configuration file:
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.xxxxx\config\machine.config.
LOG: Post-policy reference: System.Dynamic, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
LOG: GAC Lookup was unsuccessful.
LOG: Attempting download of new URL file:///C:/Users/Scott/Desktop/TechEd09/FX4/Demo 5 - DLR/Two.IronPythonInterop/bin/Debug/System.Dynamic.DLL.
LOG: Attempting download of new URL file:///C:/Users/Scott/Desktop/TechEd09/FX4/Demo 5 - DLR/Two.IronPythonInterop/bin/Debug/System.Dynamic/System.Dynamic.DLL.
LOG: Attempting download of new URL file:///C:/Users/Scott/Desktop/TechEd09/FX4/Demo 5 - DLR/Two.IronPythonInterop/bin/Debug/System.Dynamic.EXE.
LOG: Attempting download of new URL file:///C:/Users/Scott/Desktop/TechEd09/FX4/Demo 5 - DLR/Two.IronPythonInterop/bin/Debug/System.Dynamic/System.Dynamic.EXE.
LOG: All probing URLs attempted and failed.

　　

You can see that it's looking all over for the file, first in the GAC< then all over the local folders before it gives up. Hm. Why isn't this working? I can see the file sitting right there.

Well, what's the public key token for this signed assembly? I can run "sn -T" on the file:

C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0>
sn -T System.Dynamic.dll 

Microsoft (R) .NET Framework Strong Name Utility  Version 4.0.xxxxx.1
Copyright (c) Microsoft Corporation.  All rights reserved. 

Public key token is b03f5f7f11d50a3a

　　

Wha? What's b03whatever? That's not b77whatever like the one my app is looking for! Looks like my sample app had reference not only an old version of System.Dynamic, but one with a completely different public key. That's what I get for not cleaning out my obj directories between daily builds.

This could have been just by setting the registry keys and watching the c:\fusionlogs folder, but the Fusion Log Viewer makes the process more user-friendly.

If you're debugging version number mismatches or strong-name mismatches, Fusion will ALWAYS tell you what's really going on. There is no Black Box.

Related Links

• Detailing Failed Assembly Loads with the Assembly Binding Log Viewer ("The Fusion Logger")
• More on Assembly Binding, Strong Naming, the GAC, Publisher Policy and Dynamically Loaded Assemblies
• Gotcha in .NET Framework handling of Versions and Publisher Policies
• Be aware of your DLLs and who's loading them - know about assembly binding redirects
• Richard Grimes' most excellent .NET Fusion Workshop