2005 2006 2007 2008 2009 2010 2011 2015 2016 2017 aspnet azure csharp debugging elasticsearch exceptions firefox javascriptajax linux llblgen mongodb powershell projects python security services silverlight training videos wcf wpf xag xhtmlcss

Silverlight Feature Requests

Silverlight is probably the most important technology to be released since Firefox 1.0 and that since .NET 1.0.  However, as with most early-edition pieces of software, there are some major things missing.  Below is a list of things that I request that Microsoft add with a quick note on each.  It ranges from a few UI things to the more hardcore mechanical things (...things that many developers will say are meaningless until they learn how to spell C-L-R).

  • GIF Support - Obvious.  It's very odd that some manager somewhere said "no one ever uses GIF anymore".
  • Style Inheritance - Another obvious one.  We who knew WPF long before Silverlight was around THRIVE on this.
  • DockPanel - COME ON! You MUST be joking! I shouldn't have to use a third party CodePlex control for this.
  • VisualBrush - The ability to paint controls on a background and in other places will go a long way in helping people creating more interactive help files and training software.
  • RelativeSource - The ability to bind one element's property to another is EXTREMELY important (i.e. my height == my parent's height).
  • MultiView - It takes almost no code to write this; just make it native so we don't have dozens of versions all over.
  • WebHttpBinding for JSON - POX is less important, but would be nice; JSON is the de facto web standard.
  • System.Security.Cryptography.RSA with providers - This would greatly increase the security of client-server communication.
  • Workflow Foundation - Even a light version with only the basic activities (for, if, while, etc...) would be nice.
  • SVG Viewer - This should shut up the anti-Microsoft cult.
  • Better font rendering - It's HORRIBLE in 2.0.
  • iPhone Support - I think this may take a miracle.
  • MultiTouch - Need this for iPhone support.
  • CSS3 DOM Selector (http://www.w3.org/TR/selectors-api/) - So we use it instead of jQuery's version for closer-to-native speed on EVERY browser without even needing JQuery.
  • A XMLHttpRequest object - This would allow us to use Silverlight's CrossDomainPolicy.xml stuff for cross domain stuff instead of playing games with browser detection nonsense and adding hacks for browsers which don't support it.
  • API for browser abstraction - Web developers HATE having to do lame detection to figure out if we are on IE so we can talk to it like a child.  Give us the ability to create managed extensions to Silverlight so we can add in our pain-in-the-butt browser differences in Silverlight instead of relying on painfully slow JScript rendering and browser detection.
  • Service for extensions - This goes with the last one.  We should have the ability to 1) extend the hardcore functionality of Silverlight and 2) have a centralize service that Silverlight calls to download our functionality when our application requests it.  This would be very just like what Firefox does when it needs a plugin.  It would also do auto-updating like with Firefox extensions.
  • JSON deserializer that actually works to deserialize data serialized by the JSON serializer - JSON is critically important to the web; it should be supported better.
  • The ability to get 500 HTTP status codes from Firefox, Chrome, and IE - This would allow us to use fault contracts in WCF.  This would have to be a pain to implement, though. So, I wouldn't expect it for a while.
  • The ability for NPCTRL to communicate CLR configuration information to the CoreCLR to tweak GC things that are normally in configuration - If the hosting NPCTRL control would just allow us to give it configuration parameters to send to the CoreCLR, that would RULE (like we can already do in .NET with configuration).
  • Hosting documentation that doesn't SUCK - Using Win32/ATL to host a Silverlight control is insane right now.  The documentation is very incomplete and there are NO samples (and the IDL is wrong!)
  • Background GC on Windows - I suspect that this would increase performance in many scenarios.
  • The missing GC methods - The System.GC class is missing a PUBLICLY ACCESSIBLE GetCollectionCount.  This will help us with profiling.  It's marked as internal now and if you try to access it via reflection or other fancy tricks you get security errors.
  • ICorProfilerCallback interface - This is my NUMBER ONE FEATURE REQUEST!  This is REQUIRED to create a Silverlight profiler.  Silverlight will NOT be a technology ready for enterprise usage without this.  PERIOD.
  • Documentation on Silverlight's ETW - it's there and it's working in SL2, but I see absolutely zero docs on how to access it.
  • The native AGCORE and NPCTRL source code - This will aide in performance and feature creation
  • COM interface for hosting the CoreCLR - not really a Silverlight feature, but this would be REALLY nice.

Project Sneak Peek

Today I thought I would give a teaser of my latest side project (I'm about a month into it).

Since I'm not a general "business application" developer, but am a specialist in various areas of technology, most of the projects I work on are awesome.  Two of my favorite projects are the Brainbench C# 2.0 exam and the Brainbench C# 2.0 Fundamentals exam, both as the author.  Immediately after finishing the later project, I ended my 2 year long on-and-off consultancy with Brainbench as author and technical editor.

Now, nearly a year later, I've gone on my own (more teasers in Q2 2009):

Which of the following statements regarding a .NET object's sync block index are true?
 
When an ephemeral segment GC occurs, the synch block index contains a pointer to the next object.
If a root is found by a GC, a bit will be enabled in the synch block index.
Constructed objects have the sync block index initialized to a negative value.
Value types have their synch block index set to the value of their location on the stack.
When a monitor lock is entered, the index of a free sync block is recorded in the synch block index.

Setting a Silverlight 2 Startup Breakpoint using WinDBG

As I've mentioned on a previous occasion, hardcore debugging is done with WinDBG and SOS.  Visual Studio's debugger just isn't enough for the super interesting scenarios.  I've also mentioned that I have no intention of giving a step-by-step introduction into how to use SOS.  However, given that debugging is directly related to internals and that most people with advanced Silverlight issues will search for "advanced Silverlight debugging" or the like, I'll provide a little information here and there to help people find what they are looking for quicker.

Here I would like to explain how you can set a break point very early in the Silverlight load process, before your application even loads.  This technique will give you a glimpse into the internals of Silverlight to see how things work.  It will also help you to step through your method calls from the very beginning in WinDBG.

First, however, understand that Silverlight has three major components that you need to be aware of:

  • NpCtrl.dll
  • AgCore.dll
  • CoreCLR.dll

Each of these components is critical to loading and running a Silverlight application.  NpCtrl.dll is a browser plugin which follows the Netscape Plugin API (NPAPI).  When your browser sees that you are trying to load Silverlight 2, it will call the NP_Initialize export (a public DLL function).  Later, this plug-in will load the AgCore.dll and create a instance of the COM class XcpControl2 to allow NpCtrl to interact with Silverlight (you too can use XcpControl in your COM development).  AgCore is the unmanaged part of Silverlight 2.  It handles everything from DOM interaction, to rendering, to media play.

At this point you will see the spinning XAP-loading progress bar.  When this is complete, NpCtrl will finally load CoreCLR.dll.  Upon load, the DllMain function for this DLL is called.  This function will start up the CoreCLR, the CLR that Silverlight 2 uses.  Upon completion, NpCtrl then calls the GetCLRRuntimeHost DLL export to get the loaded instance of the CLR.  The rest of your application is ready to run (as a side note, the order of a few of these things changes under certain circumstances, so you may actually see CoreCLR load AgCore.)

With this information you're ready to start a fresh instance of a web browser (it's crucial that Silverlight 2 isn't already loaded) and attach WinDBG to your browser's process.  Upon attach you are given control to the command interface.  To set a breakpoint when Silverlight 2 starts, you set a breakpoint when a specific DLL is loaded.  Depending on your needs this may be the npctrl.dll, agcore.dll, or coreclr.dll.

You set a breakpoint for a DLL load in WinDBG by using the following command:

sxe ld:DLLNAME

Thus, if you wanted to break at a specific point in the Silverlight loading procedure, you use one of the following commands:

sxe ld:npctrl

sxe ld:agcore

sxe ld:coreclr

If you want to remove these breakpoints, just type the the following:

sxr

Once you have one or more DLL load breakpoints these and resume your application (the 'g' command or F5 in WinDBG) you can test it our by going to http://themelia.netfxharmonics.com/. If successful, WinDBG will break when your DLL loads.  In the call stack window (or when you type the 'k' command), you will see a call stack that looks something like the following:

kernel32!LoadLibraryW+0x11
npctrl!GetCLRRuntimeHost+0x112
npctrl!CWindowsServices::CreateCLRRuntimeHost+0x19
agcore!CCLRHost::Initialize+0x12a
agcore!CRuntimeHost::Initialize+0x60
agcore!CCoreServices::CLR_Startup+0x95
agcore!CDeployment::Start+0x27
agcore!CCoreServices::StartDeploymentTree+0x53
npctrl!CXcpBrowserHost::put_Source+0x10f
npctrl!CommonBrowserHost::OnGotSourceDownloadResponse+0x4b
npctrl!CXcpDispatcher::OnReentrancyProtectedWindowMessage+0x284
npctrl!CXcpDispatcher::WindowProc+0x51
USER32!InternalCallWinProc+0x28
USER32!UserCallWinProcCheckWow+0x150

Chances are, though, that you won't see this information like this at first.  Instead, you will probably see a series of memory addresses.  To view the actual function names, you must configure your symbol paths in WinDBG.  These symbols, often called PDBs, will relate the machine information with information from the original source.  In our case, we need them to view the function names.

To setup symbols, use the following command, replacing C:\_SYMBOL with a symbol location of your choice (symbols are very important and, therefore, should not be thrown into a "temp" folder):

. symfix SRV*C:\_SYMBOL*http://msdl.microsoft.com/download/symbols;SRV*C:\_SYMBOL*http://symbols.mozilla.org/firefox

You may also enter this information in the window that is shown when you hit Ctrl-S.  Either way, this will configure the symbols for both Microsoft's symbol server and Mozilla's.  The latter will help you view exactly how far the loading processing has gone while in Firefox.

With the symbols configured, you will then be able see something like the above call stack.  If you set the symbols after you had already loaded Silverlight 2 in your browser, then you need to exit the browser, start a new instance, and reattach WinDBG.  Regardless, every time a DLL is loaded, WinDBG will automatically access the appropriate Microsoft and Mozilla and download the symbol file for that DLL.  This will take time for every DLL that is to be loaded.  Expect the WinDBG status bar to read *BUSY* for a long time.  It will only load the symbol for each DLL once.

The method I've just described works perfectly for breaking in WinDBG Firefox and IE.  But, what about Chrome?  Depending on settings, Google Chrome uses either a process-per-tab model or a process-per-site model.  While this flexibility is great for daily use, for debugging we can't have processes bouncing around like jack rabbits on cocaine.  You need a process that you can attach to that won't run away as soon as you use it.  So, to successfully attach WinDBG to Chrome, you should start Chrome up in single process mode.  You do this as follows:

chrome.exe --single-process

At this point, you can attach to Firefox, IE, and Chrome for Silverlight 2 debugging in Silverlight.

Silverlight SOS Commands

Today I became rather curious of what commands Silverlight's version of SOS provided (see my Learning WinDBG/SOS and Advanced Debugging post for more information on SOS).  I didn't really have any guess to whether there would be more, less, or the same.  So I ran Silverlight's SOS.dll through .NET's dumpbin.exe application with the /exports switch to get a list of what you can do with it.  In this case, the lower case DLL exports are the actual SOS commands you can use.  I did dumpbin.exe on the .NET 2.0 version and did a diff between them.  The results?  The Silverlight version of SOS actually has more commands than the .NET version.

Here is a list of SOS commands that aren't in the .NET version:

analyzeoom histobj vh
ao histobjfind vo
dumpsigelem histroot  
findroots histstats  
fq hof  
gcheapstat listnearobj  
gcwhere lno  
heapstat t  
histclear tp  
histinit verifyobj  

You can type these into your debugger to see the specific syntax for each of these.  For detailed information on most of these, you'll have to wait until something gets posted on MSDN or one of the SOS developers post something.  For a few, however, Sasha Goldshtein has provided some information and examples.  Here are some posts from this person's web site where you can find information no some of these new commands:

In case you're wondering, the .NET version has a few commands that the Silverlight version didn't have too.  This isn't a big deal as the commands that are missing don't really have much meaning for Silverlight (or have an alternative).  Here's a list of these:

comstate
dumpmethodsig
rcwcleanuplist
tst

Also, for the sake of completeness, below is the completely list of all Silverlight commands.  Just type then into WinDBG with the ! prefix to play with each of them.  For many of them you can type "!sos.help COMMANDNAME" to get help for a specific COMMANDNAME.

analyzeoom dumpsigelem histclear syncblk
ao dumpstack histinit t
bpmd dumpstackobjects histobj threadpool
clrstack dumpvc histobjfind threads
da eeheap histroot token2ee
do eestack histstats tp
dso eeversion hof traverseheap
dumparray ehinfo ip2md u
dumpassembly finalizequeue listnearobj verifyheap
dumpclass findappdomain lno verifyobj
dumpdomain findroots minidumpmode vh
dumpheap fq name2ee vmmap
dumpil gchandleleaks objsize vmstat
dumplog gchandles pe vo
dumpmd gcheapstat printexception  
dumpmodule gcinfo procinfo  
dumpmt gcroot savemodule  
dumpobj gcwhere soe  
dumpruntimetypes heapstat sosflush  
dumpsig help stoponexception  

Then there's the complete list of .NET SOS commands:

bpmd dumpstack procinfo
clrstack dumpstackobjects rcwcleanuplist
comstate dumpvc savemodule
da eeheap soe
do eestack sosflush
dso eeversion stoponexception
dumparray ehinfo syncblk
dumpassembly finalizequeue threadpool
dumpclass findappdomain threads
dumpdomain gchandleleaks token2ee
dumpheap gchandles traverseheap
dumpil gcinfo tst
dumplog gcroot u
dumpmd help verifyheap
dumpmethodsig ip2md vmmap
dumpmodule minidumpmode vmstat
dumpmt name2ee  
dumpobj objsize  
dumpruntimetypes pe  
dumpsig printexception  

If you haven't noticed yet, most of these aren't even documented commands. However, if you type them into SOS, you will not only see that they exist, you will be given the syntax for how to use them (and, then, there's !sos.help).

kick it on DotNetKicks.com

Learning WinDBG/SOS and Advanced Debugging

In my daily R&D work as well as in my general development, I always keep WinDBG open so I can quickly debug major problems in a system or just to take a look under the covers. WinDBG is short for Windows Debugger and it's what you would use if you were to debug a Windows driver or figure out why your system blue screened.  It's an advanced unmanaged debugger.  If you're into internals and eat up books like Windows Internals and Windows via C/C++, then you will or probably already do love Windows Debugger.

You can use it for more than unmanaged debugging though.  The .NET framework ships with a product called SOS, which you can load into WinDBG to enable advanced managed debugging.  Actually, with the proper settings ("Enable unmanaged code debugging" to true) you can sometimes load SOS in Visual Studio.  Using either, you can do anything from break when a particular method is called, dump out the IL at a particular place in your call stack (yes, this means you can view the code at runtime without needing the code!), view the methods of a type, or even break when the CLR first loads.  It's incredibly powerful.  You don't even need to be at the system to use it.  Just have someone send you a memory dump and you can use that just as easily as if you were physically at the system.

You can even use it to debug .NET applications inside of unmanaged applications.  For example, WinDBG is the tool I used to figure out why Visual Studio 2008 didn't allow .NET assemblies to be referenced in Silverlight.  Visual Studio was simply loading a managed assembly to do all of its assembly reference dirty work.  Since .NET has this awesome thing called the CTS (Common Type System), types are actual types instead of just chunks of memory.  So when you pause a process and walk through memory, you don't just see memory addresses like 0x018271927, but you see things like System.String with a value of "ctl02".

In addition to being able to debug unmanaged code, .NET code, and .NET code in unmanaged code, you can also use WinDBG to debug Silverlight code.  As it turns out, when you install Silverlight, sos.dll is installed in your %ProgramFiles%\Microsoft Silverlight\2.0.31005.0\sos.dll folder (or on a 64-bit system, %ProgramFiles(x86)%\Microsoft Silverlight\2.0.31005.0\sos.dll).  Just attach your debugger to the unmanaged browser to debug the managed Silverlight application.

At this point you might expect me to announce that I'm going to do some long series on how to effectively use SOS, but that's not the case for two reasons: first, I'm not going to dump out a series of screen shots when many others have done this already.  Second, 90%+ of advanced debugging is all about understanding internals; something that takes many hours of study over many months.  Don't think that learning some tool will help you to effectively debug serious problems.

Thus, remember this: your ability to debug something is directly proportional to your knowledge of the system.  As I say regularly, if you ever try to fix something without understanding it, you are, at best, doing little more than hacking.  Therefore, instead of writing a series, I'm going to refer you to a series of places where you can easily learn all about WinDBG and, more importantly, advanced debugging and system internals.

Before you go diving into a list of resources, though, you need to realize that WinDBG, like Visual Studio's built-in, toned-down debugger, is only a tool.  As I've already mentioned, no tool will ever replace the need for the human to know how to use the tool or how to interpret the information provided by the tool.  Knowledge of debugging is required before you can use a debugging tool effectively.  Logic is also always required when debugging any issue.  Not only that, but as previously mentioned, deep knowledge of the internals of the system you are debugging is non-negotiable.  If you don't know what a MethodDesc, what a method table, or even what a module is, then your debugging abilities will be severely limited.  You won't have a clue how to find or interpret your debug output.

The more you know about the CLR, memory management, threading, garbage collection, the CLR's type system, and Windows the better off you are.  If you understand why .NET finalizers should only be used once every 10 years, you're on the right track.  If you don't even know how many generations are in .NET's garbage collector, you have some serious research ahead of you.  The more you know, the better off you are.  This applies to the whole of the system.  I hear people all the time talking about the "stack" and the "heap" as if they are some ethereal concept beyond human grasp.  These people typically have no idea that these are concepts that predate .NET by decades.  If you can't explain the stack and heap from a C/C++ perspective, then you don't truly understand the concepts.

Also, if you understand memory management in C/C++, you're way ahead of the game.  Even though you don't normally see them in a typical ASP.NET web form, you should know your pointers as they are the foundation of all memory management.  Without awareness of pointers, the entire concept of "passing by reference" is little more than magic.  You need to know why something is a value type and why something is a reference type.  If you don't understand the concept of a sync block index, then you're always wonder why you can't lock on a value type and why you can lock on a value type cast to an interface.  This type of information will go a long way to help you not only debug a system, but, equally important, to make sure it has optimal performance.

You should also not be afraid of IL.  This is the language of .NET.  You don't need to care about becoming fluent in writing in IL.  I promise you that you won't be required by any fascist employer to write a full-scale WPF system in IL.  However, if you don't even know what boxing, you need to hit the books.  Knowledge of IL dramatically aides in your understanding of both the framework class library and the CLR.  You should be aware of what is actually happening under the covers.  Don't simply assume that the csc.exe or vbc.exe compilers will fix your coding flaws.  Some times the code created by these compilers isn't optimal.  You should also understand what the evaluation stack is and how stack-based programming works as this is the foundation for IL development.

Fortunately, there are resources for these prerequisites of advanced debugging in addition to the following debugging resources.  Not only that, but there's two books that explains just about every thing I've just mentioned!  It's Jeffery Richter's CLR via C# and Joe Duffy's .NET Framework 2.0.  Buy these books and a stack of highlighters and study until every page has marks, notes, and coffee stains on them.  In addition to this, you should probably also drop by your local book store, grab a book on C/C++ and read a chapter or two on memory management so you can see the types of things the CLR has to deal with so you don't have to.

For even deeper debugging, though, you may want to study the "Processor Architecture" section of the WinDBG help files, get the aforementioned Windows/C++ books, and buy Advanced Windows Debugging as well.  Everything you study is just "code", so the more you know, the deeper you can debug.  Not just debugging though, but knowledge of internals help you get more knowledge of internals.  By understanding C++, assembly language, and Windows internals, you already have just about everything you need to do effective reverse engineering.  This will help you to understand the undocumented internals and features of various technologies.

These things will help further your knowledge of what's actually going on in .NET's brain.  It's also important to remember that you won't learns this stuff over night.  As I said, your debugging skills are directly related to your knowledge of internals.  Therefore, as pick up more prerequisites in your career, you will become better at debugging (and performance optimization).  Every few months I take some time to further my understanding of the internals of new (and even sometimes old) technologies for these specific reasons.  For example, in Q1 and Q2 of 2009, I'm spending the bulk of my time studying assembly language and Windows architecture to further my own understanding.

Now on to the resources (which will be updated every now and again):

Advanced Debugging Resources

Powered by
Python / Django / Elasticsearch / Azure / Nginx / CentOS 7

Mini-icons are part of the Silk Icons set of icons at famfamfam.com