10 Things Most Developers Didn't Know in 2007

To end 2007, I thought I would make a list of things which I found that most developers didn't know.  To make things more interesting, this list is actually a series of 10 mini-articles that I wrote today.  Since this article has several sub-articles, here's a table of contents to help you out (these aren't really in any order of importance):

#1  SQL Server supports powerful subqueries as anonymous sets.

Many developers don't take the time to seriously look at T-SQL or SQL Server internals.  As such, they miss many of SQL Server's more powerful features.  In January 2007, when co-worker saw he write the following query, he about fell out of his seat: 

select MemberName, m.MemberId, count(*) from (select 
    distinct MemberId, 
    VisitUserAgent 
    from VisitSession 
    where MemberId is not null) a 
inner join Member m on a.MemberId = m.MemberId 
group by m.MemberId, MemberName, VisitUserAgent 
having count(*) > 1 
order by count(*) desc

For starters, the guy didn't know you could do a filter after a group by, but that's not my point.  He had no idea that SQL Server (2000) allows you to use subqueries or use subqueries as anonymous sets.  As you can see, you can select from the list as well as use it in a join.  This tidbit alone should toss many painfully slow cursor-based stored procedures into the trash.  It's a simple SQL feature, but it's a powerful one.

#2  Firefox has an operating-system style console for web application debugging.

It's incredibly hard to find an ASP.NET web developer who knows this one.  It's a feature that knocks people right off their seats.  Instead of throwing alerts all over your AJAX applications, you can use the Firefox console and the dump( ) function.  Did I mention this has been a native feature since Firefox 1.0?

Step 1 (start Firefox with -console switch)

Step 2 (add the boolean key 'browser.dom.window.dump' to the Firefox configuration an set it to true)

Then simply call dump( ), instead of alert( ) and you're done. Your output will go to the Firefox console window (which looks almost exactly like a cmd window).

With this technique you can entirely avoid any possibility of having an infinite loops of alerts.  Personally, I like to track all the output of my web applications.  This comes in very handy when I'm using event capturing or need to watch the progressive state of my application.  When I do this, I also like to write an output identifier to each data dump.  Here's a sample of what I usually use for debugging:

var Configuration = { 
    Debug: false
}; 

var Debug = { 
    counter: 0, 
    write: function(text) { 
        if(Configuration && Configuration.Debug) { 
            dump(text); 
        } 
    }, 
    writeLine: function(text) { 
        if(Configuration && Configuration.Debug) { 
            Debug.counter++;        
            dump(Debug.counter + ':'+ text + '\n'); 
        } 
    } 
};

Here's some sample output using the Debug.writeLine( ) abstraction:

Leaves alert( ) in the dust, doesn't it? You can actually learn more about this technique and others from my Firefox for ASP.NET Web Developer video series found on my blog.  These topics are crucial to your understanding of modern web development.

#3  JavaScript has natively handled loosely-coupled multi-cast events for years.

This isn't something just for the Firefox, Opera, Safari world.  Even IE6 has native support for this feature. I'm not sure why this is, but in September 2007 when I was designing the AJAX exam for Brainbench, not a single one of the reviewers knew that JavaScript natively supported loosely-coupled multi-cast events.  I actually comments from almost all of the reviewers telling me that I should "leave server-side questions out of the exam".

JavaScript loosely-coupled multi-cast events are one of the most important core features of AJAX applications. They allow you to quickly and efficiently attach multiple event handlers to the XHTML same element. This becomes critically important when you are with multiple AJAX components, each of which that want to have an event handler attached to the load event of the window object.

I wrote an article about this in September 2007, so I'm not going to go into any kind of details here.  You my also opt to view this file from my SolutionTemplate, which supplements that blog entry.

#4  Not all image formats are created equal.

A few months ago, I came in as lead architect about half way through a project.  After having a few people fired for absolute incompetence, I did find a few people (PHP guys) who were ready, willing, and actually able to learn ASP.NET.  Everything was going well until the designer came back with his new theme and my associate whom I was training implemented it.  Everyone thought the project was going fine until I stepped in the room.  It didn't take but 10 seconds for a red flag to go up.  Just looking at the web site I could tell that this theme implementation was a disaster.  I noticed that there were signs of JPEG compression all over every single one of the images.  However, being a scientist and part-engineer I knew that measurement was a major key to success.  So, I whipped out Firebug, hit refresh and felt my jaw drop.  The landing page was 1.5MB.  Ouch.

You absolutely can not use one single image format for ever image on your web site, especially not the deadly JPEG format which does little more than destroy your images.  There are rules which web developers must need to follow or else a project is doomed to failure.  First off, you need to be using PNG24s for the all important images, while comparing their file sizes and quality with PNG8 compression.  Using Adobe Photoshop's Save For Web feature is very helpful for this.  If the image is a photo or something with many "real life" colors and shades, perhaps you want to do a size and quality comparison against a JPEG version as well.  If you absolutely need to have transparent images for IE6, then you need to take extreme care and either make special PNG versions for each background or, if you don't care too much about quality and the image is small with very few colors, use a GIF with transparencies.  The same goes for Firefox and printing.  Firefox (as of 2.0) does not print transparent PNG images.  So, if you want to support printing in Firefox, then you need to either make special PNG images for each background or make low-quality GIF images.

Needless to say, the designers theme had to go under severe reconstruction.  Not just because of the image sizes, but because he felt the need to design special input box, textarea, and button controls.  His design would have worked well for a WPF application, but this is the web (... but don't even get me started on the fact that his design for a wide screen monitor at over 1300x800.  The design was useless anyhow!)  The next project I ran as lead architect went much smoother.  Because it was extremely AJAX intensive, everything was minimized to the absolute core.  Each page had the minimal default.css plus it's own CSS sheet and only included the JavaScript it needed.  The web site landing page included barely anything and even had it's own extremely stripped down version of the JavaScript files.  For this project, I went from 350K in development to 80k in production.

#5  Custom server controls are not esoteric, complicated, or take too long to create.

  This seems to be a very common misconception amongst ASP.NET developers.  The reality, however, is that creating server controls is often a very trivial task.  Yet, many developers will use a GridView or other canned control for everything.  The GridView is awesome for basic tabular in a very simple, data-driven applications, but I can rarely use it.  On the other hand, I love the repeater and rely on it for almost everything.  Actually, it and the Literal are my two favorite controls.  I have to rely on these two controls to ensure that my AJAX applications are extremely optimized.  One of the beautiful things about .NET is that every ASP.NET control is simply a .NET class, which means that you can programmatically reuse them, inherit from them, and override their internals.  Thus, allowing us to create some powerful and elegant custom server controls.

On the same project with the overly sizes image files, we had an interesting meeting about how to show a media play list on a web page.  There was all kinds of talk about using Flash to create a media play list.  The conversation was quickly giving me an allergic reaction.  So, after hearing all kinds of absolutely insane quotes of time for creating a Flash play list, I decided to take matters in to my own hands.  Two hours later I handed the client a complete play list from A to Z.  To be clear, I had built this one something I had already had, but the grand total of time was them about 3 hours.  It's amazing what you can do when you understand the .NET framework design guidelines and aren't afraid to follow best-practices.

Here is how you would use a similar control:

<%@ Register Assembly="Jampad.Web" Namespace="Jampad.Web.Controls" TagPrefix="j" %>

<j:Media id="media01" runat="server" />

In your code behind, you would have something that looked like this:

media01.DataSource = MediaAdapter.GetContent(this.MemberGuid);

Upon loading the page, the data was bound and the output was a perfect XHTML structure that could them be customized in any number of ways using the power of CSS.  How do you make something like this happen?  It's simple, here is a similar control (Media.cs) placed in a class library (WebControls.csproj):

using System;
using System.Web;
using System.Web.UI;
using System.Web.UI.HtmlControls;
using System.Web.UI.WebControls;

namespace Jampad.Web.Controls
{
    [ToolboxData("<:Media runat=\"server\"></:Media>")]
    public class Media : CompositeControl
    {
        private Repeater repeater;

        public Media( ) {
        }

        private Object dataSource;

        public Object DataSource {
            get { return dataSource; }
            set { dataSource = value; }
        }

        protected override void CreateChildControls( ) {
            HtmlGenericControl div = new HtmlGenericControl("div");
            div.Attributes.Add("class", "media-list");

            try {
                repeater = new Repeater( );
                repeater.DataSource = this.DataSource;
                repeater.ItemTemplate = new MediaTemplate(ListItemType.Item);
                repeater.HeaderTemplate = new MediaTemplate(ListItemType.Header);
                repeater.FooterTemplate = new MediaTemplate(ListItemType.Footer);
                div.Controls.Add(repeater);
                repeater.DataBind( );
            }
            catch (Exception ex) {
                Literal error = new Literal( );
                error.Text = "<span class=\"error-message\">" + ex.Message + "</a>";
                div.Controls.Add(error);
            }

            this.Controls.Add(div);
            base.CreateChildControls( );
        }
    }
}

Notice the use of the repeater control.  This is the same control we use in ASP.NET as <asp:Repeater />.  Since this is .NET, we can use it programmatically to create our own powerful controls.  Also notice the various templates that are being set on the Repeater.  These are the same templates you would set declaratively in an ASPX page.  In this case, I'm programmatically assigning to these templates an instance of MediaTemplate (in MediaTemplate.cs).  This MediaTemplate.cs is just another file thrown in a class library, in our case the same WebControls.csproj, though since it's just a class, it could be in a different assembly and namespace altogether. Here's what the MediaTemplate.cs looks like:

using System;
using System.Collections.Generic;
using System.Text;
using System.Web.UI.WebControls;
using System.Web.UI;

namespace Jampad.Web.Controls
{
    internal class MediaTemplate : ITemplate
   {
        ListItemType type = new ListItemType( );

        public MediaTemplate(ListItemType type) {
            this.type = type;
        }

        public void InstantiateIn(Control container) {
            Literal lit = new Literal( );
            switch(type) {
                case ListItemType.Header:
                    break;

                case ListItemType.Item:
                    lit.DataBinding += new EventHandler(delegate(Object sender, System.EventArgs ea) {
                        Literal literal = (Literal)sender;
                        RepeaterItem item = (RepeaterItem)literal.NamingContainer;
                        literal.Text += String.Format("<div class=\"media-item\">\n");
                        literal.Text += String.Format("  <div class=\"media-item-inner\">\n");
                        literal.Text += String.Format("    <a href=\"\"><img src=\"\" alt=\"Media\" class=\"media-thumb\" /></a>\n", (String)DataBinder.Eval(item.DataItem, "mediaPath"), (String)DataBinder.Eval(item.DataItem, "thumbPath"));
                        literal.Text += String.Format("  </div>\n");
                        literal.Text += String.Format("  <div class=\"media-item-bottom\"></div>\n");
                        literal.Text += String.Format("</div>\n\n");
                    });
                    break;

                case ListItemType.AlternatingItem:
                    break;

                case ListItemType.Footer:
                    break;
            }
            container.Controls.Add(lit);
        }
    }
}


Simply compile those to together and you're set.  You can even embed (hopefully tiny) images in your project to make things even more seamless.  Using this simple pattern, I've created all kinds of things.  You can see a real example of this, including image embedding, in my SQL Feed Framework (formerly known as Data Feed Framework).  It's InfoBlock controls follow this same pattern.  For much better examples, whip out reflector and start digging around the System.Web namespaces.

It's actually rather astonishing to learn of some of the attituted some developers have about custom controls. When I was one of the editors for an ASP.NET 2.0 exam last year, I noticed one of the questions ask which type of control was "harder" to create. The answers were something like "User Control", "Custom Control", and a few others. They were looking for the answer "Custom Control". Since "harder" is not only a relative term, but also a subjective and an abstract one, the question had no actual meaning. Custom controls aren't "harder" than user controls.

#6  Most developers I worked with in 2007 had never heard of an O/R mapper.

Why do most developers still absolutely insist on wasting their time writing a chain of SqlConnection, SqlCommand, and SqlDataAdapter?  Perhaps it's just an addiction to being busy instead of actually being productive that causes this.  I don't know.  I would, however, expect these developers have to have some curiosity that there may be an easier way.  ADO.NET is awesome stuff and it is the foundation for all .NET O/R mappers, but if I'm not throwing around 1,000,000 records at a time with SqlBulkCopy, I'm not interested in working with ADO.NET directly.  We need to have a system that allows us to get what we want instead of forcing us to screw about with low-level mechanics.  It's no secret that I'm a huge supporter of Frans Bouma's work with LLBLGen Pro and I also use LINQ in most of my .NET 3.5 applications.  For a corporate .NET 2.0 project, there's absolutely no excuse to not pay the $300 for LLBLGen Pro.  Managers!  Open the wallets!  It will save you money.

However, it's not always about the money.  Even if the developers knew about O/R mapping, and the company isn't from in a poverty-stricken 3rd world country, sometimes extreme pride, lack of personal integrity, and political alignment can destroy any chance of being productive.  A long time ago I worked at a company where I thought I would be productive.  Five or so weeks into the design phase of the project, we received a politically-focused project manager as big brother.  He was absolutely against the use of any modern technology and despised the idea of an O/R mapper.  He instead told us that we were to write a stored procedure for every possible piece of interaction that would happen.  He also wanted us to use Microsoft's data application block to access the stored procedures.  At one point he said that this was their O/R mapper, showing that he had no idea what an O/R mapper was.

A few days after his reign had started, I took an hour or so to write up a 12 page review document covering various aspects of LLBLGen Pro and how they would work on the project.  I thought it was a very convincing document.  In fact, one guy looked at it and was convinced that I took it from the LLBLGen web site.  The project manager, however, was beginning to be annoyed (this is not uncommon with me and old-school project managers!)  The project manager decided to call together a panel of his "best" offshore developers and put me in what basically amounted to be a doctoral defense.  Prior to the meeting I sent out my "dissertation" and asked everyone to read it before they arrived at the meeting so that they would be prepared for the discussion.  When it was time for the meeting, I was told to sit at one side of a large meeting table and the project manager and his team sat at the other.  Then the disaster began.  First off, not one single person on that team had read my document.  Secondly, for the next 45 minutes they asked me basic questions that the document would have answered.  Even after they admitted that I had answered all of their concerns to their satisfaction and being told by their team that LLBLGen Pro was obviously a very productive tool, they reached the conclusion that they still weren't going to use it.  It was a waste of my time and I still want those 45 minutes of my life back.

What was really interesting about my defense was the developer's code.  In the meeting, the developers had showed me their [virtually unreadable, anti-.NET framework design guidelines, inefficient, insecure] .NET project code and I was shocked to see how much time they wasted on writing the same stuff over and over and over again.  When they showed me their stored procedures, I about passed out.  It's a wonder how any of their systems run.  They were overridden with crazy dynamic SQL and cursors.  They even had most of the business logic in the data access tier.  The concept of N-tier architecture was not something that they understood at all.  I think that's the point where I gave up on my defense.  If a developer doesn't even understand the critical need for N-layer and N-tier architecture, there's just no way they will be able to understand the need for an O/R mapper.  It's actually one of the fastest way to find a coder hiding amongst professionals.  Their SQL/ADO.NET code was also obviously not strongly-typed.  This was one of the core points of an O/R mapper and these developers could not understand that.  They could not see the benefit of having an entity called Person in place of the string "Persno" (deliberate misspelling).

This project didn't really take off at all, but for what parts I was involved, I used the next best thing to an O/R mapper: a strongly-typed data-set.  Read this carefully: there is no shame in using a strongly-typed data set if you don't have an O/R mapper.  They are no where near as powerful, but they are often good enough to efficiently build your prototypes so that the presentation layer can be built   You can replace the data access components later.

The training of developers in the use of LLBLGen Pro and LINQ O/R mapping was one of the main reasons I publicly released both my Minima Blog Engine and my Minima 3.5 Blog Engine source code to the public in 2007.  You are free to use these examples in your own training as you see fit. 

For more information and for some example of using an O/R mapper, please some of my resources below:

#7  You don't need to use SOAP for everything.

This is one of the reasons I wrote my XmlHttp Service Interop series in March and May 2007.  Sometimes straight up HTTP calls are good enough.  They are quick, simple, and light-weight.  If you want more structure, you can simply use XML serialization to customize the smallest possible data format you can think of.  No SOAP envelope required.

Here are the parts to my series:

Also keep in mind that you don't need to keep JSON to JavaScript.  It's a beautiful format that could easily be an amazing structured replacement for flat CSV files.  RESTful interfaces using GET or POST with HTTP headers are also a great way to communication using very little bandwidth.  My AJAX applications rely heavily on these techniques, but I've also used them for some behind the scenes work as well.

One great example of how you can use RESTful services is by looking at the interface of the ESV Bible Web Service V2. In November 2007, I wrote a .NET 3.5-based framework to abstract the REST calls from the developer. By looking at my freely available source code, you can see how I'm interacting with the very light-weight REST service.

#8  A poor implementation of even the most beautiful database model can lead to a disast