2005 2006 2007 2008 2009 2010 2011 2015 2016 2017 aspnet azure csharp debugging elasticsearch exceptions firefox javascriptajax linux llblgen mongodb powershell projects python security services silverlight training videos wcf wpf xag xhtmlcss

Coders and Professional Programmers

I have to say it: there is a severe lack of skill in the technology world today, even among senior-level developers.  Perhaps I just have too high of standards, but when I see a person working in technology, I kind of expect them to have the requisite skill set for the job.  It always amazes me when I see a developer, even a senior-level developer, who doesn't know the basics of their own system (i.e. an ASP.NET developer who doesn't understand CSS positioning or a .NET developer who doesn't understand the framework design guidelines).  They may have degrees and have years of experience, but they have no actual skill.  It's always concerned me a bit that people have so little skill, but I wasn't able to put the problem into words until recent.  Well, that's not completely true.  An old friend of mine put it into words years ago, but it wasn't until this week that I was reminded of my own story.

Last week I was in a bookstore looking at a few design patterns books and a "1960s NASA engineer" type of guy in a white shirt and tie came up next to me, dressed in extremely casual blue jeans and a black shirt, looking at similar books.  I noticed he picked up a rather lame design patterns book, so I took it upon myself to hand him the GoF book saying "This one is the classic".  He thanked me and started to flip through it.  A minute later I realized that I couldn't resist asking: "So, what exactly are you looking for?"  He mentioned he was looking for a book demonstrating a state machine.  I grabbed another book, flipped the pages a bit and handed it to him saying "page 486".  His response was "Wow, thanks... so, where did you go to school?"  I have to admit that I've gotten many questions in bookstores, but I've never been asked that one FIRST.  The way he asked the question wasn't so much out of sincere curiosity, but more out of arrogance.  It felt like wanted to compare his degrees to my own.  I simply replied "School?  What?  You're looking at it...", pointing to the wall of computer books.  He immediately shut up and stepped away.  I'm not sure if that qualified me as a leper or if I just castrated the man's sense of academic accomplishment, but, regardless, that conversation was over.  However, a few moments later I found a book I was looking for and said "AHA! This will make them think!"  The same man curiously asked "What do you mean?"  I explained that I was looking for a book to recommend to developers to help then get a clue about software architecture.  The man gave the look of familiar with my situation and said something I haven't thought about in years: "Oh yeah... I know exactly what you mean.  Some people just don't get the difference between a coder and a professional."  At that point I had my flash back to a time of my life almost 6 years earlier... back when I was a coder.  It was a lifetime ago and I had all but repressed until recently.

The year was 2001 and I was a PHP programmer, Linux hacker and Math major at Kansas State University.  My idea of programming was looking at the PHP documentation, reading the PHP documentation comments and copy/pasting until my application was built.  I figured I knew what I was doing because I had already been in the field for 5 years; I was only now going back to school after a 3 year academic hiatus.  I would mix PHP, HTML, JavaScript, and tiny bits of CSS all in the same page in an absolutely unreadable pile of slop, "but it worked".  A friend of mine from high school, who by this time was a Microsoft employee, looked at what I was doing and mentioned to me in his usual blunt style: "Dude, that's not programming, that's slop coding.  You don't even have a data access layer, do you?"  Completely ignorant of the first clue of what he was talking about I said "Well, sure I do, PHP has a connector to mySQL.  I'm not accessing mySQL directly; I'm using the PHP API."  Thinking back on it I think he wanted to punch me in the face from that painfully ignorant comment."  Truth be told, I was not a professional programmer even though I had 5 years of crazy JavaScript development, 2 years of Ajax development, 2 years of ASP development, 2 years of SQL Server development, and 1 year of PHP development under my belt; I was only a coder.

After that, I spent a tremendous amount of time going back to the books and learning more about technology.  In high school I had some of the lowest grades in my classes because I spent my nights reading video game programming,  graphics programming, C/C++ programming, computer history, networking theory, Novell NetWare, and web development books.  In college, the pattern was being repeated in a the same familiar way.  I spend a ton of time studying all kinds of new technologies, one of which being XHTML/CSS design, all to the destruction of my meaningless academic career.  After finally breaking free from the tight bondage of academia I went to work for a few places as a PHP developer.  After one of two complete waste-of-time contractor positions I finally realized that it was time to cross over to .NET.  So, I spend an incredible amount of time studying .NET concepts, ASP.NET, the Framework, and C# technologies.  After a month I took and passed the 70-315 (ASP.NET) exam and the 70-229 SQL Design exam a week later.  From there I went to study the 70-320 (.NET components) and 70-228 material (SQL Administration).  Around that same time I realized that Mozilla finally released a product that didn't suck.  Not only did this product not suck, it was by far and away the most revolutionary piece of technology since the Internet itself: Mozilla Firefox 1.0.  With that I also started studying web standards much more deeply.  Being former Linux hacker, I wasn’t and still am not brained washed by the Microsoft extremist cult, so I've always been open to having the best technology regardless of vendor.

At this same time I realized that I really, really wanted to be a professional programmer and not a coder in any way.  So, I said goodbye to Visual Studio and took back my EditPlus to do all my .NET work.  I was not about to rely on Intellisense to do my work for me and already knew the danger of learning a new technology by error messages.  For the next year I deliberately did all my development without color syntax and without Intellisense.  I've never had any respect for "drag-n-drop" property developers, so I never used the designers anyhow, but now I wasn't even going to let Intellisense help me either.  If I needed to do something, guess what, I had to actually read the manual.  There would be no copy/paste slop coding and no hitting the forums every single time I hit the slightest bump.  If I had a problem, I would think about it and solve it using that technology, not simply a hack to throw some hack together.  I had enough years of wasting my time doing that.  Yes, at first, this got me into trouble, but I was determined to be a professional programmer at any cost and not a coder at all.  I wanted to internalize the systems as I had internalized driving my car.  After the initial slowdown (and the initial "being let go" from a project), I became much faster than all the other developers from the forced memorization of much of the documentation.  Whereas others had to look up how to use the Sqlconnection, SqlCommand, SqlDataAdapter, and DataSet combination, I was typing them out as if I were writing a sentence.  Around the same time I spent much time studying LLBLGen Pro, unit testing, object-oriented design (not to be confused with the entry-level concept of object-oriented programming I learned 10 years ago in C++), N-tier architecture, Modern JavaScript, and a year after that I took a block of time to study web services, MSMQ, COM+, service interop, and other communication mechanisms (most of which were later rolled up into WCF) all to the point of either complete or partial internalization.  In 2005 I was finally able to say that I am a professional programmer, not simply a coder.

As you can see from my story I know what it’s like to be a coder and I know the temptation to remain as a coder.  You think “your way” works and therefore you shouldn’t change it.  In fact, you probably hate other technologies, because their way isn’t "your way".  I can understand that too, but really it’s just a misunderstanding of that other technology.  My first ASP.NET application (a photo gallery) had absolutely no declarative code and was written with 100% Response.Write statements (I often tell people "make sure your first project in any technology is NOT a production one"; this is why).  I didn’t have the first clue what I was doing and, being a longtime classic web developer, I could not get that “magical” ASP.NET postback through my head.  Building that same application today would consist of an elegant ASP.NET custom control and two or three web forms.  I also understand what it’s like to say “...but this is how I do it in this other technology”, when in a reality that’s a completely meaningless statement.  Each technology has its own paradigms, naming schemes, guidelines, concepts, and languages.  Just because you are a Java rock star, doesn’t mean you will be able to do anything in .NET for at least 6 months.  The same goes for VB rock stars and PHP rock stars (as odd as it sounds, in my experience PHP developers have the hardest time learning .NET where as VB developers excel rather quickly-- this goes to show how hard it can be to un-link and old technology from your mind). I also know what it’s like to ask yourself “how can I do this?”  That’s what a coder asks where as a professional asks “How has this been built before in this technology; what is my precedence?”  In case you didn’t know, software development has been around for decades.  If you are running around trying to come up with a way to do something, you may be wasting your time as it’s probably been done before, the debates have probably already happened, people have probably already learned from their mistakes; you just have to accept what they've done for you, within limits, of course (i.e. C++ design patterns transfer to C#, but obviously not OOP style as .NET doesn’t allow multiple implementation inheritance).

So, if you find yourself in a place where most of your development is done doing things “your way” or the way “you have always done it” or if you ever ask “how can I do this”, maybe it’s time to break down and take some time out of your life to convert from being a coder to a professional.  It is definitely an investment, but with many great books out today (not all are so great), you could probably convert from being a coder to some level of professional programmer rather quickly (it took me a bit longer because I didn't have any role models to work under and I had no idea what the scope of the training was). There is an old computer science saying that goes something like this: "Anyone can write code that a computer can read, it takes a professional to write code a human can use."  We should all think about that and possibly post it on our walls.

To aide in the process of converting from a coder to a professional programmer, a few months ago I wrote up my requirements for a Senior-Level developer or Architect on my blog.  It should give you a shell outline for some of the technologies and topics you seriously need to have under your belt to be a professional.

Minima .NET 3.5 Blog Engine (a.k.a. Minima 2.0)

Since the original release of Minima, I've added a ton of new features to my own blog (which ran on the original Minima).  That plus the release of the .NET Framework 3.5 Beta 2, which includes LINQ, has prompted me to work further on the public release of Minima.  Thus, the creation of the Minima .NET 3.5 Blog Engine.  This is major upgrade to Minima.  Here are some of the new features...

LINQ

Though I love LLBLGen Pro (the worlds only enterprise-class O/R mapper and database abstractor), I thought it would be tremendously beneficial to build the new version of Minima on something that many more people would be able to appreciate.  Thus, I rewrote all data access components to use LINQ instead of LLBLGen Pro.  In the Minima .NET 3.5 Blog Engine, there is no LLBLGen Pro left over and thus you do not need the LLBLGen Pro assemblies.

If you are new to LINQ, then the Minima .NET 3.5 Blog Engine is something you may want to checkout.

Windows Live Writer Support (with RSD Support)

The previous version of Minima didn't have any supported blog client.  That was left as a training exercise to those using Minima as a training tool.  It did however have an extensive WCF interface that allowed simple web-service interaction with Minima.  However, since the release of Windows Live Writer (WLW) Beta 2, the need for a WCF interface and the need to write your own blog client is gone.  With this release of Minima, you simply setup your blog , point WLW at it and it will detect everything for you via the provided rsd.xml and wlwmanifest.xml files.

Metaweblog API

As previously stated, the Minima .NET 2.0 Blog Engine had a WCF interface, but this new release doesn't.  In it's place is the XML-RPC based Metaweblog API to allow for seamless interaction by WLW.

Simplified File Structure

The file structure of Minima has been greatly simplified and the number of files in the actual website is very minimal now.  If you recall, the entire idea behind Minima was that it is very minimalistic.  This doesn't mean it doesn't have many features, but it rather means that everything is organized in such a way that makes it look minimalistic.

SQL Server-based HttpHandler Management

A few weeks ago I published a simplified version of the universal HttpHandlerFactory technique, which relies on SQL Server instead of XML configuration files for HttpHandler management.  Under this model you can simply go to the HttpHandler table and put in an HttpHandler, the text used to match against a URL, and a matching type ("Contains", "EndsWith", "StartsWith", or "Default").  So, if you want to move your API from /xml-rpc/ to /962c8b59-97dc-490b-a1d1-09b55e47455b/, just go into the table and paste in that GUID over the text XML-RPC and you're done (you will probably have to kill the cache as well since Minima caches HttpHandler mappings-- just open and save the web.config file.)  Using this same HttpHandler table you can change the comment moderation endpoint (which uses the MinimaCommentHttpHandler).

Google Sitemap Creator

Minima now allows you to create a Google Sitemap simply by using the MinimaSiteMapHttpHandler registered in the HttpHandler SQL Server table.  By default it's registered to anything ending with "sitemap.xml", but by changing the path in the HttpHandler table you can instantaneously change the sitemap path.

File Mapping Support

One of the features I found that I really, really needed a few months ago was a way to symbolically link virtual files to physical files.  Thus I built a simple system that would allow me to store symbolic links in SQL Server.  Initially there is a default "catch all" folder to which all files are naturally mapped.  Internally, this is by setting MinimaFileHttpHandler to a certain path in the HttpHandler table; /materials/ by default.  For specific files that aren't in that folder, you simply add the mappings in the FileMapping table.  Theoretically, you could have multiple links to the same file using this system.

User Rights System

Since there is a public API, there needs to be some type of rights management system.  In the Minima .NET 3.5 Blog Engine, rights are done at the system and blog level and you assign rights to blog authors.  For example, an author needs the 'R' (retrieve) system right to get a list of blogs on the system (what WLW will try to do when you point it at this release of Minima) and would need the 'C' (create) blog right to post to a particular blog.  There are 'C', 'R', 'U', and 'D' rights that can be set in the UserRight table.  If it's a blog right, there is a blogId associated with the right, if it's a system right the blogId is null.  It's actually a really simple, yet effective rights system.  That the entire point of minimalism.

Access Control System (via IP address, UserAgent, and HTTP Referrer)

In a recent blog entry I alluded to an access control system I added to my blog.  This access control system allowed me to control access by IP address, UserAgent, or HTTP Referrer and based up on one of those three things I could either write a message to the browser or forward them.  So, for example, if I want to block someone trying to download my entire site I can put the following data in the SQL Server 'Access' table: 1, 'I', 10.29.2.100, 'You have been banned', null, true.  That is, on BlogId 1, for an IP Address of 10.29.2.100, show a message of 'You have been banned', don't forward them anywhere (that's the null), and enable the rule ('true').  If you want to have them forwarded to another address, just put the destination address in the AccessHttpForward column and the rule is activated immediately.

This is implemented by an HttpModule, so it's going to apply to absolutely every request coming across the system.  They won't be able to download anything off your website, not even images or CSS files.  Also, this system works on a "first hit" model.  That is, the first rule that is hit is the one that is applied.

Tracing via Reflection

This release of Minima also allows you to trace just about anything to SQL Server.  The tracing mechanism works by serializing what you give it into XML and then stores that XML in a table.  Doing something like this comes in tremendously handy when troubleshooting APIs.  Since Minima's TraceManager has a RecordMethodCall method that accepts a params array of Object types, you can send as many parameters as you want to the method and it will serialize them into XML for storage in SQL Server.  Some types are obviously no serializable, but you can always send each property of a type to the method instead of sending the entire object.

Other Features

As with all my web projects, exceptions are e-mailed to the person monitoring the system.  With this release of Minima, I added a feature that I've been using for a while: support for Gmail subjects.  Usually in Gmail, messages that look a like will be grouped together in the same conversation.  This is a tremendously time saving feature for most everything, but not when it comes to exception monitoring (or comment notification).  So, when enhanced Gmail subjects are enabled, e-mail subjects are suffixed with a long random number (actually, the time in ticks) so that no two messages are in the same conversation.  The same is done for comment notification.

Previous Features

Previous features were fairly basic and included labeling, commenting, and other blog stuff.  The biggest feature of the previous release of Minima was its ability to have more than one URL for a blog entry.  So, for example, if you accidentally blog as /Blog/2007/08/LINQ-ruels.aspx, you can add another URL mapping to that entry so /Blog/2007/08/LINE-rules.aspx, which would set as the default mapping, goes to the same place.  Both are still accessible, but the new default will be the one that shows up at the blog entry's permanent home.  The Minima .NET 3.5 Blog Engine retains this feature.

As a Training Tool

As with all my project there is an "As a Training Tool" section in the release notes.  This release of Minima can be used to train concepts such as ASP.NET, CSS theming, proper use of global.asax, integrating with Windows Live Writer, framework design guidelines, HttpModules, HttpHandlers, HttpHandlerFactories, LINQ, type organization, proper-SQL Server table design and naming scheme, XML serialization, and XML-RPC.NET usage.

To download the Minima .NET 3.5 Blog Engine simply access the following Subversion repository and download the associated sample SQL Server 2005 database:

Fast Visual Studio 2008 Beta 2 Downloads

WOW! I heard that these downloads were fast, but I had no idea they were this fast!  MSTorrent [theoretically] rules!

Hopefully this will actually work.  The first two times I've tried MSCD with Standard I got a "verified" download, but it was completely corrupted.  Then I downloaded and installed Web Developer Express from the web site and then downloaded and installed Standard from the web site to find out that they have completely incompatible versions of the .NET Framework 3.5.  No idea.

If you want to give it a try, head on over to the link below.

Related Links

NetFXHarmonics Subversion Viewer

In an attempt to make my SolutionTemplate/EBook more accessible and readable, I went ahead and built a simple ASP.NET-based Subversion viewer to allow readers to view and read my SolutionTemplate/EBook online.  This should allow easier navigation of the project without actually forcing anyone to download it.  Furthermore, this really applies to all my projects in Subversion: Minima, Data Feed Framework and SolutionTemplate/EBook.

You can access the NetFXHarmonics Subversion Viewer at one of the following locations:

Basically you just access http://viewer.netfxharmonics.com/PROJECT_NAME/trunk.  It's that simple.  Hopefully these will make referencing these resources and reading SolutionTemplate/EBook a lot easier.

By the way, in case you are wondering, this viewer was created by an HttpHandler that uses the URL being accessed to figure out what path or file to look at in the Apache front-end for Subversion.  After a quick HttpWebRequest, a few regular expressions are done and a header and footer are tacked on to give the final version of the page.

Real World HttpModule Examples

Back when I first discovered HttpHandlers I remember being ecstatic that the control that I thought I lost when moving from PHP to ASP.NET was finally returned, but when I first discovered HttpModules I remember almost passing out at the level of power and control you get over the ASP.NET request pipeline.  Since then, I've used HttpHandlers, HttpHandlerFactories, and HttpModules in many projects, but I'm noticing that while many people have heard of them, many have no idea what you would ever use them for.  So, I would like to give a few examples.

The first example is really simple.  On my blog, I didn't ant anyone to alias my web site or access it in any other way than by going to www.netfxharmonics.com.  Since HttpModules allow you to plug into the ASP.NET request pipeline, I was able to quick write an HttpModule to do exactly what I wanted:

public class FixDomainHttpModule : IHttpModule
{
    public void Dispose( ) {
    }

    public void Init(HttpApplication context) {
        context.BeginRequest += delegate(Object sender, EventArgs ea) {
            HttpApplication ha = sender as HttpApplication;
            String absoluteUrl = ha.Context.Request.Url.ToString( ).ToLower( );
            if (ha != null) {
                if (MinimaConfiguration.ForceSpecifiedDomain) {
                    if (!absoluteUrl.StartsWith(MinimaConfiguration.Domain.ToLower( ))) {
                        context.Response.Redirect(MinimaConfiguration.Domain);
                    }
                }
            }
        };
    }
}

...with this web.config:

<httpModules>
  <add name="FixDomainHttpModule" type="Minima.Web.HttpExtensions.FixDomainHttpModule" />
</httpModules>

By doing this I don't need to put a check in each page or in my MasterPage.  Since HttpModules are for the entire application, even URLs accessing my images are forced to be done by www.netfxharmonics.com.

Another example is a simple authentication system for a system I was working on a while back.  The application allowed anyone logged into active directory to access it's resources, but only certain people logged into active directory would be authorized to the use application (i.e. anyone could access images and CSS, but only a few people could use the system).  Knowing that the .NET framework is the model for all .NET development, I looked at the machine's web.config to see how ASP.NET implemented its windows and form authentication.  As it turns out, it does so by HttpModules.  So, I figured that the best way to solve this problem was by creating an HttpModule, not by throwing a hack into each of my WebForms or my MasterPages.  Furthermore, since ASP.NET uses the web.config for its configuration, including authentication configuration, I wanted to allow configuration of my authentication module to be via the web.config.  The general way I wanted to configure my HttpModule would be by a custom configuration section like this:

<Jampad>
    <Security RegistrationPage="~/Pages/Register.aspx" />
</Jampad>

The code for the HttpModule was extremely simple and required only a few minutes to throw together.  If the page being accessed is a WebForm and is not the RegistrationPage set in web.config, then the system's Person table is checked to see if the user logged into the machine has an account in the application.  If not, then there is a redirect to the RegistrationPage.  Simple.  Imagine how insane that would have been if you wanted to test for security on each page.

public class JampadSecurityModule : IHttpModule
{
    public void Dispose( ) {
    }

    public void Init(HttpApplication context) {
        context.BeginRequest += delegate(Object sender, EventArgs ea) {
            HttpApplication ha = sender as HttpApplication;

            if (ha != null) {
                CheckSecurity(context);
            }
        };
    }

    private void CheckSecurity(HttpApplication context) {
        SecurityConfigSection cs = (SecurityConfigSection)ConfigurationManager.GetSection("Jampad/Security");

        if (String.IsNullOrEmpty(cs.Security.RegistrationPage)) {
            throw new SecurityException("Security RegistrationPage is required.");
        }

        if (cs == null) {
            return;
        }

        if (!context.Request.Url.AbsoluteUri.Contains(cs.Security.RegistrationPage) &&
            context.Request.Url.AbsoluteUri.EndsWith(".aspx")
            ) {
            PersonCollection pc = new PersonCollection( );
            pc.GetMulti(new PredicateExpression(PersonFields.NTLogin==ActiveDirectoryFacade.NTUserName));

            if(pc.Count < 1){
                context.Response.Redirect(cs.Security.RegistrationPage);
            }
        }
    }
}

Again, plugging this into the web.config makes everything automatically happen:

<httpModules>
    <add name="JampadSecurityModule " type="Jampad.Security.JampadSecurityModule" />
</httpModules>

Recently I had to reuse my implementation in an environment that would not allow me to use LLBLGen Pro, so I had to rewrite the above 2 lines of LLBLGen Pro code into a series of Strongly-Typed DataSet tricks.  This implementation also had a LoginPage and an AccessDeniedPage in the custom configuration section, but other than that it was the same idea.  You could actually take the idea further by checking if the person is currently authenticated and if they aren't do a check on the Person table.  If they have access to the application, then PersonLastLoginTime column to the current time.  You could do many things with this implementation that would be rather crazy to do in a WebForm or MasterPage.

Another example of an HttpModule would be my custom access control system I built into my blog.  I'm not going to paste the code here as it's just the same idea as the other examples, but I will explain the concept.  Basically I created a series of tables in my blog's SQL Server database that held information on access control.  In the Access table I put columns such as AccessDirection (allow, deny), AccessType (IP Address, UserAgent, HTTP Referral), AccessText, AccessMessage, and AccessRedirect.  My HTTPModule would filter every ASP.NET request through this table to figure out what to do with it.  For example, I could block an IP address by creating a table record for 'deny', 'IP Address', '10.2.1.9', 'Your access has been denied.', NULL.  Immediately upon inserting the row, that IP address is blocked.  I could also block certain UserAgents (actually this was the original point of this HttpModule--bots these days have little respect for the robots.txt file).  I could also block requests that were from a different web site.  This would allow me to stop people from leaching images off my web site for use on their own.  With a simple HttpModule I was able to do all this in about an hour.  By the way, one record I was tempted to create was the following: 'deny', 'UserAgent', 'MSIE', NULL, 'http://www.getfirefox.com/'.  But I didn't :)

Now when would you use an HttpModule versus an HttpHandler?  Well, just think about what the difference is.  An HttpHandler handles a specific address pattern for a specific set of HTTP verbs, while every request in an application goes through an HttpModule.  So, if you wanted to have an image creator at /ImageCreator.imgx, then you need to register .imgx to IIS and then register your image creation HttpHandler in your web.config to handle that address (in case you forgot, web browsers care about the Content-Type, not the file extension.  In this example, your HttpHandler would set the Content-Type as 'image/png' or whatever your image type is.  That's how a web browser will know what to do with a file.  It has nothing to do with the file extensions; that's just for IIS.)  On the other hand, if you wanted to block all traffic from a specific web site, then you would create an HttpModule, becayse HttpModules handle all traffic on an application.  So, if you just remember this fundamental difference in purpose between the two, then shouldn't have my problems in the future.