2005 2006 2007 2008 2009 2010 2011 2015 2016 2017 2018 aspnet azure csharp debugging docker elasticsearch exceptions firefox javascriptajax linux llblgen mongodb powershell projects python security services silverlight training videos wcf wpf xag xhtmlcss

ESV Bible Web Service Client for .NET 3.5

A while back, the guys over at the ESV Bible web site announced their new REST-based interface to replace their old SOAP interface.  This new interface provides the same functionality as the old, but allows for 5,000 queries per day instead of 500 previously and is based on REST architectural principles.  Because the service is fundamentally chatty, it made sense to switch to REST.  In the context of a Bible web service, it's hard to justify a 200-byte XML message when your actual request is 6 bytes ("John 1").  Also, because the method call is in the URI, the entire call is simplified all the more.

For those of you who are completely unfamiliar with REST interfaces, all you really need to know is that it's a resource (or noun) based architecture.  That is to say instead of calling, for example, a "GetItem" method, you simply access an "item" entity.  You access what the thing is, not what the thing does; kind of a web-based reversal of encapsulation.  In other words, instead of giving a server a command (a verb), you are accessing the resource directly (a noun).  There's obviously more to REST than this and you can get more information from this nice article titled "Building Web Services the REST Way".

RESTful architecture really is a nice way to telling a system what you want, not how to get it.  This is really the point of framework design and abstraction in general.  In light of this it's obvious to see that, as awesome as REST is, it's not how .NET developers want to think when working working on a project.  When I'm working with something I want to focus on the object at hand, not on the URLs and parameters.  For this reason, I built a .NET 3.5 framework that allows easy and efficient access to the new ESV Bible REST web service.  Here are some samples of how to use it:

Here's a simple passage query returning HTML data:

ESVBibleServiceV2 service = new ESVBibleServiceV2( );
String output = service.PassageQuery("Galatians 3:11");

With the flip of a switch you can turn it into plain text:

ESVBibleServiceV2 service = new ESVBibleServiceV2(OutputFormat.PlainText);
String output = service.PassageQuery("Galatians 3:11");

For more flexibility, you may use the provided parameter objects.  Using these in C# 3.0 is seamless thanks to object initializers:

PassageQueryParameters pqp = new PassageQueryParameters( ) { Passage = "John 14:6" };
ESVBibleServiceV2 service = new ESVBibleServiceV2(new PlainTextSettings( )
    LineLength = 100,
    Timeout = 30
String output = service.PassageQuery(pqp);

Here is a simple sample of accessing the verse of the day (in HTML without the audio link -- optional settings):

ESVBibleServiceV2 service = new ESVBibleServiceV2(new HtmlOutputSettings( )
    IncludeAudioLink = false
String output = service.DailyVerse( );

You can also access various reading plans via the provided .NET enumeration:

ESVBibleServiceV2 service = new ESVBibleServiceV2( );
String output = service.ReadingPlanQuery(new ReadingPlanQueryParameters( )
    ReadingPlan = ReadingPlan.EveryDayInTheWord

Searching is also streamlined:

ESVBibleServiceV2 service = new ESVBibleServiceV2( );
String output = service.Query("Justified");

Here is a length example showing how you can use the QueryInfoAsObject method to get information about a query as a strongly-type object:

ESVBibleServiceV2 service = new ESVBibleServiceV2( );
QueryInfoData result = service.QueryInfoAsObject("Samuel");

if (result.QueryType == QueryType.Passage) {
    Console.WriteLine("Passage: " + result.Readable);
    Console.WriteLine("Complete Chapter?: " + result.IsCompleteChapter);
    if (result.AlternateQueryType != QueryType.None) {
        Console.WriteLine(String.Format("Alternate: , ", result.AlternateQueryType, result.AlternateResultCount));

if (result.HasWarnings) {
    foreach (Warning w in result.Warnings) {
        Console.WriteLine(String.Format(": ", w.Code, w.Readable));

Here is the output:

QueryInfoAsObject Example Output

For more advanced users, the Crossway XML format is also available:

ESVBibleServiceV2 service = new ESVBibleServiceV2(new CrosswayXmlVersion10Settings( )
    IncludeWordIds = true,
    IncludeXmlDeclaration = true
String output = service.PassageQuery(new PassageQueryParameters( )
    Passage = "Galatians 3"

That same XML data is also retrievable as an XmlDocument for pure XML interaction:

ESVBibleServiceV2 service = new ESVBibleServiceV2( );
XmlDocument output = service.PassageQueryAsXmlDocument("Galatians 3");

For more flexible XML interaction, you may use XPath:

ESVBibleServiceV2 service = new ESVBibleServiceV2( );

String output = service.PassageQueryValueViaXPath(new PassageQueryParameters( )
    Passage = "Gal 3:4-5",
    XPath = "//crossway-bible/passage/surrounding-chapters/current"

Sometimes, however, you will want more than one result from XPath:

String[] output = service.PassageQueryValueViaXPathMulti(new PassageQueryParameters( )
    Passage = "Gal 3:4-5",
    XPathSet = new[]

Here's what the result looks like the debugger:

XPathSet Example Output

I've also copied the documentation for functions and parameters into the .NET XML comments, so you can quickly and easily see what a certain function or parameter does and it's default:


The new API uses your existing ESV Bible Web Service access key.  To use this key in this framework you simply add an element called ESVBibleServiceKey to the addSettings in your configuration file (a sample is provided with the framework).  You may also set it in any one of the parameter objects (i.e. PassageQueryParameters, QueryParameters, etc...), which override the key in the configuration file.  Per the API, you can use TEST for testing and IP for for general purpose queries.

Lastly, I would like to mention that this framework minimizes traffic by only sending options that deviate from defaults. So, for example, if you set IncludeWordIds to false and IncludeXmlDeclaration to true, only the IncludeXmlDeclaration value will be sent over the wire since IncludeWordIds is false by default.

You can access this ESV Bible Web Service 2.0 framework on CodePlex at the address in the links section.  Enjoy!


XmlHttp Service Interop - Part 3 (XML Serialization)

In this third part of the series, I discuss how to manually create an XML service and a SOAP service using XML serialization to maximize the flexibility in your service interop work. The technique allows you to very quickly accept just about any XML format from the wire and work with it in a strongly typed manner in your .NET applications.

XmlHttp Service Interop - Part 2 (Utilizing WCF)

Today I'm publishing the second in an article series regarding XmlHttp Service Interop. In this article, entitled "Utilizing WCF", I explain how to build WCF services, how to trace their XML messages, and how to communicate with them and other SOAP services via raw XmlHttp.

Instead of posting the entire thing on my blog, from here on out I'm going to be posting articles in their own pages to make them easier to read and easier to access. This will also make it easier for me to update and edit as well. You may access this article as well as others in the series by the links below.

XmlHttp Service Interop - Part 1 (Simple Service Creation)

This entry is the first is a series on XmlHttp Service Interop.

In my day job I am constantly making diverse systems communicate. I make classic ASP talk to the .NET 2.0 Framework, .NET systems talk with Java systems, a Pascal-based system communicate with a custom .NET mail server, and even make .NET components access systems from the stone age. I love interop, but I'm finding that many people don't. From what I see, it seems to be a lack of understanding more so than a lack of desire. It's actually some pretty cool stuff and you can find great books on various interop topics.

While I do work a lot with COM, ES, and web service interop, my favorite communication mechanism is via straight XmlHttp calls. It's just so easy to do and support for it is just about universal. You can take JavaScript and make a call to ASP.NET, go to a VBS WScript and make a call to a web service, or force the oldest, nastiest product in your company to communicate with WS-* services. In this part of the series, we are going to discuss XmlHttp in general and see a call from JavaScript to a manually created service endpoint.

To start off with lets make it clear what we are and are not talking about. We are not talking about sockets or direct TCP calls nor are we talking about a service framework. XmlHttp is a way to transfer XML over HTTP. However, even though we talk about XML, in your using XmlHttp you'll find that this isn't a requirement at all, so, at root what we are doing is making simple HTTP calls.

To see what we're talking about in action, lets create a simple XML service endpoint that accepts a well-defined XML format to allow the sending of e-mail. Then, lets access the service via JavaScript. This is very similar to something I recently created to allow a very old system to send e-mails using the .NET Framework. Of course, in that situation I used the programming language of the system in question (Pascal), not JavaScript.

To begin with lets create the client. I know it seems a bit backwards, but lets look at this from the standpoint of a framework designer: look at how it will be used first, then implement mechanics. Now, the first thing we need to do this is a simple "htm" document. I want the page to be "htm" for the sake of this demonstration, simply to show that there is no server-side processing at all in this page.

Next, we need a way to access our endpoint. I'm not going to get into severe detail about how to do this in every single browser in the world, but, rather, I'm only going to show the standardized way. You can quickly do a search online to see how to extent this behavior to IE5 and IE6.

Skipping the lame setup and stuff many 6th graders can do, let's get right to the core of what we are going to do. The full implementation of everything seen here is in an accompanying VS2005 solution. It would probably be a good idea to have that open as you go through this.

To send a request to a server, simply use syntax similar to the following:

var xmlhttp = new XMLHttpRequest( ); xmlhttp.open('POST', ' Service.aspx', true); xmlhttp.onreadystatechange = function ( ) { if(xmlhttp.readyState == 4) { alert(xmlhttp.responseText); } }; xmlhttp.send(data);

This syntax works in any version of Firefox, the newer versions of Opera, and IE7 (or what I like to call " IE6.5"). Basically, what's happening here is this: you are creating a new instance of an HTTP requestor, giving it some connection information, setting a callback function, and sending some data to the service.

The part that you should look at closely is the XMLHttpRequest::open (note: my double colon syntax is from C++ and simply means Class::Member)function. This obviously takes three parameters: the HTTP method, the HTTP endpoint, and a boolean stating asynchronous communication. I want this to be an asynchronous call, so I'm setting the third parameter to true. I'll come back to the HTTP method in a moment and the HTTP endpoint is just the service address.

After that we see that the property XMLHttpRequest::onreadystatechange is being assigned a JavaScript anonymous function. If you are unfamiliar with these, just think of them as anonymous delegates in C# 2.0. This is the function that's going to be called when the state of the XmlHttp call changed. When inside this function, there are a few parameters you can look at when this function gets calls, but here I'm only testing one: readyState. This property basically states the status of the call. Notice the XMLHttpRequest property is called "onreadystatechange", not "oncomplete". This function is actually called when the state of the HTTP request changes. When I test for readyState == 4 I'm looking for round trip completion. Frankly, you probably never touch the values 1, 2, and 3 though you could check for 0, which means that the XMLHttpRequest::open function has not yet been called. In this situation, if the readyState is 4, then I want to display a message box showing the response content, which is accessible via XMLHttpRequest::responseText. One other very important property you will definately be using a lot is the XMLHttpRequest::status. This property gives values like 404, 415, 500, and so on. If the request did a successful round trip the status will be 200, so that's something you'll probably be testing for quite a bit.

Finally we see the XMLHttpRequest::send method. This simply sends a set of data to the service... well, kind of. In the XMLHttpRequest::open, the first parameter, the HTTP method, is very important. Depending on what you want to do you will either set it to GET or POST. If you are calling a pre-existing page that has no idea what a HTTP stream is, but knows about querystrings, then you will want to use GET. In this situation, you will want to put parameters in the querystring in the HTTP end point, that is, in the second parameter of XMLHttpRequest::open. However, if you are creating your own service, you may want to use POST instead as using POST makes the development on both the client and service simplier. On the client, you don't pack stuff in the querystring (though you still could) and on the server, you can access the data via a stream rather than via parsing the URL or doing any iteration. As that last sentence implies, by using the POST method you send the data you want to submit to the HTTP endpoint as a parameter of the XMLHttpRequest::send function. For those of you who understand WCF terminology, you can think of the HTTP method as being analogous to the WCF binding and the HTTP endpoint as being analogous to the WCF address. The only thing analogous to the WCF contract is the XML schema you use to create your data stream.

Now, since we are sending the information in the data variable to the service, we need to put something in it. For this service, I'm using the following XML, though it doesn't have to be XML at all.

var data = ''; data += '<Mail>'; data += '<ToAddresses>'; data += '<ToAddress>johndoe@tempuri.org</ToAddress>'; data += '</ToAddresses>'; data += '<CcAddresses>'; data += '</CcAddresses>'; data += '<BccAddresses>'; data += '</BccAddresses>'; data += '<FromAddress>no-reply@tempuri.org</FromAddress>'; data += '<Subject>XmlHttp Service Interop - Part 1</Subject>'; data += '<DateTime>03-08-07 2:26PM'; data += '</DateTime>'; data += '<Body>This is the message body.</Body>'; data += '</Mail>';

Given proper event management in JavaScript, you have everything you need to have a fully functional client. Now onto the server.

As we saw when we looked at the client code, the service endpoint is Service.aspx . To help us focus on the task at hand, we aren't going to do anything fancy like using URL aliasing (aka URL rewriting) to make it look cooler, though in reality you would probably do that.

In the code behind for the Service.aspx, we have code that starts like this:

XmlDocument doc = new XmlDocument( ); doc.Load(Request.InputStream); XmlNode documentRoot = doc.DocumentElement; XmlNode mailRoot = documentRoot.SelectSingleNode("//Mail");

Here what we're doing is creating a new XmlDocument and loading the data streamed from the client into it. Then we are getting the root of the document via XPath. The rest of the code is in the accompanying project and simply consists of a bunch of XPath queries to get the information from the document.

After we found all the values we needed in the XML document, either via XPath or another mechanism, we can do whatever we want with what we found. The important point is this: all we have to do is a simple Response.Write ("") to send data sent back to the client, which in turn changes the readyState in the previously seen JavaScript to 4, thus allowing the client to display the output in the alert window. It's really just as simple as this: the client sends stuff to the service, the service does something and sends stuff back.

Now, we could beef this up a bit by adding some HTTP headers. This is something you may find yourself doing often. To do this, use XMLHttpRequest::setRequestHeader to set a key/value pair on the connection. Here's an example.

xmlhttp.setRequestHeader('X-App-Source', 'My Client');

That 'X-App-Source' was completely made up. You could use 'My Awesome Service Caller' if you wanted. That doesn't matter, what does matter however is that you put this after the call to XMLHttpRequest::open or else you will seriously want to throw something across the room, because it's a painfully subtle error that will cause the call will fail every time.

On the server side, to access a header, simply do this:

String xAppSource = Request.Headers["X-App-Source"];

I know. You were expecting something a bit more profound. Okay, I can satisfy that need. If you have many headers and you want them all, here's what you can do.

foreach (String header in Request.Headers) { // Do whatever you want... }

Whatever you do, try to fight the temptation to do this:

Dictionary<String, String> headers = new Dictionary<String, String>( );
foreach (String header in Request.Headers) {
    headers.Add(header, Request.Headers [header]);

As nice as that looks, if you really want to have a headers key/value pair collection, you can just do this:

NameValueCollection headers = Request.Headers;

Regardless of what you do with the headers, remember that they are there for a reason and if you do a lot of service calls, you will find yourself using HTTP headers a lot. This is something you will see in the next part of the series.

So, that's a quick overview of XmlHttp. Please see the solution provided with this post for the full code. The next part of the series discusses making manual XmlHttp calls to WCF.


Troelsen's COM and .NET Interoperability Book

Now this is cool... Andrew Troelsen's book "COM and .NET Interoperability" is FREE in e-book format at apress.com (click on Free E-Books under one of the headings). I highly recommend this book to anyone who wants to either continue to use their current COM components in the .NET world or for people who want to use their .NET abilities in COM-based technologies.

Many times I hear people say they would have to need for this. Well, let me give you some ideas of what I do at the office. Every now and again the web developer at the office will get overloaded with work and it will bubble up to the architect (me!) Well... the older websites are all in classic ASP and while I spend my share of time in the trenches with classic web application development, there is NO way I'm going to do any VBScript development. (heck, even back then I did most of my work in PerlScript!)

My solution is simple: .NET to COM interop. Most of my work is done in .NET and I just expose the interfaces to COM. You really think I'm going to consume a web service with manual XML structures in VBScript? Uh... think again.

By the way, for those web services I consume and expose to COM I wrote my own web service consuming framework (because so many people expose COMPLETELY INVALID services to the world in pseudo-"SOAP" that .NET won't get near it). I then expose these new "services" to the COM world via component services. Oh yes, the book has an entire chapter just on serviced components (COM+ Interop).


OK, so this week I found out that that there currently is no support the the Jet database provider on the x64 platform. That didn't change the fact that I was determined to do it (or create the illusion of it). I had to create a user-approachable report generator which exported the data into Excel and sent it on to the user all via a web interface. I also didn't want to use the XML version of Excel spreadsheets (it surprises me how many times people think I'm talking about Excel 12 when I say that. You can do XML-based sheets in old Excel versions too!) So I wanted this done via ADO.NET and the Jet provider creating a "real" xls file. Obviously what I wanted to do isn't even work with the framework, but...doing it on x64 is.

The solution? Simple, WSE3 provides you with the ability to use SOAP Message Transmission Optimization Mechanism (MTOM) very seamlessly. MTOM is a new W3C recommendation used to optimize messaging scenarios that involve transmitting binary data. Technical gibberish aside, MTOM is just AWESOME. Using WSE3/MTOM, all I had to do was create a WSE3 service on a different server which created the XLS binary data and returned to back to the caller which then sent it on to the client.

Want specifics? MSDN has a document which provides you with the hands-on training you sufficient to propel you into WSE3 services: WSE3 Hands-on Lab: Messaging. MTOM is the third lab, but basically all you do is enable WSE3 (and MTOM) on the client and server, set the server up to return a byte array to the caller, call the server, and stream that byte array into a file. In my case I then sent it out to the user...

Here's my code for sending the Excel worksheet out to the user...

Response.Buffer = true;
Response.Clear( );
Response.ClearContent( );
Response.ClearHeaders( );

Response.AddHeader("Content-Disposition", "attachment; filename=" +
Response.AddHeader("Content-Length", file.Length.ToString( ));
Response.ContentType = "application/vnd.ms-excel";
Response.Flush( );

I'll have to beef up on WSE3 a bit for a future SOA lecture in my WinFX/.NET course.