Blog Archives

A question that I’ve been seeing more frequently these days is how to extract a JavaScript object from an HTML list or table, given no data or information other than the markup. It’s not ideal to work backwards from HTML, but sometimes you just don’t have a lot of choice in the matter.

Whether you’re enhancing legacy elements that have been generated on the server-side or want to parse the output of a third-party DHTML widget, there are a variety of situations where converting HTML to raw data is a legitimate need. You may have seen iterative solutions to this problem before. However, nested looping code gets messy fast, doesn’t feel much like idiomatic jQuery, and certainly isn’t as concise as you’d probably like.

Luckily, one of JavaScript’s lesser-known utility methods and jQuery’s implementation of it can improve the situation quite a bit. In this post, I’m going to show you how to use this method, jQuery’s cross-browser solution, and how to use it to extract data objects from arbitrary HTML lists and tables.

Array.map()

It turns out that there’s a tool perfectly suited to the task of coercing one data structure into another: map.

Map is a higher-order function that allows you to transform the contents of a collection by applying a function to each item, capturing the result, and building a new collection of those results.

Map is a perfect tool for translating a collection full of extraneous data into a tightly-focused collection of exactly the desired subset. Even better, JavaScript 1.6 even includes a native implementation of map, which is exposed as a method on the Array prototype.

For example, this is how you could use JavaScript 1.6?s Array.map() to analyze an array of strings and create a new array containing each string’s length:

var sites = ['Encosia', 'jQuery', 'ASP.NET', 'StackOverflow'];
 
// For each site in the array, apply this function 
//  and build an array of the results.
var lengths = sites.map(function(site, index) {
  // Use the length of each name as its value in the new array.
  return site.length;
});
 
// This outputs: [7, 6, 7, 13]
console.log(lengths);

This is a very simple example, but you can probably already imagine applying that same technique to an array of list elements or table rows. The concise expressiveness of the map approach is great for paring away extraneous markup and extracting just underlying data.

Mapping uncharted territory

Unfortunately, JavaScript 1.6 and its map implementation is not something that you can count on being available in older browsers. Notably, Internet Explorer doesn’t provide an Array.map() implementation until IE9.

Though that is disappointing, map isn’t difficult to manually implement. For example, this is a polyfill that the MDC recommends for patching Array.map() into older browsers:

if (!Array.prototype.map) {
  Array.prototype.map = function(fun /*, thisp */)  {
    "use strict";
 
    if (this === void 0 || this === null)
      throw new TypeError();
 
    var t = Object(this);
    var len = t.length >>> 0;
    if (typeof fun !== "function")
      throw new TypeError();
 
    var res = new Array(len);
    var thisp = arguments[1];
    for (var i = 0; i < len; i++)  {
      if (i in t)
        res[i] = fun.call(thisp, t[i], i, t);
    }
 
    return res;
  };
}

That’s a workable solution, but I doubt you’re very excited about the prospect of including all this code in your page. I know I wouldn’t be.

jQuery has you covered

If you’re already including jQuery in your pages, the good news is that jQuery has a built-in map implementation that works in every browser. In fact, jQuery provides two separate map methods: one that’s specially suited to working with jQuery selections and a general utility method that’s more similar to the polyfill shown above.

For working with HTML, I’m going to focus on using the former: .map().

To replicate the JavaScript 1.6 dependent example shown earlier, using jQuery’s implementation instead, the code would look like this:

var sites = ['Encosia', 'jQuery', 'ASP.NET', 'StackOverflow'];
 
// Same as before, using jQuery's map() implementation.
var lengths = $(sites).map(function(index, site) {
  // Use the length of each site name as its value in the new array.
  return site.length;
});
 
// This outputs: [7, 6, 7, 13]
console.log(lengths);

Making “this” approach more concise

To condense the code a bit, we can take advantage of the execution context within the callback function. During each callback, this holds the value of the array item currently being operated on. So, there’s no need to bother capturing the callback’s two input parameters:

var lengths = $(sites).map(function() {
  // "this" refers to the current array element as this callback is
  //  applied to each array element.
  return this.length;
});

That isn’t a huge improvement, but every little bit helps and I’ll be using this in the examples throughout the rest of this post. So, I wanted to make sure what’s happening there is clear.

Unwrapping the result of jQuery’s .map()

The one quirk when using jQuery’s .map() method is that it sometimes returns a jQuery wrapped set; specifically, when you apply it to the result of a jQuery DOM selection. Even if your mapping function returns scalar values like strings and numbers, the end result of .map() will include the jQuery object prototype on each element.

That isn’t really a problem if you only intend to use that result immediately in your JavaScript code. However, the jQuery object prototype hanging off each element throws a wrench in the works if you try to use JSON.stringify() on the result of .map(). Since JSON serialization is such a common task when storing or transmitting JavaScript data, this quirk turns out to be a real issue.

The solution is to call jQuery’s get() method on those wrapped-array results, which boils them down to plain arrays. When you see .get() tagged onto the end of the examples ahead, that’s why it’s there.

Now, let’s take a look at applying .map() to HTML and using it to extract data.

Mapping the data within HTML unordered lists

Using .map() against an unordered list is one of the most straightforward examples to start with. Imagine you had this simple HTML markup:

<ul>
  <li>Item 1</li>
  <li>Item 2</li>
  <li>Item 3</li>
</ul>

To extract each of those items’ displayed value, you could use .map() like this:

// Returns ['Item 1', 'Item 2', 'Item 3']
$('li').map(function() {
  // For each <li> in the list, return its inner text and let .map()
  //  build an array of those values.
  return $(this).text();
}).get();

Complicating things slightly, maybe the list items also have an HTML5 data- attribute that you need to collect in addition to their values:

<ul>
  <li data-id="123">Item 1</li>
  <li data-id="456">Item 2</li>
  <li data-id="789">Item 3</li>
</ul>

Using .map() to extract that more complex data is just as easy:

// Returns [{id: 123, text: 'Item 1'}, 
//          {id: 456, text: 'Item 2'},
//          {id: 789, text: 'Item 3'}]
$('li').map(function() {
  // $(this) is used more than once; cache it for performance.
  var $item = $(this);
 
  return {
    // Note: using .data() to read HTML5 data- attributes 
    //  requires jQuery 1.4.3+. Use attr() in older versions.
    id: $item.data('id'),
    text: $item.text()
  };
}).get();

As you can see, .map() is a powerful tool for concisely pulling arbitrary bits of data together into a useful structure. You could certainly do this with a temp variable and for-loop, but it’s hard to beat the clean expressiveness this approach lends your code.

Extracting data from HTML tables

Working with the lists is good for a simple example, but what if we need to apply this technique to an HTML structure that’s more complex than an unordered list?

HTML tables are one of the most common targets for this technique. It’s not unusual to end up with a pre-rendered table that was generated off-page and to desire a client-side data structure representing that table’s data.

For example, here’s a tabular representation of the same data contained in the second list example:

<table id="myTable">
  <thead>
    <tr>
      <th>id</th>
      <th>text</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>123</td>
      <td>Item 1</td>
    </tr>
    <tr>
      <td>456</td>
      <td>Item 2</td>
    </tr>
    <tr>
      <td>789</td>
      <td>Item 3</td>
    </tr>
  </tbody>
</table>

If you wanted to boil that table down to exactly the same JavaScript object shown in the second list example, this .map() usage would do the trick:

// Returns [{id: 123, text: 'Item 1'}, 
//          {id: 456, text: 'Item 2'},
//          {id: 789, text: 'Item 3'}]
$('#myTable tbody tr').map(function() {
  // $(this) is used more than once; cache it for performance.
  var $row = $(this);
 
  // For each row that's "mapped", return an object that
  //  describes the first and second <td> in the row.
  return {
    id: $row.find(':nth-child(1)').text(),
    text: $row.find(':nth-child(2)').text()
  };
}).get();

The key to making this approach work is using the :nth-child selector to index into each row and retrieve the contents of the cells we’re interested in. This is very similar to how we handled the unordered list earlier, but can be applied to arbitrarily large structures such as wide HTML tables.

If you use this approach, one thing to keep in mind is that :nth-child uses one-based indexing. So, you must use :nth-child(1) to select the first cell, not :nth-child(0) as you might expect.

A general solution for tables

Using hard coded :nth-child selectors works well enough in simple scenarios, but it’s brittle. If the table structure changes, relying on a certain table layout will break. Hard coding the selectors for each column also becomes tedious when dealing with wider tables that have many columns.

So, as you apply this technique to larger or less predictable tables, you may desire a more general solution for extracting the data. One way of doing that is using the table’s column heading cells to build a basic schema of the table’s data.

Assuming your table has a proper <thead>, this is how you could extract an array of its column headings to use as a schema for mapping the rest of the table’s data:

var columns = $('#myTable thead th').map(function() {
  // This assumes that your headings are suitable to be used as
  //  JavaScript object keys. If the headings contain characters 
  //  that would be invalid, such as spaces or dashes, you should
  //  use a regex here to strip those characters out.
  return $(this).text();
});

With that column list handy, we can determine which column name any cell in the table should be filed under, given nothing more than its index in the row. Now we can automate the process that previously required those :nth-child selectors:

var tableObject = $('#myTable tbody tr').map(function(i) {
  var row = {};
 
  // Find all of the table cells on this row.
  $(this).find('td').each(function(i) {
    // Determine the cell's column name by comparing its index
    //  within the row with the columns list we built previously.
    var rowName = columns[i];
 
    // Add a new property to the row object, using this cell's
    //  column name as the key and the cell's text as the value.
    row[rowName] = $(this).text();
  });
 
  // Finally, return the row's object representation, to be included
  //  in the array that $.map() ultimately returns.
  return row;
 
// Don't forget .get() to convert the jQuery set to a regular array.
}).get();

That’s it.

With all the comments, that looks like more work than it actually is. Eleven lines of code for the entire ordeal isn’t bad considering that it will automatically handle the majority of tables you throw at it.

Conclusion

I’m going to stop here, before this gets any longer. I hope that you found this helpful and/or interesting.

Even if you don’t often convert HTML markup to JavaScript objects, do keep .map() in mind when you’re working with collections of any type. When you need it, the notion of map is an extremely useful aspect of JavaScript’s functional nature, but often goes overlooked.

Related posts:

1. Use jQuery and quickSearch to interactively search any data
2. How to easily enhance your existing tables with simple CSS
3. In JavaScript, curly brace placement matters: An example

You’ve been reading Use jQuery to extract data from HTML lists and tables, originally posted at Encosia. I hope you enjoyed it, and thanks for reading.

If you’ve got any feedback, please click through and leave a comment; I’d love to hear from you. You can click here to jump directly to the comment section of this post.

In my last post about jQuery Templates, I showed you how to use template composition to build a template out of simple sub-templates. These composite templates are a great way to address the complexity that creeps into real-world UIs, as they inevitably grow and become more intricate. However, one feature missing from my last example was the ability to store those composite templates in external files and load them asynchronously for rendering.

I’ve described how to accomplish that with single templates in the past, using jQuery’s AJAX utilities and a particular usage of tmpl(). Unfortunately, remotely loading a group of composite templates from a single file is not quite as simple, and the technique I’ve described previously will not work.

Not to worry though, it’s still relatively easy.

In this post, I’ll show you how to move a group of composite templates to an external file, how to load and render them with jQuery Templates, and how to take advantage of an expected benefit to improve separation of concerns.

Caution: If you haven’t read my previous posts about remotely loading jQuery Templates definitions and using {{tmpl}} to achieve template composition, read them before continuing with this post. I’m not going to cover that material again here, and this may not make much sense without those prerequisites.

Moving the templates to an external file

Breaking the invoice template apart helped make it more approachable and maintainable, but I don’t like leaving the template embedded in the page’s markup. The larger a single file becomes, the more difficult it is to understand and work with – especially over time.

Disentangling chunks of the presentation tier and moving them to separate files is a great way to attack the problem of bloated pages and views. We’ve been doing that since the beginning of the web, from seemingly-ancient techniques like SSI includes, to file includes in scripting frameworks like ASP and PHP, to partial views in MVC frameworks. So, why stop now just because the templates are rendered in the browser?

Moving the previous example’s template definitions to a separate file is as simple as it sounds. Just take the invoice template and both its row templates, script wrappers included, and move them into a new file of your choosing. I’m going to move them to a file named _invoice.tmpl.htm:

<!-- Tip: It's safe to use HTML comments in the file -->
 
<!-- Invoice container template -->
<script id="invoiceTemplate" type="x-jquery-tmpl">
  <table class="invoice">
  {{each lineItems}}
    {{tmpl($value) get_invoiceRowTemplateName(type)}}
  {{/each}}
  </table>
</script>
 
<!-- Invoice row templates -->
<script id="serviceRowTemplate" type="x-jquery-tmpl">
  <tr class="service">
    <td colspan="2">${service}</td>
    <td colspan="2">${price}</td>
  </tr>
</script>
 
<script id="itemRowTemplate" type="x-jquery-tmpl">
  <tr class="item">
    <td>${item}</td>
    <td>${description}</td>
    <td>${price}</td>
    <td>${qty}</td>
  </tr>
</script>

Naming the template file

If you’re wondering why I chose that somewhat convoluted filename for the template, I’ll explain:

Ideally, I would like to call the template something like invoiceTemplate.tmpl, but most popular web servers refuse to serve files with non-standard extensions by default. You can circumvent that with a bit of manual configuration, but it’s not worth the unending hassle of extra configuration work on every server and/or site where you use this technique. So, .tmpl is out.

I really liked Nathan Smith’s suggestion for a naming compromise on my previous post about remote loading, which boils down to this:

• Prefix the filename with an underscore. This denotes a partial view in many modern view engines and is a useful convention for indicating that the file is not a valid/complete HTML document.
• Nathan suggested following _templateName with .tpl, to indicate that it’s a template. I’m going to tweak that slightly and use .tmpl, so it’s more clear that the template is intended for use with jQuery Templates.
• The file should end in .htm, to be sure the file will be readily served up under almost any web server’s default configuration. Using .htm or another text/html extension also improves the odds that the template definitions will be served with appropriate compression and caching.

The first two are just suggestions, of course. You can use any arbitrary naming scheme you prefer and the approach described in this post will still work fine.

I do recommend sticking with an .htm or .html extension though. I (stubbornly) went through the configuration hassle of using .tpl when I was originally working with jTemplates’ remote loading feature a couple years ago, and eventually had to give up on it. The ongoing configuration hassles ultimately outweighed the benefit of a uniquely descriptive extension.

Loading and rendering the template

With the templates moved to an external file, we need a way to load and render them. As I mentioned earlier, the approach I’ve previously described for remote template loading isn’t viable in this scenario. Treating the external file as a simple template string only works if the file contains a single template definition and no extraneous markup. Our file fails on both counts.

Another disadvantage my previous approach is caching. When you render a string-based template, jQuery Templates doesn’t cache the compiled template. If you end up rendering the same template more than once per pageview, the string-based approach was slower than it could have been.

To solve both those issues, we can use jQuery to load the contents of the template file, inject all of it into the document, and then work with the templates as if they had been embedded in the page all along.

// The invoice object to render (see previous post).
var invoice = {
  invoiceItems: [
    { type: 'item',
      part: '99Designs', description: '99 Designs Logo',
      price: 450.00, qty: 1 },
    { type: 'service',
      service: 'Web development and testing',
      price: 25000.00 },
    { type: 'item',
      part: 'LinodeMonthly', description: 'Monthly site hosting',
      price: 40.00, qty: 12 }
  ]
};
 
// Asynchronously load the template definition file.
$.get('_invoice.tmpl.htm', function(templates) {
  // Inject all those templates at the end of the document.
  $('body').append(templates);
 
  // Select the newly injected invoiceTemplate and use it
  //  render the invoice data.
  $('#invoiceTemplate').tmpl(invoice).appendTo('body');
});

Injecting the template definitions into the page’s markup allows you to render them with the same $('#templateId').tmpl(data) syntax that you’ve seen in most jQuery Templates examples.

Bringing the row template resolver along for the ride

One remaining annoyance is the now-orphaned row template name resolver, get_invoiceRowTemplateName() (covered in my composite template post). Keeping that function with the rest of the page’s JavaScript does work, but I’m not happy with it. To get the full benefit of encapsulating the template in an external file, we should keep any dependent JavaScript code with the template itself.

That’s especially true if the template is used from more than one page.

As it turns out, accomplishing that is much easier than it may seem at first. All we need to do is move the resolver function right into _invoice.tmpl.htm:

<!-- Invoice container template -->
<script id="invoiceTemplate" type="x-jquery-tmpl">
  <table class="invoice">
  {{each lineItems}}
    {{tmpl($value) get_invoiceRowTemplateName(type)}}
  {{/each}}
  </table>
</script>
 
<!-- Invoice row templates -->
<script id="serviceRowTemplate" type="x-jquery-tmpl">
  <tr class="service">
    <td colspan="2">${service}</td>
    <td colspan="2">${price}</td>
  </tr>
</script>
 
<script id="itemRowTemplate" type="x-jquery-tmpl">
  <tr class="item">
    <td>${item}</td>
    <td>${description}</td>
    <td>${price}</td>
    <td>${qty}</td>
  </tr>
</script>
 
<!-- This function is used by the invoice container to determine -->
<!--  which row template to render for a given line item -->
<script type="text/javascript">
  function get_invoiceRowTemplateName(type) {
    // Return a template selector that matches our
    //  convention of <type>RowTemplate.
    return '#' + type + 'RowTemplate';
  }
</script>

When you inject markup into the DOM, browsers will immediately parse and evaluate that markup just as they do during the initial page load – including any JavaScript code in the markup. Since browsers do this in the same single-thread that they execute JavaScript, it’s safe to assume that embedded functions are available immediately after the markup has been injected.

It seems almost too good to be true, like one of those handy techniques that works in every browser except some version of IE. However, embedding the supporting function inside the template file works in every browser I’ve tested, even including the full complement of JavaScript-enabled options on BrowserShots.org.

Conclusion

I’ve been using this exact technique in production for about a month now. I initially came up with several approaches, but wanted to try them in real-world code before I recommended one. I’m happy to report that this has been working great for me so far. I hope you’ll find it helpful too.

One caveat is that you should be careful about loading and injecting the external template more than once. Injecting multiple copies of the template definitions over and over actually doesn’t break the rendering process in browsers I’ve tested, but it will unecessarily impact performance as the DOM grows larger with every new injection.

I’m going to cover that and another templating concurrency issue soon, but if you want a head start: test $.template('#templateName') to determine whether a particular template is already loaded and cached.

You’ve been reading jQuery Templates, composite rendering, and remote loading, originally posted at Encosia. I hope you enjoyed it, and thanks for reading.

Related posts:

1. Using external templates with jQuery Templates
2. Composition with jQuery Templates: Why and How
3. A few thoughts on jQuery templating with jQuery.tmpl

Now that jQuery Templates is official and definitely will not include remote template loading, I wanted to publish a quick guide on implementing that yourself. As I mentioned previously, there’s a handy interaction between jQuery Templates’ API and jQuery’s AJAX methods that makes this easier than you might expect.

In this post, I’ll show you how to use a plain string as a template, how to asynchronously load an external template file as a string, and how to render it with jQuery Templates once it’s loaded.

Defining an external template

Defining a template suitable for remote loading requires almost no extra effort. Simply create a new file and fill it with the same sort of jQuery Templates fragment that you might normally embed in a text/html <script> element.

For example, given this person object:

var person = { name: 'Dave' };

You might define this template for that data in a file called PersonTemplate.htm:

<p>Hello, ${name}.</p>

Be sure not to wrap the template in a <script> element. Since we’ll be loading it remotely, that obscuring measure isn’t necessary. Also be sure not to include any of the HTML boilerplate that may come along with creating a new HTML file in your editor of choice, like <html> or <body>.

An added bonus that comes along with dropping the <script> container is that you get more reliable syntax highlighting for the template’s markup. Most editors don’t provide HTML syntax highlighting within <script> elements, even when their type is text/html, but work fine in the external files.

Using strings as templates

Though most jQuery Templates examples revolve around referencing template definitions hidden inside <script> elements, it’s also possible to provide the template as a plain string. Using the $.tmpl(template, data) syntax, a simple string version of the template may be provided as that first parameter.

For example, these are both perfectly valid ways to render a template, using the person object from the previous section:

// Specifying the template inline.
$.tmpl('<p>Hello, ${name}.</p>', person);
 
// Assign that same template to a JavaScript variable, and then
//  use that string value as a template parameter to $.tmpl().
var salutationTmpl = '<p>Hello, ${name}.</p>';
 
$.tmpl(salutationTmpl, person);

With that ability to use a string as a template, we just need a way to get the external template file’s content loaded into a string variable.

Loading the external template

With that in mind, probably the easiest way to load an external template is to use jQuery’s $.get() method. When you target $.get() at a static HTML file, the result passed to its callback is a string containing the file’s content. Exactly what we’re after.

For example, if an external template were stored in a file named PersonTemplate.htm, this is how you could use $.get() to load and render that external template with jQuery Templates:

// Asynchronously our PersonTemplate's content.
$.get('PersonTemplate.htm', function(template) {
 
  // Use that stringified template with $.tmpl() and 
  //  inject the rendered result into the body.
  $.tmpl(template, person).appendTo('body');
});

That’s it.

Because you have to get involved with more of the moving parts, this is slightly more complex than jTemplates’ processTemplateURL, but it’s very manageable.

Caveats

Compilation caching – When you use this method, it’s important to keep in mind that jQuery Templates will not automatically cache the compiled template. If you intend to reuse the same template many times during a given pageview, this approach is slower than the embedded <script> technique.

It’s possible to work around that drawback by using $.template() to compile the remotely loaded template and store it manually, but I’m omitting that here to keep things simple.

If you’re interested, I’ll cover that in follow up post.

HTTP caching – In production, you should ensure that your server’s caching configuration is correct. If your server sends a proper expires header, requesting the same template multiple times during the lifespan of a page will only cause the browser to make one HTTP request. Even subsequent HTTP requests to check for a 304 status aren’t necessary if a future expires header is set on the response.

Template file naming – I recommend using a standard filename extension for your external template files, like .htm or .txt. I started out using more descriptive extensions like .tpl, but server configuration was a constant hassle. Important things like HTTP compression, expires headers, and even serving the files at all are common issues when using nonstandard extensions.

You’ve been reading Using external templates with jQuery Templates, originally posted at Encosia. I hope you enjoyed it, and thanks for reading.

Related posts:

1. A few thoughts on jQuery templating with jQuery.tmpl
2. Use jQuery and ASP.NET AJAX to build a client side Repeater
3. How you can force the Ajax Script Loader to use jQuery 1.4