Lift Weekly Recap 6: Speed, data, and elementals

Right on schedule, today we’ll be chatting about some functionality that landed in Lift 2.6 recently, as well as a couple of the most delightful changes in Lift 3… Brace yourselves, this is going to be an awesome recap. Sunglasses on.

As an ongoing rule, if you have any ideas for things that might be mentioned in something like this, drop me a line at savedfastcool AT gmail.com .

Didn’t you get the memo? #

A couple of weeks ago, an intrepid chriswebster found some inefficiencies in the way that lift-json was doing its decomposition of case classes into JValues (the step before a serialized JSON string for lift-json) and offered up a fix for it. PR 1517 memoizes the list of fields for a given class, so that getDeclaredFields need only be called once for a given class. Rather coincidentally, this past week I had to deal with a big performance bottleneck related to the same issue of repeatedly calling getDeclaredFields. The moral of the story: that method is slow.

The quick benchmark mentioned in the associated mailing list discussion indicates a >2x speedup in serialization operations from this fix, which is pretty awesome! This fix went into 2.6 snapshot builds about two weeks ago.

And with that, let’s take a step in our time warp and look at some new features in Lift 3 that landed a good while back.

Ode to data- #

Lift 2.3-M1 introduced the ability to invoke Lift snippets from the class attribute in HTML. This meant that code that used to look like:

<lift:my-snippet>
  <ul class="items">
    <li class="item">Item 1</li>
    ...
  </ul>
</lift:my-snippet>

Could now be:

  <ul class="lift:my-snippet items">
    <li class="item">Item 1</li>
    ...
  </ul>

Then, in Lift 2.4, this ability was expanded to an HTML5-friendly ability to use the data-lift attribute instead, thus:

  <ul data-lift="my-snippet" class="items">
    <li class="item">Item 1</li>
    ...
  </ul>

Lift 3 takes this functionality one step further, and allows you to define arbitrary data- attributes that will run particular processing in Lift. You can set these up much like you add custom snippet invocations, using LiftRules.dataAttributeProcessor:

LiftRules.dataAttributeProcessor.append {
  case (attribute, value, element, session) =>
    // do stuff
}

The above stub shows the four values that a data attribute processor is passed:

• The name of the attribute, without the data- prefix.
• The value of the attribute (note: HTML5 attributes that don’t need a value, such as checked in <input checked>, turn into an empty String as the value here).
• The element that the data- attribute was on, without the data- attribute that triggered this test.
• The LiftSession that everything is being processed in.

Here’s an example of the kind of thing you could do:

object currentUser extends SessionVar[Box[User]](Empty)

LiftRules.dataAttributeProcessor.append {
  case ("user-name", _, element, _) =>
    ("^" #> currentUser.is.map { user =>
      "^ *" #> user.name
    }) apply element
}

This could take:

  <a href="/session" data-user-name>My User</a>

Then, when the user is logged out, the a element will simply go away. When the user is logged in, the user’s name will be substituted in.

Two important things here:

• You can use SessionVars from an attribute processor as usual.
• The usual pattern-matching rules apply, so the above processor won’t run if data-user-last-name is present on the element.

Also worth noting, the resulting element will never have the matched data- attribute unless you explicitly add it back.

You can also use this as a good substitute for snippets that require a single value. For example, where before you might do this:

<script type="text/javascript" data-lift="script-bundle?name=base"></script>

You could now do:

<script type="text/javascript" data-script-bundle="base"></script>

And add something like:

LiftRules.dataAttributeProcessor.append {
  case ("script-bundle", bundleName, element, _) =>
    ("^ [data-lift]" #> s"script-bundle?name=$bundleName")  apply element
}

data- attributes are processed recursively, so you can attach multiple data- attributes to a single node and have them all process. If we keep the previous processor in place, we can also do:

<script type="text/javascript" data-user-script-bundle></script>

And add:

LiftRules.dataAttributeProcessor.append {
  case ("user-script-bundle", _, element, _) =>
    ("^" #> currentUser.is.map {
      "^ [data-script-bundle]" #> user.username
    }) apply element
}

If the user is logged in, this would add a bundle to the page named by the user’s username. The key here is that first the user-script-bundle handler would run, and if the user is logged in would leave a data-script-bundle attribute in the element. Then, the data-script-bundle handler would run, and leave a data-lift attribute in the element. Finally, the default data-lift handler would run, running the script-bundle snippet which would have been set up elsewhere in the code.

DataAttributeProcessorAnswer #

There’s one thing we’ve glossed over above, and that’s that what the processor partial function returns is actually an instance DataAttributeProcessorAnswer. There are a few implicit conversions to this trait, and we’ve been making use of the one that lets us just return a regular NodeSeq and have it dropped in. However, we also have two other powerful things that we can return: an LAFuture and a function that produces a NodeSeq. Both of these are run in a background thread in parallel to the main rendering, and their results are joined in before the page is sent down to the client.

For example, say a page wanted to provide both a user’s Facebook profile picture and the profile picture of a Facebook page. These are two disparate requests to Facebook’s API, and the most efficient way to do this might be in two separate threads, and leave our own interaction with our database in the main render thread. We could then do something like this:

  <section class="profile">
    <img data-fb-picture="https://graph.facebook.com/me/picture">
  </section>
  <section class="page">
    <img data-fb-picture="https://graph.facebook.com/theSoundTable/picture">
  </section>

And then in an attribute processor:

LiftRules.dataAttributeProcessor.append {
  case ("fb-picture", pictureEndpoint, element, _) =>
    val future = new LAFuture[String]()

    // do a background HTTP call on pictureEndpoint that runs:
    future.satisfy {
      ("img [src]" #> fetchedUri) apply element
    }

    future
}

Having to manage that future manually is kind of lame, so we can actually do something even cleaner if we’re not using an API that already speaks in futures:

LiftRules.dataAttributeProcessor.append {
  case ("fb-picture", pictureEndpoint, element, _) =>
    () => {
      val fetchedUri = // blocking HTTP call

      ("img [src]" #> fetchedUri) apply element
    }
}

When we return a ()=>NodeSeq, Lift knows to run that function in a background thread and merge its results at the end of the render cycle.

Note that all of this works the same way as Lift’s lift:parallel snippet. This means that the processing will only happen in the background if:

• LiftRules.allowParallelSnippets is true (by default this is true unless you are running on Google App Engine).
• This is a full page load (i.e., not an AJAX or comet request).

It also means that the parallel threads have up to LiftRules.lazySnippetTimeout to complete before their result will be discarded altogether. The default value of LiftRules.lazySnippetTimeout is 30s. The key distinction here is that these are not lift:lazy snippets—lift:lazy will wait up to LiftRules.lazySnippetTimeout and then send the page down with a placeholder to replace with the computed contents once the lazy snippet finishes rendering. lift:parallel instead waits until that timeout and then sends the page down with no content (or, if you are running in dev mode, with an error message) instead.

Elements all the way #

There’s one last even more customizable thing that Lift 3 can do beyond data- attributes: LiftRules.tagProcessor. It works basically the same way, only it lets you match on element names. For example:

LiftRules.tagProcessor.append {
  case ("fb-picture", element, _) if element.attribute("src").isDefined =>
    () => {
      val pictureEndpoint = element.attribute("src").get // guarded by isDefined above
      val fetchedUri = // blocking HTTP call

      <img src={fetchedUri} />
    }
}

This would let you use:

  <fb-picture src="https://graph.facebook.com/me/picture"></fb-picture>

Note three very important limitations:

• tagProcessor does not do any recursive processing. It runs once, and will not run again. All data- attribute processing happens before tagProcessor runs, so if you want to run snippets you’ll have to do it inside the processor.
• tagProcessor runs for every element on the page. This means the test of whether or not it will run needs to be extremely cheap, otherwise you’ll significantly slow down page rendering.
• tagProcessor does not exempt you from the HTML5 parser’s rules. Above, we need a closing tag on fb-picture because otherwise the fb-picture element will encompass any elements that come after it. This is because the HTML5 spec has an explicit list of tags that may have no end tag (adding /> does not make an element self-closing in HTML5).

And one bit of awesome news: tagProcessor also returns a DataAttributeProcessorAnswer, so you have the same futures/parallel snippet support as you do in data- attributes.

On to the next one #

A lot of powerful opportunities there folks. Next time, we’ll have a brief look at client actors and streaming promises, which have already been used extensively. Enjoy the week!