RSS

A Javascript event hub using promises

The event hub / aggregator pattern is fantastic for decoupling code modules but the emitted events are usually ‘fire and forget’ so if the event listeners want to feed back an error or status report to the event emitter, they can’t.

Adding promises solves this – if the event is published then each event listener will receive its own promise. Once the listener has performed any synchronous / async tasks that it needs to, it can choose to resolve or reject the promise it has received. If all event listeners report success using their promises, then a promise passed back to the emitter will resolve successfully. However if one or more of the listeners reject (fail) their promises, it will fail.

Because the event listeners are receiving promises, they can wait until async activities eg. ajax are complete before resolving their promise. Handy if decoupled modules need to save state or complete animations before feeding back to the event emitter.

The code is on Github, hope it helps someone!

 
Leave a comment

Posted by on April 8, 2014 in Front end development

 

BDD changed the way I work

It’s been a year and a day since my last post. Plenty has been going on in the last 12 months. I finally took the dive into mobile that I’d been putting off until the tools improved, with a rebuild at Which? in London.

Which? made some bold technology decisions to break with their legacy codebase. Previously using Java in the web app and API layers of the stack, they kept it for the API and adopted Ruby on Rails for the web application.

I’d heard great things about how fast you could build stuff with Rails. I didn’t come to front end development via a server side language so a framework like Rails, that gives an easier introduction to that layer, has great appeal. Without a doubt it’s enabling and makes you a better all-round developer.

Another big thing that learning Ruby has given me is the ability to write automated UI tests with Cucumber. I had heard about CucumberJS early last year but it looked a bit in-progress. Cucumber with Ruby looked more complete with better docs and plugins.

Regarding BDD and the practise of describing features using Given.. When.. Then syntax, that are readable by the business – I’d heard experienced consultants saying things like

“Feature files are a nice idea but I’ve never actually seen them used by a business in real-life”

I don’t buy that view – at Which? the features have been described by a business analyst in Jira then handed over to us to automate. We might make some minor tweaks in language not meaning, to make it possible to automate them.

The company, developers and QA then have a clear contract for what we’re building, and even better, we can code directly against it. When we fulfil the contract, the tests turn green.

BDD is of course an investment – in mocking up the data, working out robust ways to test the code, trying to make the tests run as fast as possible, and dealing with occasional (and sometimes inexplicable) fragile tests that work one minute then fail the next.

Finding a balance between unit and UI tests

Have you ever had someone display a diagram of a testing pyramid and declare that unit tests are far quicker to write than UI tests? I’ve not always found that. I’ve often found Javascript unit tests are at least as expensive to write, largely because you’re mocking out so many of the surrounding components – HTML fixtures, API data, other Javascript widgets, third party code. That’s all fine, and unit tests are essential, but I swear they’re not significantly quicker to write. They are of course many times quicker to run – moments after you saved a change your entire suite of hundreds of unit tests can have completed and it’s that fast feedback loop that is one big reason to write them.

You burn through a lot more unit tests because they’re more tightly tied to the implementation, which in any decently maintained codebase should be evolving constantly as it’s refactored. Parts of your code inevitably get thrown away, implementations change and the unit tests frequently turn red until you turn them green again or remove the redundant ones. The closer the tests are to the metal, the more ephemeral they are.

On the other hand, a UI test, once turned green should only turn red infrequently – if there’s a breakage in the interface, or if a refactor is significant enough to require the parts of the step definitions that touch the implementation, to need updating.

Again, am I saying don’t write unit tests? Absolutely not. They can guide the design of lower-level logic and help find flaws in it. And they can instantly expose regressions as you refactor.

I’m just questioning some commonly received wisdom that you should have orders of magnitude more unit tests than UI tests. All tests are an investment – the time spent writing and maintaining them gets paid back in fewer regressions and production bugs.

I’m totally sold on the BDD way of working now. You won’t always find it happens in a project – I’ve come across the odd very capable front end developer who didn’t see it as part of their role to write UI tests. Although it was in a technology stack that didn’t make it easy to write UI tests (hello Java).

Some QAs get nervous about developers doing it, but they shouldn’t. If the devs write UI tests for the web app that mock out services like authentication and APIs, then it frees up the QAs to write those true end-to-end tests using real copies of the services.

Writing tests and seeing them run green gives calm to the developer. You don’t get that sick feeling in the run-up to a release that you might have caused regressions with your last commit, but aren’t sure where. The better your tests, the more secure you feel. If you haven’t, try it!

 
Leave a comment

Posted by on January 12, 2014 in Front end development

 

Testing a RequireJS / Backbone component with Jasmine and jsTestDriver

If you want to use Jasmine to test something that’s inside a RequireJS module, for example a Backbone model, you need to find a way to load the Require module from your Jasmine spec, and pause the running of the test spec until the module is loaded and ready.

I need a few things loaded up front for my tests to run – Require itself, Backbone, jQuery, Underscore.

First of all I’ll load require.js from my jsTestDriver.conf.

Backbone is expressed as a dependency in my require.config, and will in turn pull in the other two libraries:

Now I can load the main RequireJS config file, and hence those dependencies, by loading an extra require.config snippet, which specifies where files can be found under the test server, and then loads the main config itself:

This extra require.config snippet is loaded directly by jsTestDriver.conf. The ‘serve’ directive below makes all files under the /app folder available to my test code under a ‘/test/app’ path. So for example /app/mymodel.js could be loaded by a Jasmine spec from the path /test/app/mymodel.js

Now if I run jsTestDriver, my dependencies are all loaded up, plus my require.config, but not my application. This matters as I should only need to load in the specific module I want to test. I can do this using a require call inside my test setup (beforeEach). Wrap it in a “runs” block then use a “waitsFor” block to halt things until it’s loaded.

These patterns took a while for me work out when I started testing Require / Backbone apps using Jasmine. Hope this helps somebody!

 
Leave a comment

Posted by on January 11, 2013 in Front end development

 

Backbone Marionette – using nested views

For the past few months I’ve been picking up Backbone, then Marionette.

I quickly liked and later cursed the fact that Backbone gives you some structure yet lots of flexibility, especially around managing views.

At the start the flexibility makes picking things up conceptually easy. As you move beyond the most basic app you need to keep the creation and destruction of views under control, especially when you start nesting them. Backbone gives you very little to handle this, but Marionette plugs the gap nicely.

Terms used in Marionette

  • A layout is a type of view that can hold other views
  • The element within that layout that holds / wraps the nested view, is called a region. A region can hold one view at a time. It can reload the same view multiple times as required, or swap in different types of view.
  • Marionette.Layout actually extends Backbone.View (not directly, but it does inherit from it). So as a layout is a type of view, you can actually nest a layout inside a region of another layout

The markup

Your template for the outer layout might be:

.. and your template for the view to be nested inside the region:

Note – I prefer that the container element for the nested view shouldn’t be in the parent layout template (it should be created dynamically by the view’s Javascript definition)

WRONG:

RIGHT:

Why? I think the view should control its container element, it’s more maintainable. You know that everything relating to that view is in one place in your codebase and there are no fragments of the view contained in other templates.

The Javascript

The outer layout can use its regions to show views, when it has been rendered itself. Use the ‘show’ method of the layout’s region. When you call ‘show’ it automatically calls the render method on the subview.

The View definition for the nested view would contain a className attribute so the view container will be created dynamically when the view is rendered.

You can hook into the onRender and onShow events.

What’s the difference between onRender and onShow?

Remember that when the parent region’s ‘show’ method is called, it automatically calls render on the subview as well, so onRender will of course be triggered after that. At this point the view has been templated and a DOM fragment created. However it hasn’t yet been attached to the parent region so it’s still only in memory and not actually part of the page yet. If you put a breakpoint in the onRender and in your browser console looked for a parent of this.$el (the view’s container element):

this.$el.parent('body').length

it would return 0.

The onShow event fires after the view has been attached to its parent region. If you ran the same line in your console during the onShow event, it would return 1.

More reading on Layouts

Marionette provides structure in the biggest areas left vacant by Backbone. One of the best things about it is its creator Derick Bailey is a prolific blogger and documenter (and Stackoverflow lifesaver).

Check out his blog post - Managing Layouts and Nested Views with Backbone.Marionette and the Marionette docs on Layout.

 
1 Comment

Posted by on December 21, 2012 in Front end development

 

Front end architecture and page load speed

“Speed is a feature” and I think the trend of templating the initial page render on the client negatively impacts performance, and also robustness and device independence.

Don’t get me wrong, Backbone, async updates and responsive web apps are all fantastic and I use them enthusiastically. But I think they should build on a solid first delivery of the page as markup – it will work more consistently on the ridiculously wide range of devices out there. You’re then taking fewer risks in the “hostile environment of the browser”. I think someone once called it Progressive Enhancement…

Twitter’s recent rebuild shows they came to the same conclusion.

 
Leave a comment

Posted by on December 18, 2012 in Front end development

 

Portfolio: News Video Hub for Associated Press

http://apvideohub.com

I used: OO Javascript | Unit testing | HTML5 | CSS3 | LESS | Selenium IDE

Associated Press is a 150 year old institution, the largest and most trusted newsgathering operation in the world. Their video archive includes over 1.5 million global news and entertainment stories dating back to the beginning of the 20th Century.

Their brand new site, AP Video Hub offers curated archive content mixed with breaking news topic pages, for preview and download in a variety of broadcast-ready formats. I was the front end developer on the project.

The beta version of the site has just gone live (April 2012) and is already generating major interest from media organisations.

Performance

The primary aims for the interface were speed and simplicity. We used server side templates and minimised the initial payload of Javascript, CSS and thumbnail images to make the above the fold render time as fast as possible. Other assets were lazily loaded after initial render, eg. instant search index, below the fold thumbnails, and the two remaining main views, to give a highly responsive experience.

Progressively enhanced single page app

I didn’t want to take the performance hit of rendering the initial page load using Javascript templating, so used server side templates (Razor). The 3 main views are switchable using left hand nav links.

I’d previously seen the History API used in the great Github interface with real URLs and ajax-powered view switching. Where the browser supports the History API, I intercept the any clicks on the links and reload just the main content panel using Ajax. The Ajax call is routed to the same server side template that’s used for the whole page load, and only the section of that template for the main content area is returned.

This setup gives the following benefits -

  • progressive enhancement eg. the nav links function normally if the browser doesn’t support javascript or the History API, and reload the whole page with the changed URL / view. If History API is supported, you get ajax content switching
  • the History API means the URL changes as you switch views so you can bookmark / revisit.
  • speed – only a portion of the page is reloading so it’s very fast
  • you only need to maintain one set of server side templates, and either the whole template is returned eg. if the user has hit a URL directly; or if an ajax request part of the template is returned

Code structure and testing

I used techniques I’d learnt on Betfair’s site rebuild to structure the code in a modular fashion.  Webstorm is firmly my favourite IDE now and brings built in Javascript unit testing with jsTestDriver. Simple to hook in Jasmine, and write tests easily, spawn test browsers and run the whole suite without leaving the IDE.

I also wanted to find a way to get higher-level functional test coverage and picked up Selenium IDE for the first time. I rate it better than Selenium 2 / Webdriver as a tool for developers to generate and run tests quickly, which I think is critical for test coverage to keep pace with development.

The site is iPad and iPhone compatible – an HTML5 video container used instead of the default Silverlight.

It’s been a great contract to work on – a talented team and a trusting and accessible client in AP. And I’ll miss the daily dive into Camden’s food stalls :)

 
Leave a comment

Posted by on May 2, 2012 in Portfolio

 

Portfolio: betfair.com rebuilt for speed

Betfair’s completely rebuilt sports betting site – http://beta.betfair.com/football

I used: Javascript | YUI3 | CSS3 | LESS | HTML5 API’s | Agile | Performance optimisation

I just completed a 12 month contract with Betfair Plc in Hammersmith to help rebuild their sports betting site from the ground up. The project had two main aims – make it robust, and blazing fast.

betfair.com is high volume – its 3.7 million customers in 140 countries have £300M on deposit. Notable spikes in usage include the 2010 World Cup Final (£68M traded) and the Grand National (30,000 bets placed in a single minute).

Performance had been shown to have a direct correlation with revenue. Make the site faster, and people place more bets. Simples.

The old site had pushed most processing and rendering duties to the browser, resulting in a morass of several MB of Javascript and a page load time of 15 – 20 seconds. The aim was to reduce that to 3s. That’s in the slowest browser, with an empty cache, on an average 4Mb connection. And ‘page loaded’ doesn’t just mean downloaded and rendered, it includes all Javascript initialised so the page is usable. It’s no fun if you make things easy is it?

Betfair's new site

I worked as front end developer on the team providing ‘core’ services including login and navigation. I also had a dedicated period to work purely on performance and get that load time closer to the magic 3 second mark.

2011 was, without doubt, the most intensive period of learning I’ve experienced in my professional career. What did I learn on the Betfair project? Well…

Testing

It’s no exaggeration to say my outlook as a developer was reconstructed.

I’ve enough to say on testing front end code for a future post. I’m not sure how other people found it, but for me this was a major change in approach from what I was used to and a bit of a journey, not a quick turnaround. In a nutshell -

  • It subverts your approach to coding, from being DOM focussed.
  • When you say a piece is dev complete, it should mean you also have test coverage and the tests are running green.

Agile

I hadn’t *done* Agile to this degree previously. Here’s how the experience unfolded for me -

Pre-launch

There was an initial construction phase lasting several months, to get the first release of the product out. You’re involved in a *lot* of user story workshops, definition and pointing. Plenty of “known unknowns” and a certainty there were several “unknown unknowns” lurking.

After a period of investigating each area of functionality to flush out problems and detail, we fell into a more formal scrum process – 2 week sprints drawing from the new backlog.

My whole perspective at this stage, with little experience of Agile to draw on, was from the ‘front’ of the project – get stuff defined, get to work on it. I’d had a lot of experience of working up to launches and deadlines, but little awareness of the hard yards needed to get stuff through to production in an ‘Agile’ manner, in other words…

Continuous delivery

So had I already experienced the main parts of Agile in defining and working through user stories? If we focused just on the execution of a series of sprints would it be enough for the shiny new features to drop off the conveyor belt?

Nope.

The critical factors, I learned, are

  • Automated tests – both javascript unit tests (we used Jasmine for this), and acceptance tests (Selenium 2 / Webdriver, scripted with Groovy). The tests have to be quick to write, quick to run, and must be run frequently. The results have to be visible to developers, ie “in your face” visible, not that you have to go looking for them. If you have this, you can develop, refactor and bug fix without that queezy feeling in your stomach that your next commit might introduce regressions, and bug fixing doesn’t resemble ‘whack-a-mole’.
  • A fast, stable delivery pipeline

In other words the stuff enabling the end of the production process, not the start. But you have to begin to address them, especially automated testing, right from the start.

YUI3

Before using YUI3 I thought “Why do you need more than jQuery?”.

Working with another library actually threw into relief jQuery’s strengths (especially the great API docs with embedded examples) and weaknesses (OK so it’s not even designed to structure your code, but it will, if you don’t know any better – as many jQuery starters don’t – ie. lots of nested callbacks).

YUI3 has some fantastic features -

  • the attribute system lets your objects have properties with getters / setters, onChange events and validation all built in
  • the loader system lets you split your YUI modules across separate files that load in parallel and may complete at different times, but will ensure all the dependencies execute in the correct order
  • you get helper methods for things like objects, arrays, console logging

Performance

This was a cornerstone of the project and integral to decision making from the outset. There was an SLA commitment that each page will load in 3 seconds or less. This is important – it gives the leverage needed to influence each technical and design decision made.

2 imagined conversations -

Between front end and server side engineers -
Q. Do we render HTML on the server or template it in the browser?
A. On the server – it’s faster, more robust and there’s less to download to the client

Between designers and front end developers -
Q. Do we use sliced graphics to do rounded corners in IE7/8?
A. No – use CSS3 for capable browsers falling back to square corners for IE. Would IE users rather have a slow site with a few rounded corners?

Some pointers -

  • Balance the application correctly between the server and browser
  • Consider performance from the beginning and at every step – don’t leave it to the end
  • Record ‘real world’ performance – eg. full round trip time. We rolled our own but for an out-the-box solution check out Yahoo Boomerang
  • Take a baseline using  a ‘private instance’ copy of Webpagetest. Make your optimisations, run the page again on your dev server in Webpagetest, see the difference. In other words make performance gains measurable.

So did all this work?

The site is being rolled out in the UK during early 2012 – currently some users are being diverted to the new experience which you can see at beta.betfair.com/football. In the coming weeks 100% of customers will get the new site.

Some early metrics from sessions using the new beta site -

  • Sessions that bet – up 2%
  • Bounce rate – improved 40%

Check out this presentation on the Betfair rebuild at Velocity Europe 2011.

 
1 Comment

Posted by on January 8, 2012 in Portfolio

 

Tags: , ,