Memoizing Network Promises for Cheap Performance Gains

I’ve recently had designers asking for various pieces of metadata to show up in list views.

What looks simple to a designer sometimes requires an extra async request for every item in a list. This came to a head today when I was asked to add a particular piece of metadata that required a two-step promise chain involving two separate REST endpoints for every single list item. It was time for some optimization.

Memoization seemed like the ideal solution. I thought I would cache the responses to each request so that as a user paged through the list, the app would check if that response had already arrived. If it had, the app would return the memoized response, but if not it would send off the real network requests.

That would have worked OK, but not amazingly. The problem is, that approach doesn’t account for in-flight requests. Let’s say there are 10 list items. Five of them will send off the exact same promise chain at about the same time. If we memoize the identical result of those five promise chains, we will have saved async calls for the future, but we still send five identical requests. All we really needed was one.

Then it hit me: instead of memoizing the result of the promises, I could memoize the promises themselves! This is brilliant.

/**
 * Sends off a set of requests to figure out the
 * credits a user has given a username. It memoizes
 * the promise to prevent duplicate requests.
 */
getCreditForUser (username) {
  let ret;
  if (this.inFlight[username]) {
    // there is already an in-flight or fulfilled promise for this username
    ret = this.inFlight[username];
  } else {
    // there is no in-flight or fulfilled promise for this username, so send 'er off
    ret = this.inFlight[username] = this.users.getByName(username)
      .then(userObj => this.creditService.getById(userObj.credit));
  }

  return ret;
}

Take the afore-mentioned example. The first of the five identical list items sends off a request for the metadata and memoizes the promise for that request. The next four items, instead of sending off their own requests, are provided with that exact same promise. This means that when those network responses are received and the promise is fulfilled, all five list items receive the result from that single pair of requests.

Those fulfilled promises can be used in exactly the same way long after the network responses were received. This provides an elegant solution to the case when you potentially have many duplicate requests and want to cut unnecessary bandwidth for performance gains.