Node.js promises inside loops tutorial

Introduction

I started using promises not too long ago. They seem like a serious alternative to using callbacks specially when you are not used to fully work asynchronously. It was really tough for me in the beginning to perfectly understand promises until I read an awesome article written by Matt Greer called JavaScript Promises … In Wicked Detail . This article helped me to understand what a Promise really is and how promises actually work.

The problem

Despite all the help the article above provided, it still left me with my problem partially unsolved. The problem I was trying to solve was: how do I manage to return an array of cached values out of a promise inside a loop? In order to cache I use a Redis database. For the purpose of this blog post I will send a list of users to my filtering method and only display the ones that are cached in my Redis database.

The preparation

In order to create a Redis database I am using redislabs. To work with Redis from Node.js I am using: node_redis. As a promise library I am using Bluebird. You can also use Q. I am also using as routing framework Express.js with an EJS view template.

The solution

I will assume that by now you already know how to start an Express basic project and how to install all the dependencies I mentioned above. Now let’s start with the beginning. First we will start by creating a cache module in our project. Inside this module we will execute all Redis specific operations. Let’s start by coding a method that will upload a hardcoded list of users. Please bear in mind that this tutorial is for showing how you can return an array of values from an array of promises inside a loop.

/cache/index.js

var redis = require('redis');
var Promise = require("bluebird");

var client = redis.createClient(process.env.REDIS_PORT, process.env.REDIS_URL);
client.auth(process.env.REDIS_PASSWORD);
client = Promise.promisifyAll(client);

module.exports.addUsersToCache = function(users) {
  return client.msetAsync(users);
}

Please note that in order to fully use promises we need to promisify the Redis client. If we do that then we must add Async after all method names used by the client. Above you can see: client.msetAsync.

In order to call the above from routes you can use the following code:

/routes/index.js

var express = require('express');
var router = express.Router();
var cache = require('../cache/index.js');

/* GET home page. */
router.get('/', function(req, res, next) {
  cache.addUsersToCache(["user1", "TestUser1", "user2", "TestUser2"]).then(function() {
    res.render('index', { title: 'Express' });
  });
});

module.exports = router;

Now, for the next step, let’s create a method that will be called with another random array of users and will return only the ones that can be found inside our cache. Add the following code to /cache/index.js:

module.exports.filterCachedUsers = function(users) {
  var cachedUsers = [];

  return Promise.any(Promise.map(users, function(user) {

    return client.getAsync(user).then(function(result) {
      if(result) {
        cachedUsers.push(result);
      }
      return cachedUsers;
    });

  }));

}

According to the documentation Promise.any initiates a competitive race between multiple promises or values (values will become immediately fulfilled promises). So, because Promise.map is asynchronous I want to return the first fulfilled promise. Now let’s display the resulting users on our home page. Please add the following code to your /routes/index.js:

router.get('/get-cached-users', function(req, res) {
  cache.filterCachedUsers(["user1", "user2", "user3", "user4"]).then(function(results) {
    console.log(results[0]);
    res.render('index', {
      title: 'Express',
      users: results});
  })
});

And finally lets modify /views/index.ejs to display our cached users:

<!DOCTYPE html>
<html>
  <head>
    <title><%= title %></title>
    <link rel='stylesheet' href='/stylesheets/style.css' />
  </head>
  <body>
    <h1><%= title %></h1>
    <p>Welcome to <%= title %></p>

    <a href="/get-cached-users">Get Cached Users</a>
    <ul>
      <% if(locals.users) { %>
        <% users.forEach( function( user ) { %>
          <li>
            <%= user %>
          </li>
        <% }) %>
      <% } %>
    </ul>
  </body>
</html> 

UPDATE 1

I have also created an implementation to use the map reduce programming model. Add the following method to /cache/index.js.

module.exports.filterCachedUsersMapReduce = function(users) {
  var cachedUsers = [];

  return Promise.map(users, function(user) {

    return client.getAsync(user).then(function(result) {
      if(result) {
        cachedUsers.push(result);
      }
      return cachedUsers;
    });

  }).reduce(function(cachedUsersResult) {
    return cachedUsersResult;
  });
}

This method can be used in /routes/index.js instead of filterCachedUsers.

UPDATE 2

As I learned more from MDN I managed to optimize a bit more the code above and make it look more understandable and readable. Add the following method to /cache/index.js:

module.exports.optimizedFilterCachedUsers = function(users) {
  return Promise.map(users, function(user) {
    return client.getAsync(user).then(function(result) {
      if(result) {
        return result;
      }
    });
  });
}

To optimize even more I would also use ioredis , used by the guys at Alibaba , instead of node-redis as it offers native support for promises so you won’t have to promisify the redis client as we did above.

Conclusion

That was it. Now we get all the cached users out of the list of users we have sent to be filtered. Maybe this example was not the most inspired one but the purpose of this tutorial was for you to understand how I managed to return an array of cached values out of a promise inside a loop. I just started learning about promises and about Node.js in general so, if you found a better solution out there for the problem above please leave it in the description box bellow. You can find the code on GitHub here .