In our previous post on functional programming we began introducing some functional themes by working through requirements for processing a typical JSON response.

Here's a recap of our requirements:

  • filter out posts older than a month (say, 30 days).
  • group the posts by their tags (this might mean posts show up in two groups if they have more than one tag.)
  • sort each tag listing by published date, descending.

That first post focused on our first requirement above - filtering out posts older than 30 days.

We also started a library of useful functional utilities, which we'll continue to add to in this post. You can view the gist to see the full source code for this series.

In this post, we'll cover our second requirement which is grouping our records in the newly filtered list by their tags.

Grouping Things

In Javascript, we can group items in a list using Array#reduce(), which we covered in a previous blog post. Definitely take a look at that now if you aren't familiar; but the basic idea is that reduce() allows us to iteratively build up a new value by doing something with each item in the array.

Usually, you think of taking some list of values and using reduce() to produce a single, new value, like:

[1,2,3,4].reduce(function(sum, n) { return sum += n; }, 0);  // 10

It's this ability to iterate over values and build up or accumulate a new value that allows us to use reduce() to perform grouping operations. For instance:

var list = [  
  { name: 'Dave', age: 40 },
  { name: 'Dan', age: 35 },
  { name: 'Kurt', age: 44 },
  { name: 'Josh', age: 33 }
];

list.reduce(function(acc, item) {  
  var key = item.age < 40 ? 'under40' : 'over40';
  acc[key] = acc[key] || [];
  acc[key].push(item);
  return acc;
}, {});
// {
//   'over40': [ 
//      { name: 'Dave', age: 40 }, 
//      { name: 'Kurt', age: 44 }
//   ],
//   'under40': [ 
//      { name: 'Dan', age: 35 }, 
//      { name: 'Josh', age: 33 }
//   ]
// }

In the above snippet, we use reduce() to iterate over a list of objects. We use an empty object as the starting point and group the records based on their age. This allows us to treat an object like a map, assigning records to groups identified by property names on the resulting object.

Let's use this ability via reduce() to create a group() function.

var toString = Object.prototype.toString;  
var isFunction = function(o) { return toString.call(o) == '[object Function]'; };

function group(list, prop) {  
  return list.reduce(function(grouped, item) {
      var key = isFunction(prop) ? prop.apply(this, [item]) : item[prop];
      grouped[key] = grouped[key] || [];
      grouped[key].push(item);
      return grouped;
  }, {});
}
// our right curried version of `group()`
var groupBy = rightCurry(group);  

group() and groupBy() divide a list into sets, grouped by the property named prop on each object in the list. If prop is a function it will use the result of passing each value through prop. This is similar to the way _.groupBy() works in the lodash and underscore libraries.

Here's our previous example, now using groupBy():

var getKey = function(item) { return item.age < 40 ? 'under40' : 'over40'; };  
groupBy(getKey)(list);  
// gives us the same results as previous example

In this case we passed a function as the prop argument that returned a string for grouping. But we can use the non-function use of prop for records like our JSON response, where we want to group by a given property on the record:

var list = [  
  { value: 'A', tag: 'letter' },
  { value: 1, tag: 'number' },
  { value: 'B', tag: 'letter' },
  { value: 2, tag: 'number' },
];
groupBy('tag')(list);  
// {
//   'letter': [ 
//      { value: 'A', tag: 'letter' },
//      { value: 'B', tag: 'letter' }
//   ],
//   'number': [ 
//      { value: 1, tag: 'number' },
//      { value: 2, tag: 'number' }
//   ]
// }

This looks like it should work well. However, the list of objects we were grouping above could only belong to one possible group: 'letter' or 'number'. The list of objects had a many-to-one relationship with the grouping key.

In our JSON response, this isn't the case, as each post's tag property is an array of one or more tag names.

Grouping Many-to-Many relationships?

So, did we go to all that trouble building a groupBy() function that won't work for our requirement? Absolutely not! This is functional programming, after all, so we'll just use that function composed with other functions to build the one we want!

Let's take a step back and look at our JSON response again; but in a different light, like a typical database table:

We need a way to explode our list so that we output a list with one record for each tag-post combination. That output list is very similar to a linking or joining table used in databases that have tables with a many-to-many relationship - in this case tags to posts.

Doing this will necessarily create duplicates on our output; but we need those since a post can show up in multiple groups given our requirements.

Our output list would resemble the following given our table example:

Given the above diagram, it's clear that what we are doing is outputing a combination. In this case, our output is the combination of the post record with each of its possible tags.

We know we'll need to map over our lists, so let's create a map() and mapWith() function we can use going forward:

// Returns a new list by applying the function `fn` to each item
// in `list`
function map(list, fn) {  
  return list.map(fn);
}
var mapWith = rightCurry(map);  

Now, let's create a pair() function that can combine the elements from two lists.

function isArray(o) { return toString.call(o) == '[object Array]'; }

function pair(list, listFn) {  
  isArray(list) || (list = [list]);
  (isFunction(listFn) || isArray(listFn)) || (listFn = [listFn]);
  return mapWith(function(itemLeft){
    return mapWith(function(itemRight) {
      return [itemLeft, itemRight];
    })(isFunction(listFn) ? listFn.call(this, itemLeft) : listFn);
  })(list);
}
var pairWith = rightCurry(pair);  

We basically take two lists, map over each item in the first list, and for each item, output the results of combining that item with each item in the second list using a nested map. We also allow a function that returns a list as the second parameter, which will be passed the item from the first list on each iteration.

Let's try this with our filtered records from the before. We'll use our right curried getWith() to pass a function as the second parameter that will return the array of tags on each post as the second set to combine against.

pair(filtered, getWith('tags'));  
// [ 
//   [ [ { /* ... */ }, 'functional programming' ] ],
//   [ [ { /* ... */ }, 'es6' ],
//     [ { /* ... */ }, 'promises' ]
//   ],
//   /* ... */
// ]

Interesting..., those are the right tag->post pairs, but they're nested in arrays within the array because we have nested mapWith() calls, each of which return an array.

We can, however, flatten that using another handy function called flatten(), which will flatten an array by one level, ie, transform [[1,2],[3,4]] into [1,2,3,4].

function flatten(list) {  
    return list.reduce(function(items, item) {
        return isArray(item) ? items.concat(item) : item;
    }, []);
}

We use reduce() here to build up the new array, concatenating any values that are arrays directly into the resulting array, removing nesting. Flattening our previous results gives us the following:

// [ 
//   [ { /* ... */ }, 'functional programming' ],
//   [ { /* ... */ }, 'es6' ],
//   [ { /* ... */ }, 'promises' ],
//   /* ... */
// ]

Now we have the data structure we want to start doing our grouping!

But, this operation is so common -- the idea of flattening nested lists as we map across them -- that it's usually combined into a single function, flatMap(list, fn).

Let's create a flatMap() function and its right curried friend flatMapWith().

function flatMap(list, fn) {  
  return flatten(map(list, fn));
}
var flatMapWith = rightCurry(flatMap);  

We can use that inside our pair() function as well to ensure the right output:

function pair(list, listFn) {  
  isArray(list) || (list = [list]);
  (isFunction(listFn) || isArray(listFn)) || (listFn = [listFn]);
  return flatMapWith(function(itemLeft){
    return mapWith(function(itemRight) {
      return [itemLeft, itemRight];
    })(isFunction(listFn) ? listFn.call(this, itemLeft) : listFn);
  })(list);
}

Now we can put our entire flow together, which consists of:

  1. creating a many-to-many list of tag->post pairs using pairWith()
  2. using that new list as input to groupBy() to group each record by its given tag (the second item in each pair).
var bytags = pairWith(getWith('tags'))(records);  // #1  
var groupedtags = groupBy(getWith(1), bytags);       // #2  
// {
//    'destructuring': [
//         [ { /* ... */ }, 'destructuring' ],
//         [ { /* ... */ }, 'destructuring' ]
//    ],
//    'es6': [
//        [ { /* ... */ }, 'es6' ],
//        [ { /* ... */ }, 'es6' ]
//    ],
//    /* ... */
// }

Cleaning Up

So, after rebuilding the list as a many-to-many joined list and then grouping by the tags as the key, we end up with a structure that still isn't quite what we want - each post record is still nested in a list pair with its group key.

We need a way to map over our output object's properties and then map over each of those arrays and replace each list pair with just the post record.

We can map over arrays already using map() and its variants; but we can also do the same thing with objects if we think of the objects as a list where each item is a property and its value.

We'll call this mapObject() and it will return an object as well.

function mapObject(obj, fn) {  
  return keys(obj).reduce(function(res, key) {
    res[key] = fn.apply(this, [key, obj[key]]);
    return res;
  }, {});
}
// A right curried version
var mapObjectWith = rightCurry(mapObject);  

The function passed to mapObject() is passed not only the item but also the property name. Now, we can use our ability to map over an object to convert our structure:

// Remove our extraneous group key and pair, replacing with the post record
var finalgroups = mapObjectWith(function(group, set){  
    return mapWith(getWith(0))(set);
})(groupedtags);
// {
//   'destructuring': [
//      { id: 2, title: 'ES6 Promises', ..., tags: ['es6', 'promises'] },
//      { id: 4, title: 'Basic Destructuring in ES6', ..., tags: ['es6', 'destructuring'] },
//   ],
//   'es6': [ /*...*/ ],
//   /*...*/
// }

Being More Declarative

The operation we used above in which we want to pull out a specific property's value from a list of objects, mapWith(getWith(prop)), is a fairly common action. So much so, that this is commonly named pluck(), and you'll find it in numerous functional libraries.

// For each object in `list`, return the value of `prop`
function pluck(list, prop) {  
  return mapWith(getWith(prop))(list);
}
// right curried version of `pluck`
var pluckWith = rightCurry(pluck);  

That's a bit more declarative and gives us another higher order function we can reuse. But we'd like our resulting code to be a bit more descriptive of the actions it is actually performing - getting the post record from each nested pair.

Let's start by being explicit with the function we're passing to mapObjectWith():

function getPostRecords(prop, pair) {  
  return pluckWith(0)(pair); 
}

Ah, that's a bit more descriptive. And when combined with our original solution, becomes much more declarative of the action we're actually performing.

var finalgroups = mapObjectWith(getPostRecords)(groupedtags);  

The full implementation

The final implementation for meeting our second requirement:

// Step 1: Build our many-to-many list 
var bytags = pairWith(getWith('tags'))(records);

// Step 2: group by the tags (pair[1]):  
var groupedtags = groupBy(getWith(1), bytags);

// Step 3: strip extra key in nested pairs: 
function getPostRecords(prop, value) {  
  return pluckWith(0)(value); 
}
var finalgroups = mapObjectWith(getPostRecords)(groupedtags);  

Summary

In this post we've added a number of utility functions to our library. We also took a circuitous route through transforming our initial data due to its many-to-many relationship between posts and tags. We were then able to output a list of posts for each tag.

We also looked at a handful of common functional programming and combinative idioms like pluck, map and mapObject. Be sure and look over the gist of the full source for this second part of our blog series.

In the next and final blog post, we'll find out why we keep making right curried versions of all our functions as we discuss composition; and we'll finish our final requirement which is sorting each group of posts.