A typed chain: exploring the limits of TypeScript

7 min readMar 30, 2017

Update (2020): it is now possible to correctly type _.chain by overloading the type of this on the wrapper interface. This is how the lodash typings work. See this thread for a simple example. The typings in sections 3–7 are no longer a good way to approach to this problem.

In my last post, I looked at how some of TypeScript’s new features made it possible to correctly type Underscore.js’s pluck method. In this post, I'll try to do the same for its chain method. In the process, we'll run up against some limitations of TypeScript and its type declaration language. We'll also make use of one of TypeScript's newest features: the (lowercase) object type, which made its debut in TypeScript 2.2.

We’ll work our way towards a typed chain in a several steps. Hopefully you’ll learn something along the way!

What is _.chain?
A simple implementation of chain in VanillaJS
An abortive attempt to implement chain in TypeScript
A type declaration file for chain with lists and scalars
Adding object types and methods to the type declaration file
Specializing pluck / map for lists of objects
Conclusions

1. What is _.chain?

Underscore and Lodash’s chain methods solve a simple problem: while we typically think of data as flowing from left to right and top to bottom (at least in the LTR world), function composition syntax goes the opposite way:

f5(f4(f3(f2(f1(x)))))  // f1 appears last but is applied first.

The chain method uses method chaining to invert this. Each operation returns a new wrapped object, which you then unwrap at the end:

_.chain(x)
 .f1()
 .f2()
 .f3()
 .f4()
 .f5()
 .value();

(In practice the functions are map, filter, reduce, groupBy and friends.)

This reads more clearly, but it has a few downsides:

It’s more verbose (you need to add the _.chain and .value()).
It’s easy to lose track of what your data looks like as you go through the chain.
It’s easy to forget the .value() call at the end.

Lodash has a partial answer to the missing .value() issue: for implicit chains, some methods auto-close the chain. There are 153 of these methods, but good luck remembering which ones they are! Do you need to call .value() or not? These are exactly the sorts of issues that static analysis can help with.

(Side note: there’s a proposal to add a |> pipe operator to EcmaScript to solve this problem in the language itself. I think this is a great idea!)

2. A simple implementation of chain in VanillaJS

Let’s start with a quick implementation of something like chain in VanillaJS:

A few notes:

It’s implemented using an ES6 class and the native map and reduce functions.
The sum method closes the chain.
The mapValues method only really makes sense for objects, not arrays. With an array you should use plain old map.
The map function acts like pluck if you pass a string argument. (This is really how Lodash works!)

3. An abortive attempt to implement chain in TypeScript

Once we try to add types to this implementation, we quickly run into some issues. Here’s a quick try at map:

Ideally we’d only provide a map method if the wrapped value is an array or object type. But, just like VanillaJS, TypeScript doesn't support method overloading. More generally it doesn't generate code that depends on types.

So what can we do? Unless we’re willing to use lots of opaque any types, we're going to have a hard time implementing chain in TypeScript. But there is another option: we can implement it in VanillaJS and write a separate type declaration file.

(Update: some astute reddit readers point out that this isn’t quite correct. It’s possible to write overloaded function declarations in an implementation file though, just as in JavaScript, you can only have a single implementation.)

4. A type declaration file for chain with lists and scalars

Whenever we write a function in TypeScript, it provides us with two related benefits:

It verifies that the function’s implementation passes static analysis and matches its declared type.
It facilitates static analysis for callers of the function by specifying its input and output types.

When we implement our functions in TypeScript, we get both of these benefits. When we use a type declaration file, we only get the second. But that’s the most important one. You care about your users more than yourself, right?

(Side note: type inference blurs these lines. But unless you love C++ template errors, it’s a good idea to write explicit types for your exported functions!)

The upside of writing a separate type declaration file is that it gives us more flexibility. Declaration files are written in a slightly different language and, unlike TypeScript implementation files, they do let you overload functions. This is just what we need to get going on chain. Here's a first try:

Note the defining characteristic of a type declaration file: none of these methods have implementations.

Overloading lets us introduce the distinction between wrapped values and wrapped arrays that was missing in our TypeScript implementation. We can verify that this works by checking a few types in vscode:

What if we wanted to add the sum method? Here's a try:

interface WrappedArray<T> extends WrappedValue<T[]> {
  sum(): T;  // ideally this would only work for string or number.
}

This will let you sum an array of numbers to a single number or concatenate an array of strings to a single string. Great! And it will auto-close the chain. Even better! But… it will also let you sum an array of Dates to a single Date and an array of regular expressions to a single regex. This doesn’t match the function’s behavior. Ideally we’d only allow sum for wrapped numbers and strings.

The simplest way to do this would be to specialize WrappedArray<number> andWrappedArray<string>. But it's currently impossible to do this in TypeScript. Here's another workaround:

Here we try to detect every way you could make an array of numbers and override it to return a distinct type. We’d have to do this with WrappedArrayOfStrings, too. This is a lot of work to get a more precise type definition! And we can't be confident we've covered every way to get a wrapped list of numbers. But crazy as this seems, it's exactly how the Lodash type declarations work.

It’s a bit frustrating that TypeScript doesn’t provide a facility for doing specialization like this. However you arrive at a type, TypeScript knows it! It’s a shame that we have to hack around the missing feature like this.

5. Adding object types and methods to the type declaration file

We can use a similar trick to add mapValues:

Here we’ve introduced a new type, WrappedObject, and specialized some of the chainable methods on it. mapValues is specific to objects (not arrays; if you have an array then you should use map). And map and filter have different behaviors for objects (they turn them into arrays):

Both types are correct. The “a” | “b” key type is particularly nice—it’s a specialization for objects, not arrays.

These declarations makes use of the object type, which is new in TypeScript 2.2. It's a type which includes any non-scalar value. A string isn't part of object, but an array and a structure are. Here we use object to create a specialized chain for types which have keys and values. Arrays are still handled specially, but so are objects. It's not entirely clear to me why you'd want to wrap a scalar, but that's the fallback.

6. Specializing pluck / map for lists of objects

What about the specialized version of map for arrays of objects, which acts like _.pluck?

_([{a: 1}, {a: 2}, {a: 3}]).map('a').value()
// [1, 2, 3]

This only makes sense for arrays of objects. So we’ll need to specialize to include those as well:

This is getting messy!

A few things to note:

We’ve introduced a new WrappedArrayOfObjects interface with a specialized map method.
The specialized map method uses keyof and index types so it's actually pretty smart! Plucking a non-existent key will be an error. (In this way we've already gone beyond the DefinitelyTyped declarations for either Underscore and Lodash. I’ll have more to say on why that might be in a future post.)
We support two ways of creating WrappedArrayOfObjects instances: directly via the chain function, and by mapping from values in an array to objects.

Here’s an example of this in action:

Using our type declarations, TypeScript is able to track the type of the chain through our map(‘str’) call.

This works well for the cases we’ve covered. But there are other ways to get an array of objects. Maybe you have an array of nested objects and you pull out a subfield. In this case, we’d need aWrappedArrayOfObjectsOfObjects type. But clearly this is madness. We'll never be able to model every way to produce an array of objects in this manner.

So WrappedArrayOfObjects isn't deep enough. And this is already deeper than the underscore typings go (they include chained arrays and objects), and even deeper than the 20,000-line Lodash typings (which include LoDashExplicitNumberArrayWrapper!).

This is as far as we’ll take our chain type declarations for this post. You can see all the declarations and examples together in this gist.

7. Conclusion

Using a type definitions file gets us close to correct types for _.chain but it still leaves some unpleasant gaps.

This will all get easier if the TypeScript team finds a good way to incorporate specialization into type declarations. I filed TypeScript#13852 to track it. It’s an advanced feature that most TS users will never touch. But they will appreciate higher-quality typings for libraries like Underscore and Lodash.

Type declaration files are an essential part of using TypeScript, but they tend to be the domain of experts. Hopefully this post has pulled back the curtain to give you a glimpse of the tricks behind the magic.