EEP 77: Assignment in Comprehensions #

Abstract #

This EEP adds assignments to comprehension qualifier lists, providing a convenient and readable alternative to other syntactical tricks.

Rationale #

It would often be useful to be able to easily bind variables in the qualifier sequence of a comprehension, for example:

[{Hash, Term} || Term <- List,
                 Hash = erlang:phash2(Term),
                 Hash rem 10 =:= 0].

or

[Char || F <- Files,
         {ok, Bin} = file:read_file(F),
         Char <- unicode:characters_to_list(Bin)].

using plain Pattern = ..., entries between qualifiers.

You can achieve the same result today by writing a singleton generator:

[Char || F <- Files,
         {ok, Bin} <- [file:read_file(F)],
         Char <- unicode:characters_to_list(Bin)].

but this has some drawbacks:

• The intent is not clear to the reader.
• You have to remember to write a single element list around the right hand side argument (which itself could be a list).
• You should probably make it a strict generator <-:- so typos don’t just silently yield no elements, but you might forget that.
• Someone editing the code later may accidentally add extra elements to the right hand side list, causing unintended Cartesian combinations.

Another trick is to piggy-back on a boolean test, using export of bindings from within a subexpression to propagate to the subsequent qualifiers:

[Char || F <- Files,
         is_tuple({ok, Bin} = file:read_file(F)),
         Char <- unicode:characters_to_list(Bin)].

which is very much not a recommended style in Erlang, making it hard to see where the bindings come from. It also relies on having a suitable test as part of the qualifiers for the value you want to bind. If you don’t, it is possible to invent a dummy always-true test:

[Char || F <- Files,
         foobar =/= ({ok, Bin} = file:read_file(F)),
         Char <- unicode:characters_to_list(Bin)].

Such tricks are very bad for readability and maintenance of the code, making the logic hard to follow. Being able to just write Pattern = Expr would be much clearer, avoiding weird workarounds.

Specification #

It is in fact already allowed syntactically to have a Pattern = ... match expression in the qualifiers. Currently however this gets interpreted as any other filter expression: it is expected to produce a boolean value, and if false, the current element will be skipped.

Hence, a match Var = Expr will only proceed with the current element if Var has the value true. We can therefore expect that no such uses exist in practice, because Var would be fully redundant. (The OTP code base has been checked and does not contain any.) For example:

[{Pid, Live}  % `Live` will always be the constant `true`
 || Pid <- erlang:processes(),
    Live = is_process_alive(Pid)].

This EEP suggests that instead, a match expression Pattern = Expr in the qualifier list should be specially treated as having the same semantics as a strict singleton generator Pattern <-:- [Expr], implying that a failure to match should be a runtime error. As in any generator, this means that all variables in Pattern are regarded as new bindings local to the comprehension.

Reference Implementation #

A reference implementation exists in the lc-match-operator branch of the author’s GitHub account, together with a GitHub pull request to the Erlang/OTP repository.

Backwards Compatibility #

To avoid incompatibilities with any possibly existing code that could still rely on the current behaviour, the new semantics have been implemented as the optional language feature compr_assign, and an error is reported for any such assignments when the feature flag is not enabled, allowing existing cases to be detected and rewritten. The feature could be enabled by default in a subsequent major release.

Copyright #

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.