Performance: IR Node Merging

[Kleptine]

One major performance downside of the vectorized engine is that the overhead of the looping can dominate very small operations. It's as if the following program were implemented per line, with separate loops. Ex:

for (i in list) {
  var x = i + 1;
  var a = x * 2;
  var b = a * x * i;
}

is executed like

for (i in list) {
   x[i] = i + 1;
}

for (i in list) { 
  a[i] = x[i] * 2;
}

for (i in list) {
  b[i] = a[i] * x[i] * i;
}

It would be ideal to merge groups of operations into one pass of the loop during execution. This is very feasible, but requires some heuristics: how much to stuff into a single loop? Stuff too much in there and I think you might regress performance because of cache effects.