Tuesday, March 16, 2010

[cjaaycrj] Recompiling pipes

A neat trick, theoretically doable with open source, is to take a command pipeline, for example "foo | bar | baz", and automatically recompile the various commands in the pipeline Just In Time into a single program. Hopefully, this allows for considerable optimization. We could skip output formatting and input parsing.

One of the interesting problems is parallelization. As a pipeline, the operating system automatically allocates processors to the various commands. Now it will need to be done manually, so that more cycles are devoted to a producer whose output is needed by a consumer which has gone idle.

No comments :