proc

An easy way to run processes like a shell script in Deno.

proc lets me write process-handling code in readable, idiomatic Typescript using async/await and AsyncIterator promisy goodness.

Documentation

deno doc --reload https://deno.land/x/proc/mod.ts 2> /dev/null

Examples

• Simple Examples for Input and Output Handlers
• Count Unique Words in War and Peace

Related Projects

• deno-asynciter

Input and Output Types

Processes really just deal with one type of data - bytes, in streams. Many programs will take this one step further and internally translate to and from text data, processing this data one line at a time.

proc treats process data as either Uint8Array or AsyncIterable<Uint8Array> for byte data, or string or AsyncIterable<string> (as lines of text) for text. It defines a set of standard input and output handlers that provide both type information and data handling behavior to the runner.

An Example

To get you started, here is a simple example where we pass a string to a process and get back a Uint8Array. The group is hidden in the gzip(...) function.

/**
 * Use `gzip` to compress some text.
 * @param text The text to compress.
 * @return The text compressed into bytes.
 */
async function gzip(text: string): Promise<Uint8Array> {
  const pg = group();
  try {
    /* I am using a string for input and a Uint8Array (bytes) for output. */
    const pr: Runner<string, Uint8Array> = runner(
      stringInput(),
      bytesOutput(),
    )(pg);

    return await pr.run({ cmd: ["gzip", "-c"] }, text);
  } finally {
    pg.close();
  }
}

console.dir(await gzip("Hello, world."));
/* prints an array of bytes to console. */

Input Types

^* - ReaderInput is a special input type that does not have a corresponding output type. It is not useful for piping data from process to process.

Output Types

^* - Special output type that mixes stdout and stderr together. stdout must be text data.

ℹ️ You must fully consume Iterable outputs. If you only partially consume Iterables, process errors will not propagate properly. For correct behavior, we have to return all the data from the process streams before we can propagate an error.

Running a Command

proc is easiest to use with a wildcard import.

import * as proc from "https://deno.land/x/proc@0.0.0/mod.ts";

First, create a template. The template is a static definition and may be reused. The input and output handlers determine the data types used by your runner.

const template = proc.runner(proc.emptyInput(), proc.stringOutput());

Next, create a runner by binding the template to a group.

const pg = proc.group();
const runner: proc.Runner<void, string> = template(pg);

Finally, use the runner to execute a command.

try {
  console.log(
    runner.run({cmd: ["ls", "-la"]});
  );
} finally {
  pg.close();
}

Key Concepts

Process Basics

Processes accept input through stdin and output data to stdout. These two streams may be interpreted either as byte data or as text data, depending on the use case.

There is another output stream called stderr. This is typically used for logging and/or details about any errors that occur. stderr is always interpreted as text. In most cases it just gets dumped to the stderr stream of the parent process, but you have some control over how it is handled.

In some cases (Java processes come to mind), stdout and stderr are roughly interchangable, with logging and error messages written to either output stream in a sloppy manner. The stderrToStdoutStringIterableOutput() output handler gives you an option for handling both streams together.

Processes return a numeric exit code when they exit. 0 means success, and any other number means something went wrong. proc deals with error conditions on process exit by throwing a ProcessExitError. You should never have to poll for process status.

Asynchronous Iterables

JavaScript introduced the AsyncIterable as part of the 2015 spec. This is an asynchronous protocol, so it works well with the streamed data to and from a process.

proc heavily relies on AsyncIterable.

See JavaScript Iteration Protocols (MDN).

Streaming code executes differently than you may be used to. Errors work differently too, being passed from iterable to iterable rather than failing directly. Bugs in this kind of code can be difficult to figure out. To help with this, proc can chain its errors. You can turn this feature on by calling a function:

proc.enableChaining(true);

This can produce some really long error chains that you may not want to work with in production, so this feature is turned off by default.

Preventing Resource Leakage

Processes are system resources, like file handles. This means they need special handling. We have to take special care to close each process, and we also have to close all the resources associated with each process - stdin, stdout, and stderr. Also, depending on how a Deno process shuts down, it may leave behind orphan child processes in certain cases (this behavior is well documented but annoying nonetheless) if measures aren't taken specifically to prevent this.

In other words, working with Deno's process API is more complicated than it looks.

To address the problem of leakage, proc uses group() to group related process lifetimes. When you are done using a group of processes, you just close the group. This cleans up everything all at once. It's easy. It's foolproof.

If you forget to close a group, or if your Deno process exits while you have some processes open, the group takes care of cleaning things up in that case too. Note that a group cannot be garbage-collected until it is explicitly closed.

const pr = runner(emptyInput(), stringOutput());
const pg = group();
try {
  console.log(
    await pr(pg).run({
      cmd: ["ls", "-la"],
    }),
  );
} finally {
  pg.close();
}