Overview

The cms-pipelines crate implements Hartmann pipelines – the VM/CMS mechanism for composing data transformations as a series of stages connected by pipes.

Pipeline Syntax

Pipelines are invoked with the PIPE command. Stages are separated by |:

PIPE literal Hello world | console
PIPE literal abc | locate /b/ | console
PIPE literal secret | nlocate /secret/ | console

Each stage reads records from its input, processes them, and writes records to its output. The first stage in a pipeline is a source (no input); the last is typically a sink.

Built-in Stages

The Stage trait can be implemented to add custom stages.

Two-Pass Executor

The pipeline executor runs in two passes:

1. Initialize – Each stage is created and connected to its neighbors via input/output channels.
2. Process + Finish – Records flow through the pipeline. Each stage processes input records and produces output. When the source is exhausted, stages receive a finish signal to flush any buffered state.

Multi-Stream Support

Stages can produce output on primary and secondary streams. The return code from a stage determines which output stream receives the record:

This allows filter stages like locate to split matching and non-matching records into separate streams.

Return Codes

Pipeline execution follows CMS conventions:

Integration with CMS

The pipeline subsystem is wired into cms-core’s CommandProcessor through the ExtCommandHandler trait. When a command starts with PIPE, the extension handler strips the prefix and passes the rest to run_pipe(). Results (return code and output messages) are returned through the standard command result path.