Introduction

The Pipeline Expression Language (PEL) or just PE (Pipeline Expression) is a powerful expression language that is used inside a pipeline to dynamically set values. It can be used for data mapping and to dynamically calculate, set and change values at processing time of a pipeline. This gives you a huge flexibility in your pipeline.

Typically a PE starts with #{ and ends with } and is placed in the value part of headers, variables or command parameters. It uses late binding: It will be executed only in case the according entry (header, variable or command parameter) is referenced somewhere.

Here is a simple example of a PE, placed inside the value of a command parameter:

Output:

It also supports interpolation in order to use the PEL like a template language inside a text string. So string concatenation is done for you:

Output:

Its also possible to access the values from one of the pipeline scopes (headers, vars and body), like shown in this example for vars:

Output:

You can also set values using a PEL in combination with the set command:

Output:

Operators

The PEL supports all common operators as also known for most programming languages. It is similar to the expressions in Microsoft Excel, although the syntax is slightly different and aligned a bit more with the traditional programming syntax.

Relational operators

Is equal (==)

Example 1

Output:

Is not equal (!=)

Example 1

Output:

Less than (<)

Example 1

Output:

Example 2

Output:

Less or equal than (<=)

Example 1

Output:

Greater than (>)

Example 1

Output:

Greater or equal than (>=)

Example 1

Output:

Detect alphabetical order with <, >, <=, >=

Example 1

Output:

Regular expression matching (matches)

Example 1

Output:

Logical operators

and

Example 1

Output:

or

Example 1

Output:

not (!)

Example 1

Output:

Mathematical operators

Addition and subtraction

Example 1 - Addition

Output:

Example 2 - Subtraction

Output:

Example 3 - Addition an subtraction

Output:

Example 4 - String concatenation

Output:

Multiplication and division

Example 1 - Multiplication

Output:

Example 2 - Negative multiplication

Output:

Example 3 - Division

Output:

Example 4 - Modulus

Output:

Example 5 - Operator precedence

Output:

Example 6 - Brackets

Output:

Assignment

Example 1

Output:

Example 2

Output:

Working with lists and maps / dictionaries

Creating a new list

Example 1 - A new empty list

Output:

Example 2 - A new list with default content

Output:

Example 3 - A new, nested list

Output:

Accessing lists and arrays

Example 1

Output:

Creating a new map / dictionary

Example 1 - A new empty map

Output:

Example 2 - A new map with default values

Output:

Example 3 - A new map with later binding

Output:

Accessing maps/dictionaries

Example 1

Output:

Navigating objects

A PE can point to values inside an object (or nested data structure), like this JSON for example:

You can navigate any structured object available inside a vars or body scope using the dot operator. For example:

To access a list/array, you can use the index operator []:

Example 1

In this more advanced example, there are different things to mention:

1. We create an inline map as initial vars value.

2. Multi line values are done by using ' (this trims any new line and space character at the beginning and end of the string, which is important to interpret it here as map instead of a string).

3. We set the body using the initial vars value using the command set.body also with a multi line value.

4. Multiple lines can also be set using |. Differently to ' in this case new lines will be kept so that the output of the body will look exactly as formatted in the value parameter. This is perfect if you want to write a template for example with exact format output as the value looks like.

5. There are comments in the configuration. A comment line starts with #.

Formatted output:

Selection Expression

With the selection syntax you can select a subset of items from a given collection to be returned as new collection by specifying a selection expression.

Similar to the WHERE part of an SQL query.

The syntax is like this:

Whereas collectionName is the variable name of the collection (can be an array, map, list, aso.) and selectionExpression is the expression which selects the items to be returned from the list.

Example 1

Lets assume we have a collection of entities like this stored in the body:

Then, we can select a subset of the entries using a selection like this:

Output would be a sublist with the entries matching the criteria:

Here is the same example but with the data set embedded into the pipeline in the vars scope:

Projection Expression

With the projection syntax you can select specific property values out from a collection of objects.

Similar to the SELECT part of an SQL query.

The syntax is like this:

Whereas collectionName is the variable name of the collection (can be an array, map, list, aso.) and projectionExpression is the expression which selects the properties to be returned from each object in the list.

Example 1

Lets assume we have a collection of entities like this stored in the body:

Then, we can select properties from this collection like this:

Output:

And here the example with embedded data set in the vars scope:

PEL Utils

Inside a PE you can use built-in Utils as helpers to simplify work.

A reference about available PEL Utils can be found in this section: PEL Utils Reference.

Example 1

Output:

Example 2

Output: