Pipeline Expression Language (PEL)
SINCE VERSION 1.0
- 1 What is the Pipeline Expression Language (PEL)?
- 2 Basic usage​
- 3 Accessing Pipeline Objects​ (headers, vars, body, request, …)
- 4 Navigating nested data structures​
- 5 Accessing attributes with special name
- 6 Relational operators​
- 7 Logical operators​
- 8 Mathematical operators​
- 9 Assignment​
- 10 Lists and Objects
- 11 Filtering​
- 12 Ternary Operator (If-Then-Else)​
- 13 Handling non-existent fields (check if field exists)
- 14 #this and #root variables
What is the Pipeline Expression Language (PEL)?
By default, the parameters and variables of an automation pipeline are fixed and static values. In some use cases this is not sufficient enough since these values must be calculated, validated, re-wired or prepared before they can be passed to a command. This is what the Pipeline Expression Language (PEL) can be used for.
The Pipeline Expression Language (PEL) or just PE (Pipeline Expression) is a powerful and easy to learn expression language based on Spring EL and adds additional features to this widely used expression language. It can be used inside an automation pipeline to access a specific portion of data and dynamically calculate, convert or set values.
The Pipeline Expression Language is optional. You do not need it for basic tasks working with PIPEFORCE. But it gives you more flexibility and power to automate and integrate nearly any process without the need to learn a complex programming language. So it is wise to learn at least the basics of it.
It depends on your basic knowledge, but in average it is a matter of about 15-30 minutes to understand and use the basic concepts of this powerful expression language.
Basic usage​
Typically a PE (Pipeline Expression) starts with ${
and ends with }
. Here is a simple example of a PE, placed inside a command parameter:
pipeline:
- body.set:
value: ${1 + 1}
Optionally, you can also use #{
to start a PE, but this must be placed inside quotes "
and "
:
pipeline:
- body.set:
value: "#{1 + 1}"
Output:
2
The PE prefix #{
is only meant as an alternative in case ${
cannot be used in some situations. Using ${
should be preferred whenever possible.
A PE can be placed in the value part of pipeline headers and variables, in parameter values of most commands and in the body of a pipeline.
It uses late binding: It will be executed only in case the according entry (header, variable, command parameter, ...) is referenced somewhere.
It also supports interpolation in order to use the PEL like a template language inside a text string. So string concatenation can be done like this:
Output:
Accessing Pipeline Objects​ (headers, vars, body, request, …)
Using a Pipeline Expression, you can access the values (= attributes) inside pipeline scopes like headers
, vars
, and body
in order to read and write them.
These scopes are provided automatically as implicit objects and therefore they're always available inside any pipeline expression even if no section like headers:
, vars:
or body:
was declared or any other allocation was done.
Also see: https://logabit.atlassian.net/wiki/spaces/PA/pages/2552856577 .
vars
(variables)
This object gives you access to all variables of the current pipeline.
Also see https://logabit.atlassian.net/wiki/spaces/PA/pages/2552856577/Pipeline+Objects#vars .
Let's assume, you have defined a variable counter
and you would like to access this counter in your expression, then you could write an expression like this:
Here is an example of a pipeline which declares this variable and uses the Pipeline Expression to output the value of counter
to the body:
This will result in an output like this:
headers
This object gives you access to all pipeline headers of the current pipeline.
Also see https://logabit.atlassian.net/wiki/spaces/PA/pages/2552856577/Pipeline+Objects#headers .
Do not mix up Pipeline headers with HTTP headers. The latter can be accessed using the ${request.headers}
object instead.
Here is an example of a pipeline which accesses the header attribute contentType
and writes it to the body:
This will result in an output like this:
body
This object gives you access to the current body of the current pipeline.
Also see https://logabit.atlassian.net/wiki/spaces/PA/pages/2552856577/Pipeline+Objects#body .
Here is an example which defines an initial body value and replaces this with another text in the pipeline:
This will result in an output like this:
Other implicit objects (request, response, exception)
Beside the scopes vars
, headers
and body
there are more implicit objects available:request
, response
and exception
.
Here is an example to access the value of a request query string like ?someKey=someValue
of a pipeline which was triggered by an HTTP request using the request
object:
This will result in an output like this:
For more information about all available implicit objects and their attributes, see:
https://logabit.atlassian.net/wiki/spaces/PA/pages/2552856577
Navigating nested data structures​
A Pipeline Expression can also be used to navigate and extract nested attributes from an object or array, like this JSON for example:
You can navigate and read any structured object available inside the scopes using the dot operator. For example:
or using the associative array navigator:
Note: The dot operator is easier to use but throws an exception in case the name
attribute doesn't exist. The associative array navigator don't.
And to access entries in a list/array, you can use the index operator []:
Example​
In this more advanced example, there are different things to mention:
We create an an initial body and set an inline JSON document there
In the pipeline we convert the JSON from the initial body in order to output the JSON values as text lines, overwriting the initial body.
Multiple lines can also be set using
|
. Differently to'
in this case new lines will be kept so that the output of the body will look exactly as formatted in the value parameter. This is perfect if you want to write a template for example with exact format output as the value looks like.There are comments in the configuration. A comment line starts with
#
.
See the official YAML documentation about how to deal with multi-line values. Here is a good summary: https://yaml-multiline.info/
Formatted output:
Accessing attributes with special name
In order to make it as easy as possible, the name of attributes you would like to access via PEL should follow the rules of common variable naming. So it should not be a number or contain any special characters like for example ;
,
-
#
+
*
?
!
§
%
&
.
Here are some example of valid standard names:
myVariable
my_variable
myvariable
And here are some example of special names which can cause problems when used inside a Pipeline Expression:
my:Variable
my-variable
my.variable
123
Sometimes it is not possible to manage and change the attribute names since they are given by external inputs. In this case you can access the attributes using the associative array notation. Example:
Relational operators​
Is equal ==
​
Example 1​
Output:
Is not equal !=
​
Example 1​
Output:
Less than <
​
Example 1​
Output:
Example 2​
Output:
Less or equal than <=
​
Example 1​
Output:
Greater than >
​
Example 1​
Output:
Greater or equal than >=
​
Example 1​
Output:
Detect alphabetical order with <
, >
, <=
, >=
​
Example 1​
Output:
Regular expression matching matches
​
Example 1​
Output:
Logical operators​
and
​
Example 1​
Output:
or
​
Example 1​
Output:
not
, !
​
Example 1​
Output:
Example 2​
Output:
Mathematical operators​
Addition +
and subtraction -
​
Example 1 - Addition​
Output:
Example 2 - Subtraction​
Output:
Example 3 - Addition an subtraction​
Output:
Example 4 - String concatenation​
Output:
Multiplication *
and division /
, %
​
Example 1 - Multiplication​
Output:
Example 2 - Negative multiplication​
Output:
Example 3 - Division​
Output:
Example 4 - Modulus​
Output:
Example 5 - Operator precedence​
Output:
Example 6 - Brackets​
Output:
Assignment​
Example 1​
Output:
Example 2​
Output:
Lists and Objects
Creating a list (JSON Array)​
Example 1 - A new empty list​
Output:
As an alternative you could also use the internal notation, but this is deprecated and is only documented here for backwards compatibility:
Output:
Example 2 - A new list with default content​
Output:
Example 3 - A new, nested list​
Output:
Accessing lists​
Example 1​
Output:
Creating a new object (JSON object)​
Example 1 - A new empty map​
Output:
Example 2 - A new map with default values​
Output:
Example 3 - A new map with later binding
Later binding means that a value is set on the object after it was declared.​
Output:
Accessing maps​
Example 1​
Output:
You can also use the associative array notation to access the max
attribute:
Save navigation - Avoid NotExists- and NullPointerException
In case you would like to access a property of an object which doesn't exist, a NullPointerException
or AttributeNotExistsException
will be thrown, indicating that the object you're trying to access doesn't exist. Let's see this example which will result in such an exception:
This will throw an exception (= error) since the object data
is null
and therefore the attribute name
cannot be found at all.
In this chapter we will discuss the options you have to avoid such errors.
Elvis operator for save navigation
Since Version 10 ​
In order to return null
instead of throwing an exception, you can use the safe navigation operator ?.
. This example will not throw an exception. Instead, it will simply put null
into the body:
It is also possible to use this operator on nested properties:
This example will also put null
into the body instead of throwing a NullPointerException
.
But this works only if the level you’re trying to access doesn’t exist at all (is null
). It doesn't work if there is an element but this element just doesn't contain an attribute with given name. This for example will not work here:
Check if object has attribute before accessing
Since version 9.0
As an alternative, you can use the @data.has
util to check whether an attribute exists before you're trying to access it. This will also work for non-existing attributes on existing elements:
You can also use it inline combined with a ternary operator:
Use an the associative array notation
Since version 9.0
Another option is to use the associative array notation to access an attribute which could potentially not exists. Example:
But this works only for one level.
Filtering​
The PEL can be used to filter lists in an easy way.
Selection Expression .?
​ (filter for objects)
With the selection syntax you can select a subset of items from a given collection to be returned as new collection by specifying a selection expression.
Similar to the WHERE
part of an SQL query.
The syntax is like this:
Whereas collectionName
is the variable name of the collection (can be an array, map, list, aso.) and selectionExpression
is the expression which selects the items to be returned from the list.
Example 1​
Lets assume we have a collection of entities like this stored in the body:
Then, we can select a subset of the entries using a selection like this:
Output would be a sublist with the entries matching the criteria:
Here is the complete example in one pipeline:
Projection Expression .!
​ (filter for attributes)
With the projection syntax you can select specific attribute values out from a collection of objects.
Similar to the SELECT
part of an SQL query.
The syntax is like this:
Whereas collectionName
is the variable name of the collection (can be an array, map, list, aso.) and projectionExpression
is the expression which selects the properties to be returned from each object in the list.
Example 1​
Lets assume we have a collection of entities like this stored in the body:
Then, we can select properties from this collection like this:
Output:
And here is the complete example:
Ternary Operator (If-Then-Else)​
The ternary operator can be used to define an if-then-else condition in your expression in a single line.
This example would output condition is false
in the body. As you can see, the part left of the question mark ?
defines the condition to be evaluated. If this condition evaluates to true
, then the part left of the colon :
is executed. Otherwise the right part:
Elvis Operator (Default Value)
You can shorten the ternary operator in case you would like to check a value for null
or false
and if given, fallback to a default value in this case. You can do so by using the so called "Elvis" operator. The syntax is this:
Whereas VALUE
must be a PEL value or variable. If VALUE
is false
or null
, then the DEFAULT_VALUE
will be returned. Otherwise VALUE
will be returned.
For example if VALUE
is null
:
Output will be in this case:
Another example, now with VALUE
not null
and not false
(it contains a string value):
Output will be in this case:
Handling non-existent fields (check if field exists)
Sometimes it is necessary to check whether a given field (attribute) on an object exists before trying to access it. To do so, you have different options:
Example 1: With associative array and null check
This will return:
The caveat of this approach is that it will treat a null
value of the attribute the same way like this attribute doesn't exist:
This will return (which has wrong semantics in this case, since field bar
exists but has null
value):
Example 2: Use @data.has(..) utility
A better approach is to use the PEL utility @data.has(...) which will check wether the field exists:
This will return:
#this
and #root
variables
The special variables #this
and #root
are available in any PEL whereas #root
always points to the root of the evaluation context which is by default usually the pipeline message with headers
, vars
and body
scope. So if you are in an iteration loop or inside a selection or projection, you can always use #root
to access the evaluation context. Example:
As you can see in this example, #root
is used here in order to put the whole root context into the utility in order to check for attribute existence by path.
The variable #this
has a similar meaning but while #root will never change for a given PEL evaluation, the value of #this
can change for example in case you are in an iteration loop.