Notifications

Owned by PIPEFORCE
Last updated: yesterday at 5:52 am
22 min read

Version 10

PIPEFORCE comes with a comprehensive multi-channel and multi-language framework which allows to generate notification messages dynamically from templates in different multiple languages and send them to multiple channels. Such a target channel for example can be : Email, Slack, Teams, SMS, WhatsApp, Message Queue and many others.

Additionally, this framework includes also push events. This means a user or group of users can subscribe to certain PIPEFORCE events. Every time such an event happens, they will be informed about this on the configured channel in the preferred language.

Target addresses of notifications can be for example email addresses, IAM usernames, IAM group names, chat groups or other custom addresses as further described below.

What is the difference between a message and a notification?

A notification is usually send to a human user. It is a subset of the message class since a message can be any kind of information send to users and or systems.

Quick Start Guides

For those not interested in the details but want to have a quick result, here are the quick start guides for the most common use cases using the notification framework.

Send a plain text email (simple mode)

In case you would like to send a simple plain text email to a given set of email addresses, you can simply use a pipeline like this:

pipeline: - notification.send: to: ["email1@domain.tld", "email2@domain.tld"] cc: ["email3@domain.tld"] bcc: ["email4@domain.tld"] subject: "This is the subject" message: "This is the plain text body." attachments: ["$uri:attachment:path/to/attachment"]

The notification framework will automatically detect the target channel (which is email in this example) and renders and sends the according mime message for you. This is the easiest approach.

Aliases to the notification.send command are notifcation.put and mail.send.

In some cases you might have the requirement to further customize the mime message. For this, see the examples below for more details.

Send a plain text email (customize mode)

In case you would like to further customize the mime headers or the multipart body of an email, you can use this approach instead:

pipeline: - notification.send: input: { "headers": { "to": ["email1@domain.tld", "email2@domain.tld"], "cc": ["email3@domain.tld"], "bcc": ["email4@domain.tld"], "subject": "This is the subject", } "body": { "contentType": "text/plain", "content": "This is the plain text body." } }

Note that the notification input is a 100% mime message (RFC2045. ) compliant format. This means it is separated into a headers and a body section. In the headers section all headers from the MIME standard are supported. They will be forwarded to the target channel, like email client for example. Also the body section is fully compliant with RFC2045 so you can construct any multipart body structure you need for the target client. For further details about such a structure please consult external documentation about how to construct valid multipart mime messages.

Send a mixed-content multipart mime message

In case you would like to compose a multipart mime message with mixed content, you can use this example as a template:

pipeline: - notification.send: input: { "headers": { "to": ["email1@domain.tld", "email2@domain.tld"], "cc": ["email3@domain.tld"], "bcc": ["email4@domain.tld"], "subject": "This is the subject", } "body": { "contentType": "multipart/mixed", "content": [ { "contentType": "multipart/alternative", "content": [ { "contentType": "text/plain; charset=UTF-8", "content": "This is the plain text body." }, { "contentType": "text/html; charset=UTF-8", "content": "$uri:property:global/app/my.app/template/body" } ] }, { "contentType": "image/jpeg", "contentDisposition": "attachment; filename=\"bild.jpg\"", "contentTransferEncoding": "base64", "content": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAHBgoJCAkLC..." }, { "contentType": "application/pdf", "contentDisposition": "attachment; filename=\"dokument.pdf\"", "contentTransferEncoding": "outbound", "content": "$uri:attachment:some-attachment-uuid" } ] } } }

For specific details about the structure of a multipart mime message, please have look at the RFC2045.

As you can see in this example, the body can be a list of nested content objects.

The content value itself can be a fixed text, a base64 string or a PIPEFORCE URI pointing to a resource to load.

Attachments

You can also add (file) attachments to notifications. Depending on the target channel they will be provided as downloads or embedded when the notification shows-up in the channel client.

Simple Mode

In order to add (file) attachments to your notification, you have multiple options. The easiest approach is to use the attachments attribute of the notification.send command:

The command will try to autodetect all required attachment information from the list of given attachments and append it to the body content in multipart mime format at the very end. It wont overwrite any existing part there.

Combined Mode

This way you can combine the simplified parameter format with a specific one. For example:

The final multipart mime message, which will be send to the email channel will be this (since the simplified parameters are merged into the input notification):

Note:

  • All given values from command parameters will overwrite the values in the mime message if there are any.

  • In case the section doesn’t exist yet in the mime message, it will be created.

  • It could be that the original mime message needs to be re-arranged to insert the parameter values: For example in case you would like to add attachments via command parameter to an existing text/plain input body. In this case, the body will be converted to multipart/mixed and the text plus attachments will be nested inside.

Customized Mode

Another option is to create the content JSON objects inside the mime message manually and set the attachments accordingly accordingly to the RFC Mime Messages specification. This is the solution which is most flexible and highly customizable. But it is also the most complex approach.

Attaching content references

You can also use content references loaded into a pipeline scope like vars or body as source for attachments. For example:

In this example the file is first loaded (implicitly) to body using the command drive.read and then linked to the attachments parameter. You can place any content reference as attachment.

Using this approach it is also possible to create the attachments dynamically on-the-fly in the pipeline (for example by using a template engine) and then add them finally to the email.

Templating Notifications

You can create dynamic notification texts and layouts using templating.

Using a Transformer​

You can use a transformer command like transform.ftl in order to render the message. For example:

In this example, the message is rendered and then stored in the body using transform.ftl. The command notification.send then picks up the rendered message from the body and uses it as email message.

And here is an example to load the template from the property store from location global/app/myapp/template/email:

Embedded template URIs

It is also possible to embed the template references inside the mime notification using PIPEFORCE URIs. For example:

In this example the $uri:template: URI is used. This will load the template from given location in property store and will render it using the template engine which fits the Content-Type of the property. By default, this is the FreeMarker Template Engine. Also all other PIPEFORCE URIs are supported here instead.

Any time the notification is sent, the elements of the mime message will be recursively scanned for PIPEFORCE URIs. In case one is found, it will be called and in case it is supported, the current mime notification object will be auto-passed to the template and can be used there to access all relevant data from the mime message to render the template. Here is an example for a message rendered using the FreeMarker Template Language:

Template Model

Inside each template the mime notification will be provided as the template root or body model, depending on the template approach used.

This root model will provide these fields:

  • headers = A key-value map which contains all header values used to configure the notification.

  • to = A list of resolved to recipient objects.

  • cc = A list of resolved cc recipient objects.

  • bcc = A list of resolved bcc recipient objects.

  • from = The from recipient object.

  • subject = Contains the (resolved) subject text.

  • content = Contains the nested mime part objects as configured in the notification configuration.

  • excludedAddresses = A list of all recipient objects excluded in this notification.

  • allAddresses = Combines all recipient objects from to, cc and bcc into a single list and removes any duplicates.

  • model = The message model provided from external. The structure of this model depends on the provider.

  • uuid = The unique id of this notification, if set.

  • attachments = A list of mime part objects extracted from the content body, defining the attachments of this notification.

Recipient

The structure of a notification recipient is like this:

  • name = The display name of the recipient.

  • locale = The locale of the recipient to be used for this notification.

  • address = The address uri of the recipient.

  • plainAddress = The plain address without any URI prefixes or query parameters.

  • channel = The channel of this recipient (for example email, slack, ...)

Part

The structure of a notification mime part is like this (also see official MIME specification for this):

  • contentType = The MIME content type of the content value.

  • contentDisposition = The disposition header as defined in the MIME specification.

  • contentTransferEncoding = The transfer encoding as defined in the MIME specification.

  • content = The content of the part which can be one of:

    • A PIPEFORCE URI if contentTransferEncoding is set to outbound-url.

    • A base64 encoded string if contentTransferEncoding is set to base64.

    • A text value with format given by contentType.

    • A list of nested Part’s in case contentType is multipart/mixed.

Addresses and address resolving

In the headers section of a mime notification you can declare the addresses of the recipients where the notification must be send to. Here is an example how to do this directly inside the mime notification JSON:

As an alternative you can also use the simple mode by declaring them as parameters directly on the command:

These address parameters will be converted to a mime notification JSON and placed in its header section for you.

Addresses prefixed with $uri:

By default all addresses are assumed to be valid email addresses. But you can also use other address types to send the notification to a different channel. In case an address is not an email address, it always must be prefixed with $uri: . Here is an example to use a Slack channel as target address instead:

In this example, the message will be send to a Slack channel instead of via email. You can also combine different addresses of different types like this example shows:

In this example the notification will be rendered as a Slack message and send to the Slack channel with name mychatgroup and additionally it will be rendered as email message and send to the email address email1@domain.tld.

Final and expandable addresses

There can be final and expandable addresses:

  • Final address: The address can be used as it is by a channel to deliver the message. For example an email address.

  • Expandable address: The address needs to resolved (expanded) to one or more final addresses first, before it can be used by a channel to deliver the notification. For example an address referring to a user in IAM must first be expanded (resolved) to the user’s email address before the notification can be send.

Here is an example how such a resolving could look like:

Expandable addresses could also be recursive.

The resolving (expanding) is automatically done by the notification framework.

Some notes about this example:

  • A: Final addresses. They will be used to send the notification directly to the channels.

  • B: The supervisor will get this notification via its default email.

  • C: Any user who is member of group sales will get this notification via its preferred notification channel. In this example, this is user manager1 and his notification channels from IAM are sales1@company.tld and private1@mail.tld .

  • D: The group sales contains another group admins and therefore all members of this role will additionally get the notification via their preferred address.

When the notification is processed, all groups and users will be resolved and the preferred channel will be used to send the notification.

Handling address duplicates

Duplicate addresses which appear after expanding will be removed (for example because a user is member of multiple groups or roles configured for the notification, so his address is used only once).

In case the same addresses is used in to, cc and bcc, then the address in the lowest visibility will be kept. The visibility is like this (from lowest to highest):

  1. bcc

  2. to

  3. cc

So for example if the same address is given in bcc, cc and to it will be removed from to and cc and only kept in bcc. Another example: If the same address is given in to and cc, it will be kept in to and removed from cc. And if the same address is given in bcc and cc, it will be kept in bcc and removed from cc. And so on.

$uri:email:

This final address will send the notification via email. The format is like this:

For example:

This is an alternative to not using the prefix at all. So this is also a valid email address:

$uri:user:

This expandable address will lookup the user with given username in IAM (user management) and be converted to the registered target address of the user which is an email address by default.

The format is like this:

Whereas <username> must be an enabled and exiting user in IAM with is username.

For example:

In case the user is disabled or doesn’t exist, this address will be ignored and removed from the address fields.

For example:

Let’s assume as default address of the user maxhuber in IAM the email address foo.bar@logabit.com is configured, then an address field like this:

"to"

["$uri:user:maxhuber"]

Will be resolved to this:

"to"

["$uri:email:foo.bar@logabit.com"]

$uri:group:

This expandable address will lookup a group with given name in IAM (user management) and it will be resolved to the default addresses of all user members of this group.

The format is like this:

Whereas <groupname> must be an enabled and existing group in IAM with this group name. For example:

In case the group is disabled or doesn’t exist, this address will be ignored and removed from the address fields.

For example:

Let’s assume there is a group sales in IAM with with these member users and their default target addresses configured in IAM:

Username

Target Address

Username

Target Address

teamlead

teamlead@mail.tld

user1

$uri:slack:mychannel

user2

$uri:email:user2@mail.tld

Then, an address field like this:

"to"

["$uri:group:sales"]

Will be expanded (resolved) to this:

"to"

["$uri:email:teamlead@mail.tld", "$uri:slack:mychannel", "$uri:email:user2@mail.tld"]

$uri:slack:

This final address will send the notification to a Slack channel as configured by the connection config id.

The format is like this:

Whereas <connection-config-id> must be an id of a valid and enabled connection configuration.

For example:

$uri:teams:

This final address will send the notification to a Microsoft Teams channel as configured by the connection config id.

The format is like this:

Whereas <connection-config-id> must be an id of a valid and enabled connection configuration.

For example:

$uri:message-route:

This is a final address: It will forward the message to the message broker using the given routing key.

The format is like this:

Whereas <routing-key> is the target routing key where the message must be forwarded to. The mime notification headers will be set as message headers and the body will be serialized as JSON into the playload of the message.

For example:

$uri:pipeline:

This expandable address will call the pipeline given by path, puts the mime notification into its body and executes it. It expects a JSON array of addresses to be returned, all starting with prefix $uri:. Otherwise, the address will be ignored. These addresses will then replace the $uri:pipeline: address in the fields.

The format is like this:

For example:

The dynamic-addresses pipeline could look like this example to return addresses dynamically:

$uri:property:

This expandable address will load the property at given path and expects the value to be a JSON array containing the addresses to be returned.

The format is like this:

For example:

The my-addresses pipeline could look like this example:

Resolving of expandable addresses

Expandable addresses are resolved (expanded) recursively until the final address appears. So for example an expandable address given like this:

"to"

["$uri:pipeline:global/app/my.app/pipeline/my-adresses"]

Could return and resolve to:

"to"

["$uri:email:user1@logabit.com", "$uri:group:sales"]

Which in term could resolve to this in the next recursion:

"to"

["$uri:email:user1@logabit.com", "$uri:user:user2", "$uri:user:user3", "$uri:user:user4"]

Which resolves to these final addresses:

"to"

["$uri:email:user1@logabit.com", "$uri:email:user2@logabit.com", "$uri:email:user3@logabit.com", "$uri:email:user4@logabit.com"]

Excluding addresses

It is possible to exclude certain addresses from a mime message. To do so, you can use the custom header x-pipeforce-exclude-addresses and specify the addresses to be excluded. For example:

In this example, the New lead notification will be send to all members of group sales excluding those users also in group vacation.

Exclude addresses can be final and expandable addresses.

Multi-Language Notifications (i18n)

The notification framework supports multi language notifications. This means a notification can be send in different languages to multiple recipients. The framework handles the way how the templates related to the selected language will be loaded and clusters the notifications based on the selected language (locale).

How to enable it

By default i18 is turned-off. In order to enable multi-language (i18n) support, you have to set the header x-pipeforce-multilang to true in your notification configuration:

How it works

The main concept behind i18n support in notifications is that any $uri:template, $uri:property or $uri:pipeline which is used in subject or body of the notification will automatically be suffixed with the selected locale (= language key): For example, if you use the URI $uri:template:global/app/com.acme.myapp/mytemplate inside your notification configuration like this:

… and the locale de (german) is selected, the URI will automatically become $uri:template:global/app/com.acme.myapp/mytemplate_de: It will be prefixed with _de so the template specific to german language can be loaded. And in case the locale en (english) is selected, this prefixed URI will be loaded instead: $uri:template:global/app/com.acme.myapp/mytemplate_en (the suffix _en is appended). This way you can organize your templates in the property store in a clear way. For example it could look like this:

  • global/app/com.acme.myapp/template/

    • mytemplate_de

    • mytemplate_en

    • mytemplate_fr

    • mytemplate_sk

Here are the steps, how this all works together:

  1. It will look for a query param ?locale=<lang> in each address in to, cc or bcc fields.
    For example: $uri:email:foo@bar.tld?locale=de. If no locale exists it will use en as default.

  2. If the address is an IAM address like $uri:group: or $uri:user: it will use the value of user attribute locale from IAM for the locale, if exists. Otherwise, it will fallback to en as default. If both are given (query param and locale attribute in IAM), the query param will have precedence.

  3. It will split a notification to multiple notifications for each new language and will manage to split also addresses according to these languages. For example if there are these TO addresses set: $uri:email:my@mail.tld?locale=de, $uri:email:another@mail.tld?locale=en this means two emails will be rendered and send: One with de as language with german text in subject and body and one with english text. Finally, there will be two emails: The DE one is sent to my@mail.tld and the EN one sent to another@mail.tld. The same is true for addresses in cc and bcc.

  4. As already mentioned above, any $uri:template, $uri:property or $uri:pipeline which is used in subject and body of the notification configuration will automatically be suffixed with the selected locale. So for example $uri:template:global/app/com.acme.myapp/mytemplate will become $uri:template:global/app/com.acme.myapp/mytemplate_de in case the detected language (locale) is set to de.

Additionally, the detected language (locale) will be set as Content-Lang header (which is a standard MIME header) so it can be used in the templates and/or at target client side as well.

Static suffix / disable suffixing

In some situations, suffixing the template resource path is not wanted or a static suffix must be used for all of such resources. For this, you can set the header x-pipeforce-multilang-suffix to a static suffix. If this is set, it will be used and the calculated suffixes from the language (for example _en) will be ignored.

In case you don’t want to use suffixes at all, set this header to empty string "". Then, always the template resource without suffix will be loaded. You can then add language specific logics inside your template by using the Content-Lang header which is also provided into the template.

Multi-Channel Notifications

The notification framework also supports sending a notification to different channels. This way it is possible beside via email, the same notification can be send to a Slack channel, via SMS and Teams for example. For each channel, different templates can be provided in order to adjust the format of the message so it fits the target channel.

How to enable it

By default multi-channel support is disabled. In order to enable multi-channel support, you have to set the header x-pipeforce-multichannel to true in your notification configuration:

How it works

If multi-channel support is enabled, the framework will handle a notification similar to the multi-language approach:

  • It will look for the final addresses in the address uri. For example:

    • $uri:email: → email channel

    • $uri:slack: → slack channel

    • $uri:teams: → teams channel

    • $uri:sms → sms channel

    • aso.

  • It will split a single notification to multiple notification messages: For each new channel and will manage to split also addresses according to these channels. For example if there are these TO addresses set: $uri:email:my@mail.tld?locale=de, $uri:slack:mygroup this means two notifications will be rendered and send: One as email and the other one as Slack message. The same is true for addresses in CC and BCC.

  • Any $uri:template, $uri:property or $uri:pipeline which is used in subject and body of the notification will automatically be suffixed with the selected channel. So for example $uri:template:mytemplate will become $uri:template:mytemplate_slack in case the channel is set to slack via address. This way, different templates can be provided in the property store and the notification framework will pick-up the one which belongs to the selected channel.

Additionally, the detected channel will be set as x-pipeforce-channel header so it can be used in the templates as well.

Static suffix / disable suffixing

In some situations, suffixing the template resource path is not wanted or a static suffix must be used for all of such resources. For this, you can set the header x-pipeforce-multichannel-suffix to a static suffix. If this is set, it will be used and the calculated suffixes from the channel (for example _email) will be ignored.

In case you don’t want to use suffixes at all, set this header to empty string "". Then, always the template resource without suffix will be loaded. You can then add channel specific logics inside your template by using the x-pipeforce-channel header which is also provided into the template.

Personalized Notifications

Lets a assume you have a list of recipient addresses configured in your notification configuration like this:

… and instead of sending the same static email to all of these recipients you would like to generate and send individual text messages for each.

So for example someUser1 should get this email text:

… and someUser2 should get this email text:

To do so, you have to set the header x-pipeforce-personalized to true:

In this case for each recipient the templating rendering is called again so the content can be personalized then. The to header will always be set to the personalized recipient and cc and bcc will be deleted.

Inside the template you can then access the personalized recipient by accessing the to header or by calling allAddresses[0]. Here is an example how such a template could look like:

This will be repeated for every recipient, regardless whether it is set on to, cc or bcc.

Finally, the text result of each template rendering will be send as individual message.

Push Notifications

Push notifications are system and business events, a user can subscribe to. Any time there is a match, such an event is automatically converted to a notification and will be send to the registered user's preferred communication channels like email or chat group for example.

A push notification always starts with a message sent with routing key pipeforce.event.#. All messages having a routing key starting with this prefix will be stored in the pipeforce_push queue.

By default the pipeforce_push queue has these restrictions:

  • Max. number of messages in queue: 20.000

  • Max. time to life: 48 hours

  • Max. size of all messages in queue: 100 MB

As soon as one of these restrictions are met, oldest messages will be deleted.

Subscribe to push notifications

In order to receive push notifications, you have to subscribe to event keys, called topics, first.

This is done in PIPEFORCE by creating a JSON document in a folder with name push inside your app. For example global/app/my.app/push/my-subscription.

This JSON document is called push subscription and has a structure like this:

The fields of this JSON are described below.

"enabled"

Is the push subscription enabled (true)? Default is true in case attribute is not given.

"includeTopics"

The routing keys of messages inside the pipeforce_push queue to get informed about.

"excludeTopics"

The routing keys to exclude explicitly (for example if it is sub-pattern of an includeTopic).

  • Both includeTopics and excludeTopics support the wildcard pattern from RabbitMQ:

    • * = A single part between two dots .

    • # = Anything which comes after.

"headers"

Since a push subscription is based on a mime notification it can contain all of its headers such as:

  • to: A list of addresses, to send the push notification to.

  • cc: A list of addresses, to send the push notification to in CC.

  • bcc: A list of addresses, to send the push notification to in BCC.

  • subject: The subject to be used for the push notification. If no subject is set, a default subject is generated.

  • es: A list of addresses to exclude from the push notification.

"body"

The multipart body message to be send as the push notification payload. If no body exists, a default body is generated. Everything described for the body of mime notification can also be applied here.

Topics / Event Keys

An event key (also known as topic) is a routing key pattern of an event message which always starts with prefix pipeforce.event, followed by the concrete event type.

It can be used to send push notifications every time such an event happens or to register a pipeline listener and listen for such an event to a trigger a pipeline execution. Here is an example how to add such an event message listener to a pipeline:

When registering a listener for an event key, also patterns can be used in the event key:

  • * = Stands for a single word up to the next period .

  • # = Stands for multiple words with multiple periods in between.

For example: pipeforce.event.workflow.# matches all workflow event messages while pipeforce.event.workflow.*.my.app matches only workflow event messages inside the app my.app.

Custom Topics

You can also send push notifications based on your custom topics.

To do so, you need to configure these steps:

  • Make sure to send event messages using a routing key format like this pipeforce.event.<yourName> using message.send command. Replace <yourName> by the custom name you would like to use. Best practise is to use the prefix app and the name of the app followed by the concrete event name to not potentially collide with other topics from other apps. For example: pipeforce.event.app.com.company.myapp.myaction
    This way it is also easy to filter later using patterns. Examples:

    • pipeforce.event.app.# = Listens to all events from all apps.

    • pipeforce.event.app.com.company.# = Listens to all events from all apps from given company.

    • pipeforce.event.app.com.company.myapp.# = Listens to all events from app myapp.

    • pipeforce.event.app.com.company.myapp.myaction = Listens to the exact myaction event.

  • Create a new push notification configuration inside the push folder of your app and make sure to map the includeTopics correctly to the recipients.

Example push notification configuration:

This push notification configuration makes sure, every time a custom event for the app myapp appears, it will be pushed to all users of group $uri:group:my-custom-group.

Exclude addresses from topics

There is an exclude list to make sure certain push notifications are not sent to these addresses. They can be configured using the header x-pipeforce-exclude-addresses as described for the mime notification.

Furthermore, it is possible to set the attribute excludeTopic for a specific user in IAM. In case the addresses $uri:group or $uri:user are used and excludeTopic exists for a matching user, it will be checked whether this user should be excluded from the push notification.

See here as an example:

image-20240324-102250.png

There can be multiple attributes with name excludeTopics or the values can be separated by comma. Both approaches work and can also be combined (for example to group certain excludeTopics).

Templating Push Notifications

Since push notifications are based on Mime Notifications, all rules of mime notifications apply.

In case no subject or body is defined, a default subject and body is created for each push notification.

In case you would like to use a custom text there, you can do so by setting it in the push subscription. For example:

It is also possible to refer to templates in subscription and content (which will be rendered later in the process):

Global Push Configuration

You can disable / enable global push notifications in admin settings.

Note: If disabled, events will not be catched from the pipeforce_push queue so outdates, too many event messages there will be deleted.

Also see the restrictions of the queue above.

Default Push Subscriptions

Inside global/app/io.pipeforce.admin/push/ there are default push subscriptions for admins, developers and users.

Testing Notifications (dryRun)

In order to test notification delivery, you can enable dryRun for such a notification by adding a marker [pipeforce.dryrun:someName] in the subject or adding an email address ending in someName@pipeforce.dryrun.

In case such a marker exists, the notification wont be delivered but instead stored to the internal cache. From there you can pickup and apply your tests.

For further information see an example for dryrun sending emails: Emailling .