SINCE VERSION 8.0

What is a Webhook?

A webhook is a unique HTTP URL in PIPEFORCE which can be called by an external system. In case the external system calls the webhook url, this will trigger any custom action inside PIPEFORCE.

This is a very effective and straight forward possibility to integrate external systems into PIPEFORCE using a push / listening approach, with a very low requirements barrier to the caller system. Nearly every modern system can be integrated this way.

The URL of such an webhook, which gets called by the external system, has a format similar to this:

You can create and manage multiple custom webhooks and URLs each with individual settings. When called, a webhook will be validated first and then an internal message gets produced, pipelines can listen to using event.listen or message.receive. This way you can make sure, webhooks comply with a given set of rules before they will be passed across multiple pipelines, queues and optional microservices:

Create a Webhook endpoint​

Before some external service can call your webhook, you have to create and endpoint for it. You can create such a webhook using the command webhook.put. Here is an example using it in a pipeline:

Or you can use the CLI to create the Webhook:

The possible parameters to create a webhook are:

• eventKey = The key, the pipelines will listen to (required).

• payloadType = (optional) What to do with the payload, before it is send to the messaging queues. Possible values are:

  • raw = The payload will be send without any conversion.

  • base64 = The payload will be encoded to base64 (default).

  • ignore = No payload will be send with the webhook message even if specified in the request.

• maxPayloadLength = (optional) The max allowed number of bytes in the payload (body) of the webhook. If bigger, the webhook will be rejected. Default value is 512000 (500KB). The maximum possible value for this is 4194304 (4MB).

As eventKey define the internal unique name of the webhook. It's good practise that this name is lower case, grouped by periods and starts with prefix webhook.. The result after executing the webhook.put command is a JSON document like this:

The uuid (also called token) is the unique identifier of your webhook. Since it is very hard to guess this token, it is used to secure your webhook. The external services can use the webhookUrl in order to call your webhook.

In case you're using the PIPEFORCE Developer Portal, you can create a Webhook with a few clicks:

Define an event key in the creation dialog and click "Add":

Finally, the Webhook gets listed and you can get its token from the list:

Calling a Webhook​

After you have setup the Webhook successfully, it can be triggered (called) from external. To do so, send a GET or POST HTTP request to the webhook url which was returned when you created it. For example:

https://hub-try.pipeforce.org/api/v3/command/webhook.receive?token=abcdef

Long polling​

Calling a webhook is by default executed async. This means a webhook call like this will start the execution of the webhook but will not wait for a response of it. Instead, long polling will be used:

Until the backend is not finished with the webhook processing, it will returnd with a redirect response so the client must again send a request to the server. This is fully transparent to the caller as long as he uses a HTTP client which supports standard HTTP redirects.

Trigger pipeline by a Webhook call​

After you have successfully setup the webhook, any time the webhook url is triggered (called) from the outside, a new message is produced inside PIPEFORCE, which can then be consumed by any pipeline. To do so, use the event.listen or message.receive command to listen for such new event messages. Here’s an example which sends an email whenever a new lead was created using a webhook with the eventKey =webhook.lead.created:

The input body of the event.listen command is the payload of the event message submitted from the outside caller.

In case the sender has sent some payload in the body of the webhook request, this payload is made available for you by default as base64 encoded string in the origin field of the event. To access this data, you have to convert this value as shown in this example:

In case the payload is a serializable format like a string or a JSON document for example, you can set payloadHandling to raw for the webhook. In this case, it is not needed to convert the payload from base64, so you can use it directly:

For security reasons, by default, the webhook pipeline is executed with very limited anonymousUser privileges. So, make sure that you use only commands in your pipeline which can be executed by this user. In case you need more privileges, you can use the iam.run.as command to switch to the privileges of the given user before executing the command. See the IAM portal for the permissions (or roles) of a given user. Also see Groups, Roles, and Permissions for more details on user privileges / permissions.

List existing Webhooks​

To list all existing webhooks, you can use the webhook.get command:

You will see a JSON / YAML list with details about all existing webhooks.

In order to get the details of a single webhook, use the webhook.get with the param uuid, where uuid is the token of the webhook you want to list:

You can also list all existing Webhooks in the Portal:

Edit a Webhook​

In order to edit an existing webhook, you can use the webhook.put command, and set the uuid (= token) of the webhook to edit. For example:

Delete a Webhook​

In order to delete an existing webhook, you can use the command webhook.delete:

And then, set the uuid of the webhook you want to delete.

You can also delete a Webhook by using the Portal in the "Webhooks" section.

Receiving uploaded files via Webhook​

It is also possible to send files as a playload with a webhook. To do so, execute the request as multipart POST with the body formatted as multipart/form-data. For example:

More information about multipart POST requests can be found here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST

Logging and Tracing of Webhooks​

Each webhook execution will be logged using the last 6 chars of the uuid/token (shortened for for security and performance reasons) in combination with a random traceId. For example, a webhook with a uuid/token like this:

Will result in a log entry like this:

The left part before the colon : is the shortened webhook uuid/token. The right part is a random id generated newly for each webhook call. So multiple calls of same webhook can be monitored separately.

This traceId is also returned to the caller of the webhook by using the command webhook.receive so he can refer to this exact call when required:

Additionally, it will also be set to the initial message sent by the webhook call:

This way you can trace a webhook all the way across different queues, pipelines and microservices.

For example, if you would like to follow all traces of a given webhook, search for the last 6 characters of the webhook uuid/token like 6936bd in all logs and you should find all traces of a given webbhook ordered by the time of their executions.

If you would like to follow only a certain webhook call instead, use the fully qualified traceId like 6936bd:efg35ee for example and search for it. You should find all traces related to exactly this single webhook call.