What is the AI Studio?

The AI Studio is a set of tools and UI components seamless integrated into PIPEFORCE to boost your process automation and data integration tasks to a next level using artificial intelligence such as LLM (Large Language Models) for example.

Configure AI Backend (secret)

Before you can start, an AI backend needs to be configured. Depending on your license, by default PIPEFORCE comes with it’s own built-in AI backend. The secret name of this is ai-default-secret.

If you do not specify otherwise, this AI backend secret is used by default.

But you can switch this default backend or use multiple AI backends in parallel. To do so, you have to configure the credentials and settings for those AI backends as secrets of type secret-text and place a JSON into the secret value which has a structure like this:

{
  "base_url": "string",
  "model": "string",
  "api_token": "string",
  "max_token": "integer",
  "custom_headers": { "name": "value" }
} 

Whereas:

• base_url: The base url of the API (requried).

• model: The AI model to be used (required).

• api_token: The security token to be used.

• max_token: The max token to be send (defaults to 800)

• custom_headers: Key-value pairs to be passed along as HTTP headers on any request. This is handy for example in case basic authentication or any other additional header setting is required.

Connect to OpenAI (ChatGPT)

In case you would like to connect to the OpenAI services for example, you could use these settings in your secretPIPEFORCE is AI-native, meaning that every component of the platform is built from the ground up to integrate seamlessly with AI. You can easily forward any type of data through an automation pipeline to AI, allowing it to perform tasks such as analysis, transformation, validation, or detection. The results can then be effortlessly integrated back into your workflows. Additionally, PIPEFORCE allows you to leverage AI to automatically create, execute and optimize workflows.

Moreover, PIPEFORCE includes a suite of advanced AI tools and UI components designed to enhance your process automation and data integration. These tools utilize cutting-edge technologies such as Generative AI and Large Language Models (LLMs), providing you with powerful capabilities to optimize your operations.

• Ready to Get Started
  No special setup is required to use AI with PIPEFORCE. You can start leveraging AI capabilities right out of the box.

• Process Intelligence
  PIPEFORCE comes with its own AI service, which integrates multiple pre-trained AI models optimized specifically for process automation and integration tasks. This service is as powerful as leading external AI platforms like OpenAI/ChatGPT or Google Gemini for process automation. In some specific use cases, it can even outperform these external services.

• Security and Privacy
  Since PIPEFORCE Cloud runs on its own AI infrastructure, fully hosted in Germany, all AI tasks are entirely GDPR-compliant. Additionally, every AI interaction is monitored and logged to ensure compliance with enterprise standards and regulatory requirements.

• Flexibility
  PIPEFORCE AI can also be integrated with other AI services, such as OpenAI/ChatGPT or Google Gemini, if needed, offering additional flexibility to meet specific requirements.

...

Table of Contents

Configure AI Backend (secret)

Before you can start, an AI backend needs to be configured. Depending on your license, by default PIPEFORCE comes with it’s own built-in AI backend which uses the default secret ai-default-secret.

If you do not specify otherwise, this AI backend secret is used by default in any AI conversation.

But you can switch this default backend or use multiple AI backends in parallel. To do so, you have to configure the credentials and settings for those AI backends as secrets of type secret-text and place a JSON into the secret value which has a structure like this:

{
  "base_url": "https://api.openai.com/v1string",
  "model": "gpt-3.5-turbostring",
  "api_token": "your_tokenstring",
  "max_token": "integer",
800 } 

Prompting - Send a question or advice to AI

One of most generic and simplest use cases is to send a prompt (= question / advice) to the AI and use the response data in your pipeline. For this you can use the ai.prompt.send command. Here is an example to return some data from the AI:

pipeline:
  - ai.prompt.send:
      prompt: |
        Return the names of the 10 biggest cities in the world as JSON array.

This will result in an entry like this in the body:

[
    "Tokyo",
    "Delhi",
    "Shanghai",
    "Sao Paulo",
    "Mumbai",
    "Beijing",
    "Mexico City",
    "Osaka",
    "Cairo",
    "Dhaka"
]

Adding context data (input) to the prompt

...

 "custom_headers": { "name": "value" }
} 

Whereas:

• base_url: The base url of the API (requried).

• model: The AI model to be used (required).

• api_token: The security token to be used.

• max_token: The max token to be send (defaults to 800)

• custom_headers: Key-value pairs to be passed along as HTTP headers on any request. This is handy for example in case basic authentication or any other additional header setting is required.

Connect to OpenAI (ChatGPT)

In case you would like to connect to the OpenAI services for example, you could use these settings in your secret:

{
  "base_url": "https://api.openai.com/v1",
  "model": "gpt-3.5-turbo",
  "api_token": "your_token",
  "max_token": 800
} 

Prompting - Send a question or advice to AI

One of most generic and simplest use cases is to send a prompt (= question / advice) to the AI and use the response data in your pipeline. For this you can use the ai.prompt.send command. Here is an example to return some data from the AI:

pipeline:
  - ai.prompt.send: |
     input: |  Return the names of the 10 biggest [cities in the world as JSON array.

This will result in an entry like this in the body:

[
    "Tokyo",
  
       "Delhi",
    
     "Shanghai",
    
     "Sao Paulo",
 
        "Mumbai",
 
        "Beijing",
          "Mexico City",

         "Osaka",
     
    "Cairo",
          "Dhaka"
]

Adding context data (input) to the prompt

You can also apply the prompt with context data. This context data can be set as input to the command:

pipeline:
  - ai.prompt.send:
   ]   input: |
  prompt: |     [
   Order the given list alphabetically.

The result of this example in the body is then:

[     "BeijingTokyo",
    "Cairo",
      "Delhi",
          "DhakaShanghai",
    "Mexico City      "Sao Paulo",
          "Mumbai",
          "OsakaBeijing",
          "Mexico "Sao PauloCity",
          "Osaka",
    "Shanghai      "Cairo",
          "Tokyo"
]

The input of the command will become the context data. It can be plain text, a file or an URI. In case it is a file (for example a PDF or Word document) or any other supported format, it will be automatically converted into an AI compatible format.

Here is an example which uses a PDF file as file context, stored in PIPEFORCE’s Drive cloud storage:

pipeline:
  - ai.prompt.send:
Dhaka"
        ]
      prompt: |
     input: $uri:drive:invoice-3662.pdf  Order the given   prompt: |list alphabetically.

The result of this example in the body is then:

[
    "Beijing",
    Check"Cairo",
the invoice to ensure it"Delhi",
is correct both in terms"Dhaka",
    "Mexico City",
   of content and calculations. If everything is fine, return "OK". 
  "Mumbai",
    "Osaka",
    "Sao Paulo",
      If not"Shanghai",
provide the reason for the error in one sentence.

See another example which converts a given input:

pipeline:
  - ai.prompt.send:
      input: |"Tokyo"
]

The input of the command will become the context data. It can be plain text, a file or an URI. In case it is a file (for example a PDF or Word document) or any other supported format, it will be automatically converted into an AI compatible format.

Here is an example which uses a PDF file as file context, stored in PIPEFORCE’s Drive cloud storage:

pipeline:
  - ai.prompt.send:
      input: $uri:drive:invoice-3662.pdf
 <person>     prompt: |
    <firstName>Max</firstName>    Check the invoice to ensure it  <lastName>Smith</lastName>
is correct both in terms 
        <age>36</age>of content and calculations. If everything is fine,  </person>return "OK". 
        prompt: "Convert to JSON"

...

If not, provide the reason for the error in one sentence.

See another example which converts a given input:

pipeline:
{  - ai.prompt.send:
      "firstName"input: "Max",|
        "lastName": "Smith",<person>
         "age": 36 <firstName>Max</firstName>
    } }

And one more example: Apply a data privacy filter:

pipeline:   - ai.prompt.send:
 <lastName>Smith</lastName>
     input: |    <age>36</age>
    {    </person>
      "person"prompt: {"Convert     to JSON"

And the result from the AI in the body will be this:

{
    "person": {
        "firstName": "Max",
   
        "lastName": "Smith",

           "age": 36
    }
}

And one more example: Apply a data privacy filter:

pipeline:
  - ai.prompt.send:
}      input: |
 }       prompt:{
|          "person": Remove{
all personal data because of  privacy and     "firstName": "Max",
     replace by randomized names and add prefix p_

As a result, a changed JSON comes back:

{
    "person": { "lastName": "Smith",
            "firstNameage": "p_Alex", 36
         "lastName": "p_Johnson", }
        "age":}
48     } }

Avanced prompting: Send multiple messages

In case you need to send multiple messages in one prompt, you can use the parameter messages like this:

pipeline:prompt: |
    - ai.prompt.send:     Remove all messages:personal data because of privacy and 
  - role: system      replace by randomized names and content:add Tell me a joke based on given user input.prefix p_

As a result, a changed JSON comes back:

{
    "person": {
        - role"firstName": user"p_Alex",
          content: I'm a 28 year old man living in New York.

The result could be like this in the body:

Why did the New York man bring a ladder to his job interview? 
Because he wanted to climb the corporate ladder!

If both parameters, messages and prompt are given, prompt will be automatically added to messages of role system at the very end.

Possible values for role are:

• system = A system message from the caller (typically the context data with basic advice).

• user = A message from the user (typically the question or advice based on the context data).

• ai = A message from the AI (typically used to enrich the context or trace the conversation).

The parameter content can be a plain text or any AI convertable object (like a PDF file for example). The conversion and preparation to an AI compatible format is done by PIPEFORCE automatically.

Intent Detection - Detect what the user wants

Another feature of the AI studio is the ability to detect the intent of the user or to classify a given information and extract additional information based on the detected intent.

Lets consider this example use case for better understanding: An employee sends an email with a PDF as attachment. This PDF can be an invoice, a termination or a documentation. The AI can detect, which intent it is and can then additionally extract all required information from the document. For the invoice this could be the invoice numer and the positions for example, for the termination it could the contract number and for the documentation it could a short summary for example. Lets model this use case now in a PIPEFORCE pipeline:

pipeline:

  # Read email from inbox
  - imap.get:
      host: "lastName": "p_Johnson",
        "age": 48
    }
}

Avanced prompting: Send multiple messages

In case you need to send multiple messages in one prompt, you can use the parameter messages like this:

pipeline:
  - ai.prompt.send:
      messages:
        - role: system
          content: Tell me a joke based on given user input.
        - role: user
          content: I'm a 28 year old man living in New York.

The result could be like this in the body:

Why did the New York man bring a ladder to his job interview? 
Because he wanted to climb the corporate ladder!

If both parameters, messages and prompt are given, prompt will be automatically added to messages of role system at the very end.

Possible values for role are:

• system = A system message from the caller (typically the context data with basic advice).

• user = A message from the user (typically the question or advice based on the context data).

• ai = A message from the AI (typically used to enrich the context or trace the conversation).

The parameter content can be a plain text or any AI convertable object (like a PDF file for example). The conversion and preparation to an AI compatible format is done by PIPEFORCE automatically.

Intent Detection - Detect what the user wants

Another feature of the AI studio is the ability to detect the intent of the user or to classify a given information and extract additional information based on the detected intent.

Lets consider this example use case for better understanding: An employee sends an email with a PDF as attachment. This PDF can be an invoice, a termination or a documentation. The AI can detect, which intent it is and can then additionally extract all required information from the document. For the invoice this could be the invoice numer and the positions for example, for the termination it could the contract number and for the documentation it could a short summary for example. Lets model this use case now in a PIPEFORCE pipeline:

pipeline:

  # Read email from inbox
  - imap.get:
      host: outlook.office365.com
      secret: my-office365-secret

  # Detect the intent in the email
  - ai.intent.detect:
      advice:
        intentCandidates:
          - intentId: "invoice"
            prompt: "Use this intent in case the attachment is an invoice."
            params:
              supplierAddress:
                prompt: Extract the supplier address from the invoice.
              invoiceNumber:
                prompt: Extract the invoice number from the invoice.
              totalAmount:
                prompt: Extract the total amount of the invoice in cents without any currency chars, separators or other special characters.
          - intentId: "temination"
            prompt: "Use this intent in case the attachment is a termination of a contract."
            params:
                contractNumber:
                  prompt: Extract the contract number.
                customerNumber:
                  prompt: Extract the customer number.
                reason:
                  prompt: Summarize in one sentence the reason for the termination.
          - intentId: "documentation"
            prompt: "Use this intent in case the attachment is a documentation."
            params:
                summary:
                  prompt: Create a short summary what this documentation is about.

...

• intentId
  This attribute is mandatory and gives the intent a unique id. This should be an explainatory, unique name without special charaters or whitespaces.

• prompt
  Each intent has an prompt in order to instruct the AI about the criteria to select this intent. In case such an intent is selected by AI, additionally the parameters will be extracted from the input. This parameter is mandatory.

• params
  The list of optional parameters to be detected in case this intent was selected.

• command
  The optional name of the command to be executed in case this intent was detected. Note: Additionally rundDetectecCommand must be set to true on the ai.intent.detect command which is false by default.

• enabled
  An intent can optionally be disabled by setting enabled = false. This is useful mainly for testing purposes for example. In this case only the other intents will be considered by the AI.

Intent Parameters

The params section on each intent lists the parameters of the intent. These parameters will be automatically set in case the given intent was detected. They can later be used for further processing by calling a command of by passing them to external systems for example.

...

All parameter attributes are explained below.

name (optional)

The name of the parameter.

This attribute is optional. If not set, the params id will be used.

required (optional)

Defines whether this parameter is required. In case it is required and its value is finally missing or cannot be detected by AI, an error is thrown and further execution stops.

The default value is false.

type (optional)

The data type of the parameter such as string, boolean, integer, number, json

...

The default value is string.

value (optional)

The value of the intent parameter.

This can be a fixed value (literal) or a template.

Templated values

The value can also be template string. By default the Mustache template syntax can be used which starts with {{ and ends with }}. The variables advice and intent are passed as model context to the template. This way you can access for example settings and values of other parameters after they have been resolved by the AI in order to formulate the final parameter for a command.

...

...
params:
  customerId:
    value: "1234567"
  message:
    value: "The customerId is: {{intent.params.customerId}}"
  ...

prompt (optional)

For each parameter, an attribute prompt instead of a value can be set. Not both!

...

As you can see in this example, there is no fixed value for parameter subject set. Instead the AI was instructured to extract the value from the given email input.

Text-to-Command - Let the AI auto-start a business process

This is the next level of automation where the AI can start a command and with this, start a business process fully automated. For this, the AI takes a non-structured text such as an email, a chat message or a PDF document for example, analyses it and then automatically detects and executes the according PIPEFORCE command in order to take action and fulfill the user’s request.

...

• Automatically forward emails with a summary to internal team
  Scan any email sent to a given inbox such as info@mycompany.tld for example, find out the intention of the sender, then forward the email to the internal team such as support, sales, … which can handle the request. The AI can find out the type of request, whether it is a support request, an order request, a question regarding an invoice or any other type just by writing an advice to the AI and without any programming. It can also detect and extract all required data such as customerId, invoiceNumber and more from the sender’s email. Furthermore, it can also create a short summary about what the core intent of the sender is to make it easier for the internal team to process the request.

• Automatically start an internal workflow by email
  Scan any email sent to a given inbox such as invoice@mycompany.tld for example and if this email matches to an existing workflow, extract all variables required for this workflow from the email, start the workflow and pass these variables along with it. For example to start an accounts payable workflow based on an given payable invoice. The AI can validate whether all required data exist and is valid in order to start the workflow.

• Automatically call endpoints of other systems by email
  Scan any email sent to a given inbox such as info@mycompany.tld for example and if this email is related to a service, offered by a third party system which provides an remote API, call this remote API (for example REST) and pass along parameters extracted from the email. For example create a new ticket on an external ticket system.

Using the command ai.intent.detect

In order to integrate Text-to-Command functionality into your automation pipelines, you can use the command ai.intent.detect. It will

...

Each intent has an prompt in order to instruct the AI about the criteria to select this intent. In case such an intent is selected by AI, there is the targetCommand field defining the name of the command which must be called. In this example this is the mail.send command.

The parameter runDetectedCommand defines whether the command should directly be executed (true) or the intent JSON should be simply returned for further processing (false).

Use Case Examples

Below you can find a list of most often requested use cases for AI in PIPEFORCE and the pipeline recipe to solve it.

UC1 - Reductive Prompt: Create a summary from an email attachment

Use Case: Given is an email with attachment as input. Create a summary from this email loaded from an IMAP inbox.

pipeline:

  - imap.get:
      host: outlook.office365.com
      secret: my-office365-secret
      
  - ai.prompt.send:
        prompt: |
            Create a short summary of the attachment in the given email.

Result:

The attachment is an invoice from ABC Software (Germany) GmbH to Musterkunde AG, 
detailing various services provided and their associated costs for the period from 
01.02.2024 to 29.02.2024. The invoice includes information on user accounts, 
transaction fees, VAT, and payment instructions.

UC2 - Classifying Prompt: Classify the type of an email attachment

Use Case: Given is an email with attachment as input. Classify the attachment of this email based on a given set of classes.

pipeline:

  - imap.get:
      host: outlook.office365.com
      secret: my-office365-secret

  - ai.prompt.send:
        prompt: |
            Given are these categories: invoice, quote, termination. 
            In which of these fits the given attachment? 
            Return only the name from the list.

Result:

invoice

UC3 - Classifying Prompt: Classify the the mood of the sender of an email

Use Case: Given is an email as input. Based on a list of moods, classify the mood of the email’s sender.

pipeline:

  - imap.get:
      host: outlook.office365.com
      secret: my-office365-secret

  - ai.prompt.send:
        prompt: |
            In which mood fits the given email? 
            Return only the one from this list which matches best: 
            happy, neutral, disappointed, angry, cannot detect

Result:

happy

UC4 - Transformative Prompt: Extract information from an PDF as JSON

Use Case: Given is an invoice as attachment PDF to an email. The goal is to extract all required information from this PDF and provide it in a valid JSON format.

pipeline:

  - imap.get:
      host: outlook.office365.com
      secret: my-office365-secret
      
  - ai.prompt.send:
      prompt: |
          Extract the invoice in the attachment to a JSON using this structure:
          {
              "invoiceNumber": "value",
              "address": "value",
              "invoiceDate": "value",
              "positions": [
                  {
                      "description": "value",
                      "pieces": "value",
                      "price": value
                  }
              ]
          }

Result:

{
    "invoiceNumber": "123100401",
    "address": "ABC Provider (Germany) GmbH - Weg 3 - 12345 Stadt",
    "invoiceDate": "01.02.2025 - 29.02.2025",
    "positions": [
        {
            "description": "Invoice WMACCESS Internet, 1 Month",
            "pieces": 1,
            "price": 130.00
        },
        {
            "description": "Router Type 3 rental",
            "pieces": 1,
            "price": 5.00
        }
    ]
}

UC5 - Detect the intent and its parameters from an user email

Use Case: Given is an email with an attachment. Based on a given set of intents, the AI should detect the one which matches and additionally extract all required information from it.

pipeline:

  # Read email from inbox
  - imap.get:
      host: outlook.office365.com
      secret: my-office365-secret

  # Detect the intent in the email
  - ai.intent.detect:
      advice:
        intentCandidates:
          - intentId: "invoice"
            prompt: "Use this intent in case the attachment is an invoice."
            params:
              supplierAddress:
                prompt: "Extract the supplier address from the invoice."
              invoiceNumber:
                prompt: "Extract the invoice number from the invoice."
              totalAmount:
                prompt: "Extract the total amount of the invoice in cents without any currency chars, separators or other special characters."
          - intentId: "termination"
            prompt: "Use this intent in case the attachment is a termination of a contract."
            params:
                contractNumber:
                  prompt: "Extract the contract number."
                customerNumber:
                  prompt: "Extract the customer number."
                reason:
                  prompt: "Summarize in one sentence the reason for the termination."
          - intentId: "documentation"
            prompt: "Use this intent in case the attachment is a documentation."
            params:
                summary:
                  prompt: "Create a short summary what this documentation is about."

Result:

{
    "params": {
        "supplierAddress": {
            "name": "supplierAddress",
            "value": "ABC Software (Germany) GmbH - Im Weg 3 - 12345 Worth",
            "required": false,
            "pass": true,
            "type": "string"
        },
        "invoiceNumber": {
            "name": "invoiceNumber",
            "value": "123100401",
            "required": false,
            "pass": true,
            "type": "string"
        },
        "totalAmount": {
            "name": "totalAmount",
            "value": "45353",
            "required": false,
            "pass": true,
            "type": "string"
        }
    },
    "enabled": true,
    "intentId": "invoice",
    "command": null
}

UC6 - Detect and start a command / workflow based on a user email

Use Case: Given is an email with attachments. The AI must detect the intent of this email and finally a command linked to this intent must be executed and all required parameters must be extracted from the input email and transformed and passed along to the command. Also the attachment file must be passed along.

vars:
    containerProp: "global/app/io.pipeforce.test/data/container-${@text.random(10)}"

pipeline:

  # Read the email from inbox
  - imap.get:
      host: outlook.office365.com
      secret: office365-testlab-secret2
      subjectContains: test123

  # Create the container property for the attachment
  - property.schema.put:
      path: ${vars.containerProp}
      input: ""
      output: false

  # Save the attachment to the container property
  - property.attachment.put:
      path: ${vars.containerProp}
      content: ${body[0].attachments[0]}
      output: ${vars.attachment}

  # Detect the command
  - ai.intent.detect:
      runDetectedCommand: true
      advice:
        intentCandidates:
        
          - intentId: "startInvoiceWorkflow"
            command: workflow.start
            prompt: "Sender has submitted a payable invoice."
            params:
                invoiceNumber:
                    prompt: "Extract the invoice number."
                    pass: false
                key: 
                    value: io.pipeforce.test_invoice-wf
                variables:
                    value:
                        "totalAmount": "{{intent.params.invoiceNumber}}"
                        "invoice": ${vars.attachment}
                        
          - intentId: "startTerminationWorkflow"
            command: workflow.start
            prompt: "Sender has submitted a contract termination."
            params:
                contractNumber:
                    prompt: "Extract the contract number."
                    pass: false
                key: 
                    value: io.pipeforce.test_termination-wf
                variables:
                    value:
                        "contractNumber": "{{intent.params.contractNumber}}"
                        "terminationLetter": ${vars.attachment}

Result:

For an invoice given in attachments of the email, this will select a the startInvoiceWorkflow intent with final workflow parameters as as listed below:

{
    "params": {
        "invoiceNumber": {
            "name": "invoiceNumber",
            "value": "45353",
            "required": false,
            "pass": false,
            "type": "string"
        },
        "key": {
            "name": "key",
            "value": "io.pipeforce.invoicetest_invoice-wf",
            "required": false,
            "pass": true,
            "type": "string"
        },
        "variables": {
            "name": "variables",
            "value": {
                "firstName": "Max",
                "lastName": "Mustermann",
                "invoiceNumber": "45353",
                "invoice": "${vars.attachment}"
            },
            "required": false,
            "pass": true,
            "type": "string"
        }
    },
    "enabled": true,
    "intentId": "startInvoiceWorkflow",
    "command": "workflow.start"
}

This JSON will then cause the command ai.intent.detect to auto-execute the command workflow.start and pass along all parameters marked "pass": true, there is the targetCommand field defining the name of the command which must be called. In this example this is the mail.send command.

The parameter runDetectedCommand defines whether the command should directly be executed (true) or the intent JSON should be simply returned for further processing (false).