{"type":"doc","content":[{"type":"paragraph","content":[{"type":"status","attrs":{"color":"blue","style":"bold","text":"SINCE VERSION 1.0","localId":"afa38f5d-dbf2-4ee5-aed3-789d8ead0791"}}]},{"type":"extension","attrs":{"layout":"default","extensionType":"com.atlassian.confluence.macro.core","extensionKey":"toc","parameters":{"macroParams":{"minLevel":{"value":"1"},"maxLevel":{"value":"3"},"outline":{"value":"false"},"type":{"value":"list"},"printable":{"value":"false"}},"macroMetadata":{"macroId":{"value":"3151f2ff-836c-40cd-8434-a578675fe82e"},"schemaVersion":{"value":"1"},"title":"Table of Contents"}},"localId":"8caef0b9-58f9-44da-a4bd-e839c46ed71d"}},{"type":"heading","attrs":{"level":1},"content":[{"text":"What is an Automation Pipeline?","type":"text"}]},{"type":"paragraph","content":[{"text":"An ","type":"text"},{"text":"Automation Pipeline","type":"text","marks":[{"type":"strong"}]},{"text":" or in short just ","type":"text"},{"text":"Pipeline","type":"text","marks":[{"type":"strong"}]},{"text":" in PIPEFORCE automates the flow of data from one endpoint to another. Between these endpoints, the data can be enriched, transformed, cleansed and so on, so it becomes compatible between these integration endpoints.","type":"text"}]},{"type":"paragraph","content":[{"text":"Such a pipeline consists typically of one or more so called ","type":"text"},{"text":"Commands","type":"text","marks":[{"type":"strong"}]},{"text":" to reach these goals. A command can also be considered as a single automation step.","type":"text"}]},{"type":"paragraph","content":[{"text":"Here is an example of such an automation pipeline flow in ","type":"text"},{"text":"Low Code","type":"text","marks":[{"type":"strong"}]},{"text":" with two simple steps: The first step calls a rest endpoint and the second step stores the result in the cloud storage:","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"pipeline:\n - http.get: https://somedomain.tld/rest/contracts\n - drive.save: contracts.json ","type":"text"}]},{"type":"paragraph","content":[{"text":"In the web portal of PIPEFORCE the automation pipeline can be designed and configured via the ","type":"text"},{"text":"No Code ","type":"text","marks":[{"type":"strong"}]},{"text":"pipeline designer using drag & drop:","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center","width":760,"widthType":"pixel"},"content":[{"type":"media","attrs":{"width":885,"alt":"image-20240111-142755.png","id":"8be06033-5f64-489b-a2b5-59a6e972c408","collection":"contentId-2542796840","type":"file","height":328},"marks":[{"type":"border","attrs":{"size":2,"color":"#091e4224"}}]}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"What is a Command?","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#what-is-a-command"}}]}]},{"type":"paragraph","content":[{"text":"A ","type":"text"},{"text":"Command","type":"text","marks":[{"type":"strong"}]},{"text":" in PIPEFORCE is a single step inside an automation pipeline. It can take optional parameters and a body as input processes a certain operation and finally produces an optional output which is the response to the caller. ","type":"text"}]},{"type":"paragraph","content":[{"text":"Every command can be a single automation step inside an automation pipeline or it can be called “standalone” using the Rest AP. Here is an example, how a command can be called via Rest API","type":"text"}]},{"type":"codeBlock","content":[{"text":"GET https://hub-NAMESPACE.pipeforce.net/api/v3/command:datetime","type":"text"}]},{"type":"paragraph","content":[{"text":"A request to this URL would call the ","type":"text"},{"text":"datetime","type":"text","marks":[{"type":"code"}]},{"text":" command at server side and finally returns the current date and time from server.","type":"text"}]},{"type":"paragraph","content":[{"text":"Multiple commands can be combined like \"LEGO\" bricks inside an automation pipeline. They will be called one after another, whereas the output of a command becomes the input of the next command. Here is an example, how such an automation pipeline can be used to call the ","type":"text"},{"text":"datetime","type":"text","marks":[{"type":"code"}]},{"text":" command from the example above and additionally send the result via email using the ","type":"text"},{"text":"mail.send","type":"text","marks":[{"type":"code"}]},{"text":" command:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - datetime\n - mail.send: \n to: some@email.tld","type":"text"}]},{"type":"paragraph","content":[{"text":"Note: This pipeline can also be designed using a graphical tool, the No Code Pipeline Designer in the PIPEFORCE WebUI.","type":"text"}]},{"type":"paragraph","content":[{"text":"Commands in PIPEFORCE follow the ","type":"text"},{"text":"Command Bus Pattern","type":"text","marks":[{"type":"link","attrs":{"href":"https://logabit.atlassian.net/wiki/spaces/PA/pages/2644705466"}}]},{"text":" .","type":"text"}]},{"type":"paragraph","content":[{"text":"There are many different commands for different auomation steps in PIPEFORCE available. For example, commands can","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Upload and download files","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Encrypt and decrypt data","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Save data into a database or read from it","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Transform and map data","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Connect to other systems, and read / write data","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"And much more...","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"You can find a all built-in commands in the ","type":"text"},{"text":"commands reference","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/api/commands"}}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Command Name","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#command-name"}}]}]},{"type":"paragraph","content":[{"text":"Each command has a unique name which is always written in lower case and follows the ","type":"text"},{"text":"dot.notation","type":"text","marks":[{"type":"code"}]},{"text":". Here are some examples of valid command names in dot notation:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"barcode.create","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"data.list.iterate","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"log","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"mail.send","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"property.put","type":"text","marks":[{"type":"code"}]}]}]}]},{"type":"paragraph","content":[{"text":"As you can see, the command name is usually structured like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"..","type":"text"}]},{"type":"paragraph","content":[{"text":"Instead, these are for example ","type":"text"},{"text":"invalid","type":"text","marks":[{"type":"strong"}]},{"text":" command names:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"data_convert","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"dataEntity","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"my command","type":"text","marks":[{"type":"code"}]}]}]}]},{"type":"paragraph","content":[{"text":"The full list of available commands and their parameters can be found in the ","type":"text"},{"text":"commands reference","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/api/commands"}}]},{"text":".","type":"text"}]},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"COMMAND NAMES VS. REST RESOURCE NAMES","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"Even if many command names do have a similar resource-based semantic like HTTP GET, POST or PUT in REST do, they do not follow this approach 100%, since a Command is typically bound to a server side procedure, not only to a resource operation. Therefore, the procedure type of a command is defined by its name, not by a method header. For example: ","type":"text"},{"text":"property.put","type":"text","marks":[{"type":"code"}]},{"text":", ","type":"text"},{"text":"service.start","type":"text","marks":[{"type":"code"}]},{"text":",","type":"text"},{"text":"property.move","type":"text","marks":[{"type":"code"}]},{"text":" or ","type":"text"},{"text":"config.get","type":"text","marks":[{"type":"code"}]},{"text":" to name just a few.","type":"text"}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Command Alias","type":"text"}]},{"type":"paragraph","content":[{"text":"Besides the default command name, a command can also have one or more aliases. These are alternative names which can be used the same way as the default names. For example the command ","type":"text"},{"text":"mail.send","type":"text","marks":[{"type":"code"}]},{"text":" has also the alias ","type":"text"},{"text":"mail","type":"text","marks":[{"type":"code"}]},{"text":". Both can be used the same way for sending emails since they point to the same command implementation.","type":"text"}]},{"type":"paragraph","content":[{"text":"For list of the alias names of a command, see the command documentation.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Command Parameters","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#command-parameters"}}]}]},{"type":"paragraph","content":[{"text":"Commands can have zero to many parameters. Whereas each parameter is a name-value pair. The parameters can be passed in different ways to the command, depending on the execution context you're working in. See ","type":"text"},{"text":"Executing a Command","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#executing-a-command"}}]},{"text":" below. ","type":"text"}]},{"type":"paragraph","content":[{"text":"Here is an example to set parameters to a command as part of a single HTTP call:","type":"text"}]},{"type":"codeBlock","attrs":{"language":"shell"},"content":[{"text":"https://hub-NAMESPACE.pipeforce.net/api/v3/command:mail.send?to=some@email.com\n &subject=Hello&message=\"Hello World!\"","type":"text"}]},{"type":"paragraph","content":[{"text":"An here you can see the same command with parameters, embedded inside a ","type":"text"},{"text":"Pipeline","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#pipeline"}}]},{"text":" script:","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"pipeline:\n - mail.send:\n to: \"some@email.com\"\n subject: \"Hello\"\n message: \"Hello World!\"","type":"text"}]},{"type":"paragraph","content":[{"text":"Command parameters can be primitive types (","type":"text"},{"text":"integer","type":"text","marks":[{"type":"code"}]},{"text":", ","type":"text"},{"text":"decimal","type":"text","marks":[{"type":"code"}]},{"text":", ","type":"text"},{"text":"boolean","type":"text","marks":[{"type":"code"}]},{"text":", ","type":"text"},{"text":"string","type":"text","marks":[{"type":"code"}]},{"text":") or complex types like YAML or JSON. Here is an example to set a JSON as parameter:","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"pipeline:\n - log:\n message: {\"key\": \"value\"}","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Default Command Parameter (short form)","type":"text"}]},{"type":"paragraph","content":[{"type":"status","attrs":{"color":"blue","style":"bold","text":"Since Version 9.0","localId":"3316dbe5-ed40-4717-8e0b-2b20358f6cd2"}},{"text":" ","type":"text"}]},{"type":"paragraph","content":[{"text":"Some commands need only one mandatory parameter to work. This is called the","type":"text"},{"text":" default parameter","type":"text","marks":[{"type":"strong"}]},{"text":". ","type":"text"}]},{"type":"paragraph","content":[{"text":"For example lets see the ","type":"text"},{"text":"log","type":"text","marks":[{"type":"code"}]},{"text":" command:","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"pipeline:\n - log:\n message: \"Hello World!\"","type":"text"}]},{"type":"paragraph","content":[{"text":"The command ","type":"text"},{"text":"log","type":"text","marks":[{"type":"code"}]},{"text":" requires the mandatory parameter ","type":"text"},{"text":"message","type":"text","marks":[{"type":"code"}]},{"text":". This parameter is also marked as the default parameter in the command docs using the ","type":"text"},{"text":"(default)","type":"text","marks":[{"type":"code"}]},{"text":" tag. Therefore, also the short form can be used:","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"pipeline:\n - log: \"Hello World\"","type":"text"}]},{"type":"paragraph","content":[{"text":"As you can see, in the short form you can omit writing the parameter name since it is obvious.","type":"text"}]},{"type":"paragraph","content":[{"text":"It depends on the command whether it supports a default parameter or not. See in the command docs to find out. The default parameters are marked with ","type":"text"},{"text":"(default)","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"Limitations","type":"text"}]},{"type":"heading","attrs":{"level":5},"content":[{"text":"Default parameter value can only be primitive","type":"text"}]},{"type":"paragraph","content":[{"text":"The values of a default parameter can only be primitives like string, number or boolean. It cannot be a JSON object or array. If you need to pass a JSON, pass it as string if the command supports this. In most cases the command will auto-convert from string to JSON if required. For example:","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"pipeline:\n\n # This doesn't work:\n - body.set: {\"text\": \"Hello World\"}\n\n # This works:\n - body.set: |\n {\"text\": \"Hello World\"}","type":"text"}]},{"type":"panel","attrs":{"panelType":"warning"},"content":[{"type":"paragraph","content":[{"text":"The default parameter value can only be primitive like string, number or boolean but no object or array.","type":"text"}]}]},{"type":"heading","attrs":{"level":5},"content":[{"text":"Mixing default and ordinary parameters not allowed","type":"text"}]},{"type":"paragraph","content":[{"text":"Note that mixing the default parameter with ordinary parameters is not possible since YAML specification is ","type":"text"},{"text":"not allowing","type":"text","marks":[{"type":"strong"}]},{"text":" this:","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"pipeline:\n\n # This doesn't work:\n - log: \"Hello World\"\n level: INFO\n \n # This works:\n - log:\n message: \"Hello World\"\n level: INFO\n \n # This works:\n - log: \"Hello World\"","type":"text"}]},{"type":"panel","attrs":{"panelType":"warning"},"content":[{"type":"paragraph","content":[{"text":"If you use the default parameter of an command, you cannot set any other ordinary parameter on it.","type":"text"}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Executing a Command","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#executing-a-command"}}]}]},{"type":"paragraph","content":[{"text":"A single Command can be executed by sending it as HTTP GET or POST request using the endpoint ","type":"text"},{"text":"/api/v3/command","type":"text","marks":[{"type":"code"}]},{"text":". The full url structure of this endpoint is always like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"https://HUB/api/v3/command:?=&=","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Replace ","type":"text"},{"text":"HUB","type":"text","marks":[{"type":"code"}]},{"text":" by the hub host name of your instance (for example ","type":"text"},{"text":"hub-mycompany.pipeforce.net","type":"text","marks":[{"type":"code"}]},{"text":").","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Replace ","type":"text"},{"text":"","type":"text","marks":[{"type":"code"}]},{"text":" by the ","type":"text"},{"text":"name","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/api/commands"}}]},{"text":" of the command you would like to execute.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Replace ","type":"text"},{"text":"","type":"text","marks":[{"type":"code"}]},{"text":",","type":"text"},{"text":"","type":"text","marks":[{"type":"code"}]},{"text":" and ","type":"text"},{"text":"","type":"text","marks":[{"type":"code"}]},{"text":",","type":"text"},{"text":"","type":"text","marks":[{"type":"code"}]},{"text":" by the optional ","type":"text"},{"text":"parameters","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/api/commands"}}]},{"text":" of your command.","type":"text"}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"HTTP GET","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#get"}}]}]},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"Calling a command via ","type":"text"},{"text":"HTTP GET","type":"text","marks":[{"type":"strong"}]},{"text":" means:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Set all ","type":"text"},{"text":"request parameters","type":"text","marks":[{"type":"strong"}]},{"text":" → ","type":"text"},{"text":"command parameters","type":"text","marks":[{"type":"strong"}]},{"text":".","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Set the command ","type":"text"},{"text":"body","type":"text","marks":[{"type":"strong"}]},{"text":" to ","type":"text"},{"text":"null","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"Here is an example to execute the ","type":"text"},{"text":"log","type":"text","marks":[{"type":"code"}]},{"text":" command as HTTP GET request, and set its ","type":"text"},{"text":"message","type":"text","marks":[{"type":"code"}]},{"text":" parameter to a string value using a HTTP request parameter:","type":"text"}]},{"type":"codeBlock","content":[{"text":"https://hub-trial.pipeforce.org/api/v3/command:log?message=HELLO","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Authentication","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/guides/security/authorization"}}]},{"text":" is not considered here for simplicity reasons.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Replace ","type":"text"},{"text":"http://hub-trial.pipeforce.org","type":"text","marks":[{"type":"code"}]},{"text":" by the url of your target system.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"In case you would like to use special characters like spaces, umlauts, ","type":"text"},{"text":"?","type":"text","marks":[{"type":"code"}]},{"text":" or ","type":"text"},{"text":"=","type":"text","marks":[{"type":"code"}]},{"text":" for example in your parameter value, you need to URL encode the URL as you would do with any such HTTP request. See Wikipedia for details: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://en.wikipedia.org/wiki/Percent-encoding"}},{"text":" ","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"Also see ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://logabit.atlassian.net/wiki/spaces/PA/pages/2542862901"}},{"text":". ","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"HTTP POST","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#post"}}]},{"text":" / File Upload","type":"text"}]},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"Calling a command via ","type":"text"},{"text":"HTTP POST ","type":"text","marks":[{"type":"strong"}]},{"text":"(default behaviour) means:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Set all ","type":"text"},{"text":"request parameters","type":"text","marks":[{"type":"strong"}]},{"text":" → ","type":"text"},{"text":"command parameters","type":"text","marks":[{"type":"strong"}]},{"text":".","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Set the ","type":"text"},{"text":"request","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"body","type":"text","marks":[{"type":"strong"}]},{"text":" → ","type":"text"},{"text":"command input body","type":"text","marks":[{"type":"strong"}]},{"text":". ","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"If ","type":"text"},{"text":"Content-Type","type":"text","marks":[{"type":"code"}]},{"text":" is ","type":"text"},{"text":"application/json","type":"text","marks":[{"type":"code"}]},{"text":" or ","type":"text"},{"text":"text/plain","type":"text","marks":[{"type":"code"}]},{"text":": The parsed JSON or plain text is set into the body of the command.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"If ","type":"text"},{"text":"Content-Type","type":"text","marks":[{"type":"code"}]},{"text":" is ","type":"text"},{"text":"multipart/form-data","type":"text","marks":[{"type":"code"}]},{"text":", then each part with name ","type":"text"},{"text":"file","type":"text","marks":[{"type":"code"}]},{"text":" will be processed into a ","type":"text"},{"text":"content object","type":"text","marks":[{"type":"link","attrs":{"href":"https://logabit.atlassian.net/wiki/spaces/PA/pages/2545320099"}}]},{"text":" and provided as content collection to the command.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Otherwise, the body is provided as single ","type":"text"},{"text":"content object","type":"text","marks":[{"type":"link","attrs":{"href":"https://logabit.atlassian.net/wiki/spaces/PA/pages/2545320099"}}]},{"text":" to the command.","type":"text"}]}]}]}]}]},{"type":"paragraph","content":[{"text":"Since ","type":"text"},{"type":"status","attrs":{"color":"blue","style":"bold","text":"Version 10","localId":"9750027f-617b-4519-9996-43e83d470126"}},{"text":": When passing the optional request parameter ","type":"text"},{"text":"?params=body","type":"text","marks":[{"type":"code"}]},{"text":":","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"If the ","type":"text"},{"text":"request body","type":"text","marks":[{"type":"strong"}]},{"text":" contains a JSON or YAML → This will be applied as ","type":"text"},{"text":"command","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"parameters","type":"text","marks":[{"type":"strong"}]},{"text":".","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Set the command ","type":"text"},{"text":"body","type":"text","marks":[{"type":"strong"}]},{"text":" to ","type":"text"},{"text":"null","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"Here is an example to execute a single command as HTTP POST request, and set the ","type":"text"},{"text":"message","type":"text","marks":[{"type":"code"}]},{"text":" parameter to a ","type":"text"},{"text":"log","type":"text","marks":[{"type":"code"}]},{"text":" command using a HTTP POST data body in ","type":"text"},{"text":"curl","type":"text","marks":[{"type":"code"}]},{"text":":","type":"text"}]},{"type":"codeBlock","content":[{"text":"curl -u \"username:password\" \\\n -X POST \"https://hub-trial.pipeforce.org/api/v3/command:log\" -d \"message=HELLO\"","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Authentication is done here by using basic authentication in the ","type":"text"},{"text":"Authorization","type":"text","marks":[{"type":"code"}]},{"text":" header. See ","type":"text"},{"text":"Authorization","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/api/headers"}}]},{"text":" and the ","type":"text"},{"text":"-u","type":"text","marks":[{"type":"code"}]},{"text":" switch of the ","type":"text"},{"text":"curl","type":"text","marks":[{"type":"code"}]},{"text":" terminal command for details.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Replace ","type":"text"},{"text":"http://hub-trial.pipeforce.org","type":"text","marks":[{"type":"code"}]},{"text":" by the url of your target system.","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"Also see ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://logabit.atlassian.net/wiki/spaces/PA/pages/2542862901"}},{"text":". ","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"CLI","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#execute-in-cli"}}]}]},{"type":"paragraph","content":[{"text":"You can also use the ","type":"text"},{"text":"PIPEFORCE CLI","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/cli"}}]},{"text":" in order to execute a single Command. Here is an example to call the ","type":"text"},{"text":"log","type":"text","marks":[{"type":"code"}]},{"text":" command and set the ","type":"text"},{"text":"message","type":"text","marks":[{"type":"code"}]},{"text":" parameter accordingly:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pi command log message=HELLO","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Body","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#command-body"}}]}]},{"type":"paragraph","content":[{"text":"Also see: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://logabit.atlassian.net/wiki/spaces/PA/pages/2592473121"}},{"text":". ","type":"text"}]},{"type":"paragraph","content":[{"text":"Beside ","type":"text"},{"text":"parameters","type":"text","marks":[{"type":"strong"}]},{"text":", a command can also consume and produce a ","type":"text"},{"text":"body","type":"text","marks":[{"type":"strong"}]},{"text":", similar to a HTTP POST request and response.","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center"},"content":[{"type":"media","attrs":{"width":1288,"id":"90786ea4-4243-47f4-9ea7-28f1706d0f11","collection":"contentId-2542796840","type":"file","height":568}}]},{"type":"paragraph","content":[{"text":"Differently to parameters, the input body is typically a more complex document and/or bigger data stream which must be modified in some way. Therefore, it is passed-in and written-out via the body by default.","type":"text"}]},{"type":"paragraph","content":[{"text":"Here is an example to pass JSON data via body to a ","type":"text"},{"text":"cache.put","type":"text","marks":[{"type":"code"}]},{"text":" command using a HTTP POST request and the ","type":"text"},{"text":"curl","type":"text","marks":[{"type":"code"}]},{"text":" tool:","type":"text"}]},{"type":"codeBlock","content":[{"text":"curl -X POST 'https://hub-trial.pipeforce.org/api/v3/command:cache.put?key=someKey' \\ \n -u 'username:password' \\\n -H 'Content-Type: application/json' \\\n -d '{\"productId\": 123, \"quantity\": 10}'","type":"text"}]},{"type":"paragraph","content":[{"text":"As you can see, the command parameter ","type":"text"},{"text":"key","type":"text","marks":[{"type":"code"}]},{"text":" has been set as request parameter here and in the HTTP POST body a JSON string is set using the ","type":"text"},{"text":"-d","type":"text","marks":[{"type":"code"}]},{"text":" switch of ","type":"text"},{"text":"curl","type":"text","marks":[{"type":"code"}]},{"text":". This JSON will become the input body (data) for the ","type":"text"},{"text":"cache.put","type":"text","marks":[{"type":"code"}]},{"text":" command. In order to automatically parse this JSON string into a JSON instance and use it as such in the command, you can optionally specify the ","type":"text"},{"text":"Content-Type: application/json","type":"text","marks":[{"type":"code"}]},{"text":" header.","type":"text"}]},{"type":"paragraph","content":[{"text":"NOTE","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Authentication is done here by using the ","type":"text"},{"text":"Authorization","type":"text","marks":[{"type":"code"}]},{"text":" header. See ","type":"text"},{"text":"Authorization","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/api/headers#authorization"}}]},{"text":" for details.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Replace ","type":"text"},{"text":"http://hub-trial.pipeforce.org","type":"text","marks":[{"type":"code"}]},{"text":" by the url of your target system.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"See ","type":"text"},{"text":"HTTP Execution Reference","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#http-execution-reference"}}]},{"text":" for an summary of all supported HTTP options here.","type":"text"}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Pipeline in Detail","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#what-is-a-pipeline"}}]}]},{"type":"paragraph","content":[{"text":"Two or more ","type":"text"},{"text":"Commands","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#command"}}]},{"text":" can be chained to a flow, called a ","type":"text"},{"text":"Pipeline","type":"text","marks":[{"type":"strong"}]},{"text":". If such a pipeline gets executed, each command in it will be executed one after another, whereas the output message of the first command will become the input message of the next command, and so on:","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center"},"content":[{"type":"media","attrs":{"width":744,"id":"b956a144-f6f4-40b8-bf00-6a76526dff5a","collection":"contentId-2542796840","type":"file","height":231},"marks":[{"type":"border","attrs":{"size":2,"color":"#091e4224"}}]}]},{"type":"paragraph","content":[{"text":"By default, such a pipeline is written in a script using the ","type":"text"},{"text":"YAML","type":"text","marks":[{"type":"link","attrs":{"href":"https://en.wikipedia.org/wiki/YAML"}}]},{"text":" format, which you can then manage by your preferred source code management tool like Git.","type":"text"}]},{"type":"paragraph","content":[{"text":"Here is an example, which connects two simple commands: The ","type":"text"},{"text":"datetime","type":"text","marks":[{"type":"code"}]},{"text":" command produces the current date and time and the ","type":"text"},{"text":"log","type":"text","marks":[{"type":"code"}]},{"text":" command finally logs it:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - datetime\n - log","type":"text"}]},{"type":"paragraph","content":[{"text":"In the YAML, a pipeline definition starts with the ","type":"text"},{"text":"pipeline:","type":"text","marks":[{"type":"code"}]},{"text":" element, followed by a list of commands. Each command to be executed is defined as an ","type":"text"},{"text":"list element","type":"text","marks":[{"type":"strong"}]},{"text":" by its name using an indent and a dash ","type":"text"},{"text":"-","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"By default, the message output (= body) of the first command (","type":"text"},{"text":"datetime","type":"text","marks":[{"type":"code"}]},{"text":" in this example) will automatically become the input message (= body) of the next command (","type":"text"},{"text":"log","type":"text","marks":[{"type":"code"}]},{"text":" in this example). Therefore no declaration and exchange of variables is required here. The exchange of body data between commands is implicitly.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Parameters","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#parameters"}}]}]},{"type":"paragraph","content":[{"text":"In case you need to configure a command by specifying parameters for it in a pipeline, you can do so by writing them below the command as ","type":"text"},{"text":"name-value-pairs","type":"text","marks":[{"type":"strong"}]},{"text":" with an additional tab indent or a at least two spaces as indent (see YAML specification for this):","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - datetime:\n format: dd-MM-YYYY\n - log","type":"text"}]},{"type":"paragraph","content":[{"text":"You can write parameter values without any quotes. Optionally, you can use single quotes for parameter values:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - datetime:\n format: 'dd-MM-YYYY'\n - log","type":"text"}]},{"type":"paragraph","content":[{"text":"or double quotes:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - datetime:\n format: \"dd-MM-YYYY\"\n - log","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Multi-line parameters","type":"text"}]},{"type":"paragraph","content":[{"text":"Parameters can also be multiple lines long:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - datetime:\n - log:\n message: This is a \n multiline log \n message. Line breaks \n wont be preserved.","type":"text"}]},{"type":"paragraph","content":[{"text":"In case you need to pass multi-line parameter values, with the ","type":"text"},{"text":"line breaks to be preserved","type":"text","marks":[{"type":"strong"}]},{"text":", you can do so by using the pipe ","type":"text"},{"text":"|","type":"text","marks":[{"type":"code"}]},{"text":" character:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - datetime\n - log:\n message: |\n This is a \n multiline log \n message. All line \n breaks are preserved.","type":"text"}]},{"type":"paragraph","content":[{"text":"There are much more options on how to format line breaks in YAML. For full details, see the YAML specification: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://yaml.org/spec/1.2.2/"}},{"text":" ","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"JSON Parameters","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#json-parameters"}}]}]},{"type":"paragraph","content":[{"text":"Furthermore, it is possible to specify a JSON as parameter, like this example shows:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - log:\n message: {\"text\": \"Hello World!\"}","type":"text"}]},{"type":"paragraph","content":[{"text":"As you can see, there is no need to escape or convert the JSON to a string. It can be placed as JSON 1:1 inside the YAML. ","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"PEL Parameters","type":"text"}]},{"type":"paragraph","content":[{"text":"Beside static string values it is also possible to pass dynamic values to the parameters. This is done by using a ","type":"text"},{"text":"Pipeline Expression","type":"text","marks":[{"type":"link","attrs":{"href":"https://logabit.atlassian.net/wiki/spaces/PA/pages/2543026496"}}]},{"text":". For example:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - log:\n message: ${@date.now()}","type":"text"}]},{"type":"paragraph","content":[{"text":"Also interpolation inside strings is possible out-of-the-box:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - log:\n message: \"Date of today: ${@date.now()}\"","type":"text"}]},{"type":"paragraph","content":[{"text":"For more information about Pipeline Expressions, see section: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://logabit.atlassian.net/wiki/spaces/PA/pages/2543026496"}},{"text":" ","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Executing a Pipeline","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#executing-a-pipeline"}}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Execute by HTTP request","type":"text"}]},{"type":"paragraph","content":[{"text":"You can execute a pipeline via HTTP request. Also see the Pipeline API docs: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://pipeforce.github.io/redoc.html#tag/Pipeline-API"}},{"text":" ","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"Ad-hoc pipeline","type":"text"}]},{"type":"paragraph","content":[{"text":"Executing an ad-hoc pipeline (ad-hoc = not persisted) with HTTP requires these steps:","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Set the pipeline YAML script in the body of a request","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Set the ","type":"text"},{"text":"Content-Type","type":"text","marks":[{"type":"code"}]},{"text":" header of the request to ","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"application/yaml","type":"text","marks":[{"type":"code"}]},{"text":" if the body is a single YAML pipeline or to","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"multipart/form-data","type":"text","marks":[{"type":"code"}]},{"text":" if the body contains a YAML pipeline and additional files to be uploaded and to be provided as content collection to the body of the ","type":"text"},{"text":"pipeline","type":"text","marks":[{"type":"code"}]},{"text":". In the multipart request body, the pipeline part must have the name ","type":"text"},{"text":"pipeline","type":"text","marks":[{"type":"code"}]},{"text":" and each file part must have the name ","type":"text"},{"text":"file","type":"text","marks":[{"type":"code"}]},{"text":". ","type":"text"}]}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Send this request as HTTP POST to the server at ","type":"text"},{"text":"/api/v3/pipeline","type":"text","marks":[{"type":"code"}]}]}]}]},{"type":"paragraph","content":[{"text":"The server then executes the pipeline and returns with the final body message of the pipeline as response body.","type":"text"}]},{"type":"paragraph","content":[{"text":"Here is an example, with a YAML pipeline in the body. This example loads a PDF file, sets a text on it and stores it back. In order to execute this pipeline, you can send it to the server using ","type":"text"},{"text":"curl","type":"text","marks":[{"type":"code"}]},{"text":" for example:","type":"text"}]},{"type":"codeBlock","content":[{"text":"curl -X POST \"https://hub-ns.pipeforce.net/api/v3/pipeline\" \\\n -H \"Content-Type: application/yaml\" \\\n -u \"username:password\" \\\n --data-binary @- << EOF\npipeline:\n - drive.read:\n path: /my.pdf\n - pdf.stamp:\n text: \"Hello World!\"\n - drive.save:\n path: /my-stamped.pdf\nEOF","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"paragraph","content":[{"text":"Summary","type":"text","marks":[{"type":"strong"}]}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"The HTTP endpoint must be ","type":"text"},{"text":"/api/v3/pipeline","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"The HTTP method must be ","type":"text"},{"text":"POST","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"The HTTP header Content-Type must be ","type":"text"},{"text":"application/yaml","type":"text","marks":[{"type":"code"}]},{"text":" or ","type":"text"},{"text":"multipart/form-data","type":"text","marks":[{"type":"code"}]}]}]}]}]},{"type":"paragraph","content":[{"text":"When using cURL, the ","type":"text"},{"text":"EOF","type":"text","marks":[{"type":"code"}]},{"text":" syntax is required here in order to preserve the intends of the YAML script. Optionally, you can write the YAML in a local file an pass this file as argument for the body. See the example below, where the pipeline is specified in a file with name ","type":"text"},{"text":"mypipeline.yaml","type":"text","marks":[{"type":"code"}]},{"text":":","type":"text"}]},{"type":"codeBlock","content":[{"text":"curl -X POST \"https://hub-ns.pipeforce.net/api/v3/pipeline\" \\\n -H \"Content-Type: application/yaml\" \\\n -u \"username:password\" \\\n -d \"@mypipeline.yaml\"","type":"text"}]},{"type":"paragraph","content":[{"text":"See ","type":"text"},{"text":"HTTP Execution Reference","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#http-execution-reference"}}]},{"text":" for an summary of all supported HTTP options.","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"Persisted pipeline","type":"text"}]},{"type":"paragraph","content":[{"text":"In order to execute a pipeline which is persisted in the property store, send a HTTP POST to the ","type":"text"},{"text":"/pipeline:{path}","type":"text","marks":[{"type":"code"}]},{"text":" endpoint whereas ","type":"text"},{"text":"{path}","type":"text","marks":[{"type":"code"}]},{"text":" is the path of the pipeline to be executed.","type":"text"}]},{"type":"paragraph","content":[{"text":"The body of the request becomes the body to the pipeline.","type":"text"}]},{"type":"paragraph","content":[{"text":"The request parameters become the variables to the pipeline.","type":"text"}]},{"type":"paragraph","content":[{"text":"Example:","type":"text"}]},{"type":"codeBlock","content":[{"text":"curl -X POST \"https://.../api/v3/pipeline:global/app/myapp/pipeline/mypipeline\" \\\n -u \"username:password\"","type":"text"}]},{"type":"paragraph","content":[{"text":"This executes the pipeline stored at path ","type":"text"},{"text":"global/app/myapp/pipeline/mypipeline","type":"text","marks":[{"type":"code"}]},{"text":" in the property store and returns the final pipeline body as response body to the caller.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Execute in Portal","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#execute-in-online-workbench"}}]}]},{"type":"paragraph","content":[{"text":"The portal offers an advanced online editor with syntax highlighting, code completion and debugging support, where you can write pipelines and execute them online. This is the easiest and most preferred way to ad-hoc execute and test your pipelines. Here you can see a simple pipeline after its ad-hoc execution in the online editor:","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center"},"content":[{"type":"media","attrs":{"width":1876,"id":"df539a9a-71e6-4e63-8a4b-bddbdcae70f9","collection":"contentId-2542796840","type":"file","height":1226}}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Execute in CLI","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#execute-in-cli-1"}}]}]},{"type":"paragraph","content":[{"text":"Another approach to execute a pipeline is by using the CLI: ","type":"text"},{"text":"Command Line Interface (CLI)","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/cli"}}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"Execute local pipeline file","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#execute-local-pipeline-file"}}]}]},{"type":"paragraph","content":[{"text":"Lets assume you have a local pipeline YAML stored at ","type":"text"},{"text":"src/global/app/myapp/pipeline/test.pi.yaml","type":"text","marks":[{"type":"code"}]},{"text":" inside of your PIPEFORCE workspace, then you can execute it via this CLI call (the path must start with ","type":"text"},{"text":"src/","type":"text","marks":[{"type":"code"}]},{"text":"):","type":"text"}]},{"type":"codeBlock","content":[{"text":"pi pipeline src/global/app/myapp/pipeline/test.pi.yaml","type":"text"}]},{"type":"paragraph","content":[{"text":"This will load the local pipeline YAML and run it by sending it to the server for execution. The result will be printed out to your terminal if there is any.","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"paragraph","content":[{"text":"NOTE","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"A pipeline YAML file must end in this suffix to be detected correctly by your workspace: ","type":"text"},{"text":".pi.yaml","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"Execute persisted remote pipeline","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#execute-persisted-remote-pipeline"}}]}]},{"type":"paragraph","content":[{"text":"In case you have stored your pipeline at server side in the ","type":"text"},{"text":"Property Store","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/propertystore"}}]},{"text":", then you can execute it using this call (the path must start with ","type":"text"},{"text":"global/","type":"text","marks":[{"type":"code"}]},{"text":"):","type":"text"}]},{"type":"codeBlock","content":[{"text":"pi pipeline global/app/myapp/pipeline/test","type":"text"}]},{"type":"paragraph","content":[{"text":"This command searches for a property in the property store with path ","type":"text"},{"text":"global/app/myapp/pipeline/test","type":"text","marks":[{"type":"code"}]},{"text":" and executes it. Finally, it sends any result back to your terminal.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Pipeline Sections","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#pipeline-sections"}}]}]},{"type":"paragraph","content":[{"text":"Every pipeline YAML script may consist of four main sections:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"headers","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"vars","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"pipeline","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"body","type":"text","marks":[{"type":"strong"}]}]}]}]},{"type":"paragraph","content":[{"text":"Here is an example of a pipeline script which defines all of these sections:","type":"text"}]},{"type":"codeBlock","content":[{"text":"headers:\n contentType: \"text/plain\"\n \nvars:\n counter: 0\n\npipeline:\n - log:\n message: \"HELLO WORLD!\"\n \nbody: \"This is text in the body\"","type":"text"}]},{"type":"paragraph","content":[{"text":"All sections except ","type":"text"},{"text":"pipeline","type":"text","marks":[{"type":"code"}]},{"text":" are ","type":"text"},{"text":"optional","type":"text","marks":[{"type":"strong"}]},{"text":" in a pipeline script. Even if not explicitly defined in the pipeline script, each scope exists implicitly. That means, you can access it and read / set values from / on it without declaring it in the pipeline. For example, by using a pipeline expression (PE).","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"headers","type":"text","marks":[{"type":"code"}]},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#headers"}}]}]},{"type":"paragraph","content":[{"text":"The headers section is optional. A header is a name-value pair to define \"global configuration\" hints and configurations for the given pipeline. Only text is allowed as content i.e. no complex objects like JSON. It is not meant to be changed during pipeline processing, even this is possible for rare cases.","type":"text"}]},{"type":"paragraph","content":[{"text":"Whether and which headers are required depends on the pipeline and its commands. There are some default headers in order to configure the pipeline processing. See the ","type":"text"},{"text":"headers reference","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/api/headers"}}]},{"text":" for details.","type":"text"}]},{"type":"paragraph","content":[{"text":"It is similar to ","type":"text"},{"text":"HTTP Request Headers","type":"text","marks":[{"type":"link","attrs":{"href":"https://en.wikipedia.org/wiki/List_of_HTTP_header_fields"}}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"You can read and set values in the headers section using the ","type":"text"},{"text":"Pipeline Expression Language (PEL)","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/guides/commands_pipelines/pel"}}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"vars","type":"text","marks":[{"type":"code"}]},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#vars"}}]}]},{"type":"paragraph","content":[{"text":"The vars section is optional and contains transient variables as name value pairs. It is meant as a transient scope for states during the pipeline processing.","type":"text"}]},{"type":"paragraph","content":[{"text":"Values can also be complex objects and documents like JSON for example.","type":"text"}]},{"type":"paragraph","content":[{"text":"Values can be changed during pipeline processing.","type":"text"}]},{"type":"paragraph","content":[{"text":"You can access values in the vars scope using the ","type":"text"},{"text":"Pipeline Expression Language (PEL)","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/guides/commands_pipelines/pel"}}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"pipeline","type":"text","marks":[{"type":"code"}]},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#pipeline"}}]}]},{"type":"paragraph","content":[{"text":"The pipeline section is mandatory and lists all commands which must be executed in given order.","type":"text"}]},{"type":"paragraph","content":[{"text":"See the ","type":"text"},{"text":"commands reference","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/api/commands"}}]},{"text":" for details about the default commands.","type":"text"}]},{"type":"paragraph","content":[{"text":"You can set dynamic parameter values on commands using the ","type":"text"},{"text":"Pipeline Expression Language (PEL)","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/guides/commands_pipelines/pel"}}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"body","type":"text","marks":[{"type":"code"}]},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#body"}}]}]},{"type":"paragraph","content":[{"text":"The body section is optional. It defines a single object to be used as “data pool” or transformation data during the pipeline processing.","type":"text"}]},{"type":"paragraph","content":[{"text":"In case a command returns a value, by default, it will write this value to the body implicitly. Whereas a previous command's value in the body will be overwritten by the command which comes next.","type":"text"}]},{"type":"paragraph","content":[{"text":"It's also possible to define an initial value for the body in the pipeline. If no such initial value is set, the body is initally ","type":"text"},{"text":"null","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"You can access values in the body scope using the ","type":"text"},{"text":"Pipeline Expression Language (PEL)","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/guides/commands_pipelines/pel"}}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Pipeline as JSON","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#pipeline-as-json"}}]}]},{"type":"paragraph","content":[{"text":"Sometimes it is necessary to use JSON as the pipeline definition language instead of YAML. Let's assume a pipeline written in YAML like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"headers:\n apiVersion: v2\nvars:\n age: 23\n name: \"Max\"\npipeline:\n - datetime:\n format: dd-MM-YYYY\n - log:\n message: \"Hello World!\"\nbody: \"Some input\"","type":"text"}]},{"type":"paragraph","content":[{"text":"You can rewrite this YAML pipeline as JSON pipeline like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"{\n \"headers\": {\n \"apiVersion\": \"v2\"\n },\n \"vars\": {\n \"age\": 23,\n \"name\": \"Max\"\n },\n \"pipeline\": [\n {\n \"datetime\": {\n \"format\": \"dd-MM-YYYY\"\n }\n },\n {\n \"log\": {\n \"message\": \"Hello World!\"\n }\n }\n ],\n \"body\": \"Some input\"\n}","type":"text"}]},{"type":"paragraph","content":[{"text":"In order to execute such a JSON pipeline, you can send it to the same POST endpoints as you would do with YAML pipelines, but with header changed to ","type":"text"},{"text":"Content-Type: application/json","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"paragraph","content":[{"text":"NOTE","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"Since ","type":"text"},{"text":"YAML is the default","type":"text","marks":[{"type":"strong"}]},{"text":" definition for pipelines and it is much easier to read, you should prefer it over JSON whenever possible. The JSON variant is mainly meant for internal cases, where YAML is not possible or hard to use and it is not compatible with any use case.","type":"text"}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Pipeline as URI","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#pipeline-as-uri"}}]}]},{"type":"paragraph","content":[{"text":"Beside YAML and JSON, a third option to define a pipeline is possible: Using a ","type":"text"},{"text":"pipeline uri","type":"text","marks":[{"type":"strong"}]},{"text":" which is an inline version of a pipeline. This is handy in case you must define a pipeline as a “one-liner”.","type":"text"}]},{"type":"paragraph","content":[{"text":"You can rewrite any pipeline YAML as a single URI request query string. ","type":"text"}]},{"type":"paragraph","content":[{"text":"Let's assume this pipeline YAML:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pipeline:\n - datetime\n format: dd.MM.YYYY \n - log","type":"text"}]},{"type":"paragraph","content":[{"text":"You can rewrite this pipeline YAML as inline using a URI request query string, which looks like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"?datetime=\"format:dd.MM.YYYY\"&log","type":"text"}]},{"type":"paragraph","content":[{"text":"You can then execute such a pipeline URI using the CLI:","type":"text"}]},{"type":"codeBlock","content":[{"text":"pi pipeline ?datetime=\"format:dd.MM.YYYY\"&log","type":"text"}]},{"type":"paragraph","content":[{"text":"So the format is like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"?command1=\"param1:value1;param2:'value2'\"&command2=\"param3:value3\"","type":"text"}]},{"type":"paragraph","content":[{"text":"The request parameter names become the command names and the parameters to this command become the request parameter values. Each single command parameter name ends with a colon ","type":"text"},{"text":":","type":"text","marks":[{"type":"code"}]},{"text":". Multiple command parameters are separated by a semicolon ","type":"text"},{"text":";","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"This is compliant to the URI syntax and especially handy for smaller pipelines which you want to execute ad-hoc from your terminal or where you can define a pipeline as one-liner only. So it consists multiple instructions in one line.","type":"text"}]},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"NOTE","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"There is also an older approach to define a pipeline in one line. This has used the pipe ","type":"text"},{"text":"|","type":"text","marks":[{"type":"code"}]},{"text":" symbol to separated the commands. Since this approach is not compatible with the URI syntax specification, it was dropped in favour of the approach defined in this chapter.","type":"text"}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Auto-completion support","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#auto-completion-support"}}]}]},{"type":"paragraph","content":[{"text":"In order to enable auto-completion support for your pipeline YAML scripts in your local development editor, you need an editor which supports YAML schema validation. Then, you can have auto-completion which shows all available commands and their parameters:","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center"},"content":[{"type":"media","attrs":{"width":740,"id":"5dd9e2ae-568d-46fc-a865-4117050ce535","collection":"contentId-2542796840","type":"file","height":314}}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Auto-completion in IntelliJ","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#intellij"}}]}]},{"type":"paragraph","content":[{"text":"To enable auto-completion in IntelliJ, open preferences and navigate to JSON Schema Mappings:","type":"text"}]},{"type":"paragraph","content":[{"text":"Preferences → Languages & Frameworks → Schemas & DTDs → JSON Schema Mappings","type":"text","marks":[{"type":"code"}]}]},{"type":"paragraph","content":[{"text":"Add a new schema mapping with these values:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Name: ","type":"text"},{"text":"pipeline-schema","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Schema URL: ","type":"text"},{"text":"https://hub-.pipeforce.net/api/v3/command:pipeline.schema.get","type":"text","marks":[{"type":"code"}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Schema version: ","type":"text"},{"text":"JSON Schema version 7","type":"text","marks":[{"type":"code"}]}]}]}]},{"type":"paragraph","content":[{"text":"Add new file path patterns for : ","type":"text"},{"text":"*.pi.yaml","type":"text","marks":[{"type":"code"}]}]},{"type":"paragraph","content":[{"text":"Now, try it out: Create a new file ","type":"text"},{"text":"foo.pi.yaml","type":"text","marks":[{"type":"code"}]},{"text":" and start typing ","type":"text"},{"text":"pipeline:","type":"text","marks":[{"type":"code"}]},{"text":". ","type":"text"}]},{"type":"paragraph","content":[{"text":"The key combination ","type":"text"},{"text":"[Ctrl]","type":"text","marks":[{"type":"code"}]},{"text":" + ","type":"text"},{"text":"[Space]","type":"text","marks":[{"type":"code"}]},{"text":" should give you a list of suggested values for your YAML.","type":"text"}]},{"type":"paragraph","content":[{"text":"A YAML pipeline script should always end in suffix ","type":"text"},{"text":".pi.yaml","type":"text","marks":[{"type":"strong"}]},{"text":" which stands for stands for pipeline scripts written in YAML.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Auto-completion in Visual Studio Code","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#visual-studio-code"}}]}]},{"type":"paragraph","content":[{"text":"If you want to also enable code-completion for your pipeline yaml files in your VS Code editor, you need to install the YAML language support plugin from Red Hat first: ","type":"text"},{"text":"https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml","type":"text","marks":[{"type":"link","attrs":{"href":"https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml"}}]}]},{"type":"paragraph","content":[{"text":"Then open ","type":"text"},{"text":"Preferences → Settings","type":"text","marks":[{"type":"code"}]},{"text":" and add this line to your configuration ","type":"text"},{"text":"settings.json","type":"text","marks":[{"type":"strong"}]},{"text":" and save it:","type":"text"}]},{"type":"codeBlock","content":[{"text":"\"yaml.schemas\": { \n \"https://hub-NAMESPACE.pipeforce.org/api/v3/command:pipeline.schema.get\": [\"/*.pi.yaml\"] \n}","type":"text"}]},{"type":"paragraph","content":[{"text":"Now, try it out: Create a new file ","type":"text"},{"text":"foo.pi.yaml","type":"text","marks":[{"type":"code"}]},{"text":" and start typing ","type":"text"},{"text":"pipeline:","type":"text","marks":[{"type":"code"}]},{"text":". ","type":"text"}]},{"type":"paragraph","content":[{"text":"The key combination ","type":"text"},{"text":"[Ctrl]","type":"text","marks":[{"type":"code"}]},{"text":" + ","type":"text"},{"text":"[Space]","type":"text","marks":[{"type":"code"}]},{"text":" should give you a list of suggested values for your YAML.","type":"text"}]},{"type":"paragraph","content":[{"text":"A local YAML pipeline script should always end in suffix ","type":"text"},{"text":".pi.yaml","type":"text","marks":[{"type":"strong"}]},{"text":" which stands for pipeline scripts written in YAML.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Auto-completion in the Portal","type":"text"},{"text":"","type":"text","marks":[{"type":"link","attrs":{"href":"https://pipeforce.github.io/docs/commands_pipelines#online-workbench"}}]}]},{"type":"paragraph","content":[{"text":"The built-in online workbench and the playground in the PIPEFORCE portal supports pipeline script completion out-of-the box.","type":"text"}]},{"type":"paragraph","content":[{"text":"To start completion simply press ","type":"text"},{"text":"[Ctrl]","type":"text","marks":[{"type":"code"}]},{"text":" + ","type":"text"},{"text":"[Space]","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center"},"content":[{"type":"media","attrs":{"width":1978,"id":"45062c0e-06c0-445a-beb6-c9645bf98dd6","collection":"contentId-2542796840","type":"file","height":842},"marks":[{"type":"border","attrs":{"size":2,"color":"#091e4224"}}]}]},{"type":"paragraph","content":[{"text":"Beside completion for available commands and their parameters, it also supports completion for other parts like utilities and variables for example:","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center","width":100.0},"content":[{"type":"media","attrs":{"width":1848,"id":"1af65ef9-febd-442e-8ba4-d5c6eae7010b","collection":"contentId-2542796840","type":"file","height":914},"marks":[{"type":"border","attrs":{"size":2,"color":"#091e4224"}}]}]},{"type":"paragraph"},{"type":"rule"},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"YOUR HELP IS NEEDED!","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"In case you're missing something on this page, you found an error or you have an idea for improvement add a comment on this page directly and put your feedback there. Many thanks for your help in order to make PIPEFORCE better!","type":"text"}]}]},{"type":"paragraph"}],"version":1}

Browser not supported