Getting Started

Owned by PIPEFORCE
Last updated: Oct 22, 2024
8 min read

Connect Systems, Processes and Teams.

PIPEFORCE is the #1 enterprise-grade hybrid cloud-native platform for application integration and business process automation (iPaaS).

It combines seamless all important elements of modern business automation and integration operations as single turnkey solution:

image-20240111-135820.png

… across the entire journey from planning and discussion to implementation and testing with full life-cycle management. Here are examples of some built-in services:

What is PIPEFORCE for?

PIPEFORCE helps you to:

  • Integrate applications and data

  • Automate processes

  • Implement and orchestrate microservices

  • Automate the testing of complex solutions

  • Monitor and report processes and messages

  • Stateful workflows

  • Train processes with users

  • And more…

All of this on a single platform running on Kubernetes.

We at PIPEFORCE believe that the digital success of new age enterprises lies in the ability to scale the development of digital solutions. And the key to this is our mission to Make digital business automation as easy as possible.

We think that Low Code / No Code platforms are important here and must seamlessly work together with workflow automation and application development and integration on a central platform. Our goal is to give your employees digital superpower in their fields of expertise. The platform brings these experts closer together, so that they can make the best possible contribution to the digital goals of your company.

PIPEFORCE helps enterprises to achieve digital superpower by clearly separating the swimlanes of expertise but defining clear interfaces between the different roles so that they can work together very effectively to reach the same goals:

This documentation is mainly for Low Coders and Professional Developers. If you are looking for user or admin manuals, please refer to the Support pages.

If you are a Low Code Developer or a Professional Developer and new to PIPEFORCE, we recommend you to walk through the Tutorials section first. After this you should have a good base understanding of PIPEFORCE.

In the API section you can find reference documentation for PIPEFORCE.

In case you don't have a PIPEFORCE account yet, we highly recommend you to create one first, so you can try out all examples and tutorials live. It's a matter of seconds to create a free account, here:

https://try.pipeforce.org

Key Concepts

What is a Namespace?​

A namespace in PIPEFORCE is the unique name to access your instance and is similar to a tenant id. All services of a certain customer are hosted inside this namespace:

Whenever you want to refer / link to services inside your own instance you need to know your namespace.

For example, to access the web portal of your instance via browser, use the namespace as subdomain part of your pipeforce.net instance url:

In case you do not know your namespace yet, feel free to ask our Support team.

What is a Service?​

Inside each PIPEFORCE namespace multiple services are hosted to run your solutions. Depending on your setup, the number and type of these services could differ. Each service is encapsulated inside a (“Docker”) container and can “talk” to each other:

We use the term “Docker” container for any type of container, for simplicity.

Although in the backend different container types are supported like: containerd, Docker, CRI-O, and Mirantis.

Public Service​

Usually these services are by default available within your namespace and can be accessed from the public internet. They are therefore called "public" services:

  • Portal: A single page application which hosts the user and developer UI. Can be accessed via: https://NAMESPACE.pipeforce.net.

  • Hub : This is the core service which manages and executes Commands & Pipelines. Can be accessed via: https://NAMESPACE-hub.pipeforce.net.

  • Drive: A data room application to store, edit and share documents and files. Can be accessed via: https://NAMESPACE-drive.pipeforce.net.

  • IAM: An identity and access management system to manage users, groups, permissions and logins. Can be accessed via: https://NAMESPACE-iam.pipeforce.net.

Private Service​

Additionally to the services accessible from the internet, there are also some default services which can be accessed only from inside the namespace, so these services can be accessed only by other services running inside the same namespace, but not from the public internet. They are called the "private" services. Examples of such public services are:

  • Postgres: A PostgreSQL database which hosts application data.

  • Messaging: A messaging broker which is part of the internal message bus.

  • Workflow: A workflow engine in order to manage and execute BPMN workflows.

Custom Service​

Beside the default services provided by PIPEFORCE as mentioned above, there could be additional, custom services depending on your license / setup. PIPEFORCE allows to deploy a container easily as an additional custom service into your namespace. This can be a container provided by the Docker Hub or developed and built by yourself.

See the section Microservice for more information about custom services (= microservices).

Resiliency and Scalability

All services run as (“Docker”) containers inside your namespace. It is possible to run multiple such container instances of of the same service with built-in load balancers. Also features like auto-scaling and auto-healing are available here so your solutions can be scaled nearly unlimited.

For example, if you have a service for which you will expect a high load, you can run multiple instances of this service out-of-the-box in order to be able to process all incoming requests accordingly.

It is even possible to auto-scale: For example to auto-run more instances in peak situations and auto-scale down if resources are no longer needed.

The default services provided by PIPEFORCE are already optimized for scaling and will be managed by the PIPEFORCE team. For your custom microservices you have to make sure that you develop them in a way that they are able to be scaled, from the ground up.

In case you have questions about how to develop resilient and scalable microservices in PIPEFORCE, feel free to ask our Support team.

What are Staging Tiers?​

Especially in case you develop your own custom microservices, you should think about a staging approach in order to increase the testability and quality of your services.

Typically, the staging process is separated into at least three tiers: DEV, QA, PROD.

You can learn more about such environments on this Wiki page.

In case you do not write custom microservices and use low code and integration, you can implement such a staging process inside a single namespace. User groups, permissions and versioning can help you to show your solution only to a certain group depending on the deployment phases.

In case you develop custom microservices, you should think about a staging approach using multiple namespaces so each staging tier operates independently from each other.

If you need help to decide your staging approach, feel free to ask our Support team.

DEV​

In this tier the development happens. It is highly dynamically and can change frequently. It is usually part of a continuous integration and deployment process with automated build and testing steps.

QA​

Once development is finished, the solution usually gets deployed to the QA tier. Sometimes also called "Staging". This is the tier where quality assurance tests are done with the system configured as close as possible similar to the production environment. Sometimes this staging tier contains a copy of the production data so migration steps and performance can be tested here as well.

PROD​

In this last tier, the system is operated in production. Once a solution has been tested successfully on the QA tier it can be moved to the PROD tier.

The standard layers of PIPEFORCE

There are standard enterprise grade layers, any business solution must address to seamless integrate, run and scale inside an enterprise, so PIPEFORCE does.

Usually, for any of these layers at least one technical expert and sometimes also a bigger team of software engineers and technical experts is required to implement and/or maintain the specific layer. But since PIPEFORCE is a turnkey solution, most of these layers are already covered by the maintenance subscription fee without extra cost or can be easily implemented and maintained by non-technical users using our Low Code, No Code and AI toolings.

Below you can find a mapping from the enterprise standard layers to the sections in the documentation for faster navigation.

Presentation Layer

This layer handles the user interactions and interfaces like forms and lists.

Business Logic & Data Processing Layer

Manages business logic and rules for data processing. Deals with processing operations on data, including computation and applying logic. Handles document evaluation, extraction, and processing, applying business logic and rules.

Data Access Layer

Responsible for data management and interaction with internal and external databases.

Integration Layer

Facilitates communication between different systems and services, often through APIs or middleware.

Workflow Layer

Coordinates stateful processes and workflows where manual steps and interactions with humans are required within the system.

Security and Management Layer

Focuses on authentication, authorization, encryption, logging, and security management.

Microservice Layer

Provides a runtime environment to execute, scales and manage microservices and containers.

Microservices

Application Infrastructure Layer

Provides the infrastructure necessary for the application, including servers and cloud services.

  • PIPEFORCE runs entirely Cloud Native inside Kubernetes.

  • Can by hosted on any hyperscaler (like Google Gloud, AWS or Azure for example) or OnPremises in a private cloud.

Caching Layer

Responsible for storing frequently accessed data to reduce latency.

  • Different types of caches in the backend.

Orchestration Layer

Coordinates complex workflows and service interactions, often in microservice architectures.

  • Built-in in infrastructure and backend architecture.

Reporting and Analytics Layer

Handles business intelligence, analytics, and reporting functions.

Identity and Access Management (IAM) Layer

Manages user identities and controls access to system resources.

Audit Layer

Captures logs and audit data for compliance.

https://logabit.atlassian.net/wiki/spaces/PA/pages/2548465885/Logging%2C+Tracing+and+Monitoring

Communication & Messaging Layer

Manages communication between services, often using messaging systems.

Security Architecture Layer

Provides security mechanisms across all system layers, including encryption and SSL/TLS.

Notification Layer

Manages notifications and real-time alerts for users or systems.

Monitoring and Logging Layer

Focuses on system health monitoring, performance metrics, and logging.

AI Layer

The artificial intelligence (AI) layer helps to speed up implementation and operation time and helps in optimizations afterwards.

  • Document Understanding Components

  • ChatBot integrations

  • Data Mapping Support

  • Security Checks

  • Automated workflow creation and optimization