...

This is especially useful for example for payable invoices in order to extract information from these PDF invoices and review them before they will be forwarded to the internal invoice payable process.

...

invoicing and approval process.

...

How to execute it?

In order to send a given PDF document to the Document Understand AI backend and to extract the required data, you can use the command ai.document.understand in your pipeline.

This command will load the document, will validate the given understand config parameters, then sends the document and the instructions to the AI and finally returns a JSON response with the values extracted from the document which can be further processed in the pipeline.

...

pipeline:
  - ai.document.understand:
      secret: google-document-understandDOCUMENT_UNDERSTANDING_GOOGLE
      provider: google
      input: $uri:drive:my-invoice.pdf
      config: {
          "projectId": "my-project",
          "location": "de",
          "processorId": "3e05c9d1c5386f42"
        }

...

• secret
  The name of the secret to be used to connect to the Document Understand AI backend.

• provider
  The AI backend to be used. There are different implementations possible:

  • google = Uses Google’s Document Understand cloud service (hosted in Germany if it is part of the enterprise plan).

  • aws = Use the AWS Document Understand cloud service.

  • <custom> = Uses a self-hosted AI model and the provided endpoint to solve this problem (coming soon).

• input
  The input document to be send to the AI. Can be any PIPEFORCE URI. If no input parameter is specified, the input document is expected in the body of the pipeline.

• config
  The configuration or prompt required by the AI backend to fulfill this document understand request and return the expected JSON. This configuration depends on the selected AI backend using the provider parameter. See the documentation of the provider. For Google Document Understand the parameters are:

  • projectId = The Google Cloud project to be used.

  • location = The location of the processor.

  • processorId = The pre-trained processor to be used for data extraction.

...

{
    "title": "Document Understand",
    "output": "...",
    "config": {
        "type": "invoice",
        "name": "My Invoice Field Detector",
        "fields": [
            {
                "id": "invoice_number",
                "mapping": "invoice_id",
                "label": "Invoice number"
            },
            {
                "id": "invoice_date",
                "mapping": "invoice_date",
                "label": "Invoice date"
            },
            {
                "id": "line_item",
                "mapping": "line_item",
                "label": "Invoice items",
                "columns": [
                    {
                        "id": "line_item/description",
                        "mapping": "line_item/description",
                        "label": "Description"
                    },
                    {
                        "id": "line_item/quantity",
                        "mapping": "line_item/quantity",
                        "label": "Quantity"
                    },
                    {
                        "id": "line_item/amount",
                        "mapping": "line_item/amount",
                        "label": "Amount"
                    }
                ]
            }
        ]
    }
}

The title defines the header to be displayed when the form is shown.

The type defines the type of document to be supported. This is by default invoice.

The output defines a PIPEFORCE URI where to write the final result JSON to after the form was confirmed.

...

You additionally add any of these fields into your fields mapping config as decribed in the example above.

The output format

After the form was submitted, a JSON which contains the extracted fields and the document embedded as base64 will be created and stored at the location specified by the output path.

Here is an example of this outout document:

{
  "fields": {
    "invoice_number": "12345",
    "invoice_date": "01.01.2024",
    "line_item": [
      {
        "line_item/description": "Some item",
        "line_item/quantity": "2",
        "line_item/amount": "13,99"
      }
    ],
    ...
  },
  "document": {
    "filename": "myinvoice.pdf",
    "contentLength": 1234,
    "contentType": "application/pdf",
    "contentEncoding": "base64",
    "content": "base64EncodedFileContent"
  }
}

The field names will be the field.id as configured.

The document is a content reference JSON with the document data base64 encoded.

Also see: Content References (Files)

Internationalizing Labels (i18n)

It is possible to translate the labels of the form fields to different languages.

If no label is given for a field, this label value will be used as default:

$uri:i18n:document-understand/<id> 

Whereas <id> is the id of the field. The UI expects this i18n key to exist. For example:

$uri:i18n:document-understand/invoice_number 

In case the label starts with prefix $uri:i18n: then the value is replaced by the current i18n message key whereas the format of the i18n URI is this:

<appName>/<contextName>/<messageKey>

So <appName> maps to the parameter app (optional), <contextName> (optional) maps to the context parameter of the command i18n.message and <messageKey> maps to the the message key inside the JSON finally returned by the command. For example if user sets this label:

"label": "$uri:i18n:io.pipeforce.myapp/invoice/invoice_number"

This would map to a message JSON which can be returned using the command i18n.message?app=io.pipeforce.myapp&context=invoice. And inside this message JSON, the value of attribute invoice_number for the currently selected language will be returned:

{
  "invoice_number": "Rechnungsnummer",
  ...
}

If no <appName> is given, io.pipeforce.common will be used by default.

If no <contextName> is given, the context default will be used.

So this example would expect the message entry invoice_number in a message JSON located in app io.pipeforce.common with context set to default:

$uri:i18n:invoice_number

And this will use the context document-understand inside the default app io.pipeforce.common (since the first part in the path is missing):

$uri:i18n:document-understand/invoice_number