What is the Property Store?​

The property store is a built-in key-value document database as integrated part of PIPEFORCE to save your application data like sources, attachments or JSON documents for example. There is no extra setup or maintenance required. You can create, update, query and delete data out-of-the-box on the property store using the property.* command from inside your automation pipeline.

Furthermore, the property store provides advanced deep query, authorization and scaling options.

What is a Property?​

Each data (for example plain text, JSON, binary data, ...) stored in the property store is wrapped into an envelope called a property.

Property Attributes​

Each property has multiple attributes (the envelope data). The most important ones are:

Property Path​

Every property has a unique path. Such a property path is structured in an hierarchical way and has this base structure:

For example:

Every property belongs to a certain namespace. If such a path starts with a slash / at the beginning, it is interpreted as absolute. Meaning, it must start with /pipeforce followed by the namespace, this property belongs to. Properties are always stored with their absolute paths in the property store.

At runtime, property paths can be defined and interpreted relatively. Here is an example of an relative path of a property:

Relative property paths do not start with a slash. They do not contain information about the current namespace. This information is automatically added when a property is loaded or stored.

WHY PROPERTY PATH?

• It defines the unique path of a property in a tree structure.

• It makes it easier to query a nested subset of properties "recursively" using path patterns.

• It allows to group properties together under a "path-like" structure.

• It is also used for declaring nested read and write permissions.

• It defines the storage location of the property for performance optimization (for example archive is stored on a different location)

Property Path Pattern​

Since the path of a property is similar to a path in a file tree, it is possible to use path matching techniques in order to easily find and load a nested subset of properties "recursively" from the store. By default, PIPEFORCE uses the so called "ant-style" path matcher. This matcher uses the following rules, applied to a path:

• The double asterisk ** matches any character in the path at given position.

• The single asterisk * matches zero or more charachters inside a "directory" of the path (up to the next slash /).

The asterisk * is often also called wildcard. A property path containing wildcards are called a path pattern.

Example 1

This path pattern will select all properties having a path starting with global/app/myapp at any sublevel. So it will match these paths:

But it won't mach these paths:

Example 2

This path pattern will select all properties having a path starting with global/app/myapp inside this level (meaning inside the "directory" myapp) but not at any sublevel. So it will match these paths:

But it won't match these paths:

Example 3

It is also possible to combine multiple wildcards inside a pattern.

This path pattern will match any path starting with prefix global/app, followed by any app name (without any slash), followed by /data/, followed by any further charachters at any level. It will match these paths:

But it won't match these paths:

Example 4

This pattern would match any relative path at top level, but not any sub-level. So it will match these paths:

But it won't match these paths:

Example 5

This pattern will match anything at any level. So it will match these paths:

There is no path, this pattern will not match.

Property Value and Type​

The value attribute of a property defines the payload of a property. Such a payload could be a JSON document or a plain text for example. It is always stored as string (text) value. Which type the text value represents, is defined by the type attribute which could be of any supported mime type e.g. application/json or text/plain. The latter is the default, if no type is specified. Important mime types, used by PIPEFORCE are:

See the http://IANA.org websites for a full list of official mime types: Media Types

Property types can additionally contain parameters. Example:

Multiple parameters are separated by a colon ;. See the IANA specification for details. PIPEFORCE uses custom type parameters in some situations. Some examples:

Binary Value​

The value attribute contains always values of type string. In order to store binary data there, it must be base64 encoded before it gets persisted. In this case, the property must be marked as a base64 value. This is done in the type attribute using the mime type parameter encoding=base64. Example:

In this case the property can be automatically converted back into a "binary" file when loaded.

Create a Property​

To create a new property in the property store, you need to define a property schema first using the command property.schema.put. Example:

This schema defines the base envelope data for the new property like its path, uuid and type. After the schema has been created, the property payload can be added and edited.

To simplify these steps, you can create the schema and add the payload value in one single step:

Property created event​

Whenever you create a new a property, an event with key property.created is fired with the created property stored in payload.target of the event object. This way you can listen in your pipelines for properties created newly. See the reference documentation for details. Here is an example how to listen to such an event in a persisted pipeline:

This persisted pipeline gets automatically executed whever a new property with a path containing global/app/myapp/data/mydata has been created.

Update a Property​

To change the value of the property, you can use the command property.put. The property must already exist beforehand. Example:

In order to change the envelope data (attributes) of a property, you need to use the property.schema.put command:

If you would like to change the path of an existing property, you can use the property.move command:

Property changed event​

Whenever you update a property value or one of its other attributes, an event with key property.updated is fired with the origin property stored in origin of the event object and the final version of the property in target. This way you can listen in your pipelines for property changes. See the reference documentation for details. Here is an example how to listen to such an event in a persisted pipeline:

Property moved event​

Whenever you move a property by changing its origin path, an event with key property.moved is fired with the origin path of the property stored in origin of the event object and the final path of the property in target. This way you can listen in your pipelines for property movements. See the reference documentation for details. Here is an example how to listen to such an event in a persisted pipeline:

Delete a Property​

In order to delete a property from the property store, you can use the command property.schema.delete, which deletes the property value and all metadata that belongs to this property. Example:

You can also delete multiple properties at once using a pattern. Example:

This example will delete all properties of the app myapp recursively.

Move to trash bin

Instead of deleting a property, you can also move it to a trash bin. In this case the property wont be deleted immediately, instead it will be deleted in case the trash bin is cleaned up or in case the time to live in the trash bin has been expired. Example:

In this example, the property wont be deleted. Instead it will be moved to the trash bin for 30 days and then deleted after this time.

For more information see: TODO

Property deleted event​

Whenever you delete a property, an event with key property.deleted is fired with the origin property stored in payload.origin of the event object. This way you can listen in your pipelines for property deletions. See the reference documentation for details. Here is an example how to listen to such an event in a persisted pipeline:

Loading Properties​

If you want to query for property data, you have multiple options in PIPEFORCE.

Listing properties​

One of the simplest approaches to query for multiple properties is by using the command property.list. Example:

This command will return a result similar to the result shown below, which describes all important attributes of the property:

You can also use a path pattern in order to load multiple properties matching this path pattern. This example will load all properties inside the pipeline folder:

This will return "recursively" all properties below the path global/app/myapp/.... The properties attributes including the value will be returned.

Getting property value​

In case you would like to load only the value (payload) of a property but not its metadata, you can use the command property.value.get. Example:

Advanced / Deep Querying​

In case the base commands for loading properties are not sufficient for your case, you can use JSON Property Querying and Searching which supports also complex search operations on JSON documents.