Prerequisites

• To execute the following actions, you have to be assigned to the Admin role.

• The Pipeline App has the status STOPPED unless otherwise stated.

Changing the basic information

You can change the Pipeline App Name and/or Pipeline Description in the Basic Information pane of App Settings.

Proceed as follows

1. Enter a new name in the Pipeline App Name field.

2. Enter a new description in the Pipeline Description field.

3. Click Save.

→ The name and/or description of the Pipeline App has been changed.
→ A new pipeline version has been automatically created. Refer to Managing pipeline versions.

Configuring processes and instances

The table in the Processes and Instances pane lists the individual instances together with their CPU, Memory Usage, and Disk Usage. If an instance is currently not running, the corresponding columns display the icon.

The table displays also the Global Processing Timeout in minutes, as well as the Multithreaded Processing status, along with the number of threads.

You can scale your Pipeline App by creating instances if it requires more memory or computation power. You can either increase the amount of instances that should run or order more memory per instance.

You can also configure the global processing timeout value and choose between single threaded or multithreaded processing.

Prerequisites

For updating the number of instances, memory usage and disk limit (points 2, 3 and 4 below), the Pipeline App must have the status STARTED.

Proceed as follows

1. Click the Settings icon in the Processes and Instances pane to scale the Pipeline App and to configure Advanced Settings.
   → The Scale Pipeline App dialog is displayed.
   

   

2. In the Instances field, enter a number of instances you want to add.

3. In the Memory Usage drop-down list, select a value to adjust the memory usage.

4. In the Disk Limit drop-down list, select a value to adjust the disk limit.

5. Click Advanced Settings to access the following additional configuration options:

   1. In the Default processing step timeout (in minutes) field, enter a value in minutes, which will be applied to the processing steps if no other timeout is defined.
      The default value is one minute. The minimum value is also one minute, whereas the maximum value is 24 hours, defined in minutes.

   2. In the Threading configuration area, choose between Single threaded or Multithreaded (default) processing.
      The default number of threads is five, which equals five input files.
      

      Although these advanced configuration settings are allowed, it is not recommended to change them, as this could lead to unforeseen behavior and issues.

6. Click the Apply Changes button to save your changes.
   → The Pipeline App has been scaled.
   → The Advanced configuration settings for processing timeout and threading are applied.
   → A new pipeline version has been automatically created. Refer to Managing pipeline versions.

Adding environment variables

Environment variables behave like standard system variables. Using environment variables, you can set parameters in the User-Provided Environment Variables pane to have them available in your application code.

If you enter a valid environment variable, it is saved automatically to the backend.

Proceed as follows

1. Click the Add Environment Variable button in the User-Provided Environment Variables pane.
   → A new line is added.

2. Enter a Name for the environment variable.

3. Enter a Value for the environment variable.
   → The environment variable has been set.

4. Click the Save button.
   → A new pipeline version has been automatically created. Refer to Managing pipeline versions.

Example

In the following image, you see that the environment variable APP_ENVIRONMENT=DEV has been set. The SFDE_PIPELINE environment variable is set by default and cannot be changed.

Depending on the programming language you use, this environment variable can be used inside the application.

In a Java application, you can print the variable as follows:

In a python script, you can print the variable as follows:

Configuring the pipeline steps

Each pipeline consists of three steps:

• Input trigger
  The input trigger step is the first step of every pipeline. It can only exist once per pipeline. The defined query filters the incoming data for processing.

• Parser
  The parser is used to transform the input data. A default parser is available that automatically detects the file type of the input data and transforms the input data accordingly. Multiple parser steps can be added.

• Output
  The output step is the last step of every pipeline. It can only exist once per pipeline.

Each pipeline can be expanded by the following steps:

• Parser
  See above.

• Custom
  In a custom step, you can upload your own code written in Python or Java to interact with and transform the input data.
  In the Examples chapter, you can find information on how to configure the custom step, refer to Pipelines: Configuring the custom step.

  The custom processing logic and its Intellectual Property (IP) belong to the project owner.
  The project owner is liable for damages which are caused by the custom processing logic.
  We advise you to perform security and validity scans (especially when using open source code).
  We also advise you to verify open source license compliance of your custom processing logic.

• Device Position
  In a device position step, the GPS coordinates of a device are extracted and stored in its digital twin representation.

Proceed as follows

1. Open the Configuration tab and configure or update the relevant step.

2. When ready, click the Save button.
   → The configuration is saved.
   → A pipeline version is automatically created and can be tracked in the Change History tab. Refer to Managing pipeline versions.
   

   

   
   

{     "tags": { "$not": { "$elemMatch": { "$eq": "duplicate" } } } ,    "metaData.yourKey": "yourMatchText"}

{     "metaData.yourKey": "yourMatchText"}