Kibana Connect

Kibana Connect

Properties

To be configured in {home}/appserver/conf/custom.properties

kibanaConnector.license

type: String, required: yes, default: -

The license key for the plugin (product: kibanaConnector), provided by brix.

kibanaConnector.elasticUrl

type: URL, required: maybe, default: localhost:9200

Where and on which port your elastic search instance resides - required if not running on the appserver itself.

kibanaConnector.threadPoolSize

type: int, required: no, default: 10

Number of threads used for indexing. A higher value can make indexing faster, but it also requires more resources.

kibanaConnector.pageSize

type: int, required: no, default: 200

This setting determines how many assets are processed simultaneously during indexing.

kibanaConnector.lastIdReset

type: boolean , required: no, default: false

This property can be set to true (preferably only temporarily via the configuration manager), to reset the Event-Id, from which the data will be sent to Kibana.

kibanaConnector.maxCacheSize

type: int, required: no, default: 50000

Number of Assets that are stored in the cache while indexing. A higher value can make indexing faster, but it also requires more resources.

kibanaConnector.eventCronExpression

type: Quartz-expression, required: no, default: 0 */5 * * * ?

How often to send new events over to Kibana

kibanaConnector.workflowCronExpression

type: Quartz-expression, required: no, default: 0 0 0 * * ?

How often to send new workflows to Kibana

Installation

Install Elasticsearch

On CentOS and older Red Hat based distributions:

1. Download and install the public signing key:

   rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

2. Create a file called elasticsearch.repo in the /etc/yum.repos.d/ containing:

   [elasticsearch-7.x]
   name=Elasticsearch repository for 7.x packages
   baseurl=https://artifacts.elastic.co/packages/7.x/yum
   gpgcheck=1
   gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
   enabled=1
   autorefresh=1
   type=rpm-md

3. You can now install Elasticsearch with the following command:

   yum install elasticsearch

4. Start Elastisearch with:

     service elasticsearch start

Install Kibana

On CentOS and older Red Hat based distributions:

1. Download and install the public signing key:

   rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

2. Create a file called kibana.repo in the /etc/yum.repos.d/ containing:

   [kibana-7.x]
   name=Kibana repository for 7.x packages
   baseurl=https://artifacts.elastic.co/packages/7.x/yum
   gpgcheck=1
   gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
   enabled=1
   autorefresh=1
   type=rpm-md

3. You can now install Kibana with the following command:

   yum install kibana

4. Start Kibana with:

   service kibana start

Set up Celum Connector

1. Place the kibanaConnector-x.jar in the celum lib folder and configure the necessary properties.
2. After an appserver restart run all the tasks in the kibana task-tab. (Make sure Elasticsearch and Kibana are running.)

Note: The Event-Tasks tracks its own progress. It can be paused or stopped and will continue where it has been stopped. If there is the need to reindex all the events, the property kibanaConnect.lastIdReset can be set to 'true' and the indexing will start from the first Event. Remember to set the property back to false the first run.

Set up Kibana

Once the first index-run in CELUM has started, go to Stack Management > Kibana > Index Patterns. Create a new index pattern (e.g. event) and select date as the time field. Leave the rest as-is.

To see if you really have any events, go to Kibana > Discover and select a reasonably large time range in the upper right. Hit refresh. If you see some events, you're now ready to set up Kibana > Visualizations and Kibana > Dashboards to your heart's content.

Kibana Index

Asset-Index

Asset
│   
└───Active From (date)
└───Active To (date)
└───Active Type (string)
└───Asset Id (number)
└───Asset Type Id (number)
└───Asset Type Name (string)
└───Created By
│   └───Date (date)
│   └───User Id (number)
└───Direct Parents (number)
└───File Category (string)
└───File Extension(string)
└───Infofield Entries
│   └───Informationfield Id (number)
│   └───Value (string)
│   └───Label (string)
└───Last Modified By
│   └───Date (date)
│   └───User Id (number)
└───Name (string)
└───Original File Name (string)
└───Original File Size (number)
└───Parents (number)
└───Status (string)
└───Versioned By
    └───Date (date)
    └───User Id (number)

Node-Index

Node
│   
└───Has Children (boolean)
└───Name (string)
└───Node Id (number)
└───Node Type Id (number)
└───Node Type Name (string)
└───Parent Id (number)
└───Parent Ids (number)
└───Validation Level Inherited (boolean)

User-Index

User
│   
└───Company (string)
└───Created (date)
└───Deactivated (boolean)
└───First Name (string)
└───Last Name (string)
└───Middle Name (string)
└───Must Change Password (boolean)
└───System (boolean)
└───User Id (number)
└───User Kind (string)

Event-Index

Event
│   
└───Asset Id (number)
└───Asset Type Id (number)
└───Asset
└───Date (date)
└───Event Id (number)
└───Event Type (string)
└───Event Type Id (number)
└───Node Id (number)
└───Node Type Id (number)
└───Node
└───User Id (number)
└───User

Workflow-Index

Workflow
│   
└───processDefinitionId (string)
└───processDefinitionKey (string)
└───processDefinitionName_de (string)
└───processDefinitionName_en (string)
└───processDefinitionName_fr (string)
└───processInstanceBusinessKey (string)
└───processInstanceCreationDate (string)
└───processInstanceId (string)
└───processInstanceName (string)
└───processInstanceRequesterId (string)
└───processInstanceRequesterName (string)
└───processInstanceRequesterUsername (string)
└───processInstanceRoleUserIds_manager (string)
└───processInstanceStatus (number)
└───processInstanceStatusName_de (string)
└───processInstanceStatusName_en (string)
└───processInstanceStatusName_fr (string)
└───processVariable_boolean_isRevised (boolean)
└───processVariable_boolean_receiveNotificationSent (boolean)
└───processVariable_longCollection_assetIds (number)
└───processVariable_longCollection_uploadNodeId (number)
└───processVariable_long_nrOfAccepted (number)
└───processVariable_long_nrOfCompleted (number)
└───processVariable_long_nrOfRejected (number)
└───processVariable_stringCollection_initialApprovalCandidates (string)
└───processVariable_string_activeTaskId (string)
└───processVariable_string_approval (string)
└───processVariable_string_approvalComment (string)
└───processVariable_string_artwork (string)
└───processVariable_string_channel (string)
└───processVariable_string_commentHistory (string)
└───processVariable_string_comments (string)
└───processVariable_string_finalAdvertisementFormat (string)
└───processVariable_string_globalMarketingGroup (string)
└───processVariable_string_medium (string)
└───processVariable_string_newComment (string)
└───processVariable_string_requesterEmail (string)
└───processVariable_string_requesterFunction (string)
└───processVariable_string_requesterGivenName (string)
└───processVariable_string_requesterPin (string)
└───processVariable_string_requesterSubsidiary (string)
└───processVariable_string_requesterSurname (string)
└───processVariable_string_reviewerName (string)
└───processVariable_string_summaryTaskAssignee (string)
└───taskStageIds (string)
└───taskStageNames_de (string)
└───taskStageNames_en (string)
└───taskStageNames_fr (string)

Compatibility Matrix

Release Notes

1.0

Released 2019-10-09

Initial version

1.1

Released 2019-12-17

Added workflow indexing

1.2

Released 2020-03-18

Code optimizations

1.3

Released 2021-02-22

Optimized indexing behavior, deliver date in millisecond accuracy