Remote Hotfolder

Remote Hotfolder is a console application that allows to upload all files of a local directory to a node of a remote CELUM server.

Note that this relies on file system events to detect when files are added. If your file system does not support that, consider using a cronjob with the -o flag, or the Local Hotfolder.

1. Requirements

• api-endpoint
• api-remote
• Java 11

2. Options

--help, -h

argument: -, required: no, default: -

Displays helping information.

--url, -u

argument: String, required: yes default: -

URL of the celum server.

--client-id, -i

argument: String, required: yes default: -

CELUM client id (defined in the remote.yml file in the CELUM config folder).

--client-secret -c

argument: String, required: yes default: -

CELUM client secret (defined in the remote.yml file in the CELUM config folder).

--server-secret -s

argument: String, required: yes default: -

CELUM server secret (defined in the remote.yml file in the CELUM config folder).

--target-node, -t

argument: Number, required: yes default: -

Celum Target node where the files will be uploaded.

--source-folder, -f

argument: Number, required: yes default: -

Source folder on the local system from where the files will be uploaded.

--mail-configuration, -m

argument: Number, required: yes default: -

Path to the properties file with general and mail properties. License (key: remoteHotfolder) is required, the other properties are optional.

license=
mail.host=
mail.port=
mail.from=
mail.to=
mail.username=
mail.password=
logProperties.path=

--locale, -l

argument: Number, required: required: no default: 'en'

Locale for creation of CELUM Nodes, used in Nodefinders.

--node-finder, -n

argument: String, required: no default: -

Name of the NodeFinder, if empty the assets will be uploaded directly to the target node.

Known implementations:

• pluginVersionNodeFinder - expects filenames to follow semantic versioning, e.g. my-plugin-1.0.0, and creates two levels of subfolders, e.g. my-plugin on the first level and 1.0 on the second. Yes this is oddly specific, but this powers our plugin delivery pipeline:

--delete-files, -d

argument: -, required: no default: -

With this argument the files will be deleted after the upload on the local system.

--prevent-duplicates, -p

argument: -, required: no default: -

With this argument, if an asset with the same name as an uploading file exists already, it will be uploaded as a new version of the existing asset instead, creating a second asset. Note that as of 1.1, every matching asset will be updated (instead of just the first one we find).

--global-search, -g

argument: -, required: no default: -, since: 1.1

When using preventDuplicates, search globally(!) instead of just in the target node, and links it to the target node instead. Be careful with this one ;-)

--dry-run, -y

argument: -, required: no default: -, since: 1.1

Don't actually do anything, but log what should have happened.

--one-shot, -o

argument: -, required: no, default: -, since: 1.2

Don't listen for new files, just upload the ones currently in the sourceFolder and then exit. Useful for batch processing.

-- blacklist, -b

argument: Regex, required: no, default: "^[.].*|Thumbs.db$", since: 1.2

Ignore files with certain file names, which by default are dot-files (e.g. .DS_Store) and Thumbs.db (crap that the OS generated)

--execute-as, -e

argument: long, required: no, default: API-User, since: 1.2.1

Specify a different user-ID to do the upload with. Permissions are assumed to be provided.

--relink, -r

argument: -, required: no, default: API-User, since: 1.3.0

When using -g, link assets that were found somewhere else in the system to the target node, rather than uploading a new one.

--wait-delay, -w

argument: int, required: no, default: 1, since: 1.5.5

Wait for n seconds before uploading, and delay the upload when the file size is bigger than before (indicates that the file is still being written).

3. Compatibility Matrix

4. Usage

command line:

java -jar remote-hotfolder.jar -u "https://[my celum url]/remoteapi" -i "[my client id]" -c "[my client secrete]" -s "[my server secret]" -t 26621 -f "[path to my local directory]" -l "en" -d -p

systemd:

/etc/systemd/system/remote-hotfolder.service:

[Unit]
Description=remote-hotfolder
After=syslog.target network.target

[Service]
User=celum
Restart=always
RestartSec=5
WorkingDirectory=/opt/remote-hotfolder
ExecStart=/usr/bin/java -jar /opt/remote-hotfolder/remote-hotfolder.jar -u "http://[my celum url]:8765/api" -i "[my client id]" -c "[my client secrete]" -s "[my server secret]" -t 26621 -f "/home/celum/data/import" -p -d -r

[Install]
WantedBy=multi-user.target

Release Notes

1.0.0

Released 2020-02-18

• Initial version

1.1.0

Released 2020-07-03

• Added -g and -y options, updated api-endpoint to 3.1.1

1.2.0

Released 2020-07-14

• Added -o, -b and -e options
• Long args now use the more standard cebab-case instead of camelCase

1.3.0

Released 2021-02-29

• Check if an added file is still growing in size (i.e. is still in transport) and wait until it doesn't anymore

1.4.0

Released 2021-04-22

• Added license mechanism

Re-Released 2022-12-16 for 6.12.x

1.5.0

Released 2023-04-03

• added file size vs. asset size check (in case transport fails silently), and delete the asset on mismatch