The Replicator Swiftlet supports the replication of files and directories via a SwiftMQ router network from arbitrary definable sources to arbitrary definable sinks.

A replication source supervises a source directory and uses SwiftMQ's Filetransfer API to replicate the content and subsequent changes into an associated file cache on the local or a remote router. Any number of replication sinks can be defined anywhere within the SwiftMQ Universal Router Network. They synchronized the file cache with a local directory and get subsequent updates from the replication source via a topic. The result is an exact and up-to-date copy of the replication source, fully automatic.

The Replicator Swiftlet is well-suited for the replication of configuration files or entire websites, the distribution of software, the centralized storage of log files, and much more.

Replication Sources

Create a File Cache

A file cache needs to be created on the local or any remote router. This file cache will be associated with a replication source that replicates its files into it.

Assume we want to replicate log files, so we create a new file cache at the local router and name it logs:

Because the replication source will not use expiration, it can be disabled at the file cache declaration by setting it to -1.

Create a Replication Source

To create a replication source, go to node Replication Sources:

Click Create a New Entity:

Field description:

Attribute

Meaning

Name

Some unique name of your choice.

Check Interval

The interval in which the directory is checked for changes.

Control File

A registry file where the replication source stores links to transferred files. This should be stored outside of the source directory.

Directory

The directory which shall be replicated.

Enabled

Activates/deactivates the replication source.

File Cache Name

The name of the associated file cache.

File Cache Router Name

The router name of the associated file cache.

Recursive

Enables/disables the replication of subdirectories.

Update Topic

The name of the topic where updates events published to.

The replication source starts replication into the associated file cache if it is enabled.

Replication Sinks

To create a replication sink, go to node Replication Sinks :

Click Create a New Entity:

Field description:

Attribute

Meaning

Name

Some unique name of your choice.

Check Interval

The interval in which the topic is checked for changes or a retry of a cache sync takes place if the previous sync failed or if there was any other error.

Control File

A registry file where the replication sink stores links to transferred files. This should be stored outside of the sink directory.

Directory

The destination directory.

Enabled

Activates/deactivates the replication sink.

File Cache Name

The name of the associated file cache.

File Cache Router Name

The router name of the associated file cache.

Update Topic

The name of the topic where updates events are received from.

The replication sink starts replication from the associated file cache into the destination directory if it is enabled. It starts with synchronizing with the file cache and subsequently receives update events from the replication source on the update topic.

Event Generation

A replication sink can be configured to generate appropriate events and send it as a JMS TextMessage containing the event as an XML document to configured queues and/or topics. This is useful in situations where a file should be processed once it has arrived, for example. In that case, a simple JMS client would listen to a particular queue (or topic) and would perform particular actions (e.g. process an arrived file) on the events it receives.

Event generation is configured per replication sink.

Select Event Sinks:

Select Create a new Entity and enter the destination, persistence mode, and where you can select which type of events you like to send to that event sink:

An event is sent as a JMS TextMessage. The text is an XML document like the following example:

Example:

<?xml version="1.0" encoding="UTF-8"?>
<replication-event
  event-type="deleted"
  sink="customer-orders"
  sink-path="/usr/local/orders"
  path="/"
  filename="PO-63446.pdf"
  modification-time="1040245494822"/>
XML

Field description:

Attribute

Meaning

event-type

The event type. Possible values are "created" or "deleted".

sink

The name of the replication sink.

sink-path

The path of the sink as specified in the replication sink configuration.

path

The path of the file/directory that caused the event. The path is relative to the "sink-path". A value of "/" is equal to the "sink-path" itself. The path seperator character is always "/".

filename

The name of the file/directory that caused the event.

modification-time

The modification time of the file at the replication source. This attribute is just informational.

Note that events are generated after the associated operation is completed. For example, an event created for a file is generated after the file has been completely received by the replication sink.

Job Scheduling

The Replicator Swiftlet registers jobs in the job group Replicator at the Scheduler Swiftlet:

These jobs can be scheduled via the Scheduler Swiftlet to run at specific times or in intervals, based on calendars and so on.

Source

The Source job activates the replication source on job start and deactivates it on job stop. The name of the replication source must be specified as a parameter. The job requires setting a maximum runtime to deactivate the source. Otherwise, it would be active forever.

The implementation of the Source job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a replication source that should run as a job should be initially disabled.

Parameter

Mandatory

Description

Source Name

Yes

Name of the replication source.

Sink

The Sink job activates the replication source on job start and deactivates it on job stop. The name of the replication sink must be specified as a parameter. The job requires setting a maximum runtime to deactivate the source. Otherwise, it would be active forever.

The implementation of the Sink job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a replication sink that should run as a job should be initially disabled.

Parameter

Mandatory

Description

Sink Name

Yes

Name of the replication sink.

Installation

The installation takes place via 'hot deployment'. See the documentation of the Deploy Swiftlet for further information.

Configuration

The configuration of the Replicator Extension Swiftlet is defined within the element

      <swiftlet name="xt$replicator" .../>
XML

of the router's configuration file.

Element List "sources", Parent Element: "swiftlet"

Replication Source Definitions. This element list contains zero or more "source" elements with this template definition:

Definition

Attribute

Type

Mandatory

Description

name

java.lang.String

Yes

Name of this Replication Source

filecache-routername

java.lang.String

Yes

File Cache Router Name

filecache-name

java.lang.String

Yes

File Cache Name

ivm-username

java.lang.String

No

IntraVM Connection User Name

ivm-password

java.lang.String

No

IntraVM Connection Password

enabled

java.lang.Boolean

No

Enabled

controlfile

java.lang.String

Yes

Control File

directory

java.lang.String

Yes

Directory

recursive

java.lang.Boolean

No

Recursive

check-interval

java.lang.Long

Yes

Check Interval (ms)

update-topic

java.lang.String

Yes

Update Topic

Values

Attribute

Values

filecache-routername

filecache-name

ivm-username

ivm-password

enabled

Default: false

controlfile

directory

recursive

Default: true

check-interval

Min: 1000
Default: 60000

update-topic

Element List "sinks", Parent Element: "swiftlet"

Replication Sink Definitions. This element list contains zero or more "sink" elements with this template definition:

Definition

Attribute

Type

Mandatory

Description

name

java.lang.String

Yes

Name of this Replication Sink

filecache-routername

java.lang.String

Yes

File Cache Router Name

filecache-name

java.lang.String

Yes

File Cache Name

ivm-username

java.lang.String

No

IntraVM Connection User Name

ivm-password

java.lang.String

No

IntraVM Connection Password

enabled

java.lang.Boolean

No

Enabled

controlfile

java.lang.String

Yes

Control File

directory

java.lang.String

Yes

Directory

check-interval

java.lang.Long

Yes

Check Interval (ms)

update-topic

java.lang.String

Yes

Update Topic

Values

Attribute

Values

filecache-routername

filecache-name

ivm-username

ivm-password

enabled

Default: false

controlfile

directory

check-interval

Min: 1000
Default: 60000

update-topic

Element List "event-sinks", Parent Element: "sink"

Event Sinks. This element list contains zero or more "event-sink" elements with this template definition:

Definition

Attribute

Type

Mandatory

Description

name

java.lang.String

Yes

Name of this Event Sink

destination-name

java.lang.String

Yes

Name of the Queue/Topic

destination-type

java.lang.String

Yes

Destination Type

delivery-mode

java.lang.String

Yes

Delivery Mode

event-file-created

java.lang.Boolean

No

Event File Created

event-file-deleted

java.lang.Boolean

No

Event File Deleted

Values

Attribute

Values

destination-name

destination-type

Choice: queue topic
Default: queue

delivery-mode

Choice: non_persistent persistent
Default: non_persistent

event-file-created

Default: true

event-file-deleted

Default: true

CLI Commands During Deployment

This Swiftlet performs the following CLI commands during deployment:

cc /sys$threadpool/pools
new replicator
cc /sys$threadpool/pools/replicator/threads
new extension.xt$replicator.%
CODE

CLI Commands During Undeployment

This Swiftlet performs the following CLI commands during undeployment:

cc /sys$threadpool/pools
delete replicator
CODE