Replicator Extension Swiftlet

The Replicator Swiftlet supports the replication of files and directories via a SwiftMQ router network from arbitrary definable sources to arbitrary definable sinks.

A replication source supervises a source directory and uses SwiftMQ's Filetransfer API to replicate the content and subsequent changes into an associated file cache on the local or a remote router. Any number of replication sinks can be defined anywhere within the SwiftMQ Universal Router Network. They synchronized the file cache with a local directory and get subsequent updates from the replication source via a topic. The result is an exact and up-to-date copy of the replication source, fully automatic.

The Replicator Swiftlet is well-suited for the replication of configuration files or entire websites, the distribution of software, the centralized storage of log files, and much more.

Replication Sources

Create a File Cache

A file cache needs to be created on the local or any remote router. This file cache will be associated with a replication source that replicates its files into it.

Assume we want to replicate log files, so we create a new file cache at the local router and name it logs:

Because the replication source will not use expiration, it can be disabled at the file cache declaration by setting it to -1.

Create a Replication Source

To create a replication source, go to node Replication Sources:

Click Create a New Entity:

Field description:

The replication source starts replication into the associated file cache if it is enabled.

Replication Sinks

To create a replication sink, go to node Replication Sinks :

Click Create a New Entity:

Field description:

The replication sink starts replication from the associated file cache into the destination directory if it is enabled. It starts with synchronizing with the file cache and subsequently receives update events from the replication source on the update topic.

Event Generation

A replication sink can be configured to generate appropriate events and send it as a JMS TextMessage containing the event as an XML document to configured queues and/or topics. This is useful in situations where a file should be processed once it has arrived, for example. In that case, a simple JMS client would listen to a particular queue (or topic) and would perform particular actions (e.g. process an arrived file) on the events it receives.

Event generation is configured per replication sink.

Select Event Sinks:

Select Create a new Entity and enter the destination, persistence mode, and where you can select which type of events you like to send to that event sink:

An event is sent as a JMS TextMessage. The text is an XML document like the following example:

Example:

<?xml version="1.0" encoding="UTF-8"?>
<replication-event
  event-type="deleted"
  sink="customer-orders"
  sink-path="/usr/local/orders"
  path="/"
  filename="PO-63446.pdf"
  modification-time="1040245494822"/>

Field description:

Note that events are generated after the associated operation is completed. For example, an event created for a file is generated after the file has been completely received by the replication sink.

Job Scheduling

The Replicator Swiftlet registers jobs in the job group Replicator at the Scheduler Swiftlet:

These jobs can be scheduled via the Scheduler Swiftlet to run at specific times or in intervals, based on calendars and so on.

Source

The Source job activates the replication source on job start and deactivates it on job stop. The name of the replication source must be specified as a parameter. The job requires setting a maximum runtime to deactivate the source. Otherwise, it would be active forever.

The implementation of the Source job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a replication source that should run as a job should be initially disabled.

Sink

The Sink job activates the replication source on job start and deactivates it on job stop. The name of the replication sink must be specified as a parameter. The job requires setting a maximum runtime to deactivate the source. Otherwise, it would be active forever.

The implementation of the Sink job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a replication sink that should run as a job should be initially disabled.

Installation

The installation takes place via 'hot deployment'. See the documentation of the Deploy Swiftlet for further information.

Configuration

The configuration of the Replicator Extension Swiftlet is defined within the element

      <swiftlet name="xt$replicator" .../>

of the router's configuration file.

Element List "sources", Parent Element: "swiftlet"

Replication Source Definitions. This element list contains zero or more "source" elements with this template definition:

Definition

Values

Element List "sinks", Parent Element: "swiftlet"

Replication Sink Definitions. This element list contains zero or more "sink" elements with this template definition:

Definition

Values

Element List "event-sinks", Parent Element: "sink"

Event Sinks. This element list contains zero or more "event-sink" elements with this template definition:

Definition

Values

CLI Commands During Deployment

This Swiftlet performs the following CLI commands during deployment:

cc /sys$threadpool/pools
new replicator
cc /sys$threadpool/pools/replicator/threads
new extension.xt$replicator.%

CLI Commands During Undeployment

This Swiftlet performs the following CLI commands during undeployment:

cc /sys$threadpool/pools
delete replicator