Scheduler Swiftlet

Introduction

The Scheduler Swiftlet is an enterprise job scheduler to run registered jobs at a configurable schedule that can be based on stackable calendars.

Job Groups and Jobs

Jobs are registered from other Swiftlets at the Scheduler Swiftlet. The registration takes place in job groups, one for each Swiftlet. Each registered job consists of metadata that describes the job and its job parameters.

Schedules

A schedule must be created to run a job. This is done by the user/administrator via SwiftMQ Explorer/CLI. After the schedule has been enabled, the Scheduler Swiftlet takes care to create and run job instances on the base of the schedule's time expression. A job is stopped when it reaches the maximum runtime or the job instance marks itself as finished.

A schedule can be based on a calendar. e.g. to run a job on business days only. See next section. The time when a job should start is specified as a time expression. It can be a 'at' or a 'repeat' expression to start the job at fixed time points during a day or in a repeat interval, respectively. If the job requires parameters, they must be specified in the schedule and will be passed to the job instance during the job start. Optionally, a maximum runtime can be specified. If this amount of time is reached, the job instance is stopped by the Scheduler Swiftlet. A schedule has a date range in which it is valid. For example, one can specify a start date 15th February 2003 and end date of 14th April 2003. The defaults for start/end dates are now and forever, respectively.

Calendars

A calendar decides at which day a job will run whereas a schedule's time expression decides at which time. A schedule can be optionally based on a calendar. If not, the job will run every day.

A calendar has multiple parts which can be enabled separately. Parts are weekdays, month days, last day in a month, annual days, date ranges. The type of calendar is either include or exclude. An include calendar includes the calendar dates whereas exclude excludes it.

One or more parts of a calendar can be enabled. It is also possible to stack calendars by using another calendar as a base calendar for a new calendar. For example, one may create a calendar Business Days with Mo-Fr enabled and another calendar Bank Holidays which is based on Business Days to exclude bank holidays from business days. Arbitrary trees of calendars can be built with stacked calendars to create whatever final custom calendar.

Custom Jobs

To create your own jobs with other scheduler products, you have to use their scheduler classes, implement their interfaces, and you become locked into their product. Not so with SwiftMQ! All jobs in SwiftMQ are provided by Swiftlets. They are already there. No need to implement anything. A bunch of functionality is already implemented, for example, a job to move the content of a local queue to a queue on a remote router, a job to perform a backup on the store, or a job to schedule a routing connector, and much more.

However, that's not all. The JMS Application Container Swiftlet provides a generic Application Invokerjob that can invoke any static JMS application (one which runs embedded in SwiftMQ). Thus, any Java application, no matter what it does (it doesn't need to use JMS at all), is a potential job. You only have to configure it in the JMS Application Container Swiftlet as a static application and you can schedule it as a job out of the box. The only pre-condition is to have a custom shutdown method in your application to ensure it can be stopped. This method is discovered by Java reflection, so no vendor lock-in, no proprietary interfaces!

Job Monitoring

The Scheduler Swiftlets provides 2 scheduler tables to view the status of all scheduled jobs. Each job contains a history of the last 20 job events. A schedule can be log-enabled to log each start/stop in the router's log file.

Job Groups and Jobs

All available jobs are provided by Swiftlets. They are registered in job groups, one for each Swiftlet. A job group can have multiple defined jobs. The registration is added dynamically, usually when a Swiftlet starts (e.g. during hot deployment), and removed when the Swiftlet stops (e.g. during undeployment).

Job Groups

All job groups are shown in the folder Registered Job Groups below the Usage folder of the Scheduler Swiftlet:

Jobs

Within each job group, you'll find all jobs a particular Swiftlet provides. They are shown in the folder Registered Jobs below a job group.

Each entry there describes a job; it is metadata to show you what the job does and what parameters it requires. The parameters, if any, are listed under the folder Parameters of the job. A parameter has a name, an optional default value, and is marked as mandatory or optional.

Basically, if you like to schedule a job, you check the registration of the job, note the job group, job name, and parameters, define a schedule and you're done.

Calendars

A calendar decides at which day a job will run whereas a schedule's time expression decides at which time. A schedule can be optionally based on a calendar. Otherwise, the job will run every day.

Parts of a Calendar

A calendar consists of the following parts:

• Week days

• Month days

• Last month day

• Annual days

• Date ranges

Each part must be enabled separately, only enabled parts are respected. The type of a calendar, includeor exclude, decides whether the marked dates in an enabled part are included or excluded. Instead of using multiple parts of a single calendar, it is recommended to stack calendars (described in a further section) which is much more flexible.

Part: Week Days

This part can be used to set the weekdays of a calendar.

The above example enables Monday to Friday. If the calendar is of type include, then Monday to Friday are valid but Saturday and Sunday are invalid. If it is excluded, then Saturday and Sunday are valid but Monday to Friday are invalid.

If the above calendar is saved in the routerconfig.xml, it looks like this:

<calendar name="Business Days" enable-weekdays="true" type="include">
   <weekdays day-02="true" day-03="true" day-04="true" day-05="true" day-06="true"/>
</calendar>

As you see, Monday is day-02, so the first day in a week is always Sunday (day-01). You might consider it if you configure a calendar directly in the routerconfig.xml.

Part: Month Days

This part can be used to set the month days of a calendar.

The above example enables the days 5, 10, and 15 of a month. If the calendar is of type include, only these days are valid and the rest days of the month are invalid. If it is exclude, all days of the month are valid, except the marked days.

Part: Last Month's Days

The last day of a month varies. It might be the 28th of February, the 30th of April, and so on. Some jobs might run on every last day in a month, e.g. end of month processing. This can be specified in a calendar by enabling the part Month Days AND Last Month Day (both!):

The above example enables the last day of a month. If the calendar is of type include, the day is valid and all other days of a month are invalid. If it is exclude, all days of a month are valid, except the last day.

Part: Annual Days

This part of a calendar can be used to specify fixed dates of a year like bank holidays:

The above example defines 2 annual days, the 1st of January (New Year) and the 3rd of October (German Reunion Day). Since it is of type exclude, these days are excluded.

Part: Date Ranges

This part of a calendar can be used to specify date ranges like company holidays and so on:

The above example defines 2 date ranges, Summer Holiday and Winter Holiday. Since it is of type exclude, these date ranges are excluded.

Stacking Calendars

The real power of calendars is that you can build stacks of calendars, that is, you can create a calendar that is based on another calendar which might be based on another calendar, and so on. Since the calendars are only connected by their name, you can build any combination of calendars, re-use existing calendars, and so on. Only the sky is the limit!

Modifying Calendars

All changes you perform on calendars (create a new calendar, modify or delete a calendar) are immediately applied to schedules. That is, all schedules that have a reference to the calendar (directly or indirectly via a stacked calendar) are determined. If a job instance of these schedules is currently running, it is stopped and the schedule will be re-scheduled on the base of the changes.

Therefore, if you don't want that a running job will be stopped due to a change, wait with your changes until your job(s) are finished, then disable the resp. schedule(s) and apply your calendar changes. Thereafter, enable the schedule(s).

Schedules

Jobs are started and stopped on the base of schedules. Before you create a schedule, you should know:

• the name of the calendar to use

• the name of the job group

• the name of the job

• the job parameters

Then select folder Job Schedules to see all active job schedules:

Click on “Create new Entity” to create a new job schedule:

The fields have the following meanings:

Time Expressions

The time expression can be specified in 2 different formats: as an 'at' expression or a 'repeat' expression.

'at' Expression

The 'at' expression specifies fixed time points during the day. The format is:

'at' HH:mm[:ss][, HH:mm[:ss]...]

Example:

at 10:00, 12:30:45, 15:15

'repeat' Expression

The 'repeat' expression specifies a repeat interval. The format is:

'start' HH:mm[:ss] 'stop' HH:mm[:ss] 'delay' <n>('h'|'m'|'s') ['repeat' <m>]

Example:

start 09:10:15 stop 18:00 delay 15s

This starts the job at 09:10:15 and repeats it every 15 seconds till 18:00.

Example:

start 09:10:15 stop 18:00 delay 15s repeat 100

This starts the job at 09:10:15 and repeats it every 15 seconds till 18:00 or until it reaches 100 invocations per day. The "repeat 100" is for convenience. You can also compute the absolute stop time.

System Time Changes during Execution

If the system time is changed during the execution of a schedule (e.g. due to Summer/Winter time change, NTP synchronization), the following applies:

• A schedule with an 'at' time expression is handled as absolute time. For example, if the time expression is "at 17:15" and the current time is changed from 16:00 to 15:00, the schedule will be executed at 17:15 of the new time.

• A schedule with a 'repeat' time expression is handled as a relative time. That is, the delta of a time change is applied to the next execution time of the schedule. For example, if the time expression is start 00:00 stop 23:59 delay 30m and the current time is 17:15 then the next execution will be at 17:30. If the system time is changed to 18:15, the next execution will be at 18:30 and then at 19:00, 19:30, etc. Note that until the current execution is finished, the Usage section of the schedule will display the next execution time on the base of the system time before the change (in this example it will display 17:30).

Message Jobs

We have removed the old-fashioned message job scheduler from this Swiftlet.

Please have a look here:

Message Scheduler

which is much more scalable.

Monitoring

Overview

Active schedules are shown in the Usage folder of the Scheduler Swiftlet. There are 2 subfolders, Active Job Schedules and Active Message Schedules:

Monitoring Job Schedules

Select folder Active Job Schedules:

Monitoring a Schedule's History

Each active schedule has a history with the last 20 state changes. This history is located in the folder History of the resp. entry:

Select the folder and you will see the history as a table:

Schedule States

An active schedule has a state. The following table shows all states a schedule can have:

Registry

This job registry contains all job groups and jobs provided from Standard, Kernel, and Swiftlets.

Job Group: File Cache

Job: File Cache Cleanup

The File Cache Cleanup job removes expired files from file caches. The file caches are determined from a single parameter that contains a SQL-Like predicate. For example, a predicate of order% performs a cleanup on all file caches whose names start with order. The File Cache Cleanup job provides more flexibility than using the interval-based cleanup per file cache. Keep in mind that you should set the cleanup intervals to 0 if you use the File Cache Cleanup job.

Since 9.4.0.

Job: File Copier

The File Copier job copies files from a source cache, which must reside on the router where the job runs, to a target cache, which can be the local or a remote router. Transferred files at the target cache are identified by a new link.

Since 9.4.0.

Job: File Purger

The File Purger job removes messages from a single file cache, optionally based on a message selector.

Since 9.4.0.

Job Group: Queue Manager

Job: Queue Cleanup

The Queue Cleanup job removes expired messages from queues. The queues are determined from a single parameter that contains a SQL-Like predicate. For example, a predicate of tmp$% performs a cleanup on all temporary queues, rt$% on all routing queues, test% on all queues starting with test like testqueue and so on. The Queue Cleanup job provides more flexibility than using the interval-based cleanup processes. Keep in mind that you should set the cleanup intervals (default or per queue) to 0 if you use the Queue Cleanup job.

Job: Queue Cleanup DLQ

This job works like the Queue Cleanup job except that it moves expired messages to the router's dead-letter queue routerdlq for further processing.

Job: Queue Mover

The Queue Mover job moves messages from a source queue to a target queue. The target queue may reside on a remote router. An optional message selector can be specified to select messages on the base of JMS properties out of the source queue. This job is excellent for tasks like the end of day processing to move all unprocessed messages to a queue on another router for further processing.

Job: Queue Purger

The Queue Purger job removes messages from a single queue, optionally based on a message selector.

Job: Multi Queue Purger

The Multi Queue Purger job removes messages from multiple queues, optionally based on a message selector. Use this job carefully!

Job: Queue Reset

The Queue Reset job resets the consumed/produced message counter on the matching queues to zero. This can be used e.g. to reset the counters every day at midnight to get daily counts.

Job Group: Streams

Job: Stream Activator

The Stream Activator job activates a stream on job start and deactivates it on job stop. The name of the stream must be specified as a parameter. The job requires setting a maximum runtime to deactivate the stream. Otherwise, it would be active forever.

The implementation of the Stream Activator job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a stream that should run as a job should be initially disabled.

Job Group: Topic Manager

Job: Delete Durable

The Delete Durable job deletes durable subscribers (incl. all messages) which are selected by 2 SQL-Like predicates, client id and durable name, specified as job parameters. For example, a client id predicate of sales and a durable name predicate of % will delete all durable subscribers of client id sales whereas a client id predicate of s__es% and a durable name predicate of dur% will delete those from saukesta and dur2003. If you set both parameters to %, it will match all durable subscribers and therefore all durable subscribers are deleted.

Job Group: Routing

Job: Routing Connector

This job takes the name of the routing connector as a parameter. It enables the connector at the job start and disables it at the job stop. Therefore, a connector should be configured as being disabled if it should be used in a job:

Example:

<connectors>
    <connector name="acme"
               enabled="false"
               hostname="chb.acme.com"
               port="17884"
               socketfactory-class="com.swiftmq.net.JSSESocketFactory"
               password="let_me_in"
               retry-time="120000"/>
</connectors>

The Routing Connector should have a specified maximum runtime in its schedule, otherwise, it will not be disabled. If the connector will be enabled during the job start and the connection cannot be established, the retry behavior will be executed as described in the section above. Retry ends at the job stop.

Job Group: Store

Job: Backup

This job can be used to schedule a backup at a specific time of the day. It has no parameters.

Job: Shrink

This job can be used to schedule a shrink of the data store at a specific time of the day. It has no parameters.

Job Group: JavaMail

Job: Outbound Bridge

The Outbound Bridge job activates an outbound bridge on job start and deactivates it on job stop. The name of the bridge must be specified as a parameter. The job requires setting a maximum runtime to deactivate the bridge. Otherwise, it would be active forever.

The implementation of the Outbound Bridge job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, an outbound bridge which should run as a job should be initially disabled.

Job: Inbound Bridge

The Inbound Bridge job activates an inbound bridge on job start and deactivates it on job stop. The name of the bridge must be specified as a parameter. The job requires setting a maximum runtime to deactivate the bridge. Otherwise, it would be active forever.

The implementation of the Inbound Bridge job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, an inbound bridge which should run as a job should be initially disabled.

Job Group: JMS Bridge

Job: Server Bridge

The Server Bridge job activates a server-bridge on the job start and deactivates it on the job stop. The name of the bridge must be specified as a parameter. The job requires setting a maximum runtime to deactivate the bridge. Otherwise, it would be active forever.

The implementation of the Server Bridge job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a server bridge that should run as a job should be initially disabled.

Job Group: AMQP Bridge

Job: Bridge

The Bridge job activates an AMQP bridge on the job start and deactivates it on the job stop. The name of the bridge must be specified as a parameter. The job requires setting a maximum runtime to deactivate the bridge. Otherwise, it would be active forever.

The implementation of the Bridge job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a server bridge that should run as a job should be initially disabled.

Job Group: Replicator

Job: Source

The Source job activates the replication source on job start and deactivates it on job stop. The name of the replication source must be specified as a parameter. The job requires setting a maximum runtime to deactivate the source. Otherwise, it would be active forever.

The implementation of the Source job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a replication source that should run as a job should be initially disabled.

Job: Sink

The Sink job activates the replication source on job start and deactivates it on job stop. The name of the replication sink must be specified as a parameter. The job requires setting a maximum runtime to deactivate the source. Otherwise, it would be active forever.

The implementation of the Sink job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a replication sink that should run as a job should be initially disabled.

Configuration

The configuration of the Scheduler Swiftlet is defined within the element

      <swiftlet name="sys$scheduler" .../>

of the router's configuration file.

Element List "calendars", Parent Element: "swiftlet"

Calendars. This element list contains zero or more "calendar" elements with this template definition:

Definition

Values

Element "weekdays", Parent Element: "calendar"

Week Days.

Definition

Values

Element "monthdays", Parent Element: "calendar"

Month Days.

Definition

Values

Element List "annualdays", Parent Element: "calendar"

Annual Days. This element list contains zero or more "annualday" elements with this template definition:

Definition

Values

Element List "date-ranges", Parent Element: "calendar"

Date Ranges. This element list contains zero or more "date-range" elements with this template definition:

Definition

Values

Element List "schedules", Parent Element: "swiftlet"

Job Schedules. This element list contains zero or more "schedule" elements with this template definition:

Definition

Values

Element List "parameters", Parent Element: "schedule"

Parameters. This element list contains zero or more "parameter" elements with this template definition:

Definition

Values

Element "usage", Parent Element: "swiftlet"

Usage.

Element List "job-groups", Parent Element: "usage"

Registered Job Groups. This element list contains zero or more "job-group" elements with this template definition:

Definition

Element List "jobs", Parent Element: "job-group"

Registered Jobs. This element list contains zero or more "job" elements with this template definition:

Definition

Values

Element List "parameters", Parent Element: "job"

Job Parameters. This element list contains zero or more "parameter" elements with this template definition:

Definition

Values

Element List "active-job-schedules", Parent Element: "usage"

Active Job Schedules. This element list contains zero or more "active-schedule" elements with this template definition:

Definition

Values

Element List "history", Parent Element: "active-schedule"

History. This element list contains zero or more "history" elements with this template definition:

Definition

Values

Element List "active-message-schedules", Parent Element: "usage"

Active Message Schedules. This element list contains zero or more "active-message-schedule" elements with this template definition:

Definition

Values

Element List "history", Parent Element: "active-message-schedule"

History. This element list contains zero or more "history" elements with this template definition:

Definition

Values