Configuring Task Manager (or WorkflowMaster)¶
The task manager generates tasks for each stage of the workflow and inserts them in the CometCloud space, which allows different sites query and retrieve the tasks they need. It offers an API for users to programmatically define the way tasks are generated. The Task Manager a service built using CometCloud master/worker programming interface. Hence, it requires deploying CometCloud core components, such as CometCloud space, request handler, and proxy. CometCloud can be deployed across multiple machines, providing scalability for the management, query, and delivery of tasks. For simplicity in this section we deploy all CometCloud services within one single machine. More information about how to configure and deploy CometCloud can be found in the CometCloud website
This service requires several configuration files: master.properties, comet.properties, nodeFile, exceptionFile, RequestHandlerList, portFile, chord.properties, and squid.properties. Next we present configuration where all services are going to be deployed in the machine machine2.domain.com. The Task Manager will be listening for instructions from the Workflow Manager and Autonomic Scheduler in the port defined by workflowMasterPort. More information about this section configuration files can be found in MASTER section. For simplicity we keep scripts and examples of the configuration files in the directory simple_run/master/
This service requires Java runtime 1.7+.
Edit master.properties file¶
This file contains basic information about the Task Manager.
workflowMasterPort=7777 logFile=master.log logFileRH=requesthandler.log
Edit comet.properties file¶
This file contains information required by CometCloud. Only TaskMonitoringPeriod and IsolatedProxy require to be modified. TaskMonitoringPeriod indicates an interval of time in milliseconds that tells CometCloud when to look for missing tasks and reinsert them into the system. This can be disabled by indicating 0 as value. The IsolatedProxy indicates the machine that has the CometCloud proxy and hence enables access to query and retrieve tasks from the CometCloud Space. This will be used by the Agents. By default the port used is 12342. This can be modified as described in the Customizing CometCloud ports Section.
MasterClass=tassl.application.cometcloud.WorkflowMaster WorkerClass=tassl.application.cometcloud.AppWorker TaskClass=tassl.application.cometcloud.WorkflowTaskTuple RoutingKeys=TaskId TaskMonitoringPeriod=50000 IsolatedProxy=machine2.domain.com
Remember to send the comet.properties file to the Agents every time it is modified.
Edit nodeFie file¶
This file tells CometCloud where to start the services. The number 2 tells CometCloud that it has to start two peers (i.e. Request Handler and a Master).
Edit portFie file¶
This file is used by CometCloud to give ports to the different peers. It has to have as many ports as peers in CometCloud. In this case we only need two.
Edit exceptionFile file¶
This file is to set the roles of each CometCloud peer and their ports.
machine2.domain.com:5555 comet.NodeType=MASTER machine2.domain.com:5556 comet.NodeType=REQUEST_HANDLER
Edit RequestHandlerList file¶
Files chord.properties and squid.properties should not be modified.
Customizing Additional CometCloud Ports (Optional)¶
CometCloud makes use of additional ports to manage its different services. This step is not mandatory, but it provides information about port usage that can be useful.
Ports defined in the comet.properties file.
- IsolatedProxyPort. This port is used by Agents to contact CometCloud proxy and request tasks. Its default value is 12342.
- REQUEST_HANDLER_PORT. This port is used by the CometCloud proxy to contact a request handler and communicate Agents’ task requests. Its default value is 12343.
- TCP_CONTROL_PORT. This port is used by Agents to communicate completed tasks and their result to the Task Manager. Its default value is 12340.
- COMET_CONTROL_PORT. This port is used by CometCloud to control its services. Its default value is 12330.
Overlay Ports need to be changed in the starting command of the Proxy and Task Manager, startProxy.sh (simply indicate port at the end) and startWorkflowMaster.sh (-overlayPort option), respectively.
- Overlay Server Port. This port is used by CometCloud to start the CometCloud space and assign roles to different peers. Its default value is 4444.
Starting the Service¶
Once the configuration is ready, you can start the Task Manager and CometCloud by executing the script startAll.sh. This script contains the following code.
#!/bin/bash ./startOverlayServer.sh & ./startProxy.sh & ./startWorkflowMaster.sh
./startOverlayServer.sh starts the CometCloud overlay, which is required to create the CometCloud space. This script contains the following code.
export CLASSPATH=../../dist/*:../../lib/* java -Xms256m -Xmx1024m -cp $CLASSPATH tassl.automate.overlay.OverlayControlServer 4444
./startProxy.sh starts the CometCloud proxy, which enables access to query and retrieve tasks from the CometCloud Space. This script contains the following code.
export CLASSPATH=../../dist/*:../../lib/* java -Xms256m -Xmx1024m -cp $CLASSPATH tassl.application.node.isolate.RequestHandlerProxyWfl -propertyFile comet.properties
./startWorkflowMaster.sh starts the Task Manager as a CometCloud application.
export CLASSPATH=../../dist/*:../../lib/* java -Xms256m -Xmx1024m -cp $CLASSPATH tassl.application.cometcloud.WorkflowStarter -nodeFile nodeFile -portFile portFile -exceptionFile exceptionFile -propertyFile chord.properties -propertyFile squid.properties -propertyFile comet.properties -overlayPort 4444 -propertyFile master.properties
These services can start regardless the status of other services of our federation.