pscluster.pl

Overview
Setup
Usage summary
Context
- Inputs
- Outputs
Examples

Overview

The pscluster.pl command allows the OPUS pipeline user to check the process status of OPUS processes, including OPUS servers, for multiple nodes at the same time. The processes can be owned by the current user or by a different user. In addition, this command will allow the current user to terminate his/her own processes on these nodes.

The script will utilize the user's setting of OPUS_REMOTE_SHELL and OPUS_HOME_DIR from the opus_login.csh file.

The script was designed to work on Tru64 (no longer supported), Solaris, and Linux machines.

This script is complicated and usage is often simplified by calling it from a wrap-around script. For instance, HST OPUS users may use psopus.pl and Sample Pipeline users may call kill_oservers.csh.

This script was written as a reliable way to bring OPUS servers down.
It is presumed that OPUS processes were started in a uniform way, eg. with the PMG. Certain patterns are looked for on a machine's CPU based on this presumption.
Unlike psopus.pl, this script does NOT automatically kill the OPUS processes when the user wants to kill OPUS servers. I.e. pscluster.pl will let you kill OPUS servers while OPUS processes are still running and this action will result in servers restarting.
Likewise, this script by itself does NOT reset the OPUS environment, as it relates to the servers. Nor will it warn the user of existing PSTATs (process status files). The wrap-around scripts will reset the user's opus_corba_objs, and empty his/her opus_iors and resource.locks files, if used to bring down servers. psopus.pl will warn of existing PSTATs.
Users may kill the OPUS processes in only one path, if so desired. This is recommended for expert use only.
Likewise, pscluster.pl can also be used to kill all OPUS processes in all paths without taking down the servers. Again, this is for expert use only.

Any processes killed by this tool will be terminated with the UNIX kill command, which is different from how a process is halted via the PMG. Thus any cleanup steps (e.g. during an internal poller's HALT command handler) will not be performed. For this reason, users should only use the -kill option when unable to take processes down with the PMG.

So when it comes right down to it, when the user specifies -kill, he/she should do so with care when using pscluster.pl.

Note:
If the OPUS servers are killed before OPUS processes are finished coming down, OPUS servers may restart automatically. It is suggested that the user always verify the state of the servers with another pscluster.pl -t a before assuming all processes are terminated.

Setup:

One other script, opusgrep.pl, must be available through your PATH for pscluster.pl to work. Both will be available in any OPUS runtime tree.

Usage:

For the most current help and examples, use "pscluster.pl -help".

%pscluster.pl
        [-node node1[,node2,...]]
        [-user user]
        [-type s|p|a]
        [-path path]
        [-kill] 
        [-help] 
        [-verbose]

Context:

For the most current help and examples, use "pscluster.pl" or "pscluster.pl -help".

Inputs

Most parameters can be abbreviated to one letter.
All arguments are optional but providing none gets you this help.
    [-user user] and [-kill] are mutually exclusive.
    [-type s or -type a] and [-path pathname] are mutually exclusive.

-node node1,...
    List of nodes on which to look for processes
    Nodes should be separated by commas: no spaces.
    Default is current node.
-user user
    Name of user for whose processes to search.
    Defaults to current user.
-type s|p|a
    s Implies server pattern opus_[a-z]*_serv.
    p Implies processes running in any path
      unless [-path pathname] is used.
    a Implies processes in all paths and the servers.
    Defaults to servers.
-path pathname
    Look only for processes in specified path. Ext ".path" unnecessary.
    No default.
-linesize N
    Limit the output of unix ps command to N-characters.
    N < 80 is not useful.  More characters give more information.
    Defaults to 115 characters.
-kill
    Indicates that processes found matching pattern should be stopped.
    Only works for processes owned by the current user.
    Note that NO file cleanup occurs with pscluster and -kill option.
    (see instead -kill in "psopus.pl -help" or read kill_oservers.csh)
-help
    Print this usage statement.
-verbose
    Developer use only.  Run script verbosely.
    
[Default: none ]

Outputs

pscluster.pl

The script will print a help summary to the screen if called with -help.

Otherwise, the script will print a banner telling the user the name of the machine on which the processes are running, the output from a Unix ps command about those processes, followed by another banner.

If the -kill option is specified, then the script will also list the PIDs of the processes being killed.

Examples:

pscluster.pl
    List OPUS servers owned by current user on current node
    (Useful for making sure kill worked.)
pscluster.pl -kill  -t a
    Kill all OPUS processes AND OPUS servers owned by current user running
    on current node  (see also -kill in "psopus.pl -help")
    If there are many processes running, it is possible for servers to be killed
    before all the processes have come down.  If so, servers will be 
    restarted, and this command may need to be issued again.
    Note that NO file cleanup occurs with pscluster and -kill option.
pscluster.pl -kill
    Kill all OPUS servers owned by current user running on current node
    If there are OPUS PROCESSES running, this will NOT WORK WELL as the
    processes will restart the servers.
    Note that NO file cleanup occurs with pscluster and -kill option.
pscluster.pl -kill -t p
    Kill all processes running in OPUS paths owned by current user running
    on current node.  (Do not kill servers).
    Note that NO file cleanup occurs with pscluster and -kill option.
pscluster.pl -kill -t p -path trodden
    Kill all processes running in OPUS path named "trodden", owned by
    current user, running on current node.  (Do not kill servers).
    (Do not kill processes in any other OPUS path.)
    Note that NO file cleanup occurs with pscluster and -kill option.
pscluster.pl -u polk -n fido,max,spot
    List OPUS server processes owned by user polk on nodes fido, max,
    and spot
pscluster.pl -n fido,max,spot -type a
    List all OPUS processes owned by current user on nodes fido, max,
    and spot
pscluster.pl -t s -l 80
    Limit width of list of OPUS server information to 80-characters
pscluster.pl -l 80
    Same: Limit width of list of OPUS server information to 80-characters

 
%pscluster.pl -l 80 -t a

--- tufnel (local) -------------------------------------------------------------
  PID %CPU %MEM  VSZ    STIME COMMAND
 3546  0.0  0.1 23344 21:12:52 opus_bb_server -t OSF -i /store/tufnel/dsb/lisa/o
 3390  0.1  0.1 23280 21:11:52 osfdelete -p /home/sherbert/defs/154b/fgs_leaf.pa
 3543  0.0  0.1 23344 21:12:49 opus_bb_server -t OSF -i /store/tufnel/dsb/lisa/o
 3435  0.0  0.1 23408 21:12:06 xpoll -p /home/sherbert/defs/154b/ing_les.path -r
 3380  0.0  0.1 23408 21:11:50 xpoll -p /home/sherbert/defs/154b/cdb_styx.path -
 3422  0.1  0.1 30072 21:11:59 update_db_tool -p /home/sherbert/defs/154b/fus_io
 3566  0.0  0.1 23344 21:12:53 opus_bb_server -t OSF -i /store/tufnel/dsb/lisa/o
 3545  0.0  0.1 23344 21:12:51 opus_bb_server -t OSF -i /store/tufnel/dsb/lisa/o
 3523  0.0  0.1 23440 21:12:46 opus_bb_server -t PSTAT -i /store/tufnel/dsb/lisa
 3525  0.0  0.1 23344 21:12:49 opus_bb_server -t OSF -i /store/tufnel/dsb/lisa/o
 3437  0.1  0.1 23280 21:12:12 osfdelete -p /home/sherbert/defs/154b/sci_fur.pat
-------------------------------------------------------------------------------


%pscluster.pl -kill
   
--- drummer (local) -------------------------------------------------------------
  PID %CPU %MEM  VSZ    STIME COMMAND
No PIDs to kill.
--------------------------------------------------------------------------------
   
   
--- drummer (local) -------------------------------------------------------------
  PID %CPU %MEM  VSZ    STIME COMMAND
19938  0.0  0.1 23216 17:18:42 opus_env_server -ORBStdProfileComponents 0 &
20106  0.0  0.1 23344 17:20:03 opus_bb_server -t OSF -i /drummer/data/user/osfs/path01/ -ORBStdProfileCompone
20064  0.0  0.1 23344 17:19:56 opus_bb_server -t OSF -i /drummer/data/user/osfs/path02/ -ORBStdProfileCompon
20269  0.0  0.1 23344 17:20:23 opus_bb_server -t OSF -i /drummer/data/user/osfs/path03/ -ORBStdProfileCom
19939  0.0  0.1 23608 17:18:44 opus_bb_server -t PSTAT -i /drummer/data/user/opus_home/ -ORBStdProfileComp
20187  0.0  0.1 23344 17:20:11 opus_bb_server -t OSF -i /drummer/data/user/osfs/path04/ -ORBStdProfileCompon
19940  0.0  0.1 23072 17:18:49 opus_event_service -i /drummer/data/user/opus_home/event/ -ORBStdProfileCom
20226  0.0  0.1 23344 17:20:16 opus_bb_server -t OSF -i /drummer/data/user/osfs/path05/ -ORBStdProfileComp
20105  0.0  0.1 23392 17:20:01 opus_bb_server -t FILE -i /Files_bb/ -ORBStdProfileComponents 0 &
   
*** killing 19938 20106 20064 20269 19939 20187 19940 20226 20105
Sending to drummer: /bin/kill 19938 20106 20064 20269 19939 20187 19940 20226 20105
--------------------------------------------------------------------------------

pscluster.pl

Contents

Overview