![[OPUS]](gif/opus.gif)
From the user's perspective the most significant change is what is NOT being delivered with OPUS 4.2. The Motif managers are no longer supported on any platform. User's are expected to use the Java managers which will be fully maintained in the future.
The changes described in the detail section below should be transparent to you. However, there is one change which could affect your existing resource files and may require action on your part.
Embedded in the description for 44741: (Access to OPUS resource and path files needed within Java applications) is a discussion of the COMMAND line in resource files which use xpoll.
Changes were made to how xpoll transmits information to the command it spawns to process a blackboard event. Earlier versions of xpoll sent any password collected from the user at startup on the command-line of the external process it spawned.Consequently, if your script requires access to any of the xpoll command line arguments, you need to include the '^CL' on the COMMAND resource line in addition to any arguments which are specific to your procedure. Note that this change affects any user-supplied command line argument, which should not match any of the OPUS switches (i.e., -p, -r, -u, -i and -v). The first two arguments are the standard PATH and RESOURCE file names, while the second two are now used to supply the CORBA interface arguments.Although this functionality remains, the user must explicitly indicate that xpoll should pass information via the command-line by placing the token ^CL at the position in the value of COMMAND where *all* of xpoll's command-line arguments will be substituted.
This change gives the external process access to not only passwords, but also to the object reference needed to establish a CORBA connection to the OAPI. By forcing users to explicitly request that command-line arguments be passed to the spawned process, conflicts between OAPI command-line flags and external process command-line flags are minimized.
For example:
TASK = <xpoll -p $PATH_FILE -r myscript > COMMAND = 'myscript.sh -t $MYPARAM ^CL'It is still the responsibility of the user task (myscript) to parse its command line for the -t argument's value which may not appear in the same argument location as it has in the input. In addition you will now have access to the xpoll command line arguments including the CORBA interface arguments.
Node Process CLS Path CMD Started Status
The java PMG columns are
pid process proc_stat start_time path node proc_cmd
These are also the columns that can be used to sort the java process list.
The difference between the java and Motif lists is
The Motif PMG Options menu allows you to Sort Ascending or Descending by the following:
So, while the java PMG currently allows one to sort by PID which the Motif PMG did not allow, one has lost the ability to sort by Class. Sorting by class is useful because it allows grouping of common PSTATs. For example, we often run opus pipelines for all the instruments at the same time. Sorting by class allows us to group by class STI to see if all the expected STIS processes are running. Sorting by class ALL tends to put the archive processes together. I would like to regain that ability.
This delivery adds a new field to the PMG: the class of the task. The class is, of course, determined from the CLASS keyword in the process resource file.
This enhancement allows you to sort the display by clicking on the new class column.
These changes are made possible by adding a new return status to the SYS_spawn* functions (SYS_EXEC_FAIL), which can be used by other applications for more specific error handling.
The Java OMG now can connect to an OSF blackboard server when the location of the OSF's for that path, as specified by the value of OPUS_OBSERVATIONS_DIR, is in the form of a stretch (e.g., MY_OSF_DIRS:/blue_path).
Running the managers on odocluster1 (tru64), I have noticed that left clicking on a status, the drop down menu will be blocked by a scroll bar for some, but not all, columns in an OPUS science pipeline. For instance, in a path for an opus science pipeline (eg. ACS, MARA, OTFR_SCI)
The dialog box which drops down to allow the user to change the status flag will no longer have a scroll bar. It was the scroll bar which occupied too much geography and obscured the visibility of the optional status choices.
ENV. no longer will be prepended to user process resource file CLASS keywords for external pollers. The bug did not prevent the process from running, but it did prevent the PMG from seeing the CLASS value for the process.
SYS_void_get_file_extension has been changed to handle periods in directory names. The workaround is to not use directory names containing periods.
This enhancement will allow the user to copy any one of the fields of a selected OSF or PSTAT to the clipboard. That field can then be used in other funtions, eg: to search the Log file with a paste command (Ctrl-V), or to paste the field into a report.
Like the OAPI Msg class, JMsg is a singleton (only one instance of the object is ever instantiated) and is thread-safe to the extent that messages from different threads are not interleaved and are labeled by thread id (in parentheses at the end of the message).
Refer to the OAPI Msg class for additional functionality details.
A simple wrapper class that can handle our current authentication strategy (i.e., look up a user's database password for a given server in a known flat file) and one that can accommodate changes to the authentication strategy in the future (e.g., kerberos) should be developed.
A simple factory interface and a Sybase-specific implementation are provided that encapsulate the details of which DriverManager is used and how the connection is authenticated. Additional classes/functionality can be added as needed (e.g., additional drivers, authentication schemes, connection properties) without greatly impacting existing users of the class.
Design Summary:
class Resource_bb is derived from Blackboard and is an OAPI Blackboard abstraction of a resource file; each key/value pair in the resource file is an entry on the "blackboard" and can be locked individually (via class Opus_resource_lock). The Entry objects on this blackboard, Resource_entry, have two Field objects: Resource_key and Resource_value, whose meanings are obvious.
class Ior_bb is derived from Resource_bb and adds some specialized member functions for dealing with CORBA objects in the specific resource file OPUS_HOME_DIR:opus_iors.
class Opus_resource_lock is derived from Opus_lock and serves as the lock object for Resource_bb (and Ior_bb). It implements resource file key/value locks via a lock database file (OPUS_HOME_DIR:resource.locks).
The Thread_pool class implements a pool of threads available for processing some task. It takes care of the repetitive thread start-up and shutdown code that otherwise would need to appear in several modules. Some thread pools are used in situations where the units of work performed by the threads need to be completed in a specific order. This sort of coordination is not built directly into Posix threads, so the Thread_sequencer class performs this duty.
There are server-side and client-side pieces to the push event model used by the OPUS CORBA servers to publish blackboard changes. Note that in the push model, the code doing the pushing (the OPUS servers) is the CORBA client whereas the push consumers (Java Managers) are the CORBA servers. Thus, you will notice that the OPUS CORBA server code uses the client-side pieces of the model and the Java Managers use the server-side pieces.
Entry_event_mgr is an abstract class for event publishing used by Blackboard objects when a change to their content occurs. The only concrete implementation of Entry_event_mgr (so far) is COSEC_event_mgr, which uses a CoS Event Channel to push events to consumers. On the consumer-side (push server), the native library used by the Java OMG and PMG instantiates one JOpus_bb_connection object per event channel connection. This object maintains a CORBA servant to incarnate the CORBA push consumer object (Entry_event_consumer_servant) and a queue in which to store events pushed by the event channel (Event_container).
New CORBA IDL interfaces (OpusUser.idl) are defined to permit OPUS pipeline applications to interface with the OAPI through CORBA objects, and existing interface code has been reworked to better fit into an OAPI-like hierarchy. Each internal polling application will be passed the IOR for the user context object (UserContext) on the command-line; external pollers also can choose to have XPOLL pass this information to the COMMAND command-line (along with the rest of XPOLL's command-line) to access UserContext methods as necessary. The servant implementations of these interfaces appear in the various _impl modules (historical naming of opus_corba_impl is kept despite the naming irregularity).
A host of supporting functions/classes perform CORBA-to-OAPI-to-CORBA object translations or services (BB_Helper, CORBA_obj_helper) or are OAPI/CORBA adapter classes (CORBA_pstat_bb, CORBA_osf_bb, CORBA_entry_lock). Several classes perform specific tasks for either CORBA clients or servers: File_bb_event_poller is used by the CORBA blackboard servant in FILE mode to monitor changes on an OAPI file-based blackboard and generate events as necessary for the managers. Proc_restrictions performs process startup restrictions checking for the CORBA blackboard servant when the managers request that a process be started. JOpus_pkg.cpp contains all of the native method implementations for the Java Manager classes that deal with setting up and managing the CoS EventChannel connections.
Finally, several enhancements have been made to existing OAPI code to support these new features. As mentioned above, XPOLL will pass its command-line arguments to the COMMAND it spawns in response to a trigger, but this feature must be requested through a new token in the process resource file. A feature of both the OAPI Pstat and Osf classes has been extracted and isolated into a mixin class so that it may better be integrated into the File_entry class. This makes it possible to support file polling through a CORBA object with minimal additional support necessary. A new command-line utility (pstat_delete) is delivered that allows one to remove PSTAT's from the PSTAT blackboard, which must not be done using file system commands when running in CORBA mode.
A mechanism for interfacing with a portion of the OAPI strictly through a CORBA interface is delivered. This enhancement allows internal pollers to be developed in languages other than C++ (any language with a CORBA binding) and it allows external pollers to access key OAPI features (e.g., access to process resource and path files).
Specific examples of how to interface with the OAPI through the new CORBA interfaces are provided in the how-to document accessible here. Other significant changes to OAPI functionality include:
pstat_delete -p path -i id -n node [status]
where path, id, and node are the corresponding values in the PSTAT to be deleted, and status is the optional status of the PSTAT. If no status value is specified, pstat_delete only will remove PSTAT's in the absent or starting state.
Manager classes that are not meant to be private should be declared public. Examples pertinent to plug-ins include MgrMsg and TextDisplayer.
The workaround is to place all jar files in the CLASSPATH, then execute the managers from the command-line (i.e., not using the Install Anywhere executables).
impact:
Found in testing, an incorrectly structured resource file was hanging the java managers during startup. Overriding the bad resource file was useless as the servers would still read the bad version anyway and hang the manager. We had to wait for a redelivery of the bad resource files before the java managers were useable again.
The managers correctly use only the 'top most' version of the resource files, so there should theoretically be no impact to operations.
Due to the fix for OPR.45187, this is no longer a problem. The search dialog box does not stay active after you tell it what you are searching for. There is now a 'Next' button to access the next instance of the search string.
We now allow the user to specify the fonts in the userOMG.ini and userPMG.ini files. By default these are set at:
font=Serif fontstyle=PLAIN fontsize=12
The user can substitute any valid font, fontstyles of BOLD or ITALIC, and any fontsize.
This release fixes the highlighting of the selected text in the text window which displays the Log (or any other file).
This release implements a Warning Dialog Message when the user searches for a string which does not exist in the Log file. It will also pop up that dialog when the Next button returns no further instances.
The list of libraries that a given OPUS executable depends on should be determined for each executable and changes made to the make files to move library dependency specification down to the executable-specific make file.
Iconification and Deiconification of the OPUS JavaMgrs should now have much less an impact on the system. Events between iconification and deiconification are thoroughly ignored since we effectively disconnect the Java Manager from the Blackboard. When the user wishes to deiconify the manager, a connection to the Blackboard is re-established, and the information from the blackboard is reloaded.
The MgrMsg class now calls System.setErr and System.setOut to redirect output to the log file.
The error message printed by OPUS when an OSF poller is run in an incorrect or incomplete path, with respect to missing stage definitions, has been clarified.
This leads me to believe that there is a timing issue with the java manager as in the next attempt to monitor the path with a new java manager works just fine (the path server is already up so the java manager doesn't have to wait for the server to startup and collect all of the pre-existing osfs). It should be noted that the servers are running in CORBA mode.
This problem is repaired under 44741:
"File locked. Giving up."
This error, from the OPUS blackboard, results in osf_test exiting with an error exit status code (=1). Several pipeline steps have demonstrated failures, at least in OTFR, when this condition occurs.
Other scripts that use osf_test should be examined to make sure they are properly handling this condition.
The manual work-around for the problem is to retrigger the errant OSF, but in OTFR this is not often caught and the archive request is returned to the user (flushed) without all of the request data files.
The OAPI library was changed so that messages due to 'continually' locked files will now also contain the name of the lock file. This may be noted in output logs. Also, the exit status of the osf_* programs, on failure due to a continually locked file, will now be 2 instead of 1. In addition, to alleviate failures due to times of high lock contention, the timeout period was changed from 20 seconds to one minute. This is not intended to be a permanent solution to this problem - PR 46063 has been created to address that.
osf_update has been changed to use the state of the OSF to be updated *after* the lock on the OSF is secured.
Two internal-poller examples have been added to the Share software. This will not affect operations, and it does not impact the Sample Pipeline.