[OPUS]

Frequently Asked Questions

What's New in OPUS 4.5?

What's New in OPUS 4.5?

How does the OPUS release 4.5 differ from the last one?

Several enhancements, bug fixes, and repairs were made to both the OAPI (OPUS applications programmers interface) and the Java Managers. These modifications have made the pipeline environment more robust, more configurable, and more maintainable.

From the user's perspective there are several modifications which have been suggested by people using OPUS around the world, and we hope the current release addresses the most important of those concerns.

Problems were noted about the ease of use in bringing up the blackboard and in managing processes. OPRs 47219, 47395, and 46771 particularly address these problems.

We are happy to report that initial problems (46975) with Java 1.4 are resolved, and that we have made major progress in supporting OPUS on the IBM AIX platform, though it is not yet ready for CD. All of OPUS is now built using GCC 3.2 (41512) which now provides better support for C++ and potential 64bit processing than the older EGCS compiler. OPUS now supports secure file transfers (49367) via the freely available J2Ssh software, eliminating the need to enable FTP access (a security issue).

Several users have encountered problems with the naming (46127) and location (44317) of pipeline files, and, while we have modified the blackboards to be more general, we still encourage users to take care in naming files.

All pipeline applications at the Institute have migrated from the FITSPP package to the CFITSIO package (45889), and we heartily endorse that suite of programs which is maintained by HEASARC at Goddard:

While we included excellent documentation in previously releases on how to write internal pollers in a variety of languages, in OPUS 4.2 we forgot to include the required header files to create C++ internal pollers. These header files are now available (48405) and a new example internal poller has been created (process 'c++poller').

What is a PR?

A PR is a Problem Report. The problem reporting system affects all components at the Space Telescope Science Institute and includes enhancements, change requests, as well as documentation updates. Not all of the 46,000+ problem reports have been filed against OPUS!

47219: OPUS does not correctly substitute SUB[] values in path files

If a user puts a variable into an OPUS path file, and specifies its value as "SUB[HOME]", for example, they expect that the value of their variable will be given the value of the $HOME env variable, as the OPUS FAQ specifies.

Example PATH file: 

!----------------------------------------------------------------------------
!
 STAGE_FILE             = OPUS_DEFINITIONS_DIR:g2f_pipeline.stage
 OPUS_OBSERVATIONS_DIR  = /info/devcl/pipe/sontag/t64/g2f/obs/
 gif_data               = /info/devcl/pipe/sontag/t64/g2f/input/
 fits_data              = /info/devcl/pipe/sontag/t64/g2f/fits/
 hdr_data               = /info/devcl/pipe/sontag/t64/g2f/fits/
 sample_db              = /info/devcl/pipe/dptbuild/sample02_2_00/db/
 FRANK                  = SUB[HOME]
!----------------------------------------------------------------------------

So, if my program uses the path file above, it expects the value of the FRANK item to expand to $HOME, specifically "/home/sontag".

41512: Port OPUS to GCC3.2 on Solaris

The bulk of this work had been performed for porting the code to gcc 3.2 on Linux RedHat8.0. Note that with this PR, compilation is still 32-bit on Solaris. But, now that we are using GCC 3.2, the switch to 64-bit is a few small changes to makefiles (and apparently some debugging). The EGCS compiler did not support 64-bit compilation. Note also that with this PR, we will no longer be using the SGI STL

47240: Please supply account name for iconized pmg

The Java omg and pmg would be more user friendly if the account was included in the title bar and the iconized managers. It would also be nice if once a path is being monitored with either manager, that information would also be displayed in the title bar and icon. Something like "otfr: lewis" would do. There's no need to have "PMG" or "OMG" on the icon since the icon itself lets the user know what application it is.

Code was added to the OMG and PMG classes (and their base class, MGRFrame) to make sure the title bar is updated with the path, account-name, and the name of the Manager (e.g. "OMG" or "PMG").

The title bars for user "joe", currently viewing the "g2f" path should be: "g2f: joe: OMG", and "g2f: joe: PMG". This allows the most identifiable information (path name) to appear leftmost, and thus most easily, on the task icon.

47242: Need an easier way to view resource files from the Java/CORBA PMGs

It was decided that such a View menu-item might be confusing, since it is entirely possible (and likely) for a user to have a process task selected in the left half of the PMG while they also have a process (pstat) selected in the right half of the PMG. In that case, which selected item would the 'View -> Resource File' menu-item actually refer to ? The unknowing user would have to perform trial-and-error to find out.

So instead of a menubar menu-item for this, a menu-item was added to the right-click-popup-menu that appears over the table on the right side. The user can now view the resource file for the process via the popup. Note that the popup works only for the selected item.

47395: PMG poorly handles when (num procs > allowed by pmg_restrs.dat)

What should happen:
  • A user tries to start too many copies of 'gifin' for example. The user should get an error dialog popping up saying that they couldn't start the process (once for each extra copy), and when dismissed, the PMG should remove the pstats for the extra copies, and redraw itself appropriately.

    • What happened before the fix

  • A user tries to start too many copies of 'gifin' for example. The user gets an error dialog popping up saying that they couldn't start the process (once for each extra copy), and when dismissed, the PMG does NOT redraw itself correctly, and a traceback is found in the PMG log file. A correct-looking table is only then achieved by reloading the listener (iconify-then-restore).

    • 45889: Project to replace/update the OPUS FITS library

    The FITS library that the OPUS processes currently use, is a software package written at STScI. It was intended to be a C++ interface to a collection of C FITS routines. It is not what would be called 'elegant C++' by today's standards, and unfortunately it is, by most accounts, very poorly documented.

    Since the sole maintainer of this package is no longer available for updates, we are left with the choices of: 1) updating it ourselves, or 2) porting the OPUS codes to one of the currently available public FITS packages. Both choices have the potential to be a software effort of 'project' scope.

    With this delivery, all OPUS and SHARE applications will now use the CFITSIO libraries which are fully supported by HEASARC at Goddard. These libraries supplant the FITS++ libraries which we have been using, and which will no longer be supported.

    49367: OPUS Managers need to support secure file transfer

    The OPUS Managers transfer files for various tasks via FTP. Since this can be a security risk (e.g. passwords sent in the clear), OPUS should support the capability of secure file transfer (e.g. SFTP). Multiple external users have requested this.

    The ability to use either FTP or SFTP (via the freely available J2Ssh s/w) has been added. Note that this is coded so that users can run without having the J2SSH jar files installed (i.e. SshFtp class and its associated J2SSH code is dynamically loaded). Normal FTP is still an option. See the FAQ for more information.

    47095: PMG updates the process state incorrectly

    If multiple instances of a process are in pending states, a start pending selection updates the state of the process selected to starting and then idle. In addition to updating the selected process state, the PMG incorrectly updates the state of an unselected process to starting. The second process state stays at starting and will never change since the process has not actually started.

    If multiple instances of a process are in pending states, a start pending action now correctly updates the state of the specific process selected to starting and then to idle.

    44317: OPUS Java PMG doesn't allow pipeline files to reside in subdirectories

    It would be nice if the java PMG could display not only *.pipeline files in one's OPUS_DEFINITIONS_DIR stretch, but also sub-directory names which could be clicked to display other subdirectories and/or pipeline files (as the Motif PMG does). The work around requires a very long list of pipeline files in one directory, or adding all the subdirectories to one's OPUS_DEFINITIONS_DIR, or having to choose all the individual processes instead of a pipeline.

    It would also be nice if the pipelines were alphabetized (across the stretch): currently *.pipeline files that appear in the first directory of one's OPUS_DEFINITIONS_DIR stretch appear before *.pipeline files in the next directory of the OPUS_DEFINITIONS_DIR stretch, so finding the desired pipeline file will be more difficult because the order will no longer be alphabetical.

    I cannot even simply move a desired pipeline up from a sub-directory to the directory the PMG sees because the Java PMG does not respond to changes in this directory. Many tools provide a Refresh capability, causing a re-read of the directory being viewed, could not the PMG also do that?

    The submitter decided that the most important part of this PR was to have the pipeline files alphabetized. That has been done. Subdirectories have not. Since one's OPUS_DEFINITIONS_DIR stretch can be quite complex, a display of those directories and all the subdirectories under each of those may not be exactly what is wanted. Further requirements should be submitted before that enhancement is begun. The "refresh" idea also has not been implemented, as that seems tied to subdirectory changes.

    46975: opus java managers don't work with java 1.4.1

    I tried running the opus 14.2/sample 4.2 java managers against java 1.4.1 and they failed to start. I would get the dialogue boxes that allowed me to enter server location, username/password but once I hit enter, the splash screen would disappear and the manager failed to start. The log file claims the wrong password was entered even though the correct password was entered (I tried the password entry multiple multiple times to verify this).

    Problems associated with java 1.4.1 have now been resolved.

    46127: opus_bb_server fails to start due to filename with a ":" in home dir

    I had a file called Pipeline: in my home directory (/home/erith) and as a result, opus_bb_server (-t FILE) failed to start.

    The reason for this is that as part of the startup of this instance of the opus_bb_server a directory listing of the home directory is done. The filename with a colon makes the software (osfile_stretch_file) think that it has an environment variable it needs to expand and attempts to do so (it fails since it is a filename and not an environment variable). The result of this error is that the server never starts and therefore the user is unable to start any processes. The workaround is, of course, not to have (valid unix) filenames with colons in one's home directory.

    The OPUS blackboard code has some restrictions on file naming (for good reason), and they should likely not be changed at present. Two such restrictions are that OPUS has problems with files with colons and/or spaces in the name. If, then, such files are found in the user's HOME dir, the opus_bb_server encounters the problems listed in this PR.

    The solution employed here is to not allow './*' to be used in that particular initialization case, but rather use only the File_entry defaults (which amount to a single match for ".") without adding any wildcards.

    46771: Corba server kill script should do clean up

    The CORBA server kill script should clean up the opus_corba_objs and the resources.lock files. It is known, a priori, what should be in these files. Confirming that these files are in the proper state to support the next set of servers should be part of the kill script.

    The 'kill_oservers.csh' wrapper script has been added as a convenience for external users to perform the two steps: 

       'pscluster.pl -kill', and then
   'alter_server_files.pl'.

    48266: change OMG property "path" to "omgPath"

    Normally, the OMG uses a property named "path", often saved in the user's OMG.ini file, to indicate the path to be monitored (or the path to start monitoring upon execution).

    This creates the following problem: both of the managers allow the user to specify how they want PSTAT/OSF fields split in the display into subcolumns. The user species this through the .ini file by creating a property named after the PSTAT/OSF field to be subdivided. The PSTAT has a field called "path", which conflicts with the OMG property of the same name, leading to a renaming of the PSTAT "path" field to match the OMG path being monitored, which in turn causes problems with PSTAT filtering across the path tabs in the PMG.

    To avoid the property name clash in a combined GUI, the OMG should look both for a property "path" (for backwards compatibility) and "omgPath", but only write out the property value as "omgPath" to the OMG.ini file when prompted by the user.

    48405: OPUS CD to include header files for user-written internal pollers

    OPUS advertises that users can create their own internal pollers by linking with the OAPI libraries supplied. The libraries are supplied on the CD (for the various operating systems), but currently the necessary header files are not.

      There are two projects here:
    1. Determine which header files are necessary for outside users to be able to build their own internal pollers, and determine if they should be made public via the CD (since many header files include inlined source code).

    2. Add a test procedure to actually build and run (perhaps in the sample pipeline) a C++ internal poller such as an outside user might create. This test should be performed with each CD creation.

    A new example internal poller has been created (process 'c++poller').


    Top of What's New FAQ

    Top of OPUS FAQ