wiki:DocumentationProcess

Documentation Process

The breadth of projects supported by SSB makes a single documentation format impractical for all the projects. A range of solutions will be needed in order to meet the needs of each project, or each aspect of the development. STScI has traditionally generated its official documents for the community (instrument handbooks, call for proposals, data handbooks, etc.) using FrameMaker. Unfortunately FrameMaker appears to be slowly fading and is only really well supported on one platform (Windows). MS word has proven to be inadequate for large documents (and is not portable). Software documentation under Python can take advantage of packages that pull information straight from the code itself and generate multiple formats (HTML, LaTeX), yet does not generate documentation suitable for users to learn how to use the code. Finally, reports describing the technical details of the software such as algorithms and standards used, not API or User-Handbook information, also need to be generated and distributed as separate products. This highlights the need to not only outline the documentation formats to be used, but also where and how the documents get distributed.

This proposal outlines the various formats which should be preferentially used in generating documentation for SSB-developed projects, and how the documents will be distributed. At no time should these recommendations be considered mandatory, but rather they should indicate what should be done if documentation has not already been generated in another suitable format.

Legacy Software

This proposal primarily addresses the documentation needs for newly developed software, notably anything not developed using IRAF. Legacy software should already have documentation associated with it that gets distributed with IRAF through help files or FrameMaker based manuals. The only aspect of this proposal which may apply to them would be whether or not there is some related documentation that should be accessible from the new SSB Document Archive web page. Any applicable documents, such as User Handbooks, should simply be included in the new Document Archive.

Mediawiki

Documents: MediaWiki lends itself to collaborative documentation efforts, and with the add-ons developed for the Multidrizzle Handbook, it can be used effectively to generate multi-format user handbooks. It also can be used for quickly generating draft documentation of many other types, such as technical reports, that may require input for many developers or contributors.

Process:The primary mediawiki server set up to support SSB documentation should be considered one of two primary formats for generating user handbooks for SSB software, particularly software which involves testing or input from groups outside SSB. The proposed work flow for using mediawiki for documentation should be:

  • Create draft documentation on mediawiki server
  • Generate table of contents page for the new document
  • Have the document reviewed and approved for distribution, use the table of contents to generate snapshot of document and convert to PDF; this copy would be circulated for review
    • if all the reviewers are within SSB, the reviewers could work off the wiki directly forwarding comments to the primary author(s) or editing the wiki itself, if it will not cause too much confusion (one person reviewing something that someone else has already edited and so on).
    • approval would also only need to be obtained for more 'official' documents such as TSRs or User Handbooks
  • Once approved for public distribution (if approval was needed at all), generate a final snapshot as a PDF or HTML, or Framemaker format or create a read-only copy of the wiki pages depending on requirements of the project.
  • Publish the snapshot document (PDF, HTML, ...) by putting it on a web server (probably SSB STSDAS web server)
  • Publish the link to the static document (PDF,...) for public access through the SSB Document Archive web page
  • If appropriate, add a copy of the PDF or HTML pages(if possible or useful) to the software's 'doc' directory in the repository for inclusion in the subsequent public release of that software.

The mediawiki should primarily be considered a means for generating drafts of documents to be published publicly through other formats or perhaps as static-views of the wiki itself.

Converting MediaWiki to PDF

The task 'wiki2frame' provides the capability to translate a mediawiki page into a FrameMaker document or into a LaTeX-derived PDF document. These pages provide the information necessary to download, install and use this task. This task will work on a mediawiki page regardless of where it is hosted, as it works over an http connection.

Sphinx

Documents: This tool (as it simply interprets the ReST format) can be used as the other primary format for generating user handbook documentation for new projects that do not require significant input from outside groups. It can also be used to generate API based documentation from the source code itself as well as Technical Software Reports if the developer is familiar enough with the ReST format.

Process:The use of sphinx requires a considerable amount of expertise in using the sphinx software in order to get presentable documents. It also relies on the use of the ReST formatted ASCII files as input. Both of these factors make it less attractive for collaboratively generated documentation, since ASCII documents are not easily shared or edited by groups. The proposed process for using Sphinx to generate documents would then be:

  • Either:
    • Create ReST formatted file in the 'doc' directory of the software's installation package in the repository, or
    • Edit the docstrings in the source code to be ReST formatted
  • Run sphinx to generate PDF and/or HTML output that gets added to the 'doc' installation directory
  • Make sure that the setup.py module will also install the 'doc' directory for user access
  • Copy the PDF or HTML pages to the SSB STSDAS web server and link to it from the software's web pages

The use of sphinx will eventually replace the use of 'epydoc' in the generation of the API level documentation for the software.

Learning Sphinx: The primary online sphinx documentation can be found at:

http://sphinx.pocoo.org/

These web pages have been setup not just to document Sphinx itself, but also to serve as a example of how to format the text using reStructuredText to format the document itself. Any page other than the top level page has been setup to display the ReST formatted input by simply using the 'Show Source' link in the sidebar on the right. It also includes a primer on the formatting syntax understood by Sphinx complete. These references should provide the basic information to allow the use of Sphinx for formatting docstrings in our Python code and for creating TSRs with as easy a learning curve as possible.

Python documentation standards

In the absence of epydoc standards, we originally came up with our own scheme for a standard module/class/method/function docstring format. We are continuing to use that, but new documentation should try to implement the standards developed by scipy/numpy, should the finally settle on a standard. This web page that describes their standard, while a close reading shows that there are still loose ends yet to be tied up; that should change by the end of the year if Joe Harrington's funded documentation project makes good progress.

Note that this format is not directly supported by Sphinx, but requires that an extra package, numpydoc, be installed. -- MGD

For the time being use the scipy/numpy guidelines with SSB specific implementations described by:

SsbEpydocGuidelines

MS Word

The JWST project generates all contractually obligated documentation using MS Word, as per the contract. All standards developed for those documents by the JWST project should be followed, including the use of SOCCER for publishing and archiving the documents. Remaining documentation can be generated using the most appropriate format for that document based on the formats listed here.

OpenOffice?

The use of OpenOffice? would be suitable for some documents. However, only a PDF version, or in rare situations a HTML version, of the document will be made available for publication online.

Publishing

The term 'publishing' in this context refers to making the documentation available for access by the general community. This can mean different things depending on the nature of the documentation itself, with user guides, handbooks, or technical reports needing to be available through a web page for constant access and API or other software specific documents being distributed with the software itself. Some documents will even need to be published in multiple ways. These documents, therefore, need to be placed where the intended audience can easily find them.

The SSB group home page provides navigation to publications already. This link will need to be renamed to 'Document Archive', a term consistently used by the Instrument Teams to direct users to their documentation. The new 'Document Archive' page for SSB (under Zope) will contain links to all the types of documents we provide, from user handbooks to technical reports and software API-based documentation. Each type of document will have its own index page, and these index pages can either be Zope pages themselves or editable static HTML pages on the SSB web server (stsdas.stsci.edu). These index pages will then have the final links to the PDF or HTML pages for each document which are accessible from the SSB web server. An example of how this would work can be seen by going to the ACS team web pages and following the Document Archive link on their side-bar.

The actual documents should probably be stored on the central store in a directory under /eng/ssb/web. Currently, directories already exist there to maintain copies of the epydoc documentation generated for each public release. These directories would then be accessed from the SSB web server through soft-links in the server directories itself.

Last modified 2 hours ago Last modified on 10/31/16 12:18:05