0.8.0 release prep (#1061)

* Removing the tracking banner, since we no longer track by default * Updating changelog * Adding stubs from OAuthSSH and MonitoringHub * Revert "Adding stubs from OAuthSSH and MonitoringHub" This reverts commit 39bfafe. * Adding stubs for OAuthSSH and MonitoringHub * Changelog updates for 0.8.0 * Updating contributors * Bumping version from 0.8.0a0 -> 0.8.0 * Adding some minimal monitoring docs * minor cleanup to changelog also, isn't there a breaking change related to Files vs strings we should mention? * Updating changelog * Updating changelog again * Updating changelog with mention Files no longer acting as str issue
Parsl · Jun 13, 2019 · 12bd758eaeb4720afd04b2e7193235e246e11678 · 12bd758
1 parent fbfd22e
commit 12bd758eaeb4720afd04b2e7193235e246e11678
diff --git a/README.rst b/README.rst
@@ -3,7 +3,7 @@ Parsl - Parallel Scripting Library
 |licence| |build-status| |docs|
 Parsl is a parallel scripting library that enables easy parallelism and workflow design.
-  
-The latest version available on PyPi is v0.7.2.
+  
+The latest version available on PyPi is v0.8.0.
 .. |licence| image:: https://img.shields.io/badge/License-Apache%202.0-blue.svg
   :target: https://github.com/Parsl/parsl/blob/master/LICENSE

diff --git a/docs/devguide/changelog.rst b/docs/devguide/changelog.rst
@@ -1,6 +1,59 @@
 Changelog
 =========
+  
+
+  
+Parsl 0.8.0
+  
+-----------
+  
+
+  
+Released on June 13th, 2019
+  
+
+  
+Parsl v0.8.0 includes 58 closed issues and pull requests with contributions (code, tests, reviews and reports)
+  
+
+  
+from: Andrew Litteken @AndrewLitteken, Anna Woodard @annawoodard, Antonio Villarreal @villarrealas,
+  
+Ben Clifford @benc, Daniel S. Katz @danielskatz, Eric Tatara @etatara, Juan David Garrido @garri1105,
+  
+Kyle Chard @@kylechard, Lindsey Gray @lgray, Tim Armstrong @timarmstrong, Tom Glanzman @TomGlanzman,
+  
+Yadu Nand Babuji @yadudoc, and Zhuozhao Li @ZhuozhaoLi
+  
+
+  
+
+  
+New Functionality
+  
+^^^^^^^^^^^^^^^^^
+  
+
+  
+* Monitoring is now integrated into parsl as default functionality.
+  
+* `parsl.AUTO_LOGNAME`: Support for a special `AUTO_LOGNAME` option to auto generate `stdout` and `stderr` file paths.
+  
+* `parsl.Files` no longer behave as strings. This means that operations in apps that treated `parsl.Files` as strings
+  
+  will break. For example the following snippet will have to be updated:
+  
+
+  
+  .. code-block:: python
+  
+
+  
+     # Old style: " ".join(inputs) is legal since inputs will behave like a list of strings
+  
+     @bash_app
+  
+     def concat(inputs=[], outputs=[], stdout="stdout.txt", stderr='stderr.txt'):
+  
+         return "cat {0} > {1}".format(" ".join(inputs), outputs[0])
+  
+
+  
+     # New style:
+  
+     @bash_app
+  
+     def concat(inputs=[], outputs=[], stdout="stdout.txt", stderr='stderr.txt'):
+  
+         return "cat {0} > {1}".format(" ".join(list(map(str,inputs))), outputs[0])
+  
+
+  
+* Cleaner user app file log management.
+  
+* Updated configurations using `HighThroughputExecutor` in the configuration section of the userguide.
+  
+* Support for OAuth based SSH with `OAuthSSHChannel`.
+  
+
+  
+Bug Fixes
+  
+^^^^^^^^^
+  
+
+  
+* Monitoring resource usage bug `issue#975 <https://github.com/Parsl/parsl/issues/975>`_
+  
+* Bash apps fail due to missing dir paths `issue#1001 <https://github.com/Parsl/parsl/issues/1001>`_
+  
+* Viz server explicit binding fix `issue#1023 <https://github.com/Parsl/parsl/issues/1023>`_
+  
+* Fix sqlalchemy version warning `issue#997 <https://github.com/Parsl/parsl/issues/997>`_
+  
+* All workflows are called typeguard `issue#973 <https://github.com/Parsl/parsl/issues/973>`_
+  
+* Fix `ModuleNotFoundError: No module named 'monitoring` `issue#971 <https://github.com/Parsl/parsl/issues/971>`_
+  
+* Fix sqlite3 integrity error `issue#920 <https://github.com/Parsl/parsl/issues/920>`_
+  
+* HTEX interchange check python version mismatch to the micro level `issue#857 <https://github.com/Parsl/parsl/issues/857>`_
+  
+* Clarify warning message when a manager goes missing `issue#698 <https://github.com/Parsl/parsl/issues/698>`_
+  
+* Apps without a specified DFK should use the global DFK in scope at call time, not at other times. `issue#697 <https://github.com/Parsl/parsl/issues/697>`_
+  
+
+  
+
 Parsl 0.7.2
 -----------
@@ -50,7 +103,7 @@ David Heise @daheise, Daniel S. Katz @danielskatz, Dominic Fitzgerald @djf604, F
 Juan David Garrido @garri1105, Gordon Watts @gordonwatts, Justin Wozniak @jmjwozniak,
 Joseph Moon @jmoon1506, Kenyi Hurtado @khurtado, Kyle Chard @kylechard, Lukasz Lacinski @lukaszlacinski,
 Ravi Madduri @madduri, Marco Govoni @mgovoni-devel, Reid McIlroy-Young @reidmcy, Ryan Chard @ryanchard,
-  
-@sdustrud, Yadu Nand Babuji @yadudoc and Zhuozhao Li @ZhuozhaoLi
+  
+@sdustrud, Yadu Nand Babuji @yadudoc, and Zhuozhao Li @ZhuozhaoLi
 New functionality
 ^^^^^^^^^^^^^^^^^

diff --git a/docs/images/mon_resource_summary.png b/docs/images/mon_resource_summary.png
diff --git a/docs/images/mon_task_app_grouping.png b/docs/images/mon_task_app_grouping.png
diff --git a/docs/images/mon_task_info.png b/docs/images/mon_task_info.png
diff --git a/docs/images/mon_task_state_grouping.png b/docs/images/mon_task_state_grouping.png
diff --git a/docs/images/mon_task_states.png b/docs/images/mon_task_states.png
diff --git a/docs/images/mon_workflow_summary.png b/docs/images/mon_workflow_summary.png
diff --git a/docs/images/mon_workflows_page.png b/docs/images/mon_workflows_page.png
diff --git a/docs/index.rst b/docs/index.rst
@@ -21,15 +21,11 @@ Parsl can be used to realize a variety of workflows:
 * Interactive and dynamic workflows in which the workflow is dynamically expanded during execution by users or the workflow itself.
 * Procedural workflows in which serial execution of tasks are managed by Parsl.
-  
-.. note::
-  
-   Parsl collects anonymous usage statistics for reporting and
-  
-   improvement purposes. To understand what stats are collected and to disable
-  
-   collection please refer to the `usage tracking guide <http://parsl.readthedocs.io/en/latest/userguide/usage_tracking.html>`__
 =======
 .. libsubmit documentation master file, created by
-  
-   sphinx-quickstart on Mon Oct  2 13:39:42 2017.
-  
-   You can adapt this file completely to your liking, but it should at least
-  
-   contain the root `toctree` directive.
+  
+.. sphinx-quickstart on Mon Oct  2 13:39:42 2017
+  
+.. You can adapt this file completely to your liking, but it should at least
+  
+.. contain the root `toctree` directive.
 .. Libsubmit is responsible for managing execution resources with a Local Resource

diff --git a/docs/reference.rst b/docs/reference.rst
@@ -24,6 +24,7 @@ Reference guide
    parsl.executors.swift_t.TurbineExecutor
    parsl.channels.LocalChannel
    parsl.channels.SSHChannel
+  
+    parsl.channels.OAuthSSHChannel
    parsl.channels.SSHInteractiveLoginChannel
    parsl.providers.AWSProvider
    parsl.providers.CobaltProvider
@@ -43,7 +44,7 @@ Reference guide
    parsl.launchers.SrunMPILauncher
    parsl.launchers.GnuParallelLauncher
    parsl.launchers.MpiExecLauncher
-  
-    parsl.monitoring.db_logger.MonitoringConfig
+  
+    parsl.monitoring.MonitoringHub
 .. autosummary::
    :toctree: stubs

diff --git a/docs/userguide/monitoring.rst b/docs/userguide/monitoring.rst
@@ -72,3 +72,39 @@ If the cluster allows you to host a web server on its public IP address with a s
   $ parsl-visualize --listen 0.0.0.0 --port 55555 sqlite:///<absolute-path-to-db>
+  
+Workflows Page
+  
+^^^^^^^^^^^^^^
+  
+
+  
+The workflows page lists all instances of a Parsl workflow that has been executed with monitoring turned on.
+  
+It also gives a high level overview of workflow runs as a table as shown below:
+  
+
+  
+.. image:: ../images/mon_workflows_page.png
+  
+
+  
+Throughout the visualization pages, all blue elements are clickable. For eg, clicking a specific worklow
+  
+name from the table takes you to the Workflow Summary page described in the next section.
+  
+
+  
+Workflow Summary
+  
+^^^^^^^^^^^^^^^^
+  
+
+  
+.. image:: ../images/mon_workflow_summary.png
+  
+
+  
+The above screenshot of the workflow summary page captures the run level details such as start and end times
+  
+as well as task summary statistics. The workflow summary section is followed by the *App Summary* that lists
+  
+the various apps and count of invocations each. This is followed by three different views of the workflow:
+  
+
+  
+* Workflow DAG - colors grouped by apps: This visualization is useful to visually inspect the dependency
+  
+  structure of the workflow DAG. Hovering over the nodes in the DAG shows a tooltip for the app that the
+  
+  node represents and it's task ID.
+  
+
+  
+.. image:: ../images/mon_task_app_grouping.png
+  
+
+  
+* Workflow DAG - colors grouped by task states: This visualization is useful to identify what stages
+  
+  in the workflow are complete and what stages are pending.
+  
+
+  
+.. image:: ../images/mon_task_state_grouping.png
+  
+
+  
+* Workflow resource usage: This visualization provides resource usage information at the workflow level.
+  
+  For eg, cumulative CPU/Memory utilization across workers over time.
+  
+
+  
+.. image:: ../images/mon_resource_summary.png
+  
+
diff --git a/parsl/version.py b/parsl/version.py
@@ -3,4 +3,4 @@
 <Major>.<Minor>.<maintenance>[alpha/beta/..]
 Alphas will be numbered like this -> 0.4.0a0
 """
-  
-VERSION = '0.8.0a0'
+  
+VERSION = '0.8.0'