Skip to content
Please note that GitHub no longer supports your web browser.

We recommend upgrading to the latest Google Chrome or Firefox.

Learn more
Parsl - Parallel Scripting Library
Python HTML Shell Vim script C Dockerfile
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Removing auto-assign to yadudoc (#1352) Oct 11, 2019
docker Adding sample predict functions as python files Mar 28, 2018
docs changing BW () from Cray to NCSA (#1504) Dec 16, 2019
mypy-stubs Introduce mypy stub for typeguard, and type-check parsl.load() (#874) Apr 24, 2019
parsl Fixing repeated timeout wrap bug in python_app code (#1534) Jan 13, 2020
.flake8 Remove unused flake8 D203 exclusion (#1457) Nov 22, 2019
.gitignore Add non-version-controlled user opts override (#1134) Jul 17, 2019
.gitmodules Re-organize libsubmit subdirectories Sep 24, 2018
.travis.yml Fix and test against Python 3.8 (#1363) Dec 5, 2019
CITATION.md update inprocessdings Apr 30, 2019
CONTRIBUTING.rst Updates to testing documentation (#1488) Dec 5, 2019
CoC.md CoC (#1509) Jan 3, 2020
LICENSE Initial commit Sep 28, 2017
MANIFEST.in Merge remote-tracking branch 'viz/master' May 22, 2019
README.rst Add CoC section to README (#1513) Jan 13, 2020
codemeta.json Resurrecting Azure Execution provider (#1121) Jul 17, 2019
mypy.ini Make and test that sites/ tests directory passes mypy (#1221) Aug 27, 2019
requirements.txt Add minimum version for typeguard to fix a MonitoringHub bug (#1419) Oct 30, 2019
setup.py Make monitoring dependencies optional #1139 (#1156) Jul 22, 2019
tag_and_release.sh Fix punctuation in release script Jul 11, 2018
test-requirements.txt Pin `coverage` version in CI. (#1505) Dec 16, 2019

README.rst

Parsl - Parallel Scripting Library

Apache Licence V2.0 Build status Documentation Status

Parsl is a parallel programming library for Python. Parsl augments Python with simple, scalable, and flexible constructs for encoding parallelism. Developers annotate Python functions to specify opportunities for concurrent execution. These annotated functions, called apps, may represent pure Python functions or calls to external applications, whether sequential, multicore (e.g., CPU, GPU, accelerator), or multi-node MPI. Parsl further allows these calls to these apps, called tasks, to be connected by shared input/output data (e.g., Python objects or files) via which Parsl can construct a dynamic dependency graph of tasks.

Parsl includes a flexible and scalable runtime that allows it to efficiently execute Python programs in parallel. Parsl scripts are portable and can be easily moved between different execution resources: from laptops to supercomputers to clouds. When executing a Parsl program, developers first define a simple Python-based configuration that outlines where and how to execute tasks. Parsl supports various target resources including clouds (e.g., Amazon Web Services and Google Cloud), clusters (e.g., using Slurm, Torque/PBS, HTCondor, Cobalt), and container orchestration systems (e.g., Kubernetes). Parsl scripts can scale from a single core on a single computer through to hundreds of thousands of cores across many thousands of nodes on a supercomputer.

Parsl can be used to implement various parallel computing paradigms:

  • Concurrent execution of a set of tasks in a bag-of-tasks program
  • Procedural workflows in which tasks are executed following control logic
  • Parallel dataflow in which tasks are executed when their data dependencies are met
  • Heterogeneous many-task applications in which many different computing resources are used together to execute different types of computational tasks
  • Dynamic workflows in which the workflow is determined during execution
  • Interactive parallel programming through notebooks or another interactive mechanism

The latest Parsl version available on PyPi is v0.9.0.

QuickStart

Parsl is now available on PyPI, but first make sure you have Python3.5+

$ python3 --version

Install Parsl using pip:

$ pip3 install parsl

To run the Parsl tutorial notebooks you will need to install Jupyter:

$ pip3 install jupyter

Detailed information about setting up Jupyter with Python3.5 is available here

Note: Parsl uses an opt-in model to collect anonymous usage statistics for reporting and improvement purposes. To understand what stats are collected and enable collection please refer to the usage tracking guide

Documentation

The complete parsl documentation is hosted here.

The Parsl tutorial is hosted on live Jupyter notebooks here

For Developers

  1. Download Parsl:

    $ git clone https://github.com/Parsl/parsl
    
  2. Install:

    $ cd parsl
    $ python3 setup.py install
    
  3. Use Parsl!

Requirements

Parsl is supported in Python 3.5+. Requirements can be found here. Requirements for running tests can be found here.

Code of Conduct

Parsl seeks to foster an open and welcoming environment - Please see the Parsl Code of Conduct for more details.

Contributing

We welcome contributions from the community. Please see our contributing guide.

You can’t perform that action at this time.