Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.
Sign upHTEX broken on Mac OS X #854
Comments
This comment has been minimized.
This comment has been minimized.
|
Hi @tilmantroester, thanks for using Parsl. I just tried your script with We can help you debug that on our slack channel too (link: http://parsl-project.org/support.html) |
This comment has been minimized.
This comment has been minimized.
|
I'm running this on my laptop (macOS 10.14.4, python 3.7). All log files |
This comment has been minimized.
This comment has been minimized.
|
@tilmantroester Thanks for reporting this issue. We've unfortunately had very limited testing on Mac systems and I am not surprised that there would be some Mac specific issues. Looking through the logs, it looks like most of the executor system seems to have come online correctly. @annawoodard is a Mac user and she's going to take a crack at this issue soon. |
yadudoc
assigned
yadudoc and
annawoodard
Apr 9, 2019
yadudoc
added
the
bug
label
Apr 9, 2019
yadudoc
added this to the Parsl-0.8.0 milestone
Apr 9, 2019
This comment has been minimized.
This comment has been minimized.
|
Thanks for looking into this! Ultimately, I'd deploy it on a Linux machine but for development purposes it be nice to test things locally. Here's the MWE used to create those logs: import parsl
import os
from parsl.app.app import python_app
# from parsl.configs.htex_local import config as htex_config
from parsl.configs.local_ipp import config as ipp_config
from parsl.configs.local_threads import config as threads_config
from parsl.providers import LocalProvider
from parsl.channels import LocalChannel
from parsl.config import Config
from parsl.executors import HighThroughputExecutor
htex_config = Config(
executors=[
HighThroughputExecutor(
label="htex_Local",
worker_debug=True,
cores_per_worker=1,
provider=LocalProvider(
channel=LocalChannel(),
init_blocks=1,
max_blocks=1,
),
)
],
)
@python_app
def dummy_app():
return "Dummy app finished!"
if __name__ == "__main__":
parsl.set_stream_logger()
print(f"Using parsl version {parsl.__version__}")
parsl.clear()
# This works
# parsl.load(ipp_config)
# parsl.load(threads_config)
# This doesn't work
parsl.load(htex_config)
print("Launching dummy app")
results = dummy_app()
print("Launched dummy app")
parsl.wait_for_current_tasks()
# print(results.result())
|
This comment has been minimized.
This comment has been minimized.
|
It looks like this is because qsize is not implemented on Mac: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue.qsize
Someone else already solved this problem, their approach looks reasonable to me: vterron/lemon@9ca6b4b |
annawoodard
referenced this issue
Apr 12, 2019
Open
Merge relevant bits from `funcx_singularity_updates` branch #872
This comment has been minimized.
This comment has been minimized.
|
While this is being implemented, is there a workaround that I could use? Or should I just stick with IPP for now? |
This comment has been minimized.
This comment has been minimized.
|
@tilmantroester I suspect that fixes to HTEX will take a bit of time, so I'd recommend using IPP for local and switching to HTEX when running on the remote system. Just to be clear, you'd have to be running completely on linux based systems for HTEX to work right now. |
annawoodard
removed their assignment
Apr 25, 2019
yadudoc
modified the milestones:
Parsl-0.8.0,
Parsl-0.9.0
May 22, 2019
annawoodard
changed the title
HTEX hangs for parsl.configs.htex_local.config and config from docs
HTEX broken on Mac OS X
May 22, 2019
This comment has been minimized.
This comment has been minimized.
|
I'll confirm this is a problem that makes the last cell in the default tutorial hang on my mac as well |
danielskatz
added
the
tutorial
label
Jun 5, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
tilmantroester commentedApr 8, 2019
I have trouble getting HTEX to work. Maybe I'm doing something fundamentally wrong but the following MWE gets stuck for me for parsl 0.7.2:
The logger output is
with the last 3 messages repeating without finishing the program.
If I instead use the HTEX config from the docs (https://parsl.readthedocs.io/en/latest/parsl-introduction.html#Local-execution-with-pilot-jobs):
the program just hangs after
Using IPP (
parsl.configs.local_ipp.config) or threads (parsl.configs.local_threads.config) works, however.