-
Notifications
You must be signed in to change notification settings - Fork 54
Open
Description
I am trying to get a local setup running for debugging purposes. Data will be staged locally to reduce latency. Unfortunately, the volumes option for the docker executor does not appear to work.
Current config
[executors.docker]
type = docker
max_workers = 20
image = ####
scratch = /tmp/scratch
include_aws_env = False
volumes = [["/home", "/home"]]
interactive = False
The get_docker_executor_config does not retrieve volumes currently.
redun/redun/executors/docker.py
Lines 38 to 43 in bcc3370
| def get_docker_executor_config(config: SectionProxy) -> SectionProxy: | |
| """ | |
| Returns a config for DockerExecutor. | |
| """ | |
| keys = ["image", "scratch", "job_monitor_interval", "vcpus", "gpus", "memory"] | |
| return create_config_section({key: config[key] for key in keys if key in config}) |
It may be helpful to throw an error for additional options specified here that cannot be used with the Docker executor.
Additionally, I wonder whether mounting the workflow working directory would be beneficial?
redun/redun/executors/docker.py
Lines 278 to 285 in bcc3370
| self._default_job_options: Dict[str, Any] = { | |
| "vcpus": config.getint("vcpus", fallback=1), | |
| "gpus": config.getint("gpus", fallback=0), | |
| "memory": config.getint("memory", fallback=4), | |
| "shared_memory": config.getint("shared_memory", fallback=None), | |
| "interactive": config.getboolean("interactive", fallback=False), | |
| "include_aws_env": config.getboolean("include_aws_env", fallback=True), | |
| } |
Metadata
Metadata
Assignees
Labels
No labels