Skip to content

Docker executor cannot recognize the volumes option. #88

@danielecook

Description

@danielecook

I am trying to get a local setup running for debugging purposes. Data will be staged locally to reduce latency. Unfortunately, the volumes option for the docker executor does not appear to work.

Current config

[executors.docker]
type = docker
max_workers = 20
image = ####
scratch = /tmp/scratch
include_aws_env = False
volumes = [["/home", "/home"]]
interactive = False

The get_docker_executor_config does not retrieve volumes currently.

def get_docker_executor_config(config: SectionProxy) -> SectionProxy:
"""
Returns a config for DockerExecutor.
"""
keys = ["image", "scratch", "job_monitor_interval", "vcpus", "gpus", "memory"]
return create_config_section({key: config[key] for key in keys if key in config})

It may be helpful to throw an error for additional options specified here that cannot be used with the Docker executor.

Additionally, I wonder whether mounting the workflow working directory would be beneficial?

self._default_job_options: Dict[str, Any] = {
"vcpus": config.getint("vcpus", fallback=1),
"gpus": config.getint("gpus", fallback=0),
"memory": config.getint("memory", fallback=4),
"shared_memory": config.getint("shared_memory", fallback=None),
"interactive": config.getboolean("interactive", fallback=False),
"include_aws_env": config.getboolean("include_aws_env", fallback=True),
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions