grpc4bmi.bmi_client_singularity module

class grpc4bmi.bmi_client_singularity.BmiClientSingularity(image: str, work_dir: str, input_dirs: Iterable[str] = (), delay=0, timeout=None, capture_logs=True)[source]

Bases: BmiClient

BMI GRPC client for singularity server processes During initialization launches a singularity container with run-bmi-server as its command. The client picks a random port and expects the container to run the server on that port. The port is passed to the container using the BMI_PORT environment variable.

Parameters:

  • image –

    Singularity image.

    For Docker Hub image use docker://* or convert it to a Singularity image file.

    To convert Docker image ewatercycle/walrus-grpc4bmi with v0.3.1 tag to ./ewatercycle-walrus-grpc4bmi_v0.3.1.sif Singularity image file use:

    singularity pull ewatercycle-walrus-grpc4bmi_v0.3.1.sif \
  docker://ewatercycle/walrus-grpc4bmi:v0.3.1

  • input_dirs (Iterable[str]) –

    Directories for input files of model.

    All of directories will be mounted read-only inside Singularity container on same path as outside container.

  • work_dir (str) –

    Working directory for model.

    Directory is mounted inside container and changed into.

    To create a random work directory you could use

    from tempfile import TemporaryDirectory
from grpc4bmi.bmi_client_singularity import BmiClientSingularity

work_dir = TemporaryDirectory()

image = 'ewatercycle-walrus-grpc4bmi_v0.2.0.sif'
client =  BmiClientSingularity(image, work_dir.name)

# Write config to work_dir and interact with client

# After checking output in work_dir, clean up
work_dir.cleanup()

  • delay (int) –

    Seconds to wait for Singularity container to startup, before connecting to it

    Increase when container takes a long time to startup.

  • timeout (int) –

    Seconds to wait for gRPC client to connect to server.

    By default will try forever to connect to gRPC server inside container. Set to low number to escape endless wait.

  • capture_logs (bool) –

    Whether to capture stdout and stderr of container .

    If false then redirects output to null device never to be seen again. If true then redirects output to temporary file which can be read with BmiClientSingularity.logs(). The temporary file gets removed when this object is deleted.

Example 1: Config file already inside image

MARRMoT has an example config file inside its Docker image.

from grpc4bmi.bmi_client_singularity import BmiClientSingularity
client = BmiClientSingularity(image='docker://ewatercycle/marrmot-grpc4bmi:latest',
                              work_dir='/opt/MARRMoT/BMI/Config'))
client.initialize('/opt/MARRMoT/BMI/Config/BMI_testcase_m01_BuffaloRiver_TN_USA.mat')
client.update_until(client.get_end_time())
del client

Example 2: Config file in input directory

The config file and all other files the model needs are in a directory (/tmp/input). Use /tmp/work to capture any output files like logs generated by model.

from grpc4bmi.bmi_client_singularity import BmiClientSingularity
# Generate config file called 'config.mat' in `/tmp/input` directory
client = BmiClientSingularity(image='docker://ewatercycle/marrmot-grpc4bmi:latest',
                              input_dirs=['/tmp/input'],
                              work_dir='/tmp/work')
client.initialize('/tmp/input/config.mat')
client.update_until(client.get_end_time())
del client

Example 3: Read only input directory with config file in work directory

The forcing data is in a shared read-only location like /shared/forcings/walrus. In the config file (/tmp/work/walrus.yml) point to a forcing data file (/shared/forcings/walrus/PEQ_Hupsel.dat).

from grpc4bmi.bmi_client_singularity import BmiClientSingularity
client = BmiClientSingularity(image='ewatercycle-walrus-grpc4bmi_v0.2.0.sif',
                              input_dirs=['/shared/forcings/walrus'],
                              work_dir='/tmp/work')
client.initialize('walrus.yml')
client.update_until(client.get_end_time())
del client

Example 4: Model writes in sub directory of input directory

Some models, for example wflow, write output in a sub-directory of the input directory. If the input directory is set with the input_dirs argument then the model will be unable to write its output as input directories are mounted read-only. That will most likely cause the model to die. A workaround is to use the work_dir argument with input directory as value instead. This will make the whole input directory writable so the model can do its thing.

When input directory is on a shared disk where you do not have write permission then the input dir should be copied to a work directory (/scratch/wflow) so model can write.

from grpc4bmi.bmi_client_singularity import BmiClientSingularity
client = BmiClientSingularity(image='docker://ewatercycle/wflow-grpc4bmi:latest',
                              work_dir='/scratch/wflow')
client.initialize('wflow_sbm.ini')
client.update_until(client.get_end_time())
del client

Example 5: Inputs are in multiple directories

A model has its forcings (/shared/forcings/muese), parameters (/shared/model/wflow/staticmaps) and config file (/tmp/work/wflow_sbm.ini) in different locations. The config file should be set to point to the forcing and parameters files.

from grpc4bmi.bmi_client_singularity import BmiClientSingularity
client = BmiClientSingularity(image='docker://ewatercycle/wflow-grpc4bmi:latest',
                              input_dirs=['/shared/forcings/muese',
                                          '/shared/model/wflow/staticmaps'],
                              work_dir='/tmp/work')
client.initialize('wflow_sbm.ini')
client.update_until(client.get_end_time())
del client

Example 6: Run model twice with their own work directory

While running 2 or models at the same time you do not want the any config or output to be mixed.

from grpc4bmi.bmi_client_singularity import BmiClientSingularity
client_muese = BmiClientSingularity(image='docker://ewatercycle/wflow-grpc4bmi:latest',
                                    work_dir='/scratch/wflow-meuse')
client_muese.initialize('wflow_sbm.meuse.ini')
client_rhine = BmiClientSingularity(image='docker://ewatercycle/wflow-grpc4bmi:latest',
                                    work_dir='/scratch/wflow-rhine')
client_rhine.initialize('wflow_sbm.rhine.ini')
...
# Run models and set/get values
...
del client_muese
del client_rhine
get_value_ptr(var_name)[source]

Not possible, unable give reference to data structure in another process and possibly another machine

logs() → str[source]

Returns complete combined stdout and stderr written by the Singularity container.

When object was created with log_enable=False argument then always returns empty string.

grpc4bmi.bmi_client_singularity.check_singularity_version()[source]
grpc4bmi.bmi_client_singularity.check_singularity_version_string(version_output: str) → bool[source]