grpc4bmi package
Submodules
grpc4bmi.bmi_client_docker module
- class grpc4bmi.bmi_client_docker.BmiClientDocker(image: str, work_dir: str, image_port=50051, host=None, input_dirs: Iterable[str] = (), user=1005, remove=False, delay=5, timeout=None)[source]
Bases:
BmiClientBMI gRPC client for dockerized server processes: the initialization launches the docker container which should have the run-bmi-server as its command. Also, it should expose the tcp port 50001 for communication with this client. Upon destruction, this class terminates the corresponding docker server.
- Parameters:
image (str) – Docker image name of grpc4bmi wrapped model
image_port (int) – Port of server inside the image
host (str) – Host on which the image port is published on a random port
input_dirs (Iterable[str]) –
Directories for input files of model.
All of directories will be mounted read-only inside Docker container on same path as outside container.
work_dir (str) –
Working directory for model.
Directory is mounted inside container and changed into.
user (str) – Username or UID of Docker container. Defaults to own UID.
remove (bool) –
Automatically remove the container and logs when it exits.
Enable to get logs when container dies prematurely.
delay (int) – Seconds to wait for Docker container to startup, before connecting to it
timeout (int) – Seconds to wait for gRPC client to connect to server
extra_volumes (Dict[str,Dict]) –
Extra volumes to attach to Docker container. The key is either the hosts path or a volume name and the value is a dictionary with the keys:
bindThe path to mount the volume inside the containermodeEitherrwto mount the volume read/write, orroto mount it read-only.
For example:
{'/data/shared/forcings/': {'bind': '/forcings', 'mode': 'ro'}}
Increase when container takes a long time to startup.
timeout –
Seconds to wait for gRPC client to connect to server.
By default will try forever to connect to gRPC server inside container. Set to low number to escape endless wait.
See
grpc4bmi.bmi_client_apptainer.BmiClientApptainerfor examples using input_dirs and work_dir.
grpc4bmi.bmi_client_singularity module
- class grpc4bmi.bmi_client_singularity.BmiClientSingularity(image: str, work_dir: str, input_dirs: Iterable[str] = (), delay=0, timeout=None, capture_logs=True)[source]
Bases:
BmiClientBMI GRPC client for singularity server processes During initialization launches a singularity container with run-bmi-server as its command. The client picks a random port and expects the container to run the server on that port. The port is passed to the container using the BMI_PORT environment variable.
- Parameters:
image –
Singularity image.
For Docker Hub image use docker://* or convert it to a Singularity image file.
To convert Docker image ewatercycle/walrus-grpc4bmi with v0.3.1 tag to ./ewatercycle-walrus-grpc4bmi_v0.3.1.sif Singularity image file use:
singularity pull ewatercycle-walrus-grpc4bmi_v0.3.1.sif \ docker://ewatercycle/walrus-grpc4bmi:v0.3.1
input_dirs (Iterable[str]) –
Directories for input files of model.
All of directories will be mounted read-only inside Singularity container on same path as outside container.
work_dir (str) –
Working directory for model.
Directory is mounted inside container and changed into.
To create a random work directory you could use
from tempfile import TemporaryDirectory from grpc4bmi.bmi_client_singularity import BmiClientSingularity work_dir = TemporaryDirectory() image = 'ewatercycle-walrus-grpc4bmi_v0.2.0.sif' client = BmiClientSingularity(image, work_dir.name) # Write config to work_dir and interact with client # After checking output in work_dir, clean up work_dir.cleanup()
delay (int) –
Seconds to wait for Singularity container to startup, before connecting to it
Increase when container takes a long time to startup.
timeout (int) –
Seconds to wait for gRPC client to connect to server.
By default will try forever to connect to gRPC server inside container. Set to low number to escape endless wait.
capture_logs (bool) –
Whether to capture stdout and stderr of container .
If false then redirects output to null device never to be seen again. If true then redirects output to temporary file which can be read with
BmiClientSingularity.logs(). The temporary file gets removed when this object is deleted.
Example 1: Config file already inside image
MARRMoT has an example config file inside its Docker image.
from grpc4bmi.bmi_client_singularity import BmiClientSingularity client = BmiClientSingularity(image='docker://ewatercycle/marrmot-grpc4bmi:latest', work_dir='/opt/MARRMoT/BMI/Config')) client.initialize('/opt/MARRMoT/BMI/Config/BMI_testcase_m01_BuffaloRiver_TN_USA.mat') client.update_until(client.get_end_time()) del client
Example 2: Config file in input directory
The config file and all other files the model needs are in a directory (/tmp/input). Use /tmp/work to capture any output files like logs generated by model.
from grpc4bmi.bmi_client_singularity import BmiClientSingularity # Generate config file called 'config.mat' in `/tmp/input` directory client = BmiClientSingularity(image='docker://ewatercycle/marrmot-grpc4bmi:latest', input_dirs=['/tmp/input'], work_dir='/tmp/work') client.initialize('/tmp/input/config.mat') client.update_until(client.get_end_time()) del client
Example 3: Read only input directory with config file in work directory
The forcing data is in a shared read-only location like /shared/forcings/walrus. In the config file (/tmp/work/walrus.yml) point to a forcing data file (/shared/forcings/walrus/PEQ_Hupsel.dat).
from grpc4bmi.bmi_client_singularity import BmiClientSingularity client = BmiClientSingularity(image='ewatercycle-walrus-grpc4bmi_v0.2.0.sif', input_dirs=['/shared/forcings/walrus'], work_dir='/tmp/work') client.initialize('walrus.yml') client.update_until(client.get_end_time()) del client
Example 4: Model writes in sub directory of input directory
Some models, for example wflow, write output in a sub-directory of the input directory. If the input directory is set with the input_dirs argument then the model will be unable to write its output as input directories are mounted read-only. That will most likely cause the model to die. A workaround is to use the work_dir argument with input directory as value instead. This will make the whole input directory writable so the model can do its thing.
When input directory is on a shared disk where you do not have write permission then the input dir should be copied to a work directory (/scratch/wflow) so model can write.
from grpc4bmi.bmi_client_singularity import BmiClientSingularity client = BmiClientSingularity(image='docker://ewatercycle/wflow-grpc4bmi:latest', work_dir='/scratch/wflow') client.initialize('wflow_sbm.ini') client.update_until(client.get_end_time()) del client
Example 5: Inputs are in multiple directories
A model has its forcings (/shared/forcings/muese), parameters (/shared/model/wflow/staticmaps) and config file (/tmp/work/wflow_sbm.ini) in different locations. The config file should be set to point to the forcing and parameters files.
from grpc4bmi.bmi_client_singularity import BmiClientSingularity client = BmiClientSingularity(image='docker://ewatercycle/wflow-grpc4bmi:latest', input_dirs=['/shared/forcings/muese', '/shared/model/wflow/staticmaps'], work_dir='/tmp/work') client.initialize('wflow_sbm.ini') client.update_until(client.get_end_time()) del client
Example 6: Run model twice with their own work directory
While running 2 or models at the same time you do not want the any config or output to be mixed.
from grpc4bmi.bmi_client_singularity import BmiClientSingularity client_muese = BmiClientSingularity(image='docker://ewatercycle/wflow-grpc4bmi:latest', work_dir='/scratch/wflow-meuse') client_muese.initialize('wflow_sbm.meuse.ini') client_rhine = BmiClientSingularity(image='docker://ewatercycle/wflow-grpc4bmi:latest', work_dir='/scratch/wflow-rhine') client_rhine.initialize('wflow_sbm.rhine.ini') ... # Run models and set/get values ... del client_muese del client_rhine
grpc4bmi.bmi_client_subproc module
- class grpc4bmi.bmi_client_subproc.BmiClientSubProcess(module_name, path=None, timeout=None, delay=1)[source]
Bases:
BmiClientBMI GRPC client that owns its server process, i.e. initiates and destroys the BMI server upon its own construction or respective destruction. The server is a forked subprocess running the run_server command.
>>> from grpc4bmi.bmi_client_subproc import BmiClientSubProcess >>> mymodel = BmiClientSubProcess(<PACKAGE>.<MODULE>.<CLASS>)
grpc4bmi.bmi_grpc_client module
- class grpc4bmi.bmi_grpc_client.BmiClient(channel=None, timeout=None, stub=None)[source]
Bases:
BmiClient BMI interface, implementing BMI by forwarding every function call via GRPC to the server connected to the same port. A GRPC channel can be passed to the constructor; if not, it constructs an insecure channel on a free port itself. The timeout parameter indicates the model BMI startup timeout parameter (s).
>>> import grpc >>> from grpc4bmi.bmi_grpc_client import BmiClient >>> mymodel = BmiClient(grpc.insecure_channel("localhost:<PORT>")) >>> print(mymodel.get_component_name()) Hello world
- finalize()[source]
Perform tear-down tasks for the model.
Perform all tasks that take place after exiting the model’s time loop. This typically includes deallocating memory, closing files and printing reports.
- get_component_name()[source]
Name of the component.
- Returns:
The name of the component.
- Return type:
- get_current_time()[source]
Return the current time of the model.
- Returns:
The current model time.
- Return type:
- get_grid_edge_nodes(grid: int, edge_nodes: ndarray) ndarray[source]
Get the edge-node connectivity.
- Parameters:
grid (int) – A grid identifier.
edge_nodes (ndarray of int, shape (2 x nnodes,)) – A numpy array to place the edge-node connectivity. For each edge, connectivity is given as node at edge tail, followed by node at edge head.
- Returns:
The input numpy array that holds the edge-node connectivity.
- Return type:
ndarray of int
- get_grid_face_edges(grid: int, face_edges: ndarray) ndarray[source]
Get the face-edge connectivity.
- get_grid_face_nodes(grid: int, face_nodes: ndarray) ndarray[source]
Get the face-node connectivity.
- Parameters:
- Returns:
The input numpy array that holds the face-node connectivity.
- Return type:
ndarray of int
- get_grid_nodes_per_face(grid: int, nodes_per_face: ndarray) ndarray[source]
Get the number of nodes for each face.
- get_grid_origin(grid, origin)[source]
Get coordinates for the lower-left corner of the computational grid.
- get_input_item_count() int[source]
Count of a model’s input variables.
- Returns:
The number of input variables.
- Return type:
- get_input_var_names()[source]
List of a model’s input variables.
Input variable names must be CSDMS Standard Names, also known as long variable names.
Notes
Standard Names enable the CSDMS framework to determine whether an input variable in one model is equivalent to, or compatible with, an output variable in another model. This allows the framework to automatically connect components.
Standard Names do not have to be used within the model.
- get_output_item_count() int[source]
Count of a model’s output variables.
- Returns:
The number of output variables.
- Return type:
- get_output_var_names()[source]
List of a model’s output variables.
Output variable names must be CSDMS Standard Names, also known as long variable names.
- get_start_time()[source]
Start time of the model.
Model times should be of type float.
- Returns:
The model start time.
- Return type:
- get_time_step()[source]
Return the current time step of the model.
The model time step should be of type float.
- Returns:
The time step used in model.
- Return type:
- get_time_units()[source]
Time units of the model.
- Returns:
The model time unit; e.g., days or s.
- Return type:
Notes
CSDMS uses the UDUNITS standard from Unidata.
- get_value(name, dest)[source]
Get a copy of values of the given variable.
This is a getter for the model, used to access the model’s current state. It returns a copy of a model variable, with the return type, size and rank dependent on the variable.
- Parameters:
name (str) – An input or output variable name, a CSDMS Standard Name.
dest (ndarray) – A numpy array into which to place the values.
- Returns:
The same numpy array that was passed as an input buffer.
- Return type:
ndarray
- get_value_at_indices(name, dest, indices)[source]
Get values at particular indices.
- Parameters:
name (str) – An input or output variable name, a CSDMS Standard Name.
dest (ndarray) – A numpy array into which to place the values.
inds (array_like) – The indices into the variable array.
- Returns:
Value of the model variable at the given location.
- Return type:
array_like
- get_value_ptr(name: str) ndarray[source]
Not possible, unable give reference to data structure in another process and possibly another machine
- get_var_location(name: str) str[source]
Get the grid element type that the a given variable is defined on.
The grid topology can be composed of nodes, edges, and faces.
- node
A point that has a coordinate pair or triplet: the most basic element of the topology.
- edge
A line or curve bounded by two nodes.
- face
A plane or surface enclosed by a set of edges. In a 2D horizontal application one may consider the word “polygon”, but in the hierarchy of elements the word “face” is most common.
- Parameters:
name (str) – An input or output variable name, a CSDMS Standard Name.
- Returns:
The grid location on which the variable is defined. Must be one of “node”, “edge”, or “face”.
- Return type:
Notes
CSDMS uses the ugrid conventions to define unstructured grids.
- get_var_units(name)[source]
Get units of the given variable.
Standard unit names, in lower case, should be used, such as
metersorseconds. Standard abbreviations, likemfor meters, are also supported. For variables with compound units, each unit name is separated by a single space, with exponents other than 1 placed immediately after the name, as inm s-1for velocity,W m-2for an energy flux, orkm2for an area.- Parameters:
name (str) – An input or output variable name, a CSDMS Standard Name.
- Returns:
The variable units.
- Return type:
Notes
CSDMS uses the UDUNITS standard from Unidata.
- initialize(filename: str | None)[source]
Perform startup tasks for the model.
Perform all tasks that take place before entering the model’s time loop, including opening files and initializing the model state. Model inputs are read from a text-based configuration file, specified by config_file.
- Parameters:
config_file (str, optional) – The path to the model configuration file.
Notes
Models should be refactored, if necessary, to use a configuration file. CSDMS does not impose any constraint on how configuration files are formatted, although YAML is recommended. A template of a model’s configuration file with placeholder values is used by the BMI.
- set_value(name, values)[source]
Specify a new value for a model variable.
This is the setter for the model, used to change the model’s current state. It accepts, through src, a new value for a model variable, with the type, size and rank of src dependent on the variable.
- Parameters:
name (str) – An input or output variable name, a CSDMS Standard Name.
src (array_like) – The new value for the specified variable.
- set_value_at_indices(name, inds, src)[source]
Specify a new value for a model variable at particular indices.
- Parameters:
name (str) – An input or output variable name, a CSDMS Standard Name.
inds (array_like) – The indices into the variable array.
src (array_like) – The new value for the specified variable.
- update()[source]
Advance model state by one time step.
Perform all tasks that take place within one pass through the model’s time loop. This typically includes incrementing all of the model’s state variables. If the model’s state variables don’t change in time, then they can be computed by the
initialize()method and this method can return with no action.
- grpc4bmi.bmi_grpc_client.handle_error(exc)[source]
Parsers DebugInfo (https://github.com/googleapis/googleapis/blob/07244bb797ddd6e0c1c15b02b4467a9a5729299f/google/rpc/error_details.proto#L46-L52) from the trailing metadata of a grpc.RpcError
- Parameters:
exc (grpc.RpcError) – Exception to handle
Raises: original exception or RemoteException
grpc4bmi.bmi_grpc_legacy_server module
- class grpc4bmi.bmi_grpc_legacy_server.BmiLegacyServer02(model, debug=False)[source]
Bases:
BmiServiceServicerBMI Server class, wrapping an existing python implementation and exposing it via GRPC across the memory space (to listening client processes). The class takes a package, module and class name and instantiates the BMI implementation by assuming a default constructor with no arguments.
For models implementing the bmi interface defined https://pypi.org/project/basic-modeling-interface/0.2/
- Parameters:
model – Bmi model object which must be wrapped by grpc
debug – If true then returns stacktrace in an error response. The stacktrace is returned in the trailing metadata as a DebugInfo (https://github.com/googleapis/googleapis/blob/07244bb797ddd6e0c1c15b02b4467a9a5729299f/google/rpc/error_details.proto#L46-L52) message.
- getComponentName(request, context)[source]
Missing associated documentation comment in .proto file.
- getInputVarNames(request, context)[source]
Missing associated documentation comment in .proto file.
- getOutputVarNames(request, context)[source]
Missing associated documentation comment in .proto file.
- getValueAtIndices(request, context)[source]
Missing associated documentation comment in .proto file.
grpc4bmi.bmi_grpc_server module
- class grpc4bmi.bmi_grpc_server.BmiServer(model: BmiServer, debug: Bmi = False)[source]
Bases:
BmiServiceServicerBMI Server class, wrapping an existing python implementation and exposing it via GRPC across the memory space (to listening client processes). The class takes a package, module and class name and instantiates the BMI implementation by assuming a default constructor with no arguments.
- Parameters:
model – Bmi model object which must be wrapped by grpc
debug – If true then returns stacktrace in an error response. The stacktrace is returned in the trailing metadata as a DebugInfo (https://github.com/googleapis/googleapis/blob/07244bb797ddd6e0c1c15b02b4467a9a5729299f/google/rpc/error_details.proto#L46-L52) message.
- getComponentName(request, context)[source]
Missing associated documentation comment in .proto file.
- getGridEdgeCount(request, context)[source]
Missing associated documentation comment in .proto file.
- getGridEdgeNodes(request, context)[source]
Missing associated documentation comment in .proto file.
- getGridFaceCount(request, context)[source]
Missing associated documentation comment in .proto file.
- getGridFaceEdges(request, context)[source]
Missing associated documentation comment in .proto file.
- getGridFaceNodes(request, context)[source]
Missing associated documentation comment in .proto file.
- getGridNodeCount(request, context)[source]
Missing associated documentation comment in .proto file.
- getGridNodesPerFace(request, context)[source]
Missing associated documentation comment in .proto file.
- getInputItemCount(request, context)[source]
Missing associated documentation comment in .proto file.
- getInputVarNames(request, context)[source]
Missing associated documentation comment in .proto file.
- getOutputItemCount(request, context)[source]
Missing associated documentation comment in .proto file.
- getOutputVarNames(request, context)[source]
Missing associated documentation comment in .proto file.
- getValueAtIndices(request, context)[source]
Missing associated documentation comment in .proto file.
grpc4bmi.bmi_r_model module
grpc4bmi.reserve module
Helpers to reserve numpy arrays for use in some of the Bmi methods as output argument
- grpc4bmi.reserve.reserve_grid_edge_nodes(model: Bmi, grid_id: int) ndarray[source]
Reserve edge_nodes for
bmipy.Bmi.get_grid_edge_nodes()
- grpc4bmi.reserve.reserve_grid_face_(model: Bmi, grid_id: int) ndarray[source]
Reserve face_edges or face_node in respectivly
bmipy.Bmi.get_grid_face_edges()orbmipy.Bmi.get_grid_face_nodes()
- grpc4bmi.reserve.reserve_grid_nodes(model: Bmi, grid_id: int, dim_index: int) ndarray[source]
Reserve dest for
bmipy.Bmi.get_grid_x(),bmipy.Bmi.get_grid_y()andbmipy.Bmi.get_grid_z()The dim_index goes x,y,z and model.get_grid_shape goes z,y,x or y,x so index is inverted
- grpc4bmi.reserve.reserve_grid_nodes_per_face(model: Bmi, grid_id: int) ndarray[source]
Reserve nodes_per_face for
bmipy.Bmi.get_grid_nodes_per_face()
- grpc4bmi.reserve.reserve_grid_padding(model: Bmi, grid_id: int) ndarray[source]
Reserve dest for
bmipy.Bmi.get_grid_spacing()andbmipy.Bmi.get_grid_origin()
- grpc4bmi.reserve.reserve_grid_shape(model: Bmi, grid_id: int) ndarray[source]
Reserve shape for
bmipy.Bmi.get_grid_shape()