grpc4bmi package

Submodules

grpc4bmi.bmi_client_docker module

class grpc4bmi.bmi_client_docker.BmiClientDocker(image, image_port=55555, host=None, input_dir=None, output_dir=None, user=1005, remove=False, delay=5, timeout=None, extra_volumes=None)

Bases: grpc4bmi.bmi_grpc_client.BmiClient

BMI gRPC client for dockerized server processes: the initialization launches the docker container which should have the run-bmi-server as its command. Also, it should expose the tcp port 55555 for communication with this client. Upon destruction, this class terminates the corresponding docker server.

Parameters:

• image (str) – Docker image name of grpc4bmi wrapped model
• image_port (int) – Port of server inside the image
• host (str) – Host on which the image port is published on a random port
• input_dir (str) – Directory for input files of model
• output_dir (str) – Directory for input files of model
• user (str) – Username or UID of Docker container
• remove (bool) – Automatically remove the container and logs when it exits.
• delay (int) – Seconds to wait for Docker container to startup, before connecting to it
• timeout (int) – Seconds to wait for gRPC client to connect to server
• extra_volumes (Dict[str,Dict]) –

  Extra volumes to attach to Docker container. The key is either the hosts path or a volume name and the value is a dictionary with the keys:

  • bind The path to mount the volume inside the container
  • mode Either rw to mount the volume read/write, or ro to mount it read-only.

  For example:

  {'/data/shared/forcings/': {'bind': '/forcings', 'mode': 'ro'}}
  

get_value_ptr(var_name)

Not possible, unable give reference to data structure in another process and possibly another machine

initialize(filename)

Perform startup tasks for the model.

Perform all tasks that take place before entering the model’s time loop, including opening files and initializing the model state. Model inputs are read from a text-based configuration file, specified by filename.

Parameters:config_file (str, optional) – The path to the model configuration file.

Notes

Models should be refactored, if necessary, to use a configuration file. CSDMS does not impose any constraint on how configuration files are formatted, although YAML is recommended. A template of a model’s configuration file with placeholder values is used by the BMI.

input_mount_point = '/data/input'

logs()

Logs of the Docker container

output_mount_point = '/data/output'

exception grpc4bmi.bmi_client_docker.DeadDockerContainerException(message, exitcode, logs, *args)

Bases: ChildProcessError

Exception for when a Docker container has died.

Parameters:

• message (str) – Human readable error message
• exitcode (int) – The non-zero exit code of the container
• logs (str) – Logs the container produced

exitcode = None

Exit code of container

logs = None

Stdout and stderr of container

exception grpc4bmi.bmi_client_docker.LogsException

Bases: Exception

grpc4bmi.bmi_client_singularity module

class grpc4bmi.bmi_client_singularity.BmiClientSingularity(image, input_dir=None, output_dir=None, timeout=None, extra_volumes=None)

Bases: grpc4bmi.bmi_grpc_client.BmiClient

BMI GRPC client for singularity server processes During initialization launches a singularity container with run-bmi-server as its command. The client picks a random port and expects the container to run the server on that port. The port is passed to the container using the BMI_PORT environment variable.

>>> from grpc4bmi.bmi_client_singularity import BmiClientSingularity
>>> image = 'docker://ewatercycle/wflow-grpc4bmi:latest'
>>> client = BmiClientSingularity(image, input_dir='wflow_rhine_sbm', output_dir='wflow_output')
>>> client.initialize('wflow_rhine_sbm/wflow_sbm.ini')
>>> client.update_until(client.get_end_time())
>>> del client

Parameters:

• image – Singularity image. For Docker Hub image use docker://*.
• input_dir (str) – Directory for input files of model
• output_dir (str) – Directory for input files of model
• timeout (int) – Seconds to wait for gRPC client to connect to server
• extra_volumes (Dict[str,str]) –

  Extra volumes to attach to Singularity container.

  The key is the hosts path and the value the mounted volume inside the container. Contrary to Docker client, extra volumes are always read/write

  For example:

  {'/data/shared/forcings/': '/data/forcings'}
  

INPUT_MOUNT_POINT = '/data/input'

OUTPUT_MOUNT_POINT = '/data/output'

get_value_ptr(var_name)

Not possible, unable give reference to data structure in another process and possibly another machine

initialize(filename)

Perform startup tasks for the model.

Perform all tasks that take place before entering the model’s time loop, including opening files and initializing the model state. Model inputs are read from a text-based configuration file, specified by filename.

Parameters:config_file (str, optional) – The path to the model configuration file.

Notes

Models should be refactored, if necessary, to use a configuration file. CSDMS does not impose any constraint on how configuration files are formatted, although YAML is recommended. A template of a model’s configuration file with placeholder values is used by the BMI.

grpc4bmi.bmi_client_singularity.check_singularity_version()

grpc4bmi.bmi_client_subproc module

class grpc4bmi.bmi_client_subproc.BmiClientSubProcess(module_name, path=None, timeout=None)

Bases: grpc4bmi.bmi_grpc_client.BmiClient

BMI GRPC client that owns its server process, i.e. initiates and destroys the BMI server upon its own construction or respective destruction. The server is a forked subprocess running the run_server command.

>>> from grpc4bmi.bmi_client_subproc import BmiClientSubProcess
>>> mymodel = BmiClientSubProcess(<PACKAGE>.<MODULE>.<CLASS>)

get_value_ref(var_name)

grpc4bmi.bmi_grpc_client module

class grpc4bmi.bmi_grpc_client.BmiClient(channel=None, timeout=None, stub=None)

Bases: bmipy.bmi.Bmi

Client BMI interface, implementing BMI by forwarding every function call via GRPC to the server connected to the same port. A GRPC channel can be passed to the constructor; if not, it constructs an insecure channel on a free port itself. The timeout parameter indicates the model BMI startup timeout parameter (s).

>>> import grpc
>>> from grpc4bmi.bmi_grpc_client import BmiClient
>>> mymodel = BmiClient(grpc.insecure_channel("localhost:<PORT>"))
>>> print(mymodel.get_component_name())
Hello world

static create_grpc_channel(port=0, host=None)

finalize()

Perform tear-down tasks for the model.

Perform all tasks that take place after exiting the model’s time loop. This typically includes deallocating memory, closing files and printing reports.

get_component_name()

Name of the component.

Returns:The name of the component. Return type:str

get_current_time()

Current time of the model.

Returns:The current model time. Return type:float

get_end_time()

End time of the model.

Returns:The maximum model time. Return type:float

get_grid_edge_count(grid: int) → int

Get the number of edges in the grid.

Parameters:grid (int) – A grid identifier. Returns:The total number of grid edges. Return type:int

get_grid_edge_nodes(grid: int, edge_nodes: numpy.ndarray) → numpy.ndarray

Get the edge-node connectivity.

Parameters:

• grid (int) – A grid identifier.
• edge_nodes (ndarray of int, shape (2 x nnodes,)) – A numpy array to place the edge-node connectivity. For each edge, connectivity is given as node at edge tail, followed by node at edge head.

Returns:

The input numpy array that holds the edge-node connectivity.

Return type:

ndarray of int

get_grid_face_count(grid: int) → int

Get the number of faces in the grid.

Parameters:grid (int) – A grid identifier. Returns:The total number of grid faces. Return type:int

get_grid_face_edges(grid: int, face_edges: numpy.ndarray) → numpy.ndarray

Get the face-edge connectivity.

Parameters:

• grid (int) – A grid identifier.
• face_edges (ndarray of int) – A numpy array to place the face-edge connectivity.

Returns:

The input numpy array that holds the face-edge connectivity.

Return type:

ndarray of int

get_grid_face_nodes(grid: int, face_nodes: numpy.ndarray) → numpy.ndarray

Get the face-node connectivity.

Parameters:

• grid (int) – A grid identifier.
• face_nodes (ndarray of int) – A numpy array to place the face-node connectivity. For each face, the nodes (listed in a counter-clockwise direction) that form the boundary of the face.

Returns:

The input numpy array that holds the face-node connectivity.

Return type:

ndarray of int

get_grid_node_count(grid: int) → int

Get the number of nodes in the grid.

Parameters:grid (int) – A grid identifier. Returns:The total number of grid nodes. Return type:int

get_grid_nodes_per_face(grid: int, nodes_per_face: numpy.ndarray) → numpy.ndarray

Get the number of nodes for each face.

Parameters:

• grid (int) – A grid identifier.
• nodes_per_face (ndarray of int, shape (nfaces,)) – A numpy array to place the number of edges per face.

Returns:

The input numpy array that holds the number of nodes per edge.

Return type:

ndarray of int

get_grid_origin(grid, origin)

Get coordinates for the lower-left corner of the computational grid.

Parameters:

• grid (int) – A grid identifier.
• origin (ndarray of float, shape (ndim,)) – A numpy array to hold the coordinates of the lower-left corner of the grid.

Returns:

The input numpy array that holds the coordinates of the grid’s lower-left corner.

Return type:

ndarray of float

get_grid_rank(grid)

Get number of dimensions of the computational grid.

Parameters:grid (int) – A grid identifier. Returns:Rank of the grid. Return type:int

get_grid_shape(grid, shape)

Get dimensions of the computational grid.

Parameters:

• grid (int) – A grid identifier.
• shape (ndarray of int, shape (ndim,)) – A numpy array into which to place the shape of the grid.

Returns:

The input numpy array that holds the grid’s shape.

Return type:

ndarray of int

get_grid_size(grid)

Get the total number of elements in the computational grid.

Parameters:grid (int) – A grid identifier. Returns:Size of the grid. Return type:int

get_grid_spacing(grid, spacing)

Get distance between nodes of the computational grid.

Parameters:

• grid (int) – A grid identifier.
• spacing (ndarray of float, shape (ndim,)) – A numpy array to hold the spacing between grid rows and columns.

Returns:

The input numpy array that holds the grid’s spacing.

Return type:

ndarray of float

get_grid_type(grid)

Get the grid type as a string.

Parameters:grid (int) – A grid identifier. Returns:Type of grid as a string. Return type:str

get_grid_x(grid, x)

Get coordinates of grid nodes in the x direction.

Parameters:

• grid (int) – A grid identifier.
• x (ndarray of float, shape (nrows,)) – A numpy array to hold the x-coordinates of the grid node columns.

Returns:

The input numpy array that holds the grid’s column x-coordinates.

Return type:

ndarray of float

get_grid_y(grid, y)

Get coordinates of grid nodes in the y direction.

Parameters:

• grid (int) – A grid identifier.
• y (ndarray of float, shape (ncols,)) – A numpy array to hold the y-coordinates of the grid node rows.

Returns:

The input numpy array that holds the grid’s row y-coordinates.

Return type:

ndarray of float

get_grid_z(grid, z)

Get coordinates of grid nodes in the z direction.

Parameters:

• grid (int) – A grid identifier.
• z (ndarray of float, shape (nlayers,)) – A numpy array to hold the z-coordinates of the grid nodes layers.

Returns:

The input numpy array that holds the grid’s layer z-coordinates.

Return type:

ndarray of float

get_input_item_count() → int

Count of a model’s input variables.

Returns:The number of input variables. Return type:int

get_input_var_names()

List of a model’s input variables.

Input variable names must be CSDMS Standard Names, also known as long variable names.

Returns:The input variables for the model. Return type:list of str

Notes

Standard Names enable the CSDMS framework to determine whether an input variable in one model is equivalent to, or compatible with, an output variable in another model. This allows the framework to automatically connect components.

Standard Names do not have to be used within the model.

get_output_item_count() → int

Count of a model’s output variables.

Returns:The number of output variables. Return type:int

get_output_var_names()

List of a model’s output variables.

Output variable names must be CSDMS Standard Names, also known as long variable names.

Returns:The output variables for the model. Return type:list of str

get_start_time()

Start time of the model.

Model times should be of type float.

Returns:The model start time. Return type:float

get_time_step()

Current time step of the model.

The model time step should be of type float.

Returns:The time step used in model. Return type:float

get_time_units()

Time units of the model.

Returns:The model time unit; e.g., days or s. Return type:float

Notes

CSDMS uses the UDUNITS standard from Unidata.

static get_unique_port(host=None)

get_value(name, dest)

Get a copy of values of the given variable.

This is a getter for the model, used to access the model’s current state. It returns a copy of a model variable, with the return type, size and rank dependent on the variable.

Parameters:

• name (str) – An input or output variable name, a CSDMS Standard Name.
• dest (ndarray) – A numpy array into which to place the values.

Returns:

The same numpy array that was passed as an input buffer.

Return type:

ndarray

get_value_at_indices(name, dest, indices)

Get values at particular indices.

Parameters:

• name (str) – An input or output variable name, a CSDMS Standard Name.
• dest (ndarray) – A numpy array into which to place the values.
• indices (array_like) – The indices into the variable array.

Returns:

Value of the model variable at the given location.

Return type:

array_like

get_value_ptr(name: str) → numpy.ndarray

Not possible, unable give reference to data structure in another process and possibly another machine

get_var_grid(name)

Get grid identifier for the given variable.

Parameters:name (str) – An input or output variable name, a CSDMS Standard Name. Returns:The grid identifier. Return type:int

get_var_itemsize(name)

Get memory use for each array element in bytes.

Parameters:name (str) – An input or output variable name, a CSDMS Standard Name. Returns:Item size in bytes. Return type:int

get_var_location(name: str) → str

Get the grid element type that the a given variable is defined on.

The grid topology can be composed of nodes, edges, and faces.

node

A point that has a coordinate pair or triplet: the most basic element of the topology.

edge

A line or curve bounded by two nodes.

face

A plane or surface enclosed by a set of edges. In a 2D horizontal application one may consider the word “polygon”, but in the hierarchy of elements the word “face” is most common.

Parameters:name (str) – An input or output variable name, a CSDMS Standard Name. Returns:The grid location on which the variable is defined. Must be one of “node”, “edge”, or “face”. Return type:str

Notes

CSDMS uses the ugrid conventions to define unstructured grids.

get_var_nbytes(name)

Get size, in bytes, of the given variable.

Parameters:name (str) – An input or output variable name, a CSDMS Standard Name. Returns:The size of the variable, counted in bytes. Return type:int

get_var_type(name)

Get data type of the given variable.

Parameters:name (str) – An input or output variable name, a CSDMS Standard Name. Returns:The Python variable type; e.g., str, int, float. Return type:str

get_var_units(name)

Get units of the given variable.

Standard unit names, in lower case, should be used, such as meters or seconds. Standard abbreviations, like m for meters, are also supported. For variables with compound units, each unit name is separated by a single space, with exponents other than 1 placed immediately after the name, as in m s-1 for velocity, W m-2 for an energy flux, or km2 for an area.

Parameters:name (str) – An input or output variable name, a CSDMS Standard Name. Returns:The variable units. Return type:str

Notes

CSDMS uses the UDUNITS standard from Unidata.

initialize(filename)

Perform startup tasks for the model.

Perform all tasks that take place before entering the model’s time loop, including opening files and initializing the model state. Model inputs are read from a text-based configuration file, specified by filename.

Parameters:config_file (str, optional) – The path to the model configuration file.

Notes

Models should be refactored, if necessary, to use a configuration file. CSDMS does not impose any constraint on how configuration files are formatted, although YAML is recommended. A template of a model’s configuration file with placeholder values is used by the BMI.

static make_array(response)

set_value(name, values)

Specify a new value for a model variable.

This is the setter for the model, used to change the model’s current state. It accepts, through src, a new value for a model variable, with the type, size and rank of src dependent on the variable.

Parameters:

• var_name (str) – An input or output variable name, a CSDMS Standard Name.
• src (array_like) – The new value for the specified variable.

set_value_at_indices(name, inds, src)

Specify a new value for a model variable at particular indices.

Parameters:

• var_name (str) – An input or output variable name, a CSDMS Standard Name.
• indices (array_like) – The indices into the variable array.
• src (array_like) – The new value for the specified variable.

update()

Advance model state by one time step.

Perform all tasks that take place within one pass through the model’s time loop. This typically includes incrementing all of the model’s state variables. If the model’s state variables don’t change in time, then they can be computed by the initialize() method and this method can return with no action.

update_until(time: float) → None

Advance model state until the given time.

Parameters:time (float) – A model time later than the current model time.

exception grpc4bmi.bmi_grpc_client.RemoteException(message, tb)

Bases: grpc.RpcError

grpc4bmi.bmi_grpc_client.handle_error(exc)

Parsers DebugInfo (https://github.com/googleapis/googleapis/blob/07244bb797ddd6e0c1c15b02b4467a9a5729299f/google/rpc/error_details.proto#L46-L52) from the trailing metadata of a grpc.RpcError

Parameters:exc (grpc.RpcError) – Exception to handle

Raises: original exception or RemoteException

grpc4bmi.bmi_grpc_legacy_server module

class grpc4bmi.bmi_grpc_legacy_server.BmiLegacyServer02(model, debug=False)

Bases: grpc4bmi.bmi_pb2_grpc.BmiServiceServicer

BMI Server class, wrapping an existing python implementation and exposing it via GRPC across the memory space (to listening client processes). The class takes a package, module and class name and instantiates the BMI implementation by assuming a default constructor with no arguments.

For models implementing the bmi interface defined https://pypi.org/project/basic-modeling-interface/0.2/

Parameters:

• model – Bmi model object which must be wrapped by grpc
• debug – If true then returns stacktrace in an error response. The stacktrace is returned in the trailing metadata as a DebugInfo (https://github.com/googleapis/googleapis/blob/07244bb797ddd6e0c1c15b02b4467a9a5729299f/google/rpc/error_details.proto#L46-L52) message.

exception_handler(exc, context)

finalize(request, context)

getComponentName(request, context)

getCurrentTime(request, context)

getEndTime(request, context)

getGridOrigin(request, context)

getGridRank(request, context)

getGridShape(request, context)

getGridSize(request, context)

getGridSpacing(request, context)

getGridType(request, context)

getGridX(request, context)

getGridY(request, context)

getGridZ(request, context)

getInputVarNames(request, context)

getOutputVarNames(request, context)

getStartTime(request, context)

getTimeStep(request, context)

getTimeUnits(request, context)

getValue(request, context)

getValueAtIndices(request, context)

getValuePtr(request, context)

getVarGrid(request, context)

getVarItemSize(request, context)

getVarNBytes(request, context)

getVarType(request, context)

getVarUnits(request, context)

initialize(request, context)

setValue(request, context)

setValueAtIndices(request, context)

setValuePtr(request, context)

update(request, context)

grpc4bmi.bmi_grpc_server module

class grpc4bmi.bmi_grpc_server.BmiServer(model, debug=False)

Bases: grpc4bmi.bmi_pb2_grpc.BmiServiceServicer

BMI Server class, wrapping an existing python implementation and exposing it via GRPC across the memory space (to listening client processes). The class takes a package, module and class name and instantiates the BMI implementation by assuming a default constructor with no arguments.

Parameters:

• model – Bmi model object which must be wrapped by grpc
• debug – If true then returns stacktrace in an error response. The stacktrace is returned in the trailing metadata as a DebugInfo (https://github.com/googleapis/googleapis/blob/07244bb797ddd6e0c1c15b02b4467a9a5729299f/google/rpc/error_details.proto#L46-L52) message.

exception_handler(exc, context)

finalize(request, context)

getComponentName(request, context)

getCurrentTime(request, context)

getEndTime(request, context)

getGridEdgeCount(request, context)

getGridEdgeNodes(request, context)

getGridFaceCount(request, context)

getGridFaceEdges(request, context)

getGridFaceNodes(request, context)

getGridNodeCount(request, context)

getGridNodesPerFace(request, context)

getGridOrigin(request, context)

getGridRank(request, context)

getGridShape(request, context)

getGridSize(request, context)

getGridSpacing(request, context)

getGridType(request, context)

getGridX(request, context)

getGridY(request, context)

getGridZ(request, context)

getInputItemCount(request, context)

getInputVarNames(request, context)

getOutputItemCount(request, context)

getOutputVarNames(request, context)

getStartTime(request, context)

getTimeStep(request, context)

getTimeUnits(request, context)

getValue(request, context)

getValueAtIndices(request, context)

getValuePtr(request, context)

getVarGrid(request, context)

getVarItemSize(request, context)

getVarLocation(request, context)

getVarNBytes(request, context)

getVarType(request, context)

getVarUnits(request, context)

initialize(request, context)

setValue(request, context)

setValueAtIndices(request, context)

update(request, context)

updateUntil(request, context)

grpc4bmi.bmi_r_model module

grpc4bmi.reserve module

Helpers to reserve numpy arrays for use in some of the Bmi methods as output argument

grpc4bmi.reserve.reserve_grid_nodes(model: bmipy.bmi.Bmi, grid_id: int, dim_index: int) → numpy.ndarray

Reserve dest for bmipy.Bmi.get_grid_x(), bmipy.Bmi.get_grid_y() and bmipy.Bmi.get_grid_z()

The dim_index goes x,y,z and model.get_grid_shape goes z,y,x or y,x so index is inverted

grpc4bmi.reserve.reserve_grid_padding(model: bmipy.bmi.Bmi, grid_id: int) → numpy.ndarray

Reserve dest for bmipy.Bmi.get_grid_spacing() and bmipy.Bmi.get_grid_origin()

grpc4bmi.reserve.reserve_grid_shape(model: bmipy.bmi.Bmi, grid_id: int) → numpy.ndarray

Reserve shape for bmipy.Bmi.get_grid_shape()

grpc4bmi.reserve.reserve_values(model: bmipy.bmi.Bmi, name: str) → numpy.ndarray

Reserve dest for bmipy.Bmi.get_value()

grpc4bmi.reserve.reserve_values_at_indices(model: bmipy.bmi.Bmi, name: str, indices) → numpy.ndarray

Reserve dest for bmipy.Bmi.get_value_at_indices()

grpc4bmi.run_server module

grpc4bmi.run_server.build(name, path)

Build a model based on it’s location and name

grpc4bmi.run_server.build_parser()

grpc4bmi.run_server.build_r(class_name, source_fn)

grpc4bmi.run_server.interrupt(signum, frame)

grpc4bmi.run_server.main(argv=['-T', '-b', 'readthedocs', '-d', '_build/doctrees-readthedocs', '-D', 'language=en', '.', '_build/html'])

grpc4bmi.run_server.serve(model, port)

grpc4bmi.utils module

grpc4bmi.utils.stage_config_file(filename, input_dir, input_mount_point, home_mounted=False)

Stage config file inside container

Parameters:

• filename (str) – Path to config file
• input_dir (str) – The input directory outside the container
• input_mount_point (str) – The input directory inside the container
• home_mounted (bool) – True if home directory is mounted inside container