visionsim.simulate package

Submodules

visionsim.simulate.blender module

visionsim.simulate.blender.require_connected_client(func: Callable[Concatenate[BlenderClient, _P], Any]) → Callable[Concatenate[BlenderClient, _P], Any]

Decorator which ensures a client is connected.

Parameters:

func (Callable[Concatenate[BlenderClient, _P], Any]) – Function to decorate

Raises:

RuntimeError – raised if client is not connected.

Returns:

Decorated function.

Return type:

Callable[Concatenate[BlenderClient, _P], Any]

visionsim.simulate.blender.require_connected_clients(func: Callable[Concatenate[BlenderClients, _P], Any]) → Callable[Concatenate[BlenderClients, _P], Any]

Decorator which ensures all clients are connected.

Parameters:

func (Callable[Concatenate[BlenderClients, _P], Any]) – Function to decorate

Raises:

RuntimeError – raised if at least one client is not connected.

Returns:

Decorated function.

Return type:

Callable[Concatenate[BlenderClients, _P], Any]

visionsim.simulate.blender.require_initialized_service(func: Callable[Concatenate[BlenderService, _P], Any]) → Callable[Concatenate[BlenderService, _P], Any]

Decorator which ensures the render service was initialized.

Parameters:

func (Callable[Concatenate[BlenderService, _P], Any]) – Function to decorate

Raises:

RuntimeError – raised if client.initialize has not been previously called.

Returns:

Decorated function.

Return type:

Callable[Concatenate[BlenderService, _P], Any]

visionsim.simulate.blender.validate_camera_moved(func: Callable[Concatenate[BlenderService, _P], Any]) → Callable[Concatenate[BlenderService, _P], Any]

Decorator which emits a warning if the camera was not moved.

Parameters:

func (Callable[Concatenate[BlenderService, _P], Any]) – Function to decorate

Returns:

Decorated function.

Return type:

Callable[Concatenate[BlenderService, _P], Any]

class visionsim.simulate.blender.BlenderServer(hostname: bytes | str | None = None, port: bytes | str | int | None = 0, service: type[BlenderService] | None = None, extra_config: dict | None = None, **kwargs)

Bases: Server

Expose a BlenderService to the outside world via RPCs.

Example

Once created, it can be started, which will block and await for an external connection from a BlenderClient:

server = BlenderServer()
server.start()

However, this needs to be called within blender’s runtime. Instead one can use BlenderServer.spawn() to spawn one or more blender instances, each with their own server.

__init__(hostname: bytes | str | None = None, port: bytes | str | int | None = 0, service: type[BlenderService] | None = None, extra_config: dict | None = None, **kwargs) → None

Initialize a BlenderServer instance

Parameters:

• hostname (bytes | str | None, optional) – the host to bind to. By default, the ‘wildcard address’ is used to listen on all interfaces. If not properly secured, the server can receive traffic from unintended or even malicious sources. Defaults to None (wildcard).

• port (bytes | str | int | None, optional) – the TCP port to bind to. Defaults to 0 (bind to a random open port).

• service (type[BlenderService], optional) – the service to expose, must be a BlenderService subclass. Defaults to BlenderService.

• extra_config (dict, optional) – the configuration dictionary that is passed to the RPyC connection. Defaults to {"allow_all_attrs": True, "allow_setattr": True}.

• **kwargs – Additional keyword arguments which are passed to the rpyc.utils.server.Server constructor.

Raises:

• RuntimeError – a BlenderServer needs to be instantiated from within a blender instance.

• ValueError – the exposed service must be BlenderService or subclass.

static spawn(jobs: int = 1, timeout: float = -1.0, log_dir: str | os.PathLike | None = None, autoexec: bool = False, executable: str | os.PathLike | None = None) → Iterator[tuple[list[subprocess.Popen], list[tuple[str, int]]]]

Spawn one or more blender instances and start a BlenderServer in each.

This is roughly equivalent to calling blender -b --python blender.py in many subprocesses, where blender.py initializes and starts a server instance. Proper logging and termination of these processes is also taken care of.

Note: The returned processes and connection settings are not guaranteed to be in the same order.

Parameters:

• jobs (int, optional) – number of jobs to spawn. Defaults to 1.

• timeout (float, optional) – try to discover spawned instances for timeout (in seconds) before giving up. If negative, a port will be randomly selected and assigned to the spawned server, bypassing the need for discovery and timeouts. Note that when a port is assigned this context manager will immediately yield, even if the server is not yet ready to accept incoming connections. Defaults to assigning a port to spawned server (-1 seconds).

• log_dir (str | os.PathLike | None, optional) – path to log directory, stdout/err will be captured if set, otherwise outputs will go to os.devnull. Defaults to None (devnull).

• autoexec (bool, optional) – if true, allow execution of any embedded python scripts within blender. For more, see blender’s CLI documentation. Defaults to False.

• executable (str | os.PathLike | None, optional) – path to Blender’s executable. Defaults to looking for blender on $PATH, but is useful when targeting a specific blender install, or when it’s installed via a package manager such as flatpak. Setting it to “flatpak run –die-with-parent org.blender.Blender” might be required when using flatpaks. Defaults to None (system PATH).

Raises:

TimeoutError – raise if unable to discover spawned servers in timeout seconds and kill any spawned processes.

Yields:

tuple[list[subprocess.Popen], list[tuple[str, int]]] –

A tuple containing:

• list[subprocess.Popen]: List of subprocess.Popen corresponding to all spawned servers.

• list[tuple[str, int]]: List of connection setting for each server, where each element is a (hostname, port) tuple.

static spawn_registry() → tuple[Process, UDPRegistryClient]

Spawn a registry server and client to aid in server discovery, or return cached result. While this method can be called directly, it will be invoked automatically by discover() and spawn().

Returns:

A tuple containing:

• Process: process running the global registry server,

• rpyc.utils.registry.UDPRegistryClient: global registry client

Return type:

tuple[Process, rpyc.utils.registry.UDPRegistryClient]

static discover() → list[tuple[str, int]]

Discover any BlenderServers that are already running and return their connection parameters.

Note

A discoverable server might already be in use and can refuse connection attempts.

Returns:

List of connection setting for each server, where each element is a (hostname, port) tuple.

Return type:

list[tuple[str, int]]

class visionsim.simulate.blender.BlenderService

Bases: Service

Server-side API to interact with blender and render novel views.

Most of the methods of a BlenderClient instance are remote procedure calls to a connected blender service. These methods are prefixed by exposed_.

ALIASES: tuple[str] = ('BLENDER',)

__init__() → None

Initialize render service.

Raises:

RuntimeError – raised if not within blender’s runtime.

on_connect(conn: Connection) → None

Called when the connection is established

Parameters:

conn (rpyc.Connection) – Connection object

on_disconnect(conn: Connection) → None

Called when the connection has already terminated. Resets blender runtime. (must not perform any IO on the connection)

Parameters:

conn (rpyc.Connection) – Connection object

reset() → None

Cleans up and resets blender runtime.

De-initialize service by restoring blender to it’s startup state, ensuring any cached attributes are cleaned (otherwise objects will be stale), and resetting any instance variables that were previously initialized.

property scene: bpy.types.Scene

Get current blender scene

property tree: bpy.types.CompositorNodeTree

Get current scene’s node tree

property render_layers: bpy.types.CompositorNodeRLayers

Get and cache render layers node, create one if needed.

property view_layer: bpy.types.ViewLayer

Get current view layer

property camera: bpy.types.Camera

Get and cache active camera

get_parents(obj: bpy.types.Object) → list[bpy.types.Object]

Recursively retrieves parent objects of a given object in Blender

Parameters:

obj – Object to find parent of.

Returns:

Parent objects of obj.

Return type:

list[bpy.types.Object]

exposed_with_logger(log: Logger) → None

Use supplied logger, if logger is initialized in client, messages will log to the client.

Parameters:

log (logging.Logger) – Logger to use for messages

exposed_initialize(blend_file: str | PathLike, root_path: str | PathLike, **kwargs) → None

Initialize BlenderService and load blendfile.

Parameters:

• blend_file (str | os.PathLike) – path of scene file to load.

• root_path (str | os.PathLike) – path at which to save rendered results.

• **kwargs – Additional keyword arguments to be passed to bpy.ops.wm.open_mainfile.

exposed_iter_fcurves(actions: list[bpy.types.Action] | None = None) → Iterator[bpy.types.FCurve]

Yield fcurves of all actions.

This abstracts away the API for accessing fcurves which changed to using channelbags in v4.4, see release notes here.

Parameters:

actions (list[bpy.types.Action] | None, optional) – Only yield fcurves from these actions if specified, otherwise use all actions. Defaults to None.

Yields:

Iterator[bpy.types.FCurve] – an fcurve object from the scene or action

exposed_empty_transforms() → dict[str, Any]

Return a dictionary with camera intrinsics. Forms the basis of a transforms.json file, but contains no frame data.

Returns:

empty transforms dictionary containing only camera parameters.

Return type:

dict[str, Any]

exposed_original_fps() → int

Get effective framerate (fps/fps_base).

Returns:

Frame rate of scene.

Return type:

int

exposed_animation_range() → range

Get animation range of current scene as range(start, end+1, step).

Returns:

Range of frames in animation.

Return type:

range

exposed_animation_range_tuple() → tuple[int, int, int]

Get animation range of current scene as a tuple of (start, end, step).

Returns:

Frame start, end, and step of animation.

Return type:

tuple[int, int, int]

exposed_include_depths(debug=True, file_format='OPEN_EXR', exr_codec='ZIP') → None

Sets up Blender compositor to include depth map for rendered images.

Parameters:

• debug (bool, optional) – if true, colorized depth maps, helpful for quick visualizations, will be generated alongside ground-truth depth maps. Defaults to True.

• file_format (str, optional) – format of depth maps, one of “OPEN_EXR” or “HDR”. The former is lossless, but can require significant storage, the later is lossy and more compressed. If depth is needed to compute scene-flow, use open-exr. Defaults to “OPEN_EXR”.

• exr_codec (str, optional) – codec used to compress exr file. Only used when file_format="OPEN_EXR", options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “ZIP”.

Note

The debug colormap is re-normalized on a per-frame basis, to visually compare across frames, apply colorization after rendering using the CLI.

Raises:

ValueError – raise if file-format nor understood.

exposed_include_normals(debug=True, exr_codec='ZIP') → None

Sets up Blender compositor to include normal map for rendered images.

Parameters:

• debug (bool, optional) – if true, colorized normal maps will also be generated with each vector component being remapped from [-1, 1] to [0-255] with xyz becoming rgb. Defaults to True.

• exr_codec (str, optional) – codec used to compress exr file. Options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “ZIP”.

exposed_include_flows(direction='forward', debug=True, exr_codec='ZIP') → None

Sets up Blender compositor to include optical flow for rendered images.

Parameters:

• direction (str, optional) – One of ‘forward’, ‘backward’ or ‘both’. Direction of flow to colorize for debug visualization. Only used when debug is true, otherwise both forward and backward flows are saved. Defaults to “forward”.

• debug (bool, optional) – If true, also save debug visualizations of flow. Defaults to True.

• exr_codec (str, optional) – codec used to compress exr file. Options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “ZIP”.

Note

The debug colormap is re-normalized on a per-frame basis, to visually compare across frames, apply colorization after rendering using the CLI.

Raises:

• ValueError – raised when direction is not understood.

• RuntimeError – raised when motion blur is enabled as flow cannot be computed.

exposed_include_segmentations(shuffle=True, debug=True, seed=1234, exr_codec='ZIP') → None

Sets up Blender compositor to include segmentation maps for rendered images.

The debug visualization simply assigns a color to each object ID by mapping the objects ID value to a hue using a HSV node with saturation=1 and value=1 (except for the background which will have a value of 0 to ensure it is black).

Parameters:

• shuffle (bool, optional) – shuffle debug colors, helps differentiate object instances. Defaults to True.

• debug (bool, optional) – If true, also save debug visualizations of segmentation. Defaults to True.

• seed (int, optional) – random seed used when shuffling colors. Defaults to 1234.

• exr_codec (str, optional) – codec used to compress exr file. Options vary depending on the version of Blender, with the following being broadly available: (‘NONE’, ‘PXR24’, ‘ZIP’, ‘PIZ’, ‘RLE’, ‘ZIPS’, ‘DWAA’, ‘DWAB’). Defaults to “ZIP”.

Raises:

RuntimeError – raised when not using CYCLES, as other renderers do not support a segmentation pass.

exposed_load_addons(*addons: str) → None

Load blender addons by name (case-insensitive).

Parameters:

*addons (str) – name of addons to load.

exposed_set_resolution(height: tuple[int] | list[int] | int | None = None, width: int | None = None) → None

Set frame resolution (height, width) in pixels. If a single tuple is passed, instead of using keyword arguments, it will be parsed as (height, width).

Parameters:

• height (tuple[int] | list[int] | int | None, optional) – Height of render in pixels. Defaults to value from file.

• width (int | None, optional) – Width of render in pixels. Defaults to value from file.

Raises:

ValueError – raised if resolution is not understood.

exposed_image_settings(file_format: str | None = None, bit_depth: int | None = None, color_mode: str | None = None) → None

Set the render’s output format and bit-depth. Useful for linear intensity renders, using “OPEN_EXR” and 32 or 16 bits.

Note: A default arguments of None means do not change setting inherited from blendfile.

Parameters:

• file_format (str | None, optional) – Format to save render as. Options vary depending on the version of Blender, with the following being broadly available: (‘BMP’, ‘IRIS’, ‘PNG’, ‘JPEG’, ‘JPEG2000’, ‘TARGA’, ‘TARGA_RAW’, ‘CINEON’, ‘DPX’, ‘OPEN_EXR_MULTILAYER’, ‘OPEN_EXR’, ‘HDR’, ‘TIFF’, ‘WEBP’, ‘AVI_JPEG’, ‘AVI_RAW’, ‘FFMPEG’). Defaults to None.

• bit_depth (int | None, optional) – Bit depth per channel, also referred to as color-depth. Options depend on the chosen file format, with 8, 16 and 32bits being common. Defaults to None.

• color_mode (str | None, optional) – Typically one of (‘BW’, ‘RGB’, ‘RGBA’). Defaults to None.

exposed_use_motion_blur(enable: bool) → None

Enable/disable motion blur.

Parameters:

enable (bool) – If true, enable motion blur.

exposed_use_animations(enable: bool) → None

Enable/disable all animations.

Parameters:

enable (bool) – If true, enable animations.

exposed_cycles_settings(device_type: str | None = None, use_cpu: bool | None = None, adaptive_threshold: float | None = None, max_samples: int | None = None, use_denoising: bool | None = None) → list[str]

Enables/activates cycles render devices and settings.

Note: A default arguments of None means do not change setting inherited from blendfile.

Parameters:

• device_type (str, optional) – Name of device to use, one of “cpu”, “cuda”, “optix”, “metal”, etc. See blender docs for full list. Defaults to None.

• use_cpu (bool, optional) – Boolean flag to enable CPUs alongside GPU devices. Defaults to None.

• adaptive_threshold (float, optional) – Set noise threshold upon which to stop taking samples. Defaults to None.

• max_samples (int, optional) – Maximum number of samples per pixel to take. Defaults to None.

• use_denoising (bool, optional) – If enabled, a denoising pass will be used. Defaults to None.

Raises:

• RuntimeError – raised when no devices are found.

• ValueError – raised when setting use_cpu is required.

Returns:

Name of activated devices.

Return type:

list[str]

exposed_unbind_camera(clear_animations: bool = True) → None

Remove constraints, animations and parents from main camera.

Note: In order to undo this, you’ll need to re-initialize.

Parameters:

clear_animations (bool, optional) – If true clear animation data for camera.

exposed_move_keyframes(scale=1.0, shift=0.0) → None

Adjusts keyframes in Blender animations, keypoints are first scaled then shifted.

Parameters:

• scale (float, optional) – Factor used to rescale keyframe positions along x-axis. Defaults to 1.0.

• shift (float, optional) – Factor used to shift keyframe positions along x-axis. Defaults to 0.0.

Raises:

RuntimeError – raised if trying to move keyframes beyond blender’s limits.

exposed_set_current_frame(frame_number: int) → None

Set current frame number. This might advance any animations.

Parameters:

frame_number (int) – index of frame to skip to.

exposed_camera_extrinsics() → npt.NDArray[np.floating]

Get the 4x4 transform matrix encoding the current camera pose.

Returns:

Current camera pose in matrix form.

Return type:

npt.NDArray[np.floating]

exposed_camera_intrinsics() → npt.NDArray[np.floating]

Get the 3x3 camera intrinsics matrix for active camera, which defines how 3D points are projected onto 2D.

Note: Assumes pinhole camera model.

Returns:

Camera intrinsics matrix based on camera properties.

Return type:

npt.NDArray[np.floating]

exposed_position_camera(location: TypeAliasForwardRef('npt.ArrayLike') | None = None, rotation: TypeAliasForwardRef('npt.ArrayLike') | None = None, look_at: TypeAliasForwardRef('npt.ArrayLike') | None = None, in_order: bool = True) → None

Positions and orients camera according to specified parameters. All transformations are local, use unbind_camera to ensure position is set in world coordinates.

Note: Only one of look_at or rotation can be set at once.

Parameters:

• location (npt.ArrayLike, optional) – Location to place camera in 3D space. Defaults to none.

• rotation (npt.ArrayLike, optional) – Rotation matrix for camera. Defaults to none.

• look_at (npt.ArrayLike, optional) – Location to point camera. Defaults to none.

• in_order (bool, optional) – If set, assume current camera pose is from previous/next frame and ensure new rotation set by look_at is compatible with current position. Without this, a rotations will stay in the [-pi, pi] range and this wrapping will mess up interpolations. Only used when look_at is set. Defaults to True.

Raises:

ValueError – raised if camera orientation is over-defined.

exposed_rotate_camera(angle: float) → None

Rotate camera around it’s optical axis, relative to current orientation. All transformations are local, use unbind_camera to ensure position is set in world coordinates.

Parameters:

angle – Relative amount to rotate by (clockwise, in radians).

exposed_set_camera_keyframe(frame_num: int, matrix: TypeAliasForwardRef('npt.ArrayLike') | None = None) → None

Set camera keyframe at given frame number. If camera matrix is not supplied, currently set camera position/rotation/scale will be used, this allows users to set camera position using position_camera and rotate_camera.

Parameters:

• frame_num (int) – index of frame to set keyframe for.

• matrix (npt.ArrayLike | None, optional) – 4x4 camera transform, if not supplied, use current camera matrix. Defaults to None.

exposed_set_animation_range(start: int | None = None, stop: int | None = None, step: int | None = None) → None

Set animation range for scene.

Parameters:

• start (int | None, optional) – frame start, inclusive. Defaults to None.

• stop (int | None, optional) – frame stop, exclusive. Defaults to None.

• step (int | None, optional) – frame interval. Defaults to None.

exposed_render_current_frame(allow_skips=True, dry_run=False) → dict[str, Any]

Generates a single frame in Blender at the current camera location, return the file paths for that frame, potentially including depth, normals, etc.

Parameters:

• allow_skips (bool, optional) – if true, blender will not re-render and overwrite existing frames. This does not however apply to depth/normals/etc, which cannot be skipped. Defaults to True.

• dry_run (bool, optional) – if true, nothing will be rendered at all. Defaults to False.

Returns:

dictionary containing paths to rendered frames for this index and camera pose.

Return type:

dict[str, Any]

exposed_render_frame(frame_number: int, allow_skips=True, dry_run=False) → dict[str, Any]

Same as first setting current frame then rendering it.

Warning

Calling this has the side-effect of changing the current frame.

Parameters:

• frame_number (int) – frame to render

• allow_skips (bool, optional) – if true, blender will not re-render and overwrite existing frames. This does not however apply to depth/normals/etc, which cannot be skipped. Defaults to True.

• dry_run (bool, optional) – if true, nothing will be rendered at all. Defaults to False.

Returns:

dictionary containing paths to rendered frames for this index and camera pose.

Return type:

dict[str, Any]

exposed_render_frames(frame_numbers: Iterable[int], allow_skips=True, dry_run=False, update_fn: UpdateFn | None = None) → dict[str, Any]

Render all requested frames and return associated transforms dictionary.

Parameters:

• frame_numbers (Iterable[int]) – frames to render.

• allow_skips (bool, optional) – if true, blender will not re-render and overwrite existing frames. This does not however apply to depth/normals/etc, which cannot be skipped. Defaults to True.

• dry_run (bool, optional) – if true, nothing will be rendered at all. Defaults to False.

• update_fn (UpdateFn, optional) – callback function to track render progress. Will first be called with total kwarg, indicating number of steps to be taken, then will be called with advance=1 at every step. Closely mirrors the rich.Progress API. Defaults to None.

Raises:

RuntimeError – raised if trying to render frames beyond blender’s limits.

Returns:

transforms dictionary containing paths to rendered frames, camera poses and intrinsics.

Return type:

dict[str, Any]

exposed_render_animation(frame_start: int | None = None, frame_end: int | None = None, frame_step: int | None = None, allow_skips=True, dry_run=False, update_fn: UpdateFn | None = None) → dict[str, Any]

Determines frame range to render, sets camera positions and orientations, and renders all frames in animation range.

Note: All frame start/end/step arguments are absolute quantities, applied after any keyframe moves.

If the animation is from (1-100) and you’ve scaled it by calling move_keyframes(scale=2.0) then calling render_animation(frame_start=1, frame_end=100) will only render half of the animation. By default the whole animation will render when no start/end and step values are set.

Parameters:

• frame_start (int, optional) – Starting index (inclusive) of frames to render as seen in blender. Defaults to None, meaning value from .blend file.

• frame_end (int, optional) – Ending index (inclusive) of frames to render as seen in blender. Defaults to None, meaning value from .blend file.

• frame_step (int, optional) – Skip every nth frame. Defaults to None, meaning value from .blend file.

• allow_skips (bool, optional) – Same as render_current_frame.

• dry_run (bool, optional) – Same as render_current_frame.

• update_fn (UpdateFn, optional) – Same as render_frames.

Raises:

ValueError – raised if scene and camera are entirely static.

Returns:

transforms dictionary containing paths to rendered frames, camera poses and intrinsics.

Return type:

dict[str, Any]

exposed_save_file(path: str | PathLike) → None

Save the opened blender file. This is useful for introspecting the state of the compositor/scene/etc.

Parameters:

path (str | os.PathLike) – path where to save blendfile.

Raises:

ValueError – raised if file already exists.

class visionsim.simulate.blender.BlenderClient(addr: tuple[str, int], timeout: float = 10.0)

Bases: object

Client-side API to interact with blender and render novel views.

The BlenderClient is responsible for communicating with (and potentially spawning) separate BlenderServer`s that will actually perform the rendering via a :class:`BlenderService.

The client acts as a context manager, it will connect to it’s server when the context is entered and cleanly disconnect and close the connection in case of errors or when exiting the with-block.

Many useful methods to interact with blender are provided, such as set_resolution or render_animation. These methods are dynamically generated when the client connects to the server. Available methods are directly inherited from BlenderService (or whichever service the server is exposing), specifically any service method starting with exposed_ will be accessible to the client at runtime. For example, BlenderClient.include_depths is a remote procedure call to BlenderService.exposed_include_depths().

__init__(addr: tuple[str, int], timeout: float = 10.0) → None

Initialize a client with known address of server. Note: Using auto_connect() or spawn() is often more convenient.

Parameters:

• addr (tuple[str, int]) – Connection tuple containing the hostname and port

• timeout (float, optional) – Maximum time in seconds the client will attempt to connect to the server for before an error is thrown. Only used when entering context manager. Defaults to 10 seconds.

addr: tuple[str, int]

conn: Connection | None

awaitable: AsyncResult | None

process: Popen | None

timeout: float

classmethod auto_connect(timeout: float = 10.0) → Self

Automatically connect to available server.

Use BlenderServer.discover() to find available server within timeout.

Note: This doesn’t actually connect to the server instance, the connection happens

when the context manager is entered. This simply creates a client instance with the connection settings (i.e: hostname, port) of an existing server. The connection might still fail when entering the with-block.

Parameters:

timeout (float, optional) – try to discover server instance for timeout (in seconds) before giving up. Defaults to 10.0 seconds.

Raises:

TimeoutError – raise if unable to discover server in timeout seconds.

Returns:

client instance initialized with connection settings of existing server.

Return type:

Self

classmethod spawn(timeout: float = -1.0, log_dir: str | os.PathLike | None = None, autoexec: bool = False, executable: str | os.PathLike | None = None) → Iterator[Self]

Spawn and connect to a blender server. The spawned process is accessible through the client’s process attribute.

Parameters:

• timeout (float, optional) – try to discover spawned instances for timeout (in seconds) before giving up. If negative, a port will be randomly selected and assigned to the spawned server, bypassing the need for discovery and timeouts. Note that when a port is assigned this context manager will immediately yield, even if the server is not yet ready to accept incoming connections. Defaults to assigning a port to spawned server (-1 seconds).

• log_dir (str | os.PathLike | None, optional) – path to log directory, stdout/err will be captured if set, otherwise outputs will go to os.devnull. Defaults to None (devnull).

• autoexec (bool, optional) – if true, allow execution of any embedded python scripts within blender. For more, see blender’s CLI documentation. Defaults to False.

• executable (str | os.PathLike | None, optional) – path to Blender’s executable. Defaults to looking for blender on $PATH, but is useful when targeting a specific blender install, or when it’s installed via a package manager such as flatpak. Setting it to “flatpak run –die-with-parent org.blender.Blender” might be required when using flatpaks. Defaults to None (system PATH).

Yields:

Self – the connected client

render_animation_async(*args, **kwargs) → AsyncResult

Asynchronously call render_animation and return an rpyc.AsyncResult.

Parameters:

• *args – Same as BlendService.exposed_render_animation()

• *kwargs – Same as BlendService.exposed_render_animation()

Returns:

Result encapsulating the return value of render_animation.

After wait``ing for the render to finish, it can be accessed using the ``.value attribute.

Return type:

rpyc.AsyncResult

render_frames_async(*args, **kwargs) → AsyncResult

Asynchronously call render_frames and return an rpyc.AsyncResult.

Parameters:

• *args – Same as BlendService.exposed_render_frames()

• *kwargs – Same as BlendService.exposed_render_frames()

Returns:

Result encapsulating the return value of render_frames.

After wait``ing for the render to finish, it can be accessed using the ``.value attribute.

Return type:

rpyc.AsyncResult

wait() → None

Block and await any async results.

class visionsim.simulate.blender.BlenderClients(*objs: Iterator[BlenderClient | tuple[str, int]])

Bases: tuple

Collection of BlenderClient instances.

Most methods in this class simply call the equivalent method of each client, that is, calling clients.set_resolution is equivalent to calling set_resolution for each client in clients. Some special methods, namely the render_frames() and render_animation() methods will instead distribute the rendering load to all clients.

Finally, entering each client’s context-manager, and closing each client connection is ensured by using this class’ context-manager.

__init__(*objs) → None

Initialize collection of BlenderClient from iterable of clients, or their connection settings.

Parameters:

*objs (Iterator[BlenderClient | tuple[str, int]]) – BlenderClient instances or their hostnames and ports.

stack: ExitStack

classmethod spawn(jobs: int = 1, timeout: float = -1.0, log_dir: str | os.PathLike | None = None, autoexec: bool = False, executable: str | os.PathLike | None = None) → Iterator[Self]

Spawn and connect to one or more blender servers. The spawned processes are accessible through the client’s process attribute.

Parameters:

• jobs (int, optional) – number of jobs to spawn. Defaults to 1.

• timeout (float, optional) – try to discover spawned instances for timeout (in seconds) before giving up. If negative, a port will be randomly selected and assigned to the spawned server, bypassing the need for discovery and timeouts. Note that when a port is assigned this context manager will immediately yield, even if the server is not yet ready to accept incoming connections. Defaults to assigning a port to spawned server (-1 seconds).

• log_dir (str | os.PathLike | None, optional) – path to log directory, stdout/err will be captured if set, otherwise outputs will go to os.devnull. Defaults to None (devnull).

• autoexec (bool, optional) – if true, allow execution of any embedded python scripts within blender. For more, see blender’s CLI documentation. Defaults to False.

• executable (str | os.PathLike | None, optional) – path to Blender’s executable. Defaults to looking for blender on $PATH, but is useful when targeting a specific blender install, or when it’s installed via a package manager such as flatpak. Setting it to “flatpak run –die-with-parent org.blender.Blender” might be required when using flatpaks. Defaults to None (system PATH).

Yields:

Self – the connected clients

static pool(jobs: int = 1, timeout: float = -1.0, log_dir: str | os.PathLike | None = None, autoexec: bool = False, executable: str | os.PathLike | None = None, conns: list[tuple[str, int]] | None = None) → Iterator[multiprocess.Pool]

Spawns a multiprocessing-like worker pool, each with their own BlenderClient instance. The function supplied to pool.map/imap/starmap and their async variants will be automagically passed a client instance as their first argument that they can use for rendering.

Example

def render(client, blend_file):
    root = Path("renders") / Path(blend_file).stem
    client.initialize(blend_file, root)
    client.render_animation()

if __name__ == "__main__":
    with BlenderClients.pool(2) as pool:
        pool.map(render, ["monkey.blend", "cube.blend", "metaballs.blend"])

Note

Here we use multiprocess instead of the builtin multiprocessing library to take advantage of the more advanced dill serialization (as opposed to the standard pickling).

Parameters:

• jobs (int, optional) – number of jobs to spawn. Defaults to 1.

• timeout (float, optional) – try to discover spawned instances for timeout (in seconds) before giving up. If negative, a port will be randomly selected and assigned to the spawned server, bypassing the need for discovery and timeouts. Note that when a port is assigned this context manager will immediately yield, even if the server is not yet ready to accept incoming connections. Defaults to assigning a port to spawned server (-1 seconds).

• log_dir (str | os.PathLike | None, optional) – path to log directory, stdout/err will be captured if set, otherwise outputs will go to os.devnull. Defaults to None (devnull).

• autoexec (bool, optional) – if true, allow execution of any embedded python scripts within blender. For more, see blender’s CLI documentation. Defaults to False.

• executable (str | os.PathLike | None, optional) – path to Blender’s executable. Defaults to looking for blender on $PATH, but is useful when targeting a specific blender install, or when it’s installed via a package manager such as flatpak. Setting it to “flatpak run –die-with-parent org.blender.Blender” might be required when using flatpaks. Defaults to None (system PATH).

• conns – List of connection tuples containing the hostnames and ports of existing servers. If specified, the pool will use these servers (and jobs and other spawn arguments will be ignored) instead of spawning new ones.

Yields:

multiprocess.Pool –

A multiprocess.Pool instance which has had it’s applicator methods

(map/imap/starmap/etc) monkey-patched to inject a client instance as first argument.

common_animation_range() → range

Get animation range shared by all clients as range(start, end+1, step).

Raises:

RuntimeError – animation ranges for all clients are expected to be the same.

Returns:

Range of frames in animation.

Return type:

range

common_animation_range_tuple() → tuple[int, int, int]

Get animation range shared by all clients as a tuple of (start, end, step).

Raises:

RuntimeError – animation ranges for all clients are expected to be the same.

Returns:

Frame start, end, and step of animation.

Return type:

tuple[int, int, int]

render_frames(frame_numbers: Collection[int], allow_skips: bool = True, dry_run: bool = False, update_fn: UpdateFn | None = None) → dict[str, Any]

Render all requested frames by distributing workload across connected clients and return associated transforms dictionary.

Warning

Assumes all clients are initialized in the same manner, that is, to the same blendfile, with the same animation range, render settings, etc.

Parameters:

• frame_numbers (Collection[int]) – frames to render.

• allow_skips (bool, optional) – if true, blender will not re-render and overwrite existing frames. This does not however apply to depth/normals/etc, which cannot be skipped. Defaults to True.

• dry_run (bool, optional) – if true, nothing will be rendered at all. Defaults to False.

• update_fn (UpdateFn, optional) –

  callback function to track render progress. Will first be called with total kwarg, indicating number of steps to be taken, then will be called with advance=1 at every step. Closely mirrors the rich.Progress API. Defaults to None.

Raises:

RuntimeError – raised if trying to render frames beyond blender’s limits.

Returns:

transforms dictionary containing paths to rendered frames, camera poses and intrinsics.

Return type:

dict[str, Any]

render_animation(frame_start: int | None = None, frame_end: int | None = None, frame_step: int | None = None, allow_skips=True, dry_run=False, update_fn: UpdateFn | None = None) → dict[str, Any]

Determines frame range to render, sets camera positions and orientations, and renders all frames in animation range by distributing workload onto all connected clients.

Note: All frame start/end/step arguments are absolute quantities, applied after any keyframe moves.

If the animation is from (1-100) and you’ve scaled it by calling move_keyframes(scale=2.0) then calling render_animation(frame_start=1, frame_end=100) will only render half of the animation. By default the whole animation will render when no start/end and step values are set.

Parameters:

• frame_start (int, optional) – Starting index (inclusive) of frames to render as seen in blender. Defaults to None, meaning value from .blend file.

• frame_end (int, optional) – Ending index (inclusive) of frames to render as seen in blender. Defaults to None, meaning value from .blend file.

• frame_step (int, optional) – Skip every nth frame. Defaults to None, meaning value from .blend file.

• allow_skips (bool, optional) – Same as render_current_frame.

• dry_run (bool, optional) – Same as render_current_frame.

• update_fn (UpdateFn, optional) – Same as render_frames.

Raises:

ValueError – raised if scene and camera are entirely static.

Returns:

transforms dictionary containing paths to rendered frames, camera poses and intrinsics.

Return type:

dict[str, Any]

save_file(path: str | PathLike) → None

Save opened blender file. This is useful for introspecting the state of the compositor/scene/etc.

Note: Only saves file once (from a single connected client), assumes all clients have

been initialized in the same manner.

Parameters:

path (str | os.PathLike) – path where to save blendfile.

Raises:

ValueError – raised if file already exists.

wait() → None

Wait for all clients at once.

visionsim.simulate.config module

class visionsim.simulate.config.RenderConfig(executable: 'str | os.PathLike | None' = None, height: 'int' = 512, width: 'int' = 512, bit_depth: 'int' = 8, file_format: 'str' = 'PNG', exr_codec: 'str' = 'ZIP', depths: 'bool' = False, normals: 'bool' = False, flows: 'bool' = False, flow_direction: "Literal['forward', 'backward', 'both']" = 'forward', segmentations: 'bool' = False, debug: 'bool' = True, keyframe_multiplier: 'float' = 1.0, timeout: 'int' = -1, autoexec: 'bool' = True, device_type: "Literal['cpu', 'cuda', 'optix', 'metal']" = 'optix', adaptive_threshold: 'float' = 0.05, max_samples: 'int' = 256, use_denoising: 'bool' = True, log_dir: 'str | os.PathLike' = 'logs/', allow_skips: 'bool' = True, unbind_camera: 'bool' = False, use_animations: 'bool' = True, use_motion_blur: 'bool | None' = None, addons: 'list[str] | None' = None, jobs: 'int' = 1, autoscale: 'bool' = False, max_job_vram: 'MemSize | None' = None)

Bases: object

executable: str | PathLike | None = None

Path to blender executable

height: int = 512

Height of rendered frames

width: int = 512

Width of rendered frames

bit_depth: int = 8

Bit depth for intensity frames. Usually 8 for pngs, 32 or 16 bits for OPEN_EXR

file_format: str = 'PNG'

File format to use for intensity frames

exr_codec: str = 'ZIP'

Encoding used to compress EXRs, used for all supported ground truths

depths: bool = False

If true, enable depth map outputs

normals: bool = False

If true, enable normal map outputs

flows: bool = False

If true, enable optical flow outputs

flow_direction: Literal['forward', 'backward', 'both'] = 'forward'

Direction of flow to colorize for debug visualization. Only used when debug is true

segmentations: bool = False

If true, enable segmentation map outputs

debug: bool = True

If true, also save debug visualizations for auxiliary outputs

keyframe_multiplier: float = 1.0

2.0 will slow down time

Type:

Stretch keyframes by this amount, eg

timeout: int = -1

Maximum allowed time in seconds to wait to connect to render instance

__init__(executable: str | PathLike | None = None, height: int = 512, width: int = 512, bit_depth: int = 8, file_format: str = 'PNG', exr_codec: str = 'ZIP', depths: bool = False, normals: bool = False, flows: bool = False, flow_direction: Literal['forward', 'backward', 'both']='forward', segmentations: bool = False, debug: bool = True, keyframe_multiplier: float = 1.0, timeout: int = -1, autoexec: bool = True, device_type: Literal['cpu', 'cuda', 'optix', 'metal']='optix', adaptive_threshold: float = 0.05, max_samples: int = 256, use_denoising: bool = True, log_dir: str | PathLike = 'logs/', allow_skips: bool = True, unbind_camera: bool = False, use_animations: bool = True, use_motion_blur: bool | None = None, addons: list[str] | None = None, jobs: int = 1, autoscale: bool = False, max_job_vram: Annotated[int, ~tyro.constructors._primitive_spec.PrimitiveConstructorSpec(nargs=1, metavar=BYTES, instance_from_str=~visionsim.types._bytes_from_str, is_instance=~visionsim.types.<lambda>, str_from_instance=~visionsim.types._bytes_to_str, choices=None, _action=None)] | None = None) → None

autoexec: bool = True

potentially dangerous)

Type:

If true, allow python execution of embedded scripts (warning

device_type: Literal['cpu', 'cuda', 'optix', 'metal'] = 'optix'

Name of device to use, one of “cpu”, “cuda”, “optix”, “metal”, etc

adaptive_threshold: float = 0.05

Noise threshold of rendered images, for higher quality frames make this threshold smaller. The default value is intentionally a little high to speed up renders

max_samples: int = 256

Maximum number of samples per pixel to take

use_denoising: bool = True

If enabled, a denoising pass will be used

log_dir: str | PathLike = 'logs/'

Directory to use for logging

allow_skips: bool = True

If true, skip rendering a frame if it already exists

unbind_camera: bool = False

Free the camera from it’s parents, any constraints and animations it may have. Ensures it uses the world’s coordinate frame and the provided camera trajectory

use_animations: bool = True

Allow any animations to play out, if false, scene will be static

use_motion_blur: bool | None = None

Enable realistic motion blur. cannot be used if also rendering optical flow

addons: list[str] | None = None

List of extra addons to enable

jobs: int = 1

Number of concurrent render jobs

autoscale: bool = False

Set number of jobs automatically based on available VRAM and max_job_vram when enabled

max_job_vram: _bytes_to_str, choices=None, _action=None)] | None = None

Maximum allowable VRAM per job in bytes (limit is not enforced, simply used for autoscale)

visionsim.simulate.install module

visionsim.simulate.job module

visionsim.simulate.job.render_job(client: BlenderClient | BlenderClients, blend_file: str | PathLike, root: str | PathLike, *, config: RenderConfig = RenderConfig(executable=None, height=512, width=512, bit_depth=8, file_format='PNG', exr_codec='ZIP', depths=False, normals=False, flows=False, flow_direction='forward', segmentations=False, debug=True, keyframe_multiplier=1.0, timeout=-1, autoexec=True, device_type='optix', adaptive_threshold=0.05, max_samples=256, use_denoising=True, log_dir='logs/', allow_skips=True, unbind_camera=False, use_animations=True, use_motion_blur=None, addons=None, jobs=1, autoscale=False, max_job_vram=None), frame_start: int | None = None, frame_end: int | None = None, frame_step: int | None = None, output_blend_file: str | PathLike | None = None, dry_run: bool = False, update_fn: UpdateFn | None = None)

Render a sequence from a given blender-file.

Parameters:

• client (BlenderClient | BlenderClients) – The blender client(s) which will be used for rendering. These should already be connected to a BlenderServer, and will get automagically passed in when using this function with BlenderClients.pool or similar.

• blend_file (str | os.PathLike) – Path to blender file to use.

• root (str | os.PathLike) – Location at which to save all outputs.

• config (RenderConfig) – Render configuration.

• frame_start (int | None, optional) – Frame index to start capture at (inclusive). If None, use start of animation range.

• frame_end (int | None, optional) – frame number to stop capture at (inclusive). If None, use end of animation range.

• frame_step (int | None, optional) – Step with which to capture frames. If None, use step of animation range.

• output_blend_file (str | os.PathLike | None, optional) – If set, write the modified blend file to this path. Helpful for troubleshooting. Defaults to not saving.

• dry_run (bool, optional) – If enabled, do not render any frames or ground truth annotations.

• update_fn (UpdateFn | None, optional) –

  callback function to track render progress. Will first be called with total kwarg, indicating number of steps to be taken, then will be called with advance=1 at every step. Closely mirrors the rich.Progress API.

Module contents

visionsim.simulate.install_dependencies(executable: str | PathLike | None = None, editable: bool = False) → CompletedProcess

Install additional packages into blender`s runtime.

Parameters:

• executable (str | os.PathLike | None, optional) – Same as BlenderServer.spawn. Defaults to None.

• editable – (bool, optional): If set, install current visionsim as editable in blender. Only works if visionsim is already installed as editable locally.