xrmocap.core¶

estimation¶

evaluation¶

smplify hook¶

train¶

visualization¶

xrmocap.data¶

data_converter¶

dataloader¶

dataset¶

data_visualization¶

xrmocap.data_structure¶

class xrmocap.data_structure.Keypoints(src_dict: Optional[dict] = None, dtype: typing_extensions.Literal[torch, numpy, auto] = 'auto', kps: Optional[Union[numpy.ndarray, torch.Tensor]] = None, mask: Optional[Union[numpy.ndarray, torch.Tensor]] = None, convention: Optional[str] = None, logger: Union[None, str, logging.Logger] = None)[source]¶

A class for multi-frame, multi-person keypoints data, based on python dict.

keypoints, mask and convention are the three necessary keys, and we advise you to just call Keypoints(). If you’d like to set them manually, it is recommended to obey the following turn: convention -> keypoints -> mask.

clone() → xrmocap.data_structure.keypoints.Keypoints [source]¶

Clone a Keypoints instance as self.

Returns: Keypoints – A deep copied instance of Keypoints, with the same dtype and value as self.

dump(npz_path: str, overwrite: bool = True)[source]¶

Dump keys and items to an npz file.

Parameters

npz_path (str) – Path to a dumped npz file.
overwrite (bool, optional) – Whether to overwrite if there is already a file. Defaults to True.

Raises

ValueError – npz_path does not end with ‘.npz’.
FileExistsError – When overwrite is False and file exists.

classmethod fromfile(npz_path: str) → xrmocap.data_structure.keypoints.Keypoints [source]¶

Construct a body model data structure from an npz file.

Parameters: npz_path (str) – Path to a dumped npz file.
Returns: Keypoints – A Keypoints instance load from file.

get_convention() → str[source]¶

Get keypoints convention name.

Returns: str – convention

get_frame_number() → int[source]¶

Get frame number of keypoints.

Returns: int – frame number

get_keypoints() → Union[numpy.ndarray, torch.Tensor][source]¶

Get keypoints array.

Returns: np.ndarray – keypoints

get_keypoints_number() → int[source]¶

Get number of keypoints.

Returns: int – keypoints number

get_mask() → Union[numpy.ndarray, torch.Tensor][source]¶

Get keypoints mask.

Returns: np.ndarray – mask

get_person_number() → int[source]¶

Get person number of keypoints.

Returns: int – person number

load(npz_path: str)[source]¶

Load data from npz_path and update them to self.

Parameters: npz_path (str) – Path to a dumped npz file.

set_convention(convention: str) → None[source]¶

Set convention name of the keypoints.

Parameters: convention (str) – Convention name of the keypoints, can be found in KEYPOINTS_FACTORY.
Raises: TypeError – Type of convention is not str.

set_keypoints(kps: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set keypoints array.

Parameters

kps (Union[np.ndarray, torch.Tensor]) – A tensor or ndarray for keypoints, kps2d in shape [n_frame, n_person, n_kps, 3], kps3d in shape [n_frame, n_person, n_kps, 4]. Shape [n_kps, 3 or 4] is also accepted, unsqueezed automatically.

Raises

TypeError – Type of keypoints is wrong.
ValueError – kps.shape[-1] is wrong.
ValueError – Shape of kps is wrong.

set_mask(mask: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set mask of the keypoints. It should be called after the corresponding keypoints has been set.

Parameters

mask (Union[np.ndarray, torch.Tensor]) – A tensor or ndarray for keypoint mask, in shape [n_frame, n_person, n_kps], in dtype uint8. Shape [n_kps, ] is also accepted, unsqueezed automatically.

Raises

TypeError – Type of mask is wrong.
ValueError – Shape of mask is wrong.

to_numpy() → xrmocap.data_structure.keypoints.Keypoints [source]¶

Return all the necessary values for keypoints expression in another Keypoints instance, convert Tensor into numpy.

Returns: Keypoints – An instance of Keypoints data, whose keys are keypoints, mask, convention.

to_tensor(device: Union[torch.device, str] = 'cpu') → xrmocap.data_structure.keypoints.Keypoints [source]¶

Return all the necessary values for keypoints expression in another Keypoints instance, convert ndarray into Tensor.

Parameters: device (Union[torch.device, str], optional) – A specified device. Defaults to ‘cpu’.
Returns: Keypoints – An instance of Keypoints data, whose keys are keypoints, mask, convention.

class xrmocap.data_structure.Limbs(connections: Union[numpy.ndarray, torch.Tensor], connection_names: Optional[List[str]] = None, parts: Optional[List[List[int]]] = None, part_names: Optional[List[str]] = None, points: Optional[Union[numpy.ndarray, torch.Tensor]] = None, logger: Union[None, str, logging.Logger] = None)[source]¶

A class for person limbs data, recording connection vectors between keypoints.

Connections are the only necessary data, while human parts, points are optional.

get_connection_names() → List[str][source]¶

Get names of connection.

Returns: List[str] – A list of names

get_connections() → numpy.ndarray[source]¶

Get connections of limbs.

Returns: np.ndarray – connections

get_connections_by_names() → dict[source]¶

Get connection by names.

Returns: dict – keys are connection names and values are [start_pt_index, end_pt_index], in type ndarray.

get_connections_in_parts() → dict[source]¶

Get connection in parts. In each part, there’s a list of.

[start_pt_index, end_pt_index].

Returns: dict – keys are part names.

get_part_names() → List[str][source]¶

Get names of part. If self.part is None, an empty list will be returned.

Returns: List[str] – A list of names

get_parts() → Optional[List[List[int]]][source]¶

Get parts of limbs, which might be None.

Returns: Union[List[List[int]], None] – parts

get_points() → Optional[numpy.ndarray][source]¶

Get points array, which might be None.

Returns: Union[np.ndarray, None] – keypoints

set_connections(conn: Union[numpy.ndarray, torch.Tensor], conn_names: Optional[List[str]] = None) → None[source]¶

Set connection relations of the limbs. Names are optional.

Parameters

conn (Union[np.ndarray, torch.Tensor]) – A tensor or ndarray for connections, in shape [n_conn, 2], conn[:, 0] are start point indice and conn[:, 1] are end point indice.
conn_names (List[str], optional) – A list of connections names. If given, len(conn_names)==len(conn), else default names will be returned when getting connections. Defaults to None.

Raises

TypeError – Type of connections is not correct.
ValueError – Shape of connections is not correct.

set_parts(parts: List[List[int]], part_names: Optional[List[str]] = None) → None[source]¶

Set parts of the limbs. If parts has been set, connections can be arranged by part when getting. Names are optional.

Parameters

parts (List[List[int]]) – A nested list, len(parts) is part number, and len(parts[0]) is connection number of the first part. Each element in parts[i] is an index of one connection.
part_names (List[str], optional) – A list of part names. If given, len(part_names)==len(parts), else default names will be returned when getting parts. Defaults to None.

Raises

TypeError – Type of parts is not correct.
ValueError – Type of connection index is not correct.

set_points(points: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set points of the limbs.

Parameters

points (Union[np.ndarray, torch.Tensor]) – A tensor or ndarray for points, in shape [n_point, point_dim].

Raises

TypeError – Type of points is not correct.
ValueError – Shape of points should be [n_point, point_dim].

class xrmocap.data_structure.SMPLData(gender: Optional[typing_extensions.Literal[female, male, neutral]] = None, fullpose: Optional[Union[numpy.ndarray, torch.Tensor]] = None, transl: Optional[Union[numpy.ndarray, torch.Tensor]] = None, betas: Optional[Union[numpy.ndarray, torch.Tensor]] = None, mask: Optional[Union[numpy.ndarray, torch.Tensor]] = None, logger: Union[None, str, logging.Logger] = None)[source]¶

dump(npz_path: str, overwrite: bool = True)[source]¶

Dump keys and items to an npz file.

Parameters

npz_path (str) – Path to a dumped npz file.
overwrite (bool, optional) – Whether to overwrite if there is already a file. Defaults to True.

Raises

ValueError – npz_path does not end with ‘.npz’.
FileExistsError – When overwrite is False and file exists.

classmethod from_dict(smpl_data_dict: Union[xrmocap.data_structure.body_model.smpl_data.SMPLData, dict]) → xrmocap.data_structure.body_model.smpl_data.SMPLData [source]¶

Construct a body model data structure from a SMPLData, or a degraded smpl_data in dict type.

Parameters: smpl_data_dict (dict) – A degraded smpl_data in dict type.
Returns: SMPLData – A SMPLData instance load from dict.

from_param_dict(smpl_dict: dict) → None[source]¶

Load SMPL parameters from smpl_dict, which is the output of a body model in most cases.

Parameters: smpl_dict (dict) – A dict of ndarray|Tensor parameters. global_orient and body_pose are necessary, transl and betas are optional. Other keys are ignored.
Raises: KeyError – missing necessary keys.

classmethod fromfile(npz_path: str) → xrmocap.data_structure.body_model.smpl_data.SMPLData [source]¶

Construct a body model data structure from an npz file.

Parameters: npz_path (str) – Path to a dumped npz file.
Returns: SMPLData – A SMPLData instance load from file.

get_batch_size() → int[source]¶

Get batch size.

Returns: int – batch size of fullpose.

get_betas(repeat_betas: bool = True) → numpy.ndarray[source]¶

Get betas.

Parameters: repeat_betas (bool, optional) – Whether to repeat betas when its first dim doesn’t match batch_size. Defaults to True.
Returns: ndarray – betas in shape [batch_size, betas_dims] or [1, betas_dims].

get_body_pose() → numpy.ndarray[source]¶

Get body_pose.

Returns: ndarray – body_pose in shape [batch_size, BODY_POSE_LEN, 3].

get_fullpose() → numpy.ndarray[source]¶

Get fullpose.

Returns: ndarray – fullpose in shape [batch_size, fullpose_dim, 3].

classmethod get_fullpose_dim() → int[source]¶

Get dimension of full pose.

Returns: int – Dim value. Full pose shall be in shape (frame_n, dim, 3)

get_gender() → str[source]¶

Get gender.

Returns: str – gender in [‘neutral’, ‘female’, ‘male’].

get_global_orient() → numpy.ndarray[source]¶

Get global_orient.

Returns: ndarray – global_orient in shape [batch_size, 3].

get_mask() → numpy.ndarray[source]¶

Get mask.

Returns: ndarray – mask in shape [batch_size, ].

get_transl() → numpy.ndarray[source]¶

Get translation.

Returns: ndarray – translation in shape [batch_size, 3].

load(npz_path: str)[source]¶

Load data from npz_path and update them to self.

Parameters: npz_path (str) – Path to a dumped npz file.

set_betas(betas: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set betas data.

Parameters: betas (Union[np.ndarray, torch.Tensor]) – Body shape parameters in ndarray or tensor, in shape [batch_size, n]. n stands for any positive int, typically it’s 10.
Raises: TypeError – Type of betas is not correct.

set_fullpose(fullpose: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set full pose data.

Parameters: fullpose (Union[np.ndarray, torch.Tensor]) – Full pose in ndarray or tensor, in shape [batch_size, fullpose_dim, 3]. global_orient at [:, 0, :].
Raises: TypeError – Type of fullpose is not correct.

set_gender(gender: typing_extensions.Literal[female, male, neutral] = 'neutral') → None[source]¶

Set gender.

Parameters: gender (Literal["female", "male", "neutral"], optional) – Gender of the body model. Should be one among [“female”, “male”, “neutral”]. Defaults to ‘neutral’.
Raises: ValueError – Value of gender is not correct.

set_mask(mask: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set framewise mask data.

Parameters: mask (Union[np.ndarray, torch.Tensor]) – Visibility mask in ndarray or tensor, in shape [batch_size, ].
Raises: TypeError – Type of mask is not correct.

set_transl(transl: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set translation data.

Parameters: transl (Union[np.ndarray, torch.Tensor]) – Translation in ndarray or tensor, in shape [batch_size, 3].
Raises: TypeError – Type of transl is not correct.

to_param_dict(repeat_betas: bool = True) → dict[source]¶

Split fullpose into global_orient and body_pose, return all the necessary parameters in one dict.

Parameters: repeat_betas (bool, optional) – Whether to repeat betas when its first dim doesn’t match batch_size. Defaults to True.
Returns: dict – A dict of SMPL data, whose keys are betas, body_pose, global_orient and transl.

to_tensor_dict(repeat_betas: bool = True, device: Union[torch.device, str] = 'cpu') → dict[source]¶

It is almost same as self.to_param_dict, but all the values are tensors in a specified device. Split fullpose into global_orient and body_pose, return all the necessary parameters in one dict.

Parameters

repeat_betas (bool, optional) – Whether to repeat betas when its first dim doesn’t match batch_size. Defaults to True.
device (Union[torch.device, str], optional) – A specified device. Defaults to CPU_DEVICE. Defaults to ‘cpu’.

Returns

dict – A dict of SMPL data, whose keys are betas, body_pose, global_orient and transl.

class xrmocap.data_structure.SMPLXDData(gender: Optional[typing_extensions.Literal[female, male, neutral]] = None, fullpose: Optional[Union[numpy.ndarray, torch.Tensor]] = None, transl: Optional[Union[numpy.ndarray, torch.Tensor]] = None, betas: Optional[Union[numpy.ndarray, torch.Tensor]] = None, expression: Optional[Union[numpy.ndarray, torch.Tensor]] = None, displacement: Optional[Union[numpy.ndarray, torch.Tensor]] = None, mask: Optional[Union[numpy.ndarray, torch.Tensor]] = None, logger: Union[None, str, logging.Logger] = None)[source]¶

classmethod from_dict(smpl_data_dict: Union[xrmocap.data_structure.body_model.smplxd_data.SMPLXDData, dict]) → xrmocap.data_structure.body_model.smplxd_data.SMPLXDData [source]¶

Construct a body model data structure from a SMPLXDData, or a degraded smplxd_data in dict type.

Parameters: smplxd_data_dict (dict) – A degraded smplxd_data in dict type.
Returns: SMPLXData – A SMPLXDData instance load from dict.

from_param_dict(smplxd_dict: dict) → None[source]¶

Load SMPLX+D parameters from smplxd_dict, which is the output of a body model in most cases.

Parameters: smplx_dict (dict) – A dict of ndarray|Tensor parameters. global_orient and body_pose are necessary, jaw_pose, leye_pose, reye_pose, left_hand_pose, right_hand_pose, transl and betas are optional. Other keys are ignored.
Raises: KeyError – missing necessary keys.

get_displacement() → numpy.ndarray[source]¶

Get displacement.

Returns: ndarray – Displacement in shape [batch_size, NUM_VERTS, 3].

set_displacement(displacement: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set displacement data.

Parameters: displacement (Union[np.ndarray, torch.Tensor]) – Displacement parameters in ndarray or tensor, in shape [batch_size, NUM_VERTS, 3].
Raises: TypeError – Type of displacement is not correct.

to_param_dict(repeat_betas: bool = True, repeat_expression: bool = True) → dict[source]¶

Split fullpose into global_orient, body_pose, jaw_pose, leye_pose, reye_pose, left_hand_pose, right_hand_pose, return all the necessary parameters in one dict.

Parameters

repeat_betas (bool, optional) – Whether to repeat betas when its first dim doesn’t match batch_size. Defaults to True.
repeat_expression (bool, optional) – Whether to repeat expression when its first dim doesn’t match batch_size. Defaults to True.

Returns

dict – A dict of SMPLX data, whose keys are betas, global_orient, transl, global_orient, body_pose, jaw_pose, leye_pose, reye_pose, left_hand_pose, right_hand_pose, expression.

class xrmocap.data_structure.SMPLXData(gender: Optional[typing_extensions.Literal[female, male, neutral]] = None, fullpose: Optional[Union[numpy.ndarray, torch.Tensor]] = None, transl: Optional[Union[numpy.ndarray, torch.Tensor]] = None, betas: Optional[Union[numpy.ndarray, torch.Tensor]] = None, expression: Optional[Union[numpy.ndarray, torch.Tensor]] = None, mask: Optional[Union[numpy.ndarray, torch.Tensor]] = None, logger: Union[None, str, logging.Logger] = None)[source]¶

classmethod from_dict(smpl_data_dict: Union[xrmocap.data_structure.body_model.smplx_data.SMPLXData, dict]) → xrmocap.data_structure.body_model.smplx_data.SMPLXData [source]¶

Construct a body model data structure from a SMPLXData, or a degraded smplx_data in dict type.

Parameters: smplx_data_dict (dict) – A degraded smplx_data in dict type.
Returns: SMPLXData – A SMPLXData instance load from dict.

from_param_dict(smplx_dict: dict) → None[source]¶

Load SMPLX parameters from smplx_dict, which is the output of a body model in most cases.

Parameters: smplx_dict (dict) – A dict of ndarray|Tensor parameters. global_orient and body_pose are necessary, expression, jaw_pose, leye_pose, reye_pose, left_hand_pose, right_hand_pose, transl and betas are optional. Other keys are ignored.
Raises: KeyError – missing necessary keys.

get_expression(repeat_expression: bool = True) → numpy.ndarray[source]¶

Get expression.

Parameters: repeat_expression (bool, optional) – Whether to repeat expression when its first dim doesn’t match batch_size. Defaults to True.
Returns: ndarray – expression in shape [batch_size, expression_dims] or [1, expression_dims].

classmethod get_fullpose_dim() → int[source]¶

Get dimension of full pose.

Returns: int – Dim value. Full pose shall be in shape (frame_n, dim, 3)

set_expression(expression: Union[numpy.ndarray, torch.Tensor]) → None[source]¶

Set expression data.

Parameters: expression (Union[np.ndarray, torch.Tensor]) – Expression parameters in ndarray or tensor, in shape [batch_size, n]. n stands for any positive int, typically it’s 10.
Raises: TypeError – Type of expression is not correct.

to_param_dict(repeat_betas: bool = True, repeat_expression: bool = True) → dict[source]¶

Split fullpose into global_orient, body_pose, jaw_pose, leye_pose, reye_pose, left_hand_pose, right_hand_pose, return all the necessary parameters in one dict.

Parameters

repeat_betas (bool, optional) – Whether to repeat betas when its first dim doesn’t match batch_size. Defaults to True.
repeat_expression (bool, optional) – Whether to repeat expression when its first dim doesn’t match batch_size. Defaults to True.

Returns

dict – A dict of SMPLX data, whose keys are betas, global_orient, transl, global_orient, body_pose, jaw_pose, leye_pose, reye_pose, left_hand_pose, right_hand_pose, expression.

to_tensor_dict(repeat_betas: bool = True, repeat_expression: bool = True, device: Union[torch.device, str] = 'cpu') → dict[source]¶

It is almost same as self.to_param_dict, but all the values are tensors in a specified device. Split fullpose into global_orient and body_pose, return all the necessary parameters in one dict.

Parameters

repeat_betas (bool, optional) – Whether to repeat betas when its first dim doesn’t match batch_size. Defaults to True.
repeat_expression (bool, optional) – Whether to repeat expression when its first dim doesn’t match batch_size. Defaults to True.
device (Union[torch.device, str], optional) – A specified device. Defaults to CPU_DEVICE. Defaults to ‘cpu’.

Returns

dict – A dict of SMPLX data, whose keys are betas, body_pose, global_orient and transl, etc.

xrmocap.human_perception¶

bbox_detection¶

keypoints_estimation¶

xrmocap.io¶

class xrmocap.io.H5Helper[source]¶: Some helper function related to h5py.

xrmocap.io.get_all_color_kinect_parameter_from_smc(smc_reader: xrmocap.data_structure.smc_reader.SMCReader, align_floor: bool = True, logger: Union[None, str, logging.Logger] = None) → List[xrprimer.data_structure.camera.fisheye_camera.FisheyeCameraParameter][source]¶

Get an RGB FisheyeCameraParameter from an smc reader.

Parameters

smc_reader (SMCReader) – An SmcReader instance containing kinect and iphone camera parameters.
align_floor (bool) – Whether to rotate camera extrinsics, makes the xOz plane parallel to floor calibrated by Kinect_0’s depth camera. Defaults to True.
logger (Union[None, str, logging.Logger], optional) – Logger for logging. If None, root logger will be selected. Defaults to None.

Returns

List[FisheyeCameraParameter]

xrmocap.io.get_color_camera_parameter_from_smc(smc_reader: xrmocap.data_structure.smc_reader.SMCReader, camera_type: typing_extensions.Literal[kinect, iphone], camera_id: int, align_floor: bool = True, logger: Union[None, str, logging.Logger] = None) → xrprimer.data_structure.camera.fisheye_camera.FisheyeCameraParameter[source]¶

Get an RGB FisheyeCameraParameter from an smc reader.

Parameters

smc_reader (SMCReader) – An SmcReader instance containing kinect and iphone camera parameters.
camera_type (Literal['kinect', 'iphone']) – Which type of camera to get.
camera_id (int) – ID of the selected camera.
align_floor (bool) – Whether to rotate camera extrinsics, makes the xOz plane parallel to floor calibrated by Kinect_0’s depth camera. Defaults to True.
logger (Union[None, str, logging.Logger], optional) – Logger for logging. If None, root logger will be selected. Defaults to None.

Raises

NotImplementedError – iphone has not been supported yet.
KeyError – camera_type is neither kinect nor iphone.

Returns

FisheyeCameraParameter

xrmocap.io.get_n_frame_from_mview_src(img_arr: Union[None, numpy.ndarray] = None, img_paths: Union[None, List[List[str]]] = None, video_paths: Optional[List[str]] = None, logger: Union[None, str, logging.Logger] = None) → int[source]¶

Get number of frames from multi-view image source. It needs one images input among [img_arr, img_paths, video_paths].

Parameters

img_arr (Union[None, np.ndarray], optional) – A multi-view image array, in shape [n_view, n_frame, h, w, c]. Defaults to None.
img_paths (Union[None, List[List[str]]], optional) – A nested list of image paths, in shape [n_view, n_frame]. Defaults to None.
video_paths (Union[None, List[str]], optional) – A list of video paths, each is a view. Defaults to None.
logger (Union[None, str, logging.Logger], optional) – Logger for logging. If None, root logger will be selected. Defaults to None.

Returns

int – Number of frames.

xrmocap.io.load_camera_parameters_from_zoemotion_dir(camera_parameter_path: str, enable_camera_id: Optional[list] = None) → Tuple[list, list][source]¶

Load camera parameter and get an RGB FisheyeCameraParameter.

Parameters

camera_parameter_path (str) – path to the camera parameter
enable_camera_id (Union[list, None], optional) – camera ID(str).
to None. (Defaults) –

Returns

cam_param_list (list) – FisheyeCameraParameter
enable_camera_list (list) – enable camera list e.g.[‘0’, ‘1’]

xrmocap.io.load_clip_from_mview_src(start_idx: int, end_idx: int, img_arr: Union[None, numpy.ndarray] = None, img_paths: Union[None, List[List[str]]] = None, video_paths: Optional[List[str]] = None, logger: Union[None, str, logging.Logger] = None) → numpy.ndarray[source]¶

Get image array of a clip from multi-view image source. It needs one images input among [img_arr, img_paths, video_paths].

Parameters

img_arr (Union[None, np.ndarray], optional) – A multi-view image array, in shape [n_view, n_frame, h, w, c]. Defaults to None.
img_paths (Union[None, List[List[str]]], optional) – A nested list of image paths, in shape [n_view, n_frame]. Defaults to None.
video_paths (Union[None, List[str]], optional) – A list of video paths, each is a view. Defaults to None.
logger (Union[None, str, logging.Logger], optional) – Logger for logging. If None, root logger will be selected. Defaults to None.

Returns

np.ndarray – Image array of the selected range.

xrmocap.io.load_multiview_images(img_paths: Union[None, List[List[str]]]) → numpy.ndarray[source]¶

Load multi-view images to an ndarray.

Parameters: img_paths (Union[None, List[List[str]]]) – A nested list of image paths, in shape [view_n, frame_n].
Returns: np.ndarray – Multi-view image array, in shape [view_n, frame_n, h, w, c].

xrmocap.model¶

architecture¶

xrmocap.ops¶

projection¶

class xrmocap.ops.projection.AniposelibProjector(camera_parameters: List[xrprimer.data_structure.camera.fisheye_camera.FisheyeCameraParameter], logger: Union[None, str, logging.Logger] = None)[source]¶

project(points: Union[numpy.ndarray, list, tuple], points_mask: Optional[Union[numpy.ndarray, list, tuple]] = None) → numpy.ndarray[source]¶

Project points with self.camera_parameters.

Parameters

points (Union[np.ndarray, list, tuple]) – An ndarray or a nested list of points3d, in shape [n_point, 3].
points_mask (Union[np.ndarray, list, tuple], optional) – An ndarray or a nested list of mask, in shape [n_point, 1]. If points_mask[index] == 1, points[index] is valid for projection, else it is ignored. Defaults to None.

Returns

np.ndarray – An ndarray of points2d, in shape [n_view, n_point, 2].

project_single_point(points: Union[numpy.ndarray, list, tuple]) → numpy.ndarray[source]¶

Project a single point with self.camera_parameters.

Parameters: points (Union[np.ndarray, list, tuple]) – An ndarray or a list of points3d, in shape [3].
Returns: np.ndarray – An ndarray of points2d, in shape [n_view, 2].

class xrmocap.ops.projection.PytorchProjector(camera_parameters: List[xrprimer.data_structure.camera.fisheye_camera.FisheyeCameraParameter], logger: Union[None, str, logging.Logger] = None)[source]¶

project(points: torch.Tensor, points_mask: Optional[torch._VariableFunctionsClass.tensor] = None) → torch.Tensor[source]¶

Project points with self.camera_parameters.

Parameters

points (torch.Tensor) – points3d, in shape [n_point, 3].
points_mask (torch.Tensor, optional) – mask, in shape [n_point, 1]. If points_mask[index] == 1, points[index] is valid for projection, else it is ignored. Defaults to None.

Returns

torch.Tensor – points2d, in shape [n_view, n_point, 2].

project_single_point(points: torch.Tensor) → torch.Tensor[source]¶

Project a single point with self.camera_parameters.

Parameters: points (torch.Tensor) – points3d, in shape [3].
Returns: torch.Tensor – points2d, in shape [n_view, 2].

xrmocap.transform¶

keypoints3d.optim¶

xrmocap.utils¶

xrmocap.utils.project_point_radial(x, R, T, f, c, k, p)[source]¶

This function is to project a point in 3D space to 2D pixel space with given camera parameters.

Parameters

x – Nx3 points in world coordinates
R – 3x3 Camera rotation matrix
T – 3x1 Camera translation parameters
f – (scalar) Camera focal length
c – 2x1 Camera center
k – 3x1 Camera radial distortion coefficients
p – 2x1 Camera tangential distortion coefficients

Returns

ypixel.T – Nx2 points in pixel space

xrmocap.utils.unfold_camera_param(camera: dict)[source]¶

This function is to extract camera extrinsic, intrinsic and distorsion parameters from dictionary.

Parameters

camera (dict) – Dictionary to store the camera parameters.

Returns

R (Union[np.ndarray, torch.Tensor]) – Extrinsic parameters, rotation matrix.
T (Union[np.ndarray, torch.Tensor]) – Extrinsic parameters, translation matrix.
f (Union[np.ndarray, torch.Tensor]) – Focal length in x, y direction.
c (Union[np.ndarray, torch.Tensor]) – Camera center.
k (Union[list, torch.Tensor]) – Radial distortion coefficients.
p (Union[list, torch.Tensor]) – Tangential distortion coefficients.