spine.io.overlay

Module with methods to overlay multiple events.

Classes

Overlayer(data_types, methods, multiplicity)

Generic class to produce data overlays.

class spine.io.overlay.Overlayer(data_types: Mapping[str, str], methods: Mapping[str, str | None], multiplicity: int, mode: str = 'constant')

Generic class to produce data overlays.

This class supports three overlay modes:

• constant uses a fixed multiplicity.

• uniform samples multiplicities M_i from a uniform distribution and adjusts them so that, for a batch size B, sum_i M_i = B.

• poisson samples multiplicities from a Poisson distribution with mean set by multiplicity and adjusts them the same way.

Methods

__call__(batch)	Given a batch of data, provides an overlay batching and modifies the data in place to avoid indexing conflicts.
build_parser_tensor(ref_data, features[, ...])	Build a parser tensor while preserving overlay metadata.
cat_objects(batch, key, index)	Concatenate object lists into one, offset index attributes if needed.
get_assignments(batch_size)	Given a data product count, produce batch assignments.
get_overlay_order(batch, index)	Order reference tensors before tensors that depend on them.
merge_objects(batch, key, index)	Merge objects into one per overlay.
merge_scalars(batch, key, index)	Merge scalars into one per overlay.
stack_edge_index_data(batch, key, index, ...)	Overlay one edge-index payload.
stack_feature_tensor_data(batch, key, index, ...)	Overlay one feature-only parser payload.
stack_flat_index_data(batch, key, index, ...)	Overlay one flat index payload.
stack_index_list_data(batch, key, index, ...)	Overlay one jagged index-list payload.
stack_tensor_data(batch, key, index, ref_data)	Overlay one tensor-like parser payload.
stack_tensors(batch, key, index)	Stack parser payloads together across an overlay.

get_overlay_order(batch: Sequence[dict[str, Any]], index: ndarray | Sequence[int]) → list[str]

Order reference tensors before tensors that depend on them.

Feature-only tensors such as source IDs may be row-aligned to another tensor that drops duplicate coordinates during overlay. Processing references first lets them define the row selection reused by aligned tensors.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• index (np.ndarray or Sequence[int]) – List of indexes to merge into an overlay

Returns:

List of keys in the order they should be processed for overlay.

Return type:

List[str]

get_assignments(batch_size: int) → ndarray

Given a data product count, produce batch assignments.

Parameters:

batch_size (int) – Number of entries in the batch

Returns:

Overlay ID assignments

Return type:

np.ndarray

merge_scalars(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int]) → Any

Merge scalars into one per overlay.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Scalar data product key

• index (np.ndarray) – List of indexes to merge into an overlay

Returns:

Single scalar for the batch

Return type:

object

merge_objects(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int]) → Any

Merge objects into one per overlay.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Object data product key

• index (np.ndarray) – List of indexes to merge into an overlay

Returns:

Single object for the batch

Return type:

object

cat_objects(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int]) → ParserObjectList

Concatenate object lists into one, offset index attributes if needed.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Object list data product key

• index (np.ndarray) – List of indexes to merge into an overlay

Returns:

Concatenated obejct list

Return type:

ObjList

stack_tensors(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int]) → ParserTensor | ParserIndex | ParserIndexList | ParserEdgeIndex

Stack parser payloads together across an overlay.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Tensor data product key

• index (np.ndarray) – List of indexes to merge into an overlay

Returns:

Overlayed parser payload of the same logical type as the input.

Return type:

ParserTensor or ParserIndex or ParserIndexList or ParserEdgeIndex

stack_tensor_data(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int], ref_data: ParserTensor) → ParserTensor

Overlay one tensor-like parser payload.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Tensor data product key

• index (np.ndarray) – List of indexes to merge into an overlay

• ref_data (ParserTensor) – Reference tensor used to check metadata and index columns, and to preserve overlay metadata in the output.

Returns:

Overlayed parser tensor

Return type:

ParserTensor

stack_feature_tensor_data(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int], ref_data: ParserTensor) → ParserTensor

Overlay one feature-only parser payload.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Tensor data product key

• index (np.ndarray) – List of indexes to merge into an overlay

• ref_data (ParserTensor) – Reference tensor used to check metadata and index columns, and to preserve overlay metadata in the output.

Returns:

Overlayed parser tensor with feature-only coordinates

Return type:

ParserTensor

static build_parser_tensor(ref_data: ParserTensor, features: ndarray, coords: ndarray | None = None, index_shifts: ndarray | None = None, feats_only: bool | None = None) → ParserTensor

Build a parser tensor while preserving overlay metadata.

Parameters:

• ref_data (ParserTensor) – Reference tensor used to check metadata and index columns, and to preserve overlay metadata in the output.

• features (np.ndarray) – Stacked features for the overlay

• coords (np.ndarray, optional) – Stacked coordinates for the overlay, if present in the reference tensor

• index_shifts (np.ndarray, optional) – Stacked index shifts for the overlay, if present in the reference tensor

• feats_only (bool, optional) – Whether the output tensor should be feature-only. If not provided, will be inferred from the reference tensor.

stack_flat_index_data(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int], ref_data: ParserIndex) → ParserIndex

Overlay one flat index payload.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Index data product key

• index (np.ndarray) – List of indexes to merge into an overlay

• ref_data (ParserIndex) – Reference index used to check metadata and preserve overlay metadata in the output.

Returns:

Overlayed index data.

Return type:

ParserIndex

stack_index_list_data(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int], ref_data: ParserIndexList) → ParserIndexList

Overlay one jagged index-list payload.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Index list data product key

• index (np.ndarray) – List of indexes to merge into an overlay

• ref_data (ParserIndexList) – Reference index list used to check metadata and preserve overlay metadata in the output.

Returns:

Overlayed index list data.

Return type:

ParserIndexList

stack_edge_index_data(batch: Sequence[dict[str, Any]], key: str, index: ndarray | Sequence[int], ref_data: ParserEdgeIndex) → ParserEdgeIndex

Overlay one edge-index payload.

Parameters:

• batch (List[Dict]) – List of dictionaries of parsed information, one per event. Each dictionary matches one data key to one event-worth of parsed data.

• key (str) – Edge index data product key

• index (np.ndarray) – List of indexes to merge into an overlay

• ref_data (ParserEdgeIndex) – Reference edge index used to check metadata and preserve overlay metadata in the output.

Returns:

Overlayed edge index data.

Return type:

ParserEdgeIndex