heat.cluster

Add the clustering functions to the ht.cluster namespace

Submodules

• heat.cluster._kcluster
• heat.cluster.batchparallelclustering
• heat.cluster.kmeans
• heat.cluster.kmedians
• heat.cluster.kmedoids
• heat.cluster.metrics
• heat.cluster.spectral

Package Contents

_kmex(X, p, n_clusters, init, max_iter, tol, random_state=None, weights: torch.tensor = 1.0)

Auxiliary function: single-process k-means and k-medians in pytorch p is the norm used for computing distances: p=2 implies k-means, p=1 implies k-medians. p should be 1 (k-medians) or 2 (k-means). For other choice of p, we proceed as for p=2 and hope for the best. (note: kmex stands for kmeans and kmedians)

class DNDarray(array: torch.Tensor, gshape: Tuple[int, Ellipsis], dtype: heat.core.types.datatype, split: int | None, device: heat.core.devices.Device, comm: Communication, balanced: bool)

Distributed N-Dimensional array. The core element of HeAT. It is composed of PyTorch tensors local to each process.

Parameters:

• array (torch.Tensor) – Local array elements

• gshape (Tuple[int,...]) – The global shape of the array

• dtype (datatype) – The datatype of the array

• split (int or None) – The axis on which the array is divided between processes

• device (Device) – The device on which the local arrays are using (cpu or gpu)

• comm (Communication) – The communications object for sending and receiving data

• balanced (bool or None) – Describes whether the data are evenly distributed across processes. If this information is not available (self.balanced is None), it can be gathered via the is_balanced() method (requires communication).

__array

__gshape

__dtype

__split

__device

__comm

__balanced

__ishalo = False

__halo_next = None

__halo_prev = None

__partitions_dict__ = None

__lshape_map = None

__prephalo(start, end) → torch.Tensor

Extracts the halo indexed by start, end from self.array in the direction of self.split

Parameters:

• start (int) – Start index of the halo extracted from self.array

• end (int) – End index of the halo extracted from self.array

get_halo(halo_size: int, prev: bool = True, next: bool = True)

Fetch halos of size halo_size from neighboring ranks and save them in self.halo_next/self.halo_prev.

Parameters:

• halo_size (int) – Size of the halo.

• prev (bool, optional) – If True, fetch the halo from the previous rank. Default: True.

• next (bool, optional) – If True, fetch the halo from the next rank. Default: True.

__cat_halo() → torch.Tensor

Return local array concatenated to halos if they are available.

__array__() → numpy.ndarray

Returns a view of the process-local slice of the DNDarray as a numpy ndarray, if the DNDarray resides on CPU. Otherwise, it returns a copy, on CPU, of the process-local slice of DNDarray as numpy ndarray.

__array_ufunc__(ufunc, method, *inputs, **kwargs)

Override NumPy’s universal functions.

__array_function__(func, types, args, kwargs)

Augments NumPy’s functions.

astype(dtype, copy=True) → DNDarray

Returns a casted version of this array. Casted array is a new array of the same shape but with given type of this array. If copy is True, the same array is returned instead.

Parameters:

• dtype (datatype) – Heat type to which the array is cast

• copy (bool, optional) – By default the operation returns a copy of this array. If copy is set to False the cast is performed in-place and this array is returned

balance_() → DNDarray

Function for balancing a DNDarray between all nodes. To determine if this is needed use the is_balanced() function. If the DNDarray is already balanced this function will do nothing. This function modifies the DNDarray itself and will not return anything.

Examples

>>> a = ht.zeros((10, 2), split=0)
>>> a[:, 0] = ht.arange(10)
>>> b = a[3:]
[0/2] tensor([[3., 0.],
[1/2] tensor([[4., 0.],
              [5., 0.],
              [6., 0.]])
[2/2] tensor([[7., 0.],
              [8., 0.],
              [9., 0.]])
>>> b.balance_()
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (1, 2)
[1/2] (7, 2) (3, 2)
[2/2] (7, 2) (3, 2)
>>> b
[0/2] tensor([[3., 0.],
             [4., 0.],
             [5., 0.]])
[1/2] tensor([[6., 0.],
              [7., 0.]])
[2/2] tensor([[8., 0.],
              [9., 0.]])
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (3, 2)
[1/2] (7, 2) (2, 2)
[2/2] (7, 2) (2, 2)

__bool__() → bool

Boolean scalar casting.

__cast(cast_function) → float | int

Implements a generic cast function for DNDarray objects.

Parameters:

cast_function (function) – The actual cast function, e.g. float or int

Raises:

TypeError – If the DNDarray object cannot be converted into a scalar.

collect_(target_rank: int | None = 0) → None

A method collecting a distributed DNDarray to one MPI rank, chosen by the target_rank variable. It is a specific case of the redistribute_ method.

Parameters:

target_rank (int, optional) – The rank to which the DNDarray will be collected. Default: 0.

Raises:

• TypeError – If the target rank is not an integer.

• ValueError – If the target rank is out of bounds.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.collect_()
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)
>>> st.collect_(1)
>>> print(st.lshape)
[0/2] (50, 81, 0)
[1/2] (50, 81, 67)
[2/2] (50, 81, 0)

__complex__() → DNDarray

Complex scalar casting.

counts_displs() → Tuple[Tuple[int], Tuple[int]]

Returns actual counts (number of items per process) and displacements (offsets) of the DNDarray. Does not assume load balance.

cpu() → DNDarray

Returns a copy of this object in main memory. If this object is already in main memory, then no copy is performed and the original object is returned.

create_lshape_map(force_check: bool = False) → torch.Tensor

Generate a ‘map’ of the lshapes of the data on all processes. Units are (process rank, lshape)

Parameters:

force_check (bool, optional) – if False (default) and the lshape map has already been created, use the previous result. Otherwise, create the lshape_map

create_partition_interface()

Create a partition interface in line with the DPPY proposal. This is subject to change. The intention of this to facilitate the usage of a general format for the referencing of distributed datasets.

An example of the output and shape is shown below.

__partitioned__ = {

‘shape’: (27, 3, 2), ‘partition_tiling’: (4, 1, 1), ‘partitions’: {

(0, 0, 0): {

‘start’: (0, 0, 0), ‘shape’: (7, 3, 2), ‘data’: tensor([…], dtype=torch.int32), ‘location’: [0], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (1, 0, 0): {

‘start’: (7, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [1], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (2, 0, 0): {

‘start’: (14, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [2], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (3, 0, 0): {

‘start’: (21, 0, 0), ‘shape’: (6, 3, 2), ‘data’: None, ‘location’: [3], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}

}, ‘locals’: [(rank, 0, 0)], ‘get’: lambda x: x,

}

Return type:

dictionary containing the partition interface as shown above.

__float__() → DNDarray

Float scalar casting.

See also

flatten()

fill_diagonal(value: float) → DNDarray

Fill the main diagonal of a 2D DNDarray. This function modifies the input tensor in-place, and returns the input array.

Parameters:

value (float) – The value to be placed in the DNDarrays main diagonal

__getitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis]) → DNDarray

Global getter function for DNDarrays. Returns a new DNDarray composed of the elements of the original tensor selected by the indices given. This does NOT redistribute or rebalance the resulting tensor. If the selection of values is unbalanced then the resultant tensor is also unbalanced! To redistributed the DNDarray use balance() (issue #187)

Parameters:

key (int, slice, Tuple[int,...], List[int,...]) – Indices to get from the tensor.

Examples

>>> a = ht.arange(10, split=0)
(1/2) >>> tensor([0, 1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5, 6, 7, 8, 9], dtype=torch.int32)
>>> a[1:6]
(1/2) >>> tensor([1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5], dtype=torch.int32)
>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1]
(1/2) >>> tensor([0.])
(2/2) >>> tensor([0., 0.])

gpu() → DNDarray

Returns a copy of this object in GPU memory. If this object is already in GPU memory, then no copy is performed and the original object is returned.

__int__() → DNDarray

Integer scalar casting.

is_balanced(force_check: bool = False) → bool

Determine if self is balanced evenly (or as evenly as possible) across all nodes distributed evenly (or as evenly as possible) across all processes. This is equivalent to returning self.balanced. If no information is available (self.balanced = None), the balanced status will be assessed via collective communication.

Parameters:

force_check (bool, optional) – If True, the balanced status of the DNDarray will be assessed via collective communication in any case.

is_distributed() → bool

Determines whether the data of this DNDarray is distributed across multiple processes.

__key_is_singular(key: any, axis: int, self_proxy: torch.Tensor) → bool

__key_adds_dimension(key: any, axis: int, self_proxy: torch.Tensor) → bool

item()

Returns the only element of a 1-element DNDarray. Mirror of the pytorch command by the same name. If size of DNDarray is >1 element, then a ValueError is raised (by pytorch)

Examples

>>> import heat as ht
>>> x = ht.zeros((1))
>>> x.item()
0.0

__len__() → int

The length of the DNDarray, i.e. the number of items in the first dimension.

numpy() → numpy.array

Returns a copy of the DNDarray as numpy ndarray. If the DNDarray resides on the GPU, the underlying data will be copied to the CPU first.

If the DNDarray is distributed, an MPI Allgather operation will be performed before converting to np.ndarray, i.e. each MPI process will end up holding a copy of the entire array in memory. Make sure process memory is sufficient!

Examples

>>> import heat as ht
T1 = ht.random.randn((10,8))
T1.numpy()

_repr_pretty_(p, cycle)

Pretty print for IPython.

__repr__() → str

Returns a printable representation of the passed DNDarray, targeting developers.

ravel()

Flattens the DNDarray.

See also

ravel()

Examples

>>> a = ht.ones((2, 3), split=0)
>>> b = a.ravel()
>>> a[0, 0] = 4
>>> b
DNDarray([4., 1., 1., 1., 1., 1.], dtype=ht.float32, device=cpu:0, split=0)

redistribute_(lshape_map: torch.Tensor | None = None, target_map: torch.Tensor | None = None)

Redistributes the data of the DNDarray along the split axis to match the given target map. This function does not modify the non-split dimensions of the DNDarray. This is an abstraction and extension of the balance function.

Parameters:

• lshape_map (torch.Tensor, optional) – The current lshape of processes. Units are [rank, lshape].

• target_map (torch.Tensor, optional) – The desired distribution across the processes. Units are [rank, target lshape]. Note: the only important parts of the target map are the values along the split axis, values which are not along this axis are there to mimic the shape of the lshape_map.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> target_map = torch.zeros((st.comm.size, 3), dtype=torch.int64)
>>> target_map[0, 2] = 67
>>> print(target_map)
[0/2] tensor([[ 0,  0, 67],
[0/2]         [ 0,  0,  0],
[0/2]         [ 0,  0,  0]], dtype=torch.int32)
[1/2] tensor([[ 0,  0, 67],
[1/2]         [ 0,  0,  0],
[1/2]         [ 0,  0,  0]], dtype=torch.int32)
[2/2] tensor([[ 0,  0, 67],
[2/2]         [ 0,  0,  0],
[2/2]         [ 0,  0,  0]], dtype=torch.int32)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.redistribute_(target_map=target_map)
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)

__redistribute_shuffle(snd_pr: int | torch.Tensor, send_amt: int | torch.Tensor, rcv_pr: int | torch.Tensor, snd_dtype: torch.dtype)

Function to abstract the function used during redistribute for shuffling data between processes along the split axis

Parameters:

• snd_pr (int or torch.Tensor) – Sending process

• send_amt (int or torch.Tensor) – Amount of data to be sent by the sending process

• rcv_pr (int or torch.Tensor) – Receiving process

• snd_dtype (torch.dtype) – Torch type of the data in question

resplit_(axis: int = None)

In-place option for resplitting a DNDarray.

Parameters:

axis (int) – The new split axis, None denotes gathering, an int will set the new split axis

Examples

>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, None)
>>> a.split
None
>>> a.lshape
(0/2) (4, 5)
(1/2) (4, 5)
>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, 1)
>>> a.split
1
>>> a.lshape
(0/2) (4, 3)
(1/2) (4, 2)

__setitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Global item setter

Parameters:

• key (Union[int, Tuple[int,...], List[int,...]]) – Index/indices to be set

• value (Union[float, DNDarray,torch.Tensor]) – Value to be set to the specified positions in the DNDarray (self)

Notes

If a DNDarray is given as the value to be set then the split axes are assumed to be equal. If they are not, PyTorch will raise an error when the values are attempted to be set on the local array

Examples

>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1] = 1
>>> a
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])
(2/2) >>> tensor([[0., 1., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])

__setter(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Utility function for checking value and forwarding to :func:__setitem__

Raises:

NotImplementedError – If the type of value ist not supported

__str__() → str

Computes a string representation of the passed DNDarray.

tolist(keepsplit: bool = False) → List

Return a copy of the local array data as a (nested) Python list. For scalars, a standard Python number is returned.

Parameters:

keepsplit (bool) – Whether the list should be returned locally or globally.

Examples

>>> a = ht.array([[0, 1], [2, 3]])
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=0)
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=1)
>>> a.tolist(keepsplit=True)
(1/2) [[0], [2]]
(2/2) [[1], [3]]

__torch_function__(func, types, args=(), kwargs=None)

Supports PyTorch’s dispatch mechanism.

__torch_proxy__() → torch.Tensor

Return a 1-element torch.Tensor strided as the global self shape. Used internally for sanitation purposes.

__xitem_get_key_start_stop(rank: int, actives: list, key_st: int, key_sp: int, step: int, ends: torch.Tensor, og_key_st: int) → Tuple[int, int]

class _KCluster(metric: Callable, n_clusters: int, init: str | heat.core.dndarray.DNDarray, max_iter: int, tol: float, random_state: int)

Bases: heat.ClusteringMixin, heat.BaseEstimator

Base class for k-statistics clustering algorithms (kmeans, kmedians, kmedoids). The clusters are represented by centroids ci (we use the term from kmeans for simplicity)

Parameters:

• metric (function) – One of the distance metrics in ht.spatial.distance. Needs to be passed as lambda function to take only two arrays as input

• n_clusters (int) – The number of clusters to form as well as the number of centroids to generate.

• init (str or DNDarray, default: ‘random’) –

  Method for initialization:

  • ‘probability_based’ : selects initial cluster centers for the clustering in a smart way to speed up convergence (k-means++)

  • ‘random’: choose k observations (rows) at random from data for the initial centroids.

  • ’batchparallel’: use the batch parallel algorithm to initialize the centroids, only available for split=0 and KMeans or KMedians

  • DNDarray: gives the initial centers, should be of Shape = (n_clusters, n_features)

• max_iter (int) – Maximum number of iterations for a single run.

• tol (float, default: 1e-4) – Relative tolerance with regards to inertia to declare convergence.

• random_state (int) – Determines random number generation for centroid initialization.

n_clusters

init

max_iter

tol

random_state

_metric

_cluster_centers = None

_functional_value = None

_labels = None

_inertia = None

_n_iter = None

_p = None

_initialize_cluster_centers(x: heat.core.dndarray.DNDarray, oversampling: float, iter_multiplier: float)

Initializes the K-Means centroids.

Parameters:

• x (DNDarray) – The data to initialize the clusters for. Shape = (n_samples, n_features)

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• iter_multiplier (float) – factor that increases the number of iterations used in the initialization of centroids

_centroid_sampling_helper(x: heat.core.dndarray.DNDarray, centroids: heat.core.dndarray.DNDarray, oversampling: float, num_iters: int)

Helper function for the k-means|| initialization of centroids. Samples new centroids based on a probability distribution derived from the distance of data points to the current set of centroids.

Parameters:

• x (DNDarray) – The data to initialize the clusters for. Shape = (n_samples, n_features)

• centroids (DNDarray) – The initial set of centroids

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• num_iters (float) – number of iterations used in the initialization of centroids

_assign_to_cluster(x: heat.core.dndarray.DNDarray, eval_functional_value: bool = False)

Assigns the passed data points to the centroids based on the respective metric

Parameters:

• x (DNDarray) – Data points, Shape = (n_samples, n_features)

• eval_functional_value (bool, default: False) – If True, the current K-Clustering functional value of the clustering algorithm is evaluated

_update_centroids(x: heat.core.dndarray.DNDarray, matching_centroids: heat.core.dndarray.DNDarray)

The Update strategy is algorithm specific (e.g. calculate mean of assigned points for kmeans, median for kmedians, etc.)

Parameters:

• x (DNDarray) – Input Data

• matching_centroids (DNDarray) – Index array of assigned centroids

fit(x: heat.core.dndarray.DNDarray)

Computes the centroid of the clustering algorithm to fit the data x. The full pipeline is algorithm specific.

Parameters:

x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features)

predict(x: heat.core.dndarray.DNDarray)

Predict the closest cluster each sample in x belongs to.

In the vector quantization literature, cluster_centers_() is called the code book and each value returned by predict is the index of the closest code in the code book.

Parameters:

x (DNDarray) – New data to predict. Shape = (n_samples, n_features)

class _KCluster(metric: Callable, n_clusters: int, init: str | heat.core.dndarray.DNDarray, max_iter: int, tol: float, random_state: int)

Bases: heat.ClusteringMixin, heat.BaseEstimator

Base class for k-statistics clustering algorithms (kmeans, kmedians, kmedoids). The clusters are represented by centroids ci (we use the term from kmeans for simplicity)

Parameters:

• metric (function) – One of the distance metrics in ht.spatial.distance. Needs to be passed as lambda function to take only two arrays as input

• n_clusters (int) – The number of clusters to form as well as the number of centroids to generate.

• init (str or DNDarray, default: ‘random’) –

  Method for initialization:

  • ‘probability_based’ : selects initial cluster centers for the clustering in a smart way to speed up convergence (k-means++)

  • ‘random’: choose k observations (rows) at random from data for the initial centroids.

  • ’batchparallel’: use the batch parallel algorithm to initialize the centroids, only available for split=0 and KMeans or KMedians

  • DNDarray: gives the initial centers, should be of Shape = (n_clusters, n_features)

• max_iter (int) – Maximum number of iterations for a single run.

• tol (float, default: 1e-4) – Relative tolerance with regards to inertia to declare convergence.

• random_state (int) – Determines random number generation for centroid initialization.

n_clusters

init

max_iter

tol

random_state

_metric

_cluster_centers = None

_functional_value = None

_labels = None

_inertia = None

_n_iter = None

_p = None

_initialize_cluster_centers(x: heat.core.dndarray.DNDarray, oversampling: float, iter_multiplier: float)

Initializes the K-Means centroids.

Parameters:

• x (DNDarray) – The data to initialize the clusters for. Shape = (n_samples, n_features)

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• iter_multiplier (float) – factor that increases the number of iterations used in the initialization of centroids

_centroid_sampling_helper(x: heat.core.dndarray.DNDarray, centroids: heat.core.dndarray.DNDarray, oversampling: float, num_iters: int)

Helper function for the k-means|| initialization of centroids. Samples new centroids based on a probability distribution derived from the distance of data points to the current set of centroids.

Parameters:

• x (DNDarray) – The data to initialize the clusters for. Shape = (n_samples, n_features)

• centroids (DNDarray) – The initial set of centroids

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• num_iters (float) – number of iterations used in the initialization of centroids

_assign_to_cluster(x: heat.core.dndarray.DNDarray, eval_functional_value: bool = False)

Assigns the passed data points to the centroids based on the respective metric

Parameters:

• x (DNDarray) – Data points, Shape = (n_samples, n_features)

• eval_functional_value (bool, default: False) – If True, the current K-Clustering functional value of the clustering algorithm is evaluated

_update_centroids(x: heat.core.dndarray.DNDarray, matching_centroids: heat.core.dndarray.DNDarray)

The Update strategy is algorithm specific (e.g. calculate mean of assigned points for kmeans, median for kmedians, etc.)

Parameters:

• x (DNDarray) – Input Data

• matching_centroids (DNDarray) – Index array of assigned centroids

fit(x: heat.core.dndarray.DNDarray)

Computes the centroid of the clustering algorithm to fit the data x. The full pipeline is algorithm specific.

Parameters:

x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features)

predict(x: heat.core.dndarray.DNDarray)

Predict the closest cluster each sample in x belongs to.

In the vector quantization literature, cluster_centers_() is called the code book and each value returned by predict is the index of the closest code in the code book.

Parameters:

x (DNDarray) – New data to predict. Shape = (n_samples, n_features)

class DNDarray(array: torch.Tensor, gshape: Tuple[int, Ellipsis], dtype: heat.core.types.datatype, split: int | None, device: heat.core.devices.Device, comm: Communication, balanced: bool)

Distributed N-Dimensional array. The core element of HeAT. It is composed of PyTorch tensors local to each process.

Parameters:

• array (torch.Tensor) – Local array elements

• gshape (Tuple[int,...]) – The global shape of the array

• dtype (datatype) – The datatype of the array

• split (int or None) – The axis on which the array is divided between processes

• device (Device) – The device on which the local arrays are using (cpu or gpu)

• comm (Communication) – The communications object for sending and receiving data

• balanced (bool or None) – Describes whether the data are evenly distributed across processes. If this information is not available (self.balanced is None), it can be gathered via the is_balanced() method (requires communication).

__array

__gshape

__dtype

__split

__device

__comm

__balanced

__ishalo = False

__halo_next = None

__halo_prev = None

__partitions_dict__ = None

__lshape_map = None

__prephalo(start, end) → torch.Tensor

Extracts the halo indexed by start, end from self.array in the direction of self.split

Parameters:

• start (int) – Start index of the halo extracted from self.array

• end (int) – End index of the halo extracted from self.array

get_halo(halo_size: int, prev: bool = True, next: bool = True)

Fetch halos of size halo_size from neighboring ranks and save them in self.halo_next/self.halo_prev.

Parameters:

• halo_size (int) – Size of the halo.

• prev (bool, optional) – If True, fetch the halo from the previous rank. Default: True.

• next (bool, optional) – If True, fetch the halo from the next rank. Default: True.

__cat_halo() → torch.Tensor

Return local array concatenated to halos if they are available.

__array__() → numpy.ndarray

Returns a view of the process-local slice of the DNDarray as a numpy ndarray, if the DNDarray resides on CPU. Otherwise, it returns a copy, on CPU, of the process-local slice of DNDarray as numpy ndarray.

__array_ufunc__(ufunc, method, *inputs, **kwargs)

Override NumPy’s universal functions.

__array_function__(func, types, args, kwargs)

Augments NumPy’s functions.

astype(dtype, copy=True) → DNDarray

Returns a casted version of this array. Casted array is a new array of the same shape but with given type of this array. If copy is True, the same array is returned instead.

Parameters:

• dtype (datatype) – Heat type to which the array is cast

• copy (bool, optional) – By default the operation returns a copy of this array. If copy is set to False the cast is performed in-place and this array is returned

balance_() → DNDarray

Function for balancing a DNDarray between all nodes. To determine if this is needed use the is_balanced() function. If the DNDarray is already balanced this function will do nothing. This function modifies the DNDarray itself and will not return anything.

Examples

>>> a = ht.zeros((10, 2), split=0)
>>> a[:, 0] = ht.arange(10)
>>> b = a[3:]
[0/2] tensor([[3., 0.],
[1/2] tensor([[4., 0.],
              [5., 0.],
              [6., 0.]])
[2/2] tensor([[7., 0.],
              [8., 0.],
              [9., 0.]])
>>> b.balance_()
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (1, 2)
[1/2] (7, 2) (3, 2)
[2/2] (7, 2) (3, 2)
>>> b
[0/2] tensor([[3., 0.],
             [4., 0.],
             [5., 0.]])
[1/2] tensor([[6., 0.],
              [7., 0.]])
[2/2] tensor([[8., 0.],
              [9., 0.]])
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (3, 2)
[1/2] (7, 2) (2, 2)
[2/2] (7, 2) (2, 2)

__bool__() → bool

Boolean scalar casting.

__cast(cast_function) → float | int

Implements a generic cast function for DNDarray objects.

Parameters:

cast_function (function) – The actual cast function, e.g. float or int

Raises:

TypeError – If the DNDarray object cannot be converted into a scalar.

collect_(target_rank: int | None = 0) → None

A method collecting a distributed DNDarray to one MPI rank, chosen by the target_rank variable. It is a specific case of the redistribute_ method.

Parameters:

target_rank (int, optional) – The rank to which the DNDarray will be collected. Default: 0.

Raises:

• TypeError – If the target rank is not an integer.

• ValueError – If the target rank is out of bounds.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.collect_()
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)
>>> st.collect_(1)
>>> print(st.lshape)
[0/2] (50, 81, 0)
[1/2] (50, 81, 67)
[2/2] (50, 81, 0)

__complex__() → DNDarray

Complex scalar casting.

counts_displs() → Tuple[Tuple[int], Tuple[int]]

Returns actual counts (number of items per process) and displacements (offsets) of the DNDarray. Does not assume load balance.

cpu() → DNDarray

Returns a copy of this object in main memory. If this object is already in main memory, then no copy is performed and the original object is returned.

create_lshape_map(force_check: bool = False) → torch.Tensor

Generate a ‘map’ of the lshapes of the data on all processes. Units are (process rank, lshape)

Parameters:

force_check (bool, optional) – if False (default) and the lshape map has already been created, use the previous result. Otherwise, create the lshape_map

create_partition_interface()

Create a partition interface in line with the DPPY proposal. This is subject to change. The intention of this to facilitate the usage of a general format for the referencing of distributed datasets.

An example of the output and shape is shown below.

__partitioned__ = {

‘shape’: (27, 3, 2), ‘partition_tiling’: (4, 1, 1), ‘partitions’: {

(0, 0, 0): {

‘start’: (0, 0, 0), ‘shape’: (7, 3, 2), ‘data’: tensor([…], dtype=torch.int32), ‘location’: [0], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (1, 0, 0): {

‘start’: (7, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [1], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (2, 0, 0): {

‘start’: (14, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [2], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (3, 0, 0): {

‘start’: (21, 0, 0), ‘shape’: (6, 3, 2), ‘data’: None, ‘location’: [3], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}

}, ‘locals’: [(rank, 0, 0)], ‘get’: lambda x: x,

}

Return type:

dictionary containing the partition interface as shown above.

__float__() → DNDarray

Float scalar casting.

See also

flatten()

fill_diagonal(value: float) → DNDarray

Fill the main diagonal of a 2D DNDarray. This function modifies the input tensor in-place, and returns the input array.

Parameters:

value (float) – The value to be placed in the DNDarrays main diagonal

__getitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis]) → DNDarray

Global getter function for DNDarrays. Returns a new DNDarray composed of the elements of the original tensor selected by the indices given. This does NOT redistribute or rebalance the resulting tensor. If the selection of values is unbalanced then the resultant tensor is also unbalanced! To redistributed the DNDarray use balance() (issue #187)

Parameters:

key (int, slice, Tuple[int,...], List[int,...]) – Indices to get from the tensor.

Examples

>>> a = ht.arange(10, split=0)
(1/2) >>> tensor([0, 1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5, 6, 7, 8, 9], dtype=torch.int32)
>>> a[1:6]
(1/2) >>> tensor([1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5], dtype=torch.int32)
>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1]
(1/2) >>> tensor([0.])
(2/2) >>> tensor([0., 0.])

gpu() → DNDarray

Returns a copy of this object in GPU memory. If this object is already in GPU memory, then no copy is performed and the original object is returned.

__int__() → DNDarray

Integer scalar casting.

is_balanced(force_check: bool = False) → bool

Determine if self is balanced evenly (or as evenly as possible) across all nodes distributed evenly (or as evenly as possible) across all processes. This is equivalent to returning self.balanced. If no information is available (self.balanced = None), the balanced status will be assessed via collective communication.

Parameters:

force_check (bool, optional) – If True, the balanced status of the DNDarray will be assessed via collective communication in any case.

is_distributed() → bool

Determines whether the data of this DNDarray is distributed across multiple processes.

__key_is_singular(key: any, axis: int, self_proxy: torch.Tensor) → bool

__key_adds_dimension(key: any, axis: int, self_proxy: torch.Tensor) → bool

item()

Returns the only element of a 1-element DNDarray. Mirror of the pytorch command by the same name. If size of DNDarray is >1 element, then a ValueError is raised (by pytorch)

Examples

>>> import heat as ht
>>> x = ht.zeros((1))
>>> x.item()
0.0

__len__() → int

The length of the DNDarray, i.e. the number of items in the first dimension.

numpy() → numpy.array

Returns a copy of the DNDarray as numpy ndarray. If the DNDarray resides on the GPU, the underlying data will be copied to the CPU first.

If the DNDarray is distributed, an MPI Allgather operation will be performed before converting to np.ndarray, i.e. each MPI process will end up holding a copy of the entire array in memory. Make sure process memory is sufficient!

Examples

>>> import heat as ht
T1 = ht.random.randn((10,8))
T1.numpy()

_repr_pretty_(p, cycle)

Pretty print for IPython.

__repr__() → str

Returns a printable representation of the passed DNDarray, targeting developers.

ravel()

Flattens the DNDarray.

See also

ravel()

Examples

>>> a = ht.ones((2, 3), split=0)
>>> b = a.ravel()
>>> a[0, 0] = 4
>>> b
DNDarray([4., 1., 1., 1., 1., 1.], dtype=ht.float32, device=cpu:0, split=0)

redistribute_(lshape_map: torch.Tensor | None = None, target_map: torch.Tensor | None = None)

Redistributes the data of the DNDarray along the split axis to match the given target map. This function does not modify the non-split dimensions of the DNDarray. This is an abstraction and extension of the balance function.

Parameters:

• lshape_map (torch.Tensor, optional) – The current lshape of processes. Units are [rank, lshape].

• target_map (torch.Tensor, optional) – The desired distribution across the processes. Units are [rank, target lshape]. Note: the only important parts of the target map are the values along the split axis, values which are not along this axis are there to mimic the shape of the lshape_map.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> target_map = torch.zeros((st.comm.size, 3), dtype=torch.int64)
>>> target_map[0, 2] = 67
>>> print(target_map)
[0/2] tensor([[ 0,  0, 67],
[0/2]         [ 0,  0,  0],
[0/2]         [ 0,  0,  0]], dtype=torch.int32)
[1/2] tensor([[ 0,  0, 67],
[1/2]         [ 0,  0,  0],
[1/2]         [ 0,  0,  0]], dtype=torch.int32)
[2/2] tensor([[ 0,  0, 67],
[2/2]         [ 0,  0,  0],
[2/2]         [ 0,  0,  0]], dtype=torch.int32)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.redistribute_(target_map=target_map)
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)

__redistribute_shuffle(snd_pr: int | torch.Tensor, send_amt: int | torch.Tensor, rcv_pr: int | torch.Tensor, snd_dtype: torch.dtype)

Function to abstract the function used during redistribute for shuffling data between processes along the split axis

Parameters:

• snd_pr (int or torch.Tensor) – Sending process

• send_amt (int or torch.Tensor) – Amount of data to be sent by the sending process

• rcv_pr (int or torch.Tensor) – Receiving process

• snd_dtype (torch.dtype) – Torch type of the data in question

resplit_(axis: int = None)

In-place option for resplitting a DNDarray.

Parameters:

axis (int) – The new split axis, None denotes gathering, an int will set the new split axis

Examples

>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, None)
>>> a.split
None
>>> a.lshape
(0/2) (4, 5)
(1/2) (4, 5)
>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, 1)
>>> a.split
1
>>> a.lshape
(0/2) (4, 3)
(1/2) (4, 2)

__setitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Global item setter

Parameters:

• key (Union[int, Tuple[int,...], List[int,...]]) – Index/indices to be set

• value (Union[float, DNDarray,torch.Tensor]) – Value to be set to the specified positions in the DNDarray (self)

Notes

If a DNDarray is given as the value to be set then the split axes are assumed to be equal. If they are not, PyTorch will raise an error when the values are attempted to be set on the local array

Examples

>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1] = 1
>>> a
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])
(2/2) >>> tensor([[0., 1., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])

__setter(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Utility function for checking value and forwarding to :func:__setitem__

Raises:

NotImplementedError – If the type of value ist not supported

__str__() → str

Computes a string representation of the passed DNDarray.

tolist(keepsplit: bool = False) → List

Return a copy of the local array data as a (nested) Python list. For scalars, a standard Python number is returned.

Parameters:

keepsplit (bool) – Whether the list should be returned locally or globally.

Examples

>>> a = ht.array([[0, 1], [2, 3]])
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=0)
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=1)
>>> a.tolist(keepsplit=True)
(1/2) [[0], [2]]
(2/2) [[1], [3]]

__torch_function__(func, types, args=(), kwargs=None)

Supports PyTorch’s dispatch mechanism.

__torch_proxy__() → torch.Tensor

Return a 1-element torch.Tensor strided as the global self shape. Used internally for sanitation purposes.

__xitem_get_key_start_stop(rank: int, actives: list, key_st: int, key_sp: int, step: int, ends: torch.Tensor, og_key_st: int) → Tuple[int, int]

class KMeans(n_clusters: int = 8, init: str | heat.core.dndarray.DNDarray = 'random', max_iter: int = 300, tol: float = 0.0001, random_state: int | None = None)

Bases: heat.cluster._kcluster._KCluster

K-Means clustering algorithm. An implementation of Lloyd’s algorithm [1].

Variables:

• n_clusters (int) – The number of clusters to form as well as the number of centroids to generate.

• init (str or DNDarray) –

  Method for initialization:

  • ‘k-means++’ : selects initial cluster centers for the clustering in a smart way to speed up convergence [2].

  • ‘random’: choose k observations (rows) at random from data for the initial centroids.

  • ’batchparallel’: initialize by using the batch parallel algorithm (see BatchParallelKMeans for more information).

  • DNDarray: it should be of shape (n_clusters, n_features) and gives the initial centers.

• max_iter (int) – Maximum number of iterations of the k-means algorithm for a single run.

• tol (float) – Relative tolerance with regards to inertia to declare convergence.

• random_state (int) – Determines random number generation for centroid initialization.

Notes

The average complexity is given by \(O(k \cdot n \cdot T)\), were n is the number of samples and \(T\) is the number of iterations. In practice, the k-means algorithm is very fast, but it may fall into local minima. That is why it can be useful to restart it several times. If the algorithm stops before fully converging (because of tol or max_iter), labels_ and cluster_centers_ will not be consistent, i.e. the cluster_centers_ will not be the means of the points in each cluster. Also, the estimator will reassign labels_ after the last iteration to make labels_ consistent with predict on the training set.

References

[1] Lloyd, Stuart P., “Least squares quantization in PCM”, IEEE Transactions on Information Theory, 28 (2), pp. 129–137, 1982.

[2] Arthur, D., Vassilvitskii, S., “k-means++: The Advantages of Careful Seeding”, Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, Society for Industrial and Applied Mathematics Philadelphia, PA, USA. pp. 1027–1035, 2007.

_p = 2

_update_centroids(x: heat.core.dndarray.DNDarray, matching_centroids: heat.core.dndarray.DNDarray)

Compute coordinates of new centroid as mean of the data points in x that are assigned to this centroid.

Parameters:

• x (DNDarray) – Input data

• matching_centroids (DNDarray) – Array filled with indices i indicating to which cluster ci each sample point in x is assigned

fit(x: heat.core.dndarray.DNDarray, oversampling: float = 2, iter_multiplier: float = 1) → KMeans.fit.self

Computes the centroid of a k-means clustering. Reduce the values of the parameters ‘oversampling’ and ‘iter_multiplier’ to speed up the computation, if necessary. However, for too low values the initialization of cluster centers might fail and raise a corresponding ValueError.

Parameters:

• x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features)

• oversampling (float) – oversampling factor used for the k-means|| initializiation of centroids

• iter_multiplier (float) – factor that increases the number of iterations used in the initialization of centroids

class _KCluster(metric: Callable, n_clusters: int, init: str | heat.core.dndarray.DNDarray, max_iter: int, tol: float, random_state: int)

Bases: heat.ClusteringMixin, heat.BaseEstimator

Base class for k-statistics clustering algorithms (kmeans, kmedians, kmedoids). The clusters are represented by centroids ci (we use the term from kmeans for simplicity)

Parameters:

• metric (function) – One of the distance metrics in ht.spatial.distance. Needs to be passed as lambda function to take only two arrays as input

• n_clusters (int) – The number of clusters to form as well as the number of centroids to generate.

• init (str or DNDarray, default: ‘random’) –

  Method for initialization:

  • ‘probability_based’ : selects initial cluster centers for the clustering in a smart way to speed up convergence (k-means++)

  • ‘random’: choose k observations (rows) at random from data for the initial centroids.

  • ’batchparallel’: use the batch parallel algorithm to initialize the centroids, only available for split=0 and KMeans or KMedians

  • DNDarray: gives the initial centers, should be of Shape = (n_clusters, n_features)

• max_iter (int) – Maximum number of iterations for a single run.

• tol (float, default: 1e-4) – Relative tolerance with regards to inertia to declare convergence.

• random_state (int) – Determines random number generation for centroid initialization.

n_clusters

init

max_iter

tol

random_state

_metric

_cluster_centers = None

_functional_value = None

_labels = None

_inertia = None

_n_iter = None

_p = None

_initialize_cluster_centers(x: heat.core.dndarray.DNDarray, oversampling: float, iter_multiplier: float)

Initializes the K-Means centroids.

Parameters:

• x (DNDarray) – The data to initialize the clusters for. Shape = (n_samples, n_features)

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• iter_multiplier (float) – factor that increases the number of iterations used in the initialization of centroids

_centroid_sampling_helper(x: heat.core.dndarray.DNDarray, centroids: heat.core.dndarray.DNDarray, oversampling: float, num_iters: int)

Helper function for the k-means|| initialization of centroids. Samples new centroids based on a probability distribution derived from the distance of data points to the current set of centroids.

Parameters:

• x (DNDarray) – The data to initialize the clusters for. Shape = (n_samples, n_features)

• centroids (DNDarray) – The initial set of centroids

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• num_iters (float) – number of iterations used in the initialization of centroids

_assign_to_cluster(x: heat.core.dndarray.DNDarray, eval_functional_value: bool = False)

Assigns the passed data points to the centroids based on the respective metric

Parameters:

• x (DNDarray) – Data points, Shape = (n_samples, n_features)

• eval_functional_value (bool, default: False) – If True, the current K-Clustering functional value of the clustering algorithm is evaluated

_update_centroids(x: heat.core.dndarray.DNDarray, matching_centroids: heat.core.dndarray.DNDarray)

The Update strategy is algorithm specific (e.g. calculate mean of assigned points for kmeans, median for kmedians, etc.)

Parameters:

• x (DNDarray) – Input Data

• matching_centroids (DNDarray) – Index array of assigned centroids

fit(x: heat.core.dndarray.DNDarray)

Computes the centroid of the clustering algorithm to fit the data x. The full pipeline is algorithm specific.

Parameters:

x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features)

predict(x: heat.core.dndarray.DNDarray)

Predict the closest cluster each sample in x belongs to.

In the vector quantization literature, cluster_centers_() is called the code book and each value returned by predict is the index of the closest code in the code book.

Parameters:

x (DNDarray) – New data to predict. Shape = (n_samples, n_features)

class DNDarray(array: torch.Tensor, gshape: Tuple[int, Ellipsis], dtype: heat.core.types.datatype, split: int | None, device: heat.core.devices.Device, comm: Communication, balanced: bool)

Distributed N-Dimensional array. The core element of HeAT. It is composed of PyTorch tensors local to each process.

Parameters:

• array (torch.Tensor) – Local array elements

• gshape (Tuple[int,...]) – The global shape of the array

• dtype (datatype) – The datatype of the array

• split (int or None) – The axis on which the array is divided between processes

• device (Device) – The device on which the local arrays are using (cpu or gpu)

• comm (Communication) – The communications object for sending and receiving data

• balanced (bool or None) – Describes whether the data are evenly distributed across processes. If this information is not available (self.balanced is None), it can be gathered via the is_balanced() method (requires communication).

__array

__gshape

__dtype

__split

__device

__comm

__balanced

__ishalo = False

__halo_next = None

__halo_prev = None

__partitions_dict__ = None

__lshape_map = None

__prephalo(start, end) → torch.Tensor

Extracts the halo indexed by start, end from self.array in the direction of self.split

Parameters:

• start (int) – Start index of the halo extracted from self.array

• end (int) – End index of the halo extracted from self.array

get_halo(halo_size: int, prev: bool = True, next: bool = True)

Fetch halos of size halo_size from neighboring ranks and save them in self.halo_next/self.halo_prev.

Parameters:

• halo_size (int) – Size of the halo.

• prev (bool, optional) – If True, fetch the halo from the previous rank. Default: True.

• next (bool, optional) – If True, fetch the halo from the next rank. Default: True.

__cat_halo() → torch.Tensor

Return local array concatenated to halos if they are available.

__array__() → numpy.ndarray

Returns a view of the process-local slice of the DNDarray as a numpy ndarray, if the DNDarray resides on CPU. Otherwise, it returns a copy, on CPU, of the process-local slice of DNDarray as numpy ndarray.

__array_ufunc__(ufunc, method, *inputs, **kwargs)

Override NumPy’s universal functions.

__array_function__(func, types, args, kwargs)

Augments NumPy’s functions.

astype(dtype, copy=True) → DNDarray

Returns a casted version of this array. Casted array is a new array of the same shape but with given type of this array. If copy is True, the same array is returned instead.

Parameters:

• dtype (datatype) – Heat type to which the array is cast

• copy (bool, optional) – By default the operation returns a copy of this array. If copy is set to False the cast is performed in-place and this array is returned

balance_() → DNDarray

Function for balancing a DNDarray between all nodes. To determine if this is needed use the is_balanced() function. If the DNDarray is already balanced this function will do nothing. This function modifies the DNDarray itself and will not return anything.

Examples

>>> a = ht.zeros((10, 2), split=0)
>>> a[:, 0] = ht.arange(10)
>>> b = a[3:]
[0/2] tensor([[3., 0.],
[1/2] tensor([[4., 0.],
              [5., 0.],
              [6., 0.]])
[2/2] tensor([[7., 0.],
              [8., 0.],
              [9., 0.]])
>>> b.balance_()
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (1, 2)
[1/2] (7, 2) (3, 2)
[2/2] (7, 2) (3, 2)
>>> b
[0/2] tensor([[3., 0.],
             [4., 0.],
             [5., 0.]])
[1/2] tensor([[6., 0.],
              [7., 0.]])
[2/2] tensor([[8., 0.],
              [9., 0.]])
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (3, 2)
[1/2] (7, 2) (2, 2)
[2/2] (7, 2) (2, 2)

__bool__() → bool

Boolean scalar casting.

__cast(cast_function) → float | int

Implements a generic cast function for DNDarray objects.

Parameters:

cast_function (function) – The actual cast function, e.g. float or int

Raises:

TypeError – If the DNDarray object cannot be converted into a scalar.

collect_(target_rank: int | None = 0) → None

A method collecting a distributed DNDarray to one MPI rank, chosen by the target_rank variable. It is a specific case of the redistribute_ method.

Parameters:

target_rank (int, optional) – The rank to which the DNDarray will be collected. Default: 0.

Raises:

• TypeError – If the target rank is not an integer.

• ValueError – If the target rank is out of bounds.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.collect_()
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)
>>> st.collect_(1)
>>> print(st.lshape)
[0/2] (50, 81, 0)
[1/2] (50, 81, 67)
[2/2] (50, 81, 0)

__complex__() → DNDarray

Complex scalar casting.

counts_displs() → Tuple[Tuple[int], Tuple[int]]

Returns actual counts (number of items per process) and displacements (offsets) of the DNDarray. Does not assume load balance.

cpu() → DNDarray

Returns a copy of this object in main memory. If this object is already in main memory, then no copy is performed and the original object is returned.

create_lshape_map(force_check: bool = False) → torch.Tensor

Generate a ‘map’ of the lshapes of the data on all processes. Units are (process rank, lshape)

Parameters:

force_check (bool, optional) – if False (default) and the lshape map has already been created, use the previous result. Otherwise, create the lshape_map

create_partition_interface()

Create a partition interface in line with the DPPY proposal. This is subject to change. The intention of this to facilitate the usage of a general format for the referencing of distributed datasets.

An example of the output and shape is shown below.

__partitioned__ = {

‘shape’: (27, 3, 2), ‘partition_tiling’: (4, 1, 1), ‘partitions’: {

(0, 0, 0): {

‘start’: (0, 0, 0), ‘shape’: (7, 3, 2), ‘data’: tensor([…], dtype=torch.int32), ‘location’: [0], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (1, 0, 0): {

‘start’: (7, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [1], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (2, 0, 0): {

‘start’: (14, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [2], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (3, 0, 0): {

‘start’: (21, 0, 0), ‘shape’: (6, 3, 2), ‘data’: None, ‘location’: [3], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}

}, ‘locals’: [(rank, 0, 0)], ‘get’: lambda x: x,

}

Return type:

dictionary containing the partition interface as shown above.

__float__() → DNDarray

Float scalar casting.

See also

flatten()

fill_diagonal(value: float) → DNDarray

Fill the main diagonal of a 2D DNDarray. This function modifies the input tensor in-place, and returns the input array.

Parameters:

value (float) – The value to be placed in the DNDarrays main diagonal

__getitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis]) → DNDarray

Global getter function for DNDarrays. Returns a new DNDarray composed of the elements of the original tensor selected by the indices given. This does NOT redistribute or rebalance the resulting tensor. If the selection of values is unbalanced then the resultant tensor is also unbalanced! To redistributed the DNDarray use balance() (issue #187)

Parameters:

key (int, slice, Tuple[int,...], List[int,...]) – Indices to get from the tensor.

Examples

>>> a = ht.arange(10, split=0)
(1/2) >>> tensor([0, 1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5, 6, 7, 8, 9], dtype=torch.int32)
>>> a[1:6]
(1/2) >>> tensor([1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5], dtype=torch.int32)
>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1]
(1/2) >>> tensor([0.])
(2/2) >>> tensor([0., 0.])

gpu() → DNDarray

Returns a copy of this object in GPU memory. If this object is already in GPU memory, then no copy is performed and the original object is returned.

__int__() → DNDarray

Integer scalar casting.

is_balanced(force_check: bool = False) → bool

Determine if self is balanced evenly (or as evenly as possible) across all nodes distributed evenly (or as evenly as possible) across all processes. This is equivalent to returning self.balanced. If no information is available (self.balanced = None), the balanced status will be assessed via collective communication.

Parameters:

force_check (bool, optional) – If True, the balanced status of the DNDarray will be assessed via collective communication in any case.

is_distributed() → bool

Determines whether the data of this DNDarray is distributed across multiple processes.

__key_is_singular(key: any, axis: int, self_proxy: torch.Tensor) → bool

__key_adds_dimension(key: any, axis: int, self_proxy: torch.Tensor) → bool

item()

Returns the only element of a 1-element DNDarray. Mirror of the pytorch command by the same name. If size of DNDarray is >1 element, then a ValueError is raised (by pytorch)

Examples

>>> import heat as ht
>>> x = ht.zeros((1))
>>> x.item()
0.0

__len__() → int

The length of the DNDarray, i.e. the number of items in the first dimension.

numpy() → numpy.array

Returns a copy of the DNDarray as numpy ndarray. If the DNDarray resides on the GPU, the underlying data will be copied to the CPU first.

If the DNDarray is distributed, an MPI Allgather operation will be performed before converting to np.ndarray, i.e. each MPI process will end up holding a copy of the entire array in memory. Make sure process memory is sufficient!

Examples

>>> import heat as ht
T1 = ht.random.randn((10,8))
T1.numpy()

_repr_pretty_(p, cycle)

Pretty print for IPython.

__repr__() → str

Returns a printable representation of the passed DNDarray, targeting developers.

ravel()

Flattens the DNDarray.

See also

ravel()

Examples

>>> a = ht.ones((2, 3), split=0)
>>> b = a.ravel()
>>> a[0, 0] = 4
>>> b
DNDarray([4., 1., 1., 1., 1., 1.], dtype=ht.float32, device=cpu:0, split=0)

redistribute_(lshape_map: torch.Tensor | None = None, target_map: torch.Tensor | None = None)

Redistributes the data of the DNDarray along the split axis to match the given target map. This function does not modify the non-split dimensions of the DNDarray. This is an abstraction and extension of the balance function.

Parameters:

• lshape_map (torch.Tensor, optional) – The current lshape of processes. Units are [rank, lshape].

• target_map (torch.Tensor, optional) – The desired distribution across the processes. Units are [rank, target lshape]. Note: the only important parts of the target map are the values along the split axis, values which are not along this axis are there to mimic the shape of the lshape_map.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> target_map = torch.zeros((st.comm.size, 3), dtype=torch.int64)
>>> target_map[0, 2] = 67
>>> print(target_map)
[0/2] tensor([[ 0,  0, 67],
[0/2]         [ 0,  0,  0],
[0/2]         [ 0,  0,  0]], dtype=torch.int32)
[1/2] tensor([[ 0,  0, 67],
[1/2]         [ 0,  0,  0],
[1/2]         [ 0,  0,  0]], dtype=torch.int32)
[2/2] tensor([[ 0,  0, 67],
[2/2]         [ 0,  0,  0],
[2/2]         [ 0,  0,  0]], dtype=torch.int32)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.redistribute_(target_map=target_map)
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)

__redistribute_shuffle(snd_pr: int | torch.Tensor, send_amt: int | torch.Tensor, rcv_pr: int | torch.Tensor, snd_dtype: torch.dtype)

Function to abstract the function used during redistribute for shuffling data between processes along the split axis

Parameters:

• snd_pr (int or torch.Tensor) – Sending process

• send_amt (int or torch.Tensor) – Amount of data to be sent by the sending process

• rcv_pr (int or torch.Tensor) – Receiving process

• snd_dtype (torch.dtype) – Torch type of the data in question

resplit_(axis: int = None)

In-place option for resplitting a DNDarray.

Parameters:

axis (int) – The new split axis, None denotes gathering, an int will set the new split axis

Examples

>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, None)
>>> a.split
None
>>> a.lshape
(0/2) (4, 5)
(1/2) (4, 5)
>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, 1)
>>> a.split
1
>>> a.lshape
(0/2) (4, 3)
(1/2) (4, 2)

__setitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Global item setter

Parameters:

• key (Union[int, Tuple[int,...], List[int,...]]) – Index/indices to be set

• value (Union[float, DNDarray,torch.Tensor]) – Value to be set to the specified positions in the DNDarray (self)

Notes

If a DNDarray is given as the value to be set then the split axes are assumed to be equal. If they are not, PyTorch will raise an error when the values are attempted to be set on the local array

Examples

>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1] = 1
>>> a
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])
(2/2) >>> tensor([[0., 1., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])

__setter(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Utility function for checking value and forwarding to :func:__setitem__

Raises:

NotImplementedError – If the type of value ist not supported

__str__() → str

Computes a string representation of the passed DNDarray.

tolist(keepsplit: bool = False) → List

Return a copy of the local array data as a (nested) Python list. For scalars, a standard Python number is returned.

Parameters:

keepsplit (bool) – Whether the list should be returned locally or globally.

Examples

>>> a = ht.array([[0, 1], [2, 3]])
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=0)
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=1)
>>> a.tolist(keepsplit=True)
(1/2) [[0], [2]]
(2/2) [[1], [3]]

__torch_function__(func, types, args=(), kwargs=None)

Supports PyTorch’s dispatch mechanism.

__torch_proxy__() → torch.Tensor

Return a 1-element torch.Tensor strided as the global self shape. Used internally for sanitation purposes.

__xitem_get_key_start_stop(rank: int, actives: list, key_st: int, key_sp: int, step: int, ends: torch.Tensor, og_key_st: int) → Tuple[int, int]

class KMedians(n_clusters: int = 8, init: str | heat.core.dndarray.DNDarray = 'random', max_iter: int = 300, tol: float = 0.0001, random_state: int = None)

Bases: heat.cluster._kcluster._KCluster

K-Medians clustering algorithm [1]. Uses the Manhattan (City-block, \(L_1\)) metric for distance calculations

Parameters:

• n_clusters (int, optional, default: 8) – The number of clusters to form as well as the number of centroids to generate.

• init (str or DNDarray, default: ‘random’) –

  Method for initialization:

  • ‘k-medians++’ : selects initial cluster centers for the clustering in a smart way to speed up convergence [2].

  • ‘random’: choose k observations (rows) at random from data for the initial centroids.

  • ’batchparallel’: initialize by using the batch parallel algorithm (see BatchParallelKMedians for more information).

  • DNDarray: gives the initial centers, should be of Shape = (n_clusters, n_features)

• max_iter (int, default: 300) – Maximum number of iterations of the k-means algorithm for a single run.

• tol (float, default: 1e-4) – Relative tolerance with regards to inertia to declare convergence.

• random_state (int) – Determines random number generation for centroid initialization.

References

[1] Hakimi, S., and O. Kariv. “An algorithmic approach to network location problems II: The p-medians.” SIAM Journal on Applied Mathematics 37.3 (1979): 539-560.

_p = 1

_update_centroids(x: heat.core.dndarray.DNDarray, matching_centroids: heat.core.dndarray.DNDarray)

Compute coordinates of new centroid as median of the data points in x that are assigned to it

Parameters:

• x (DNDarray) – Input data

• matching_centroids (DNDarray) – Array filled with indeces i indicating to which cluster ci each sample point in x is assigned

fit(x: heat.core.dndarray.DNDarray, oversampling: float = 2, iter_multiplier: float = 1)

Computes the centroid of a k-medians clustering.

Parameters:

• x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features)

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• iter_multiplier (float) – factor that increases the number of iterations used in the initialization of centroids

class _KCluster(metric: Callable, n_clusters: int, init: str | heat.core.dndarray.DNDarray, max_iter: int, tol: float, random_state: int)

Bases: heat.ClusteringMixin, heat.BaseEstimator

Base class for k-statistics clustering algorithms (kmeans, kmedians, kmedoids). The clusters are represented by centroids ci (we use the term from kmeans for simplicity)

Parameters:

• metric (function) – One of the distance metrics in ht.spatial.distance. Needs to be passed as lambda function to take only two arrays as input

• n_clusters (int) – The number of clusters to form as well as the number of centroids to generate.

• init (str or DNDarray, default: ‘random’) –

  Method for initialization:

  • ‘probability_based’ : selects initial cluster centers for the clustering in a smart way to speed up convergence (k-means++)

  • ‘random’: choose k observations (rows) at random from data for the initial centroids.

  • ’batchparallel’: use the batch parallel algorithm to initialize the centroids, only available for split=0 and KMeans or KMedians

  • DNDarray: gives the initial centers, should be of Shape = (n_clusters, n_features)

• max_iter (int) – Maximum number of iterations for a single run.

• tol (float, default: 1e-4) – Relative tolerance with regards to inertia to declare convergence.

• random_state (int) – Determines random number generation for centroid initialization.

n_clusters

init

max_iter

tol

random_state

_metric

_cluster_centers = None

_functional_value = None

_labels = None

_inertia = None

_n_iter = None

_p = None

_initialize_cluster_centers(x: heat.core.dndarray.DNDarray, oversampling: float, iter_multiplier: float)

Initializes the K-Means centroids.

Parameters:

• x (DNDarray) – The data to initialize the clusters for. Shape = (n_samples, n_features)

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• iter_multiplier (float) – factor that increases the number of iterations used in the initialization of centroids

_centroid_sampling_helper(x: heat.core.dndarray.DNDarray, centroids: heat.core.dndarray.DNDarray, oversampling: float, num_iters: int)

Helper function for the k-means|| initialization of centroids. Samples new centroids based on a probability distribution derived from the distance of data points to the current set of centroids.

Parameters:

• x (DNDarray) – The data to initialize the clusters for. Shape = (n_samples, n_features)

• centroids (DNDarray) – The initial set of centroids

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• num_iters (float) – number of iterations used in the initialization of centroids

_assign_to_cluster(x: heat.core.dndarray.DNDarray, eval_functional_value: bool = False)

Assigns the passed data points to the centroids based on the respective metric

Parameters:

• x (DNDarray) – Data points, Shape = (n_samples, n_features)

• eval_functional_value (bool, default: False) – If True, the current K-Clustering functional value of the clustering algorithm is evaluated

_update_centroids(x: heat.core.dndarray.DNDarray, matching_centroids: heat.core.dndarray.DNDarray)

The Update strategy is algorithm specific (e.g. calculate mean of assigned points for kmeans, median for kmedians, etc.)

Parameters:

• x (DNDarray) – Input Data

• matching_centroids (DNDarray) – Index array of assigned centroids

fit(x: heat.core.dndarray.DNDarray)

Computes the centroid of the clustering algorithm to fit the data x. The full pipeline is algorithm specific.

Parameters:

x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features)

predict(x: heat.core.dndarray.DNDarray)

Predict the closest cluster each sample in x belongs to.

In the vector quantization literature, cluster_centers_() is called the code book and each value returned by predict is the index of the closest code in the code book.

Parameters:

x (DNDarray) – New data to predict. Shape = (n_samples, n_features)

class DNDarray(array: torch.Tensor, gshape: Tuple[int, Ellipsis], dtype: heat.core.types.datatype, split: int | None, device: heat.core.devices.Device, comm: Communication, balanced: bool)

Distributed N-Dimensional array. The core element of HeAT. It is composed of PyTorch tensors local to each process.

Parameters:

• array (torch.Tensor) – Local array elements

• gshape (Tuple[int,...]) – The global shape of the array

• dtype (datatype) – The datatype of the array

• split (int or None) – The axis on which the array is divided between processes

• device (Device) – The device on which the local arrays are using (cpu or gpu)

• comm (Communication) – The communications object for sending and receiving data

• balanced (bool or None) – Describes whether the data are evenly distributed across processes. If this information is not available (self.balanced is None), it can be gathered via the is_balanced() method (requires communication).

__array

__gshape

__dtype

__split

__device

__comm

__balanced

__ishalo = False

__halo_next = None

__halo_prev = None

__partitions_dict__ = None

__lshape_map = None

__prephalo(start, end) → torch.Tensor

Extracts the halo indexed by start, end from self.array in the direction of self.split

Parameters:

• start (int) – Start index of the halo extracted from self.array

• end (int) – End index of the halo extracted from self.array

get_halo(halo_size: int, prev: bool = True, next: bool = True)

Fetch halos of size halo_size from neighboring ranks and save them in self.halo_next/self.halo_prev.

Parameters:

• halo_size (int) – Size of the halo.

• prev (bool, optional) – If True, fetch the halo from the previous rank. Default: True.

• next (bool, optional) – If True, fetch the halo from the next rank. Default: True.

__cat_halo() → torch.Tensor

Return local array concatenated to halos if they are available.

__array__() → numpy.ndarray

Returns a view of the process-local slice of the DNDarray as a numpy ndarray, if the DNDarray resides on CPU. Otherwise, it returns a copy, on CPU, of the process-local slice of DNDarray as numpy ndarray.

__array_ufunc__(ufunc, method, *inputs, **kwargs)

Override NumPy’s universal functions.

__array_function__(func, types, args, kwargs)

Augments NumPy’s functions.

astype(dtype, copy=True) → DNDarray

Returns a casted version of this array. Casted array is a new array of the same shape but with given type of this array. If copy is True, the same array is returned instead.

Parameters:

• dtype (datatype) – Heat type to which the array is cast

• copy (bool, optional) – By default the operation returns a copy of this array. If copy is set to False the cast is performed in-place and this array is returned

balance_() → DNDarray

Function for balancing a DNDarray between all nodes. To determine if this is needed use the is_balanced() function. If the DNDarray is already balanced this function will do nothing. This function modifies the DNDarray itself and will not return anything.

Examples

>>> a = ht.zeros((10, 2), split=0)
>>> a[:, 0] = ht.arange(10)
>>> b = a[3:]
[0/2] tensor([[3., 0.],
[1/2] tensor([[4., 0.],
              [5., 0.],
              [6., 0.]])
[2/2] tensor([[7., 0.],
              [8., 0.],
              [9., 0.]])
>>> b.balance_()
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (1, 2)
[1/2] (7, 2) (3, 2)
[2/2] (7, 2) (3, 2)
>>> b
[0/2] tensor([[3., 0.],
             [4., 0.],
             [5., 0.]])
[1/2] tensor([[6., 0.],
              [7., 0.]])
[2/2] tensor([[8., 0.],
              [9., 0.]])
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (3, 2)
[1/2] (7, 2) (2, 2)
[2/2] (7, 2) (2, 2)

__bool__() → bool

Boolean scalar casting.

__cast(cast_function) → float | int

Implements a generic cast function for DNDarray objects.

Parameters:

cast_function (function) – The actual cast function, e.g. float or int

Raises:

TypeError – If the DNDarray object cannot be converted into a scalar.

collect_(target_rank: int | None = 0) → None

A method collecting a distributed DNDarray to one MPI rank, chosen by the target_rank variable. It is a specific case of the redistribute_ method.

Parameters:

target_rank (int, optional) – The rank to which the DNDarray will be collected. Default: 0.

Raises:

• TypeError – If the target rank is not an integer.

• ValueError – If the target rank is out of bounds.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.collect_()
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)
>>> st.collect_(1)
>>> print(st.lshape)
[0/2] (50, 81, 0)
[1/2] (50, 81, 67)
[2/2] (50, 81, 0)

__complex__() → DNDarray

Complex scalar casting.

counts_displs() → Tuple[Tuple[int], Tuple[int]]

Returns actual counts (number of items per process) and displacements (offsets) of the DNDarray. Does not assume load balance.

cpu() → DNDarray

Returns a copy of this object in main memory. If this object is already in main memory, then no copy is performed and the original object is returned.

create_lshape_map(force_check: bool = False) → torch.Tensor

Generate a ‘map’ of the lshapes of the data on all processes. Units are (process rank, lshape)

Parameters:

force_check (bool, optional) – if False (default) and the lshape map has already been created, use the previous result. Otherwise, create the lshape_map

create_partition_interface()

Create a partition interface in line with the DPPY proposal. This is subject to change. The intention of this to facilitate the usage of a general format for the referencing of distributed datasets.

An example of the output and shape is shown below.

__partitioned__ = {

‘shape’: (27, 3, 2), ‘partition_tiling’: (4, 1, 1), ‘partitions’: {

(0, 0, 0): {

‘start’: (0, 0, 0), ‘shape’: (7, 3, 2), ‘data’: tensor([…], dtype=torch.int32), ‘location’: [0], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (1, 0, 0): {

‘start’: (7, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [1], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (2, 0, 0): {

‘start’: (14, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [2], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (3, 0, 0): {

‘start’: (21, 0, 0), ‘shape’: (6, 3, 2), ‘data’: None, ‘location’: [3], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}

}, ‘locals’: [(rank, 0, 0)], ‘get’: lambda x: x,

}

Return type:

dictionary containing the partition interface as shown above.

__float__() → DNDarray

Float scalar casting.

See also

flatten()

fill_diagonal(value: float) → DNDarray

Fill the main diagonal of a 2D DNDarray. This function modifies the input tensor in-place, and returns the input array.

Parameters:

value (float) – The value to be placed in the DNDarrays main diagonal

__getitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis]) → DNDarray

Global getter function for DNDarrays. Returns a new DNDarray composed of the elements of the original tensor selected by the indices given. This does NOT redistribute or rebalance the resulting tensor. If the selection of values is unbalanced then the resultant tensor is also unbalanced! To redistributed the DNDarray use balance() (issue #187)

Parameters:

key (int, slice, Tuple[int,...], List[int,...]) – Indices to get from the tensor.

Examples

>>> a = ht.arange(10, split=0)
(1/2) >>> tensor([0, 1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5, 6, 7, 8, 9], dtype=torch.int32)
>>> a[1:6]
(1/2) >>> tensor([1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5], dtype=torch.int32)
>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1]
(1/2) >>> tensor([0.])
(2/2) >>> tensor([0., 0.])

gpu() → DNDarray

Returns a copy of this object in GPU memory. If this object is already in GPU memory, then no copy is performed and the original object is returned.

__int__() → DNDarray

Integer scalar casting.

is_balanced(force_check: bool = False) → bool

Determine if self is balanced evenly (or as evenly as possible) across all nodes distributed evenly (or as evenly as possible) across all processes. This is equivalent to returning self.balanced. If no information is available (self.balanced = None), the balanced status will be assessed via collective communication.

Parameters:

force_check (bool, optional) – If True, the balanced status of the DNDarray will be assessed via collective communication in any case.

is_distributed() → bool

Determines whether the data of this DNDarray is distributed across multiple processes.

__key_is_singular(key: any, axis: int, self_proxy: torch.Tensor) → bool

__key_adds_dimension(key: any, axis: int, self_proxy: torch.Tensor) → bool

item()

Returns the only element of a 1-element DNDarray. Mirror of the pytorch command by the same name. If size of DNDarray is >1 element, then a ValueError is raised (by pytorch)

Examples

>>> import heat as ht
>>> x = ht.zeros((1))
>>> x.item()
0.0

__len__() → int

The length of the DNDarray, i.e. the number of items in the first dimension.

numpy() → numpy.array

Returns a copy of the DNDarray as numpy ndarray. If the DNDarray resides on the GPU, the underlying data will be copied to the CPU first.

If the DNDarray is distributed, an MPI Allgather operation will be performed before converting to np.ndarray, i.e. each MPI process will end up holding a copy of the entire array in memory. Make sure process memory is sufficient!

Examples

>>> import heat as ht
T1 = ht.random.randn((10,8))
T1.numpy()

_repr_pretty_(p, cycle)

Pretty print for IPython.

__repr__() → str

Returns a printable representation of the passed DNDarray, targeting developers.

ravel()

Flattens the DNDarray.

See also

ravel()

Examples

>>> a = ht.ones((2, 3), split=0)
>>> b = a.ravel()
>>> a[0, 0] = 4
>>> b
DNDarray([4., 1., 1., 1., 1., 1.], dtype=ht.float32, device=cpu:0, split=0)

redistribute_(lshape_map: torch.Tensor | None = None, target_map: torch.Tensor | None = None)

Redistributes the data of the DNDarray along the split axis to match the given target map. This function does not modify the non-split dimensions of the DNDarray. This is an abstraction and extension of the balance function.

Parameters:

• lshape_map (torch.Tensor, optional) – The current lshape of processes. Units are [rank, lshape].

• target_map (torch.Tensor, optional) – The desired distribution across the processes. Units are [rank, target lshape]. Note: the only important parts of the target map are the values along the split axis, values which are not along this axis are there to mimic the shape of the lshape_map.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> target_map = torch.zeros((st.comm.size, 3), dtype=torch.int64)
>>> target_map[0, 2] = 67
>>> print(target_map)
[0/2] tensor([[ 0,  0, 67],
[0/2]         [ 0,  0,  0],
[0/2]         [ 0,  0,  0]], dtype=torch.int32)
[1/2] tensor([[ 0,  0, 67],
[1/2]         [ 0,  0,  0],
[1/2]         [ 0,  0,  0]], dtype=torch.int32)
[2/2] tensor([[ 0,  0, 67],
[2/2]         [ 0,  0,  0],
[2/2]         [ 0,  0,  0]], dtype=torch.int32)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.redistribute_(target_map=target_map)
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)

__redistribute_shuffle(snd_pr: int | torch.Tensor, send_amt: int | torch.Tensor, rcv_pr: int | torch.Tensor, snd_dtype: torch.dtype)

Function to abstract the function used during redistribute for shuffling data between processes along the split axis

Parameters:

• snd_pr (int or torch.Tensor) – Sending process

• send_amt (int or torch.Tensor) – Amount of data to be sent by the sending process

• rcv_pr (int or torch.Tensor) – Receiving process

• snd_dtype (torch.dtype) – Torch type of the data in question

resplit_(axis: int = None)

In-place option for resplitting a DNDarray.

Parameters:

axis (int) – The new split axis, None denotes gathering, an int will set the new split axis

Examples

>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, None)
>>> a.split
None
>>> a.lshape
(0/2) (4, 5)
(1/2) (4, 5)
>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, 1)
>>> a.split
1
>>> a.lshape
(0/2) (4, 3)
(1/2) (4, 2)

__setitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Global item setter

Parameters:

• key (Union[int, Tuple[int,...], List[int,...]]) – Index/indices to be set

• value (Union[float, DNDarray,torch.Tensor]) – Value to be set to the specified positions in the DNDarray (self)

Notes

If a DNDarray is given as the value to be set then the split axes are assumed to be equal. If they are not, PyTorch will raise an error when the values are attempted to be set on the local array

Examples

>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1] = 1
>>> a
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])
(2/2) >>> tensor([[0., 1., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])

__setter(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Utility function for checking value and forwarding to :func:__setitem__

Raises:

NotImplementedError – If the type of value ist not supported

__str__() → str

Computes a string representation of the passed DNDarray.

tolist(keepsplit: bool = False) → List

Return a copy of the local array data as a (nested) Python list. For scalars, a standard Python number is returned.

Parameters:

keepsplit (bool) – Whether the list should be returned locally or globally.

Examples

>>> a = ht.array([[0, 1], [2, 3]])
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=0)
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=1)
>>> a.tolist(keepsplit=True)
(1/2) [[0], [2]]
(2/2) [[1], [3]]

__torch_function__(func, types, args=(), kwargs=None)

Supports PyTorch’s dispatch mechanism.

__torch_proxy__() → torch.Tensor

Return a 1-element torch.Tensor strided as the global self shape. Used internally for sanitation purposes.

__xitem_get_key_start_stop(rank: int, actives: list, key_st: int, key_sp: int, step: int, ends: torch.Tensor, og_key_st: int) → Tuple[int, int]

class KMedoids(n_clusters: int = 8, init: str | heat.core.dndarray.DNDarray = 'random', max_iter: int = 300, random_state: int = None)

Bases: heat.cluster._kcluster._KCluster

Kmedoids with the Manhattan distance as fixed metric, calculating the median of the assigned cluster points as new cluster center and snapping the centroid to the the nearest datapoint afterwards. This is not the original implementation of k-medoids using PAM as originally proposed by in [1].

Parameters:

• n_clusters (int, optional, default: 8) – The number of clusters to form as well as the number of centroids to generate.

• init (str or DNDarray, default: ‘random’) –

  Method for initialization:

  • ‘k-medoids++’ : selects initial cluster centers for the clustering in a smart way to speed up convergence [2].

  • ‘random’: choose k observations (rows) at random from data for the initial centroids.

  • DNDarray: gives the initial centers, should be of Shape = (n_clusters, n_features)

• max_iter (int, default: 300) – Maximum number of iterations of the algorithm for a single run.

• random_state (int) – Determines random number generation for centroid initialization.

References

[1] Kaufman, L. and Rousseeuw, P.J. (1987), Clustering by means of Medoids, in Statistical Data Analysis Based on the L1 Norm and Related Methods, edited by Y. Dodge, North-Holland, 405416.

_update_centroids(x: heat.core.dndarray.DNDarray, matching_centroids: heat.core.dndarray.DNDarray)

Compute new centroid ci as closest sample to the median of the data points in x that are assigned to ci

Parameters:

• x (DNDarray) – Input data

• matching_centroids (DNDarray) – Array filled with indeces i indicating to which cluster ci each sample point in x is assigned

fit(x: heat.core.dndarray.DNDarray, oversampling: float = 2, iter_multiplier: float = 1)

Computes the centroid of a k-medoids clustering.

Parameters:

• x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features)

• oversampling (float) – oversampling factor used in the k-means|| initializiation of centroids

• iter_multiplier (float) – factor that increases the number of iterations used in the initialization of centroids

class DNDarray(array: torch.Tensor, gshape: Tuple[int, Ellipsis], dtype: heat.core.types.datatype, split: int | None, device: heat.core.devices.Device, comm: Communication, balanced: bool)

Distributed N-Dimensional array. The core element of HeAT. It is composed of PyTorch tensors local to each process.

Parameters:

• array (torch.Tensor) – Local array elements

• gshape (Tuple[int,...]) – The global shape of the array

• dtype (datatype) – The datatype of the array

• split (int or None) – The axis on which the array is divided between processes

• device (Device) – The device on which the local arrays are using (cpu or gpu)

• comm (Communication) – The communications object for sending and receiving data

• balanced (bool or None) – Describes whether the data are evenly distributed across processes. If this information is not available (self.balanced is None), it can be gathered via the is_balanced() method (requires communication).

__array

__gshape

__dtype

__split

__device

__comm

__balanced

__ishalo = False

__halo_next = None

__halo_prev = None

__partitions_dict__ = None

__lshape_map = None

__prephalo(start, end) → torch.Tensor

Extracts the halo indexed by start, end from self.array in the direction of self.split

Parameters:

• start (int) – Start index of the halo extracted from self.array

• end (int) – End index of the halo extracted from self.array

get_halo(halo_size: int, prev: bool = True, next: bool = True)

Fetch halos of size halo_size from neighboring ranks and save them in self.halo_next/self.halo_prev.

Parameters:

• halo_size (int) – Size of the halo.

• prev (bool, optional) – If True, fetch the halo from the previous rank. Default: True.

• next (bool, optional) – If True, fetch the halo from the next rank. Default: True.

__cat_halo() → torch.Tensor

Return local array concatenated to halos if they are available.

__array__() → numpy.ndarray

Returns a view of the process-local slice of the DNDarray as a numpy ndarray, if the DNDarray resides on CPU. Otherwise, it returns a copy, on CPU, of the process-local slice of DNDarray as numpy ndarray.

__array_ufunc__(ufunc, method, *inputs, **kwargs)

Override NumPy’s universal functions.

__array_function__(func, types, args, kwargs)

Augments NumPy’s functions.

astype(dtype, copy=True) → DNDarray

Returns a casted version of this array. Casted array is a new array of the same shape but with given type of this array. If copy is True, the same array is returned instead.

Parameters:

• dtype (datatype) – Heat type to which the array is cast

• copy (bool, optional) – By default the operation returns a copy of this array. If copy is set to False the cast is performed in-place and this array is returned

balance_() → DNDarray

Function for balancing a DNDarray between all nodes. To determine if this is needed use the is_balanced() function. If the DNDarray is already balanced this function will do nothing. This function modifies the DNDarray itself and will not return anything.

Examples

>>> a = ht.zeros((10, 2), split=0)
>>> a[:, 0] = ht.arange(10)
>>> b = a[3:]
[0/2] tensor([[3., 0.],
[1/2] tensor([[4., 0.],
              [5., 0.],
              [6., 0.]])
[2/2] tensor([[7., 0.],
              [8., 0.],
              [9., 0.]])
>>> b.balance_()
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (1, 2)
[1/2] (7, 2) (3, 2)
[2/2] (7, 2) (3, 2)
>>> b
[0/2] tensor([[3., 0.],
             [4., 0.],
             [5., 0.]])
[1/2] tensor([[6., 0.],
              [7., 0.]])
[2/2] tensor([[8., 0.],
              [9., 0.]])
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (3, 2)
[1/2] (7, 2) (2, 2)
[2/2] (7, 2) (2, 2)

__bool__() → bool

Boolean scalar casting.

__cast(cast_function) → float | int

Implements a generic cast function for DNDarray objects.

Parameters:

cast_function (function) – The actual cast function, e.g. float or int

Raises:

TypeError – If the DNDarray object cannot be converted into a scalar.

collect_(target_rank: int | None = 0) → None

A method collecting a distributed DNDarray to one MPI rank, chosen by the target_rank variable. It is a specific case of the redistribute_ method.

Parameters:

target_rank (int, optional) – The rank to which the DNDarray will be collected. Default: 0.

Raises:

• TypeError – If the target rank is not an integer.

• ValueError – If the target rank is out of bounds.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.collect_()
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)
>>> st.collect_(1)
>>> print(st.lshape)
[0/2] (50, 81, 0)
[1/2] (50, 81, 67)
[2/2] (50, 81, 0)

__complex__() → DNDarray

Complex scalar casting.

counts_displs() → Tuple[Tuple[int], Tuple[int]]

Returns actual counts (number of items per process) and displacements (offsets) of the DNDarray. Does not assume load balance.

cpu() → DNDarray

Returns a copy of this object in main memory. If this object is already in main memory, then no copy is performed and the original object is returned.

create_lshape_map(force_check: bool = False) → torch.Tensor

Generate a ‘map’ of the lshapes of the data on all processes. Units are (process rank, lshape)

Parameters:

force_check (bool, optional) – if False (default) and the lshape map has already been created, use the previous result. Otherwise, create the lshape_map

create_partition_interface()

Create a partition interface in line with the DPPY proposal. This is subject to change. The intention of this to facilitate the usage of a general format for the referencing of distributed datasets.

An example of the output and shape is shown below.

__partitioned__ = {

‘shape’: (27, 3, 2), ‘partition_tiling’: (4, 1, 1), ‘partitions’: {

(0, 0, 0): {

‘start’: (0, 0, 0), ‘shape’: (7, 3, 2), ‘data’: tensor([…], dtype=torch.int32), ‘location’: [0], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (1, 0, 0): {

‘start’: (7, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [1], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (2, 0, 0): {

‘start’: (14, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [2], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (3, 0, 0): {

‘start’: (21, 0, 0), ‘shape’: (6, 3, 2), ‘data’: None, ‘location’: [3], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}

}, ‘locals’: [(rank, 0, 0)], ‘get’: lambda x: x,

}

Return type:

dictionary containing the partition interface as shown above.

__float__() → DNDarray

Float scalar casting.

See also

flatten()

fill_diagonal(value: float) → DNDarray

Fill the main diagonal of a 2D DNDarray. This function modifies the input tensor in-place, and returns the input array.

Parameters:

value (float) – The value to be placed in the DNDarrays main diagonal

__getitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis]) → DNDarray

Global getter function for DNDarrays. Returns a new DNDarray composed of the elements of the original tensor selected by the indices given. This does NOT redistribute or rebalance the resulting tensor. If the selection of values is unbalanced then the resultant tensor is also unbalanced! To redistributed the DNDarray use balance() (issue #187)

Parameters:

key (int, slice, Tuple[int,...], List[int,...]) – Indices to get from the tensor.

Examples

>>> a = ht.arange(10, split=0)
(1/2) >>> tensor([0, 1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5, 6, 7, 8, 9], dtype=torch.int32)
>>> a[1:6]
(1/2) >>> tensor([1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5], dtype=torch.int32)
>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1]
(1/2) >>> tensor([0.])
(2/2) >>> tensor([0., 0.])

gpu() → DNDarray

Returns a copy of this object in GPU memory. If this object is already in GPU memory, then no copy is performed and the original object is returned.

__int__() → DNDarray

Integer scalar casting.

is_balanced(force_check: bool = False) → bool

Determine if self is balanced evenly (or as evenly as possible) across all nodes distributed evenly (or as evenly as possible) across all processes. This is equivalent to returning self.balanced. If no information is available (self.balanced = None), the balanced status will be assessed via collective communication.

Parameters:

force_check (bool, optional) – If True, the balanced status of the DNDarray will be assessed via collective communication in any case.

is_distributed() → bool

Determines whether the data of this DNDarray is distributed across multiple processes.

__key_is_singular(key: any, axis: int, self_proxy: torch.Tensor) → bool

__key_adds_dimension(key: any, axis: int, self_proxy: torch.Tensor) → bool

item()

Returns the only element of a 1-element DNDarray. Mirror of the pytorch command by the same name. If size of DNDarray is >1 element, then a ValueError is raised (by pytorch)

Examples

>>> import heat as ht
>>> x = ht.zeros((1))
>>> x.item()
0.0

__len__() → int

The length of the DNDarray, i.e. the number of items in the first dimension.

numpy() → numpy.array

Returns a copy of the DNDarray as numpy ndarray. If the DNDarray resides on the GPU, the underlying data will be copied to the CPU first.

If the DNDarray is distributed, an MPI Allgather operation will be performed before converting to np.ndarray, i.e. each MPI process will end up holding a copy of the entire array in memory. Make sure process memory is sufficient!

Examples

>>> import heat as ht
T1 = ht.random.randn((10,8))
T1.numpy()

_repr_pretty_(p, cycle)

Pretty print for IPython.

__repr__() → str

Returns a printable representation of the passed DNDarray, targeting developers.

ravel()

Flattens the DNDarray.

See also

ravel()

Examples

>>> a = ht.ones((2, 3), split=0)
>>> b = a.ravel()
>>> a[0, 0] = 4
>>> b
DNDarray([4., 1., 1., 1., 1., 1.], dtype=ht.float32, device=cpu:0, split=0)

redistribute_(lshape_map: torch.Tensor | None = None, target_map: torch.Tensor | None = None)

Redistributes the data of the DNDarray along the split axis to match the given target map. This function does not modify the non-split dimensions of the DNDarray. This is an abstraction and extension of the balance function.

Parameters:

• lshape_map (torch.Tensor, optional) – The current lshape of processes. Units are [rank, lshape].

• target_map (torch.Tensor, optional) – The desired distribution across the processes. Units are [rank, target lshape]. Note: the only important parts of the target map are the values along the split axis, values which are not along this axis are there to mimic the shape of the lshape_map.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> target_map = torch.zeros((st.comm.size, 3), dtype=torch.int64)
>>> target_map[0, 2] = 67
>>> print(target_map)
[0/2] tensor([[ 0,  0, 67],
[0/2]         [ 0,  0,  0],
[0/2]         [ 0,  0,  0]], dtype=torch.int32)
[1/2] tensor([[ 0,  0, 67],
[1/2]         [ 0,  0,  0],
[1/2]         [ 0,  0,  0]], dtype=torch.int32)
[2/2] tensor([[ 0,  0, 67],
[2/2]         [ 0,  0,  0],
[2/2]         [ 0,  0,  0]], dtype=torch.int32)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.redistribute_(target_map=target_map)
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)

__redistribute_shuffle(snd_pr: int | torch.Tensor, send_amt: int | torch.Tensor, rcv_pr: int | torch.Tensor, snd_dtype: torch.dtype)

Function to abstract the function used during redistribute for shuffling data between processes along the split axis

Parameters:

• snd_pr (int or torch.Tensor) – Sending process

• send_amt (int or torch.Tensor) – Amount of data to be sent by the sending process

• rcv_pr (int or torch.Tensor) – Receiving process

• snd_dtype (torch.dtype) – Torch type of the data in question

resplit_(axis: int = None)

In-place option for resplitting a DNDarray.

Parameters:

axis (int) – The new split axis, None denotes gathering, an int will set the new split axis

Examples

>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, None)
>>> a.split
None
>>> a.lshape
(0/2) (4, 5)
(1/2) (4, 5)
>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, 1)
>>> a.split
1
>>> a.lshape
(0/2) (4, 3)
(1/2) (4, 2)

__setitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Global item setter

Parameters:

• key (Union[int, Tuple[int,...], List[int,...]]) – Index/indices to be set

• value (Union[float, DNDarray,torch.Tensor]) – Value to be set to the specified positions in the DNDarray (self)

Notes

If a DNDarray is given as the value to be set then the split axes are assumed to be equal. If they are not, PyTorch will raise an error when the values are attempted to be set on the local array

Examples

>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1] = 1
>>> a
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])
(2/2) >>> tensor([[0., 1., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])

__setter(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Utility function for checking value and forwarding to :func:__setitem__

Raises:

NotImplementedError – If the type of value ist not supported

__str__() → str

Computes a string representation of the passed DNDarray.

tolist(keepsplit: bool = False) → List

Return a copy of the local array data as a (nested) Python list. For scalars, a standard Python number is returned.

Parameters:

keepsplit (bool) – Whether the list should be returned locally or globally.

Examples

>>> a = ht.array([[0, 1], [2, 3]])
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=0)
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=1)
>>> a.tolist(keepsplit=True)
(1/2) [[0], [2]]
(2/2) [[1], [3]]

__torch_function__(func, types, args=(), kwargs=None)

Supports PyTorch’s dispatch mechanism.

__torch_proxy__() → torch.Tensor

Return a 1-element torch.Tensor strided as the global self shape. Used internally for sanitation purposes.

__xitem_get_key_start_stop(rank: int, actives: list, key_st: int, key_sp: int, step: int, ends: torch.Tensor, og_key_st: int) → Tuple[int, int]

class Spectral(n_clusters: int = None, gamma: float = 1.0, metric: str = 'rbf', laplacian: str = 'fully_connected', threshold: float = 1.0, boundary: str = 'upper', eigen_solver: str = 'randomized', reigh_rank: int = 100, reigh_n_oversamples: int = 10, reigh_power_iter: int = 0, lanczos_n_iter: int = 300, assign_labels: str = 'kmeans', **params)

Bases: heat.ClusteringMixin, heat.BaseEstimator

Spectral clustering

Variables:

• n_clusters (int) – Number of clusters to fit

• gamma (float) – Kernel coefficient sigma for ‘rbf’, ignored for metric=’euclidean’

• metric (string) –

  How to construct the similarity matrix.

  • ’rbf’ : construct the similarity matrix using a radial basis function (RBF) kernel.

  • ’euclidean’ : construct the similarity matrix as only euclidean distance.

• laplacian (str) – How to calculate the graph laplacian (affinity) Currently supported : ‘fully_connected’, ‘eNeighbour’

• threshold (float) – Threshold for affinity matrix if laplacian=’eNeighbour’ Ignorded for laplacian=’fully_connected’

• boundary (str) – How to interpret threshold: ‘upper’, ‘lower’ Ignorded for laplacian=’fully_connected’

• eigen_solver (str) – The eigenvalue decomposition strategy to use. - ‘lanczos’ : Use Lanczos iterations to reduce the Laplacian matrix size before applying the torch eigenvalue solver. - ‘randomized’ : Use a randomized algorithm to compute the approximate eigenvalues and eigenvectors.

• reigh_rank (int) – number of samples for randomized eigenvalue decomposition. Only used if eigen_solver=’randomized’. It must hold reigh_rank >= n_clusters. If n_clusters is None (automatic selection of number of clusters), reigh_rank gives an upper bound on the number of clusters that can be found. Therefore, reigh_rank should be set high enough to capture the expected number of clusters in that case.

• reigh_n_oversamples (int) – number of oversamples for randomized eigenvalue decomposition. Only used if eigen_solver=’randomized’. Default is 10.

• reigh_power_iter (int) – number of power iterations for randomized eigenvalue decomposition. Only used if eigen_solver=’randomized’. Default is 0. Consider increasing this value if the eigen-spectrum of the Laplacian decays slowly.

• lanczos_n_iter (int) – number of Lanczos iterations for Eigenvalue decomposition. Only used if eigen_solver=’lanczos’. Default is 300.

• assign_labels (str) – The strategy to use to assign labels in the embedding space.

• **params (dict) – Parameter dictionary for the assign_labels estimator

n_clusters = None

gamma = 1.0

metric = 'rbf'

laplacian = 'fully_connected'

threshold = 1.0

boundary = 'upper'

lanczos_n_iter = 300

assign_labels = 'kmeans'

eigen_solver = 'randomized'

reigh_n_oversamples = 10

reigh_power_iter = 0

reigh_rank = 100

_labels = None

_spectral_embedding(x: heat.core.dndarray.DNDarray) → Tuple[heat.core.dndarray.DNDarray, heat.core.dndarray.DNDarray]

Helper function for dataset x embedding. Returns Tupel(Eigenvalues, Eigenvectors) of the graph’s Laplacian matrix.

Parameters:

x (DNDarray) – Sample Matrix for which the embedding should be calculated

Notes

This will throw out the complex side of the eigenvalues found during this.

fit(x: heat.core.dndarray.DNDarray)

Clusters dataset X via spectral embedding. Computes the low-dim representation by calculation of eigenspectrum (eigenvalues and eigenvectors) of the graph laplacian from the similarity matrix and fits the eigenvectors that correspond to the k lowest eigenvalues with a seperate clustering algorithm (currently only kmeans is supported). Similarity metrics for adjacency calculations are supported via spatial.distance. The eigenvalues and eigenvectors are computed by reducing the Laplacian via lanczos iterations and using the torch eigenvalue solver on this smaller matrix. If other eigenvalue decompostion methods are supported, this will be expanded.

Parameters:

x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features)

predict(x: heat.core.dndarray.DNDarray) → heat.core.dndarray.DNDarray

Return the label each sample in X belongs to. X is transformed to the low-dim representation by calculation of eigenspectrum (eigenvalues and eigenvectors) of the graph laplacian from the similarity matrix. Inference of lables is done by extraction of the closest centroid of the n_clusters eigenvectors from the previously fitted clustering algorithm (kmeans).

Parameters:

x (DNDarray) – New data to predict. Shape = (n_samples, n_features)

Warning

Caution: Calculation of the low-dim representation requires some time!

class DNDarray(array: torch.Tensor, gshape: Tuple[int, Ellipsis], dtype: heat.core.types.datatype, split: int | None, device: heat.core.devices.Device, comm: Communication, balanced: bool)

Distributed N-Dimensional array. The core element of HeAT. It is composed of PyTorch tensors local to each process.

Parameters:

• array (torch.Tensor) – Local array elements

• gshape (Tuple[int,...]) – The global shape of the array

• dtype (datatype) – The datatype of the array

• split (int or None) – The axis on which the array is divided between processes

• device (Device) – The device on which the local arrays are using (cpu or gpu)

• comm (Communication) – The communications object for sending and receiving data

• balanced (bool or None) – Describes whether the data are evenly distributed across processes. If this information is not available (self.balanced is None), it can be gathered via the is_balanced() method (requires communication).

__array

__gshape

__dtype

__split

__device

__comm

__balanced

__ishalo = False

__halo_next = None

__halo_prev = None

__partitions_dict__ = None

__lshape_map = None

__prephalo(start, end) → torch.Tensor

Extracts the halo indexed by start, end from self.array in the direction of self.split

Parameters:

• start (int) – Start index of the halo extracted from self.array

• end (int) – End index of the halo extracted from self.array

get_halo(halo_size: int, prev: bool = True, next: bool = True)

Fetch halos of size halo_size from neighboring ranks and save them in self.halo_next/self.halo_prev.

Parameters:

• halo_size (int) – Size of the halo.

• prev (bool, optional) – If True, fetch the halo from the previous rank. Default: True.

• next (bool, optional) – If True, fetch the halo from the next rank. Default: True.

__cat_halo() → torch.Tensor

Return local array concatenated to halos if they are available.

__array__() → numpy.ndarray

Returns a view of the process-local slice of the DNDarray as a numpy ndarray, if the DNDarray resides on CPU. Otherwise, it returns a copy, on CPU, of the process-local slice of DNDarray as numpy ndarray.

__array_ufunc__(ufunc, method, *inputs, **kwargs)

Override NumPy’s universal functions.

__array_function__(func, types, args, kwargs)

Augments NumPy’s functions.

astype(dtype, copy=True) → DNDarray

Returns a casted version of this array. Casted array is a new array of the same shape but with given type of this array. If copy is True, the same array is returned instead.

Parameters:

• dtype (datatype) – Heat type to which the array is cast

• copy (bool, optional) – By default the operation returns a copy of this array. If copy is set to False the cast is performed in-place and this array is returned

balance_() → DNDarray

Function for balancing a DNDarray between all nodes. To determine if this is needed use the is_balanced() function. If the DNDarray is already balanced this function will do nothing. This function modifies the DNDarray itself and will not return anything.

Examples

>>> a = ht.zeros((10, 2), split=0)
>>> a[:, 0] = ht.arange(10)
>>> b = a[3:]
[0/2] tensor([[3., 0.],
[1/2] tensor([[4., 0.],
              [5., 0.],
              [6., 0.]])
[2/2] tensor([[7., 0.],
              [8., 0.],
              [9., 0.]])
>>> b.balance_()
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (1, 2)
[1/2] (7, 2) (3, 2)
[2/2] (7, 2) (3, 2)
>>> b
[0/2] tensor([[3., 0.],
             [4., 0.],
             [5., 0.]])
[1/2] tensor([[6., 0.],
              [7., 0.]])
[2/2] tensor([[8., 0.],
              [9., 0.]])
>>> print(b.gshape, b.lshape)
[0/2] (7, 2) (3, 2)
[1/2] (7, 2) (2, 2)
[2/2] (7, 2) (2, 2)

__bool__() → bool

Boolean scalar casting.

__cast(cast_function) → float | int

Implements a generic cast function for DNDarray objects.

Parameters:

cast_function (function) – The actual cast function, e.g. float or int

Raises:

TypeError – If the DNDarray object cannot be converted into a scalar.

collect_(target_rank: int | None = 0) → None

A method collecting a distributed DNDarray to one MPI rank, chosen by the target_rank variable. It is a specific case of the redistribute_ method.

Parameters:

target_rank (int, optional) – The rank to which the DNDarray will be collected. Default: 0.

Raises:

• TypeError – If the target rank is not an integer.

• ValueError – If the target rank is out of bounds.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.collect_()
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)
>>> st.collect_(1)
>>> print(st.lshape)
[0/2] (50, 81, 0)
[1/2] (50, 81, 67)
[2/2] (50, 81, 0)

__complex__() → DNDarray

Complex scalar casting.

counts_displs() → Tuple[Tuple[int], Tuple[int]]

Returns actual counts (number of items per process) and displacements (offsets) of the DNDarray. Does not assume load balance.

cpu() → DNDarray

Returns a copy of this object in main memory. If this object is already in main memory, then no copy is performed and the original object is returned.

create_lshape_map(force_check: bool = False) → torch.Tensor

Generate a ‘map’ of the lshapes of the data on all processes. Units are (process rank, lshape)

Parameters:

force_check (bool, optional) – if False (default) and the lshape map has already been created, use the previous result. Otherwise, create the lshape_map

create_partition_interface()

Create a partition interface in line with the DPPY proposal. This is subject to change. The intention of this to facilitate the usage of a general format for the referencing of distributed datasets.

An example of the output and shape is shown below.

__partitioned__ = {

‘shape’: (27, 3, 2), ‘partition_tiling’: (4, 1, 1), ‘partitions’: {

(0, 0, 0): {

‘start’: (0, 0, 0), ‘shape’: (7, 3, 2), ‘data’: tensor([…], dtype=torch.int32), ‘location’: [0], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (1, 0, 0): {

‘start’: (7, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [1], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (2, 0, 0): {

‘start’: (14, 0, 0), ‘shape’: (7, 3, 2), ‘data’: None, ‘location’: [2], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}, (3, 0, 0): {

‘start’: (21, 0, 0), ‘shape’: (6, 3, 2), ‘data’: None, ‘location’: [3], ‘dtype’: torch.int32, ‘device’: ‘cpu’

}

}, ‘locals’: [(rank, 0, 0)], ‘get’: lambda x: x,

}

Return type:

dictionary containing the partition interface as shown above.

__float__() → DNDarray

Float scalar casting.

See also

flatten()

fill_diagonal(value: float) → DNDarray

Fill the main diagonal of a 2D DNDarray. This function modifies the input tensor in-place, and returns the input array.

Parameters:

value (float) – The value to be placed in the DNDarrays main diagonal

__getitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis]) → DNDarray

Global getter function for DNDarrays. Returns a new DNDarray composed of the elements of the original tensor selected by the indices given. This does NOT redistribute or rebalance the resulting tensor. If the selection of values is unbalanced then the resultant tensor is also unbalanced! To redistributed the DNDarray use balance() (issue #187)

Parameters:

key (int, slice, Tuple[int,...], List[int,...]) – Indices to get from the tensor.

Examples

>>> a = ht.arange(10, split=0)
(1/2) >>> tensor([0, 1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5, 6, 7, 8, 9], dtype=torch.int32)
>>> a[1:6]
(1/2) >>> tensor([1, 2, 3, 4], dtype=torch.int32)
(2/2) >>> tensor([5], dtype=torch.int32)
>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1]
(1/2) >>> tensor([0.])
(2/2) >>> tensor([0., 0.])

gpu() → DNDarray

Returns a copy of this object in GPU memory. If this object is already in GPU memory, then no copy is performed and the original object is returned.

__int__() → DNDarray

Integer scalar casting.

is_balanced(force_check: bool = False) → bool

Determine if self is balanced evenly (or as evenly as possible) across all nodes distributed evenly (or as evenly as possible) across all processes. This is equivalent to returning self.balanced. If no information is available (self.balanced = None), the balanced status will be assessed via collective communication.

Parameters:

force_check (bool, optional) – If True, the balanced status of the DNDarray will be assessed via collective communication in any case.

is_distributed() → bool

Determines whether the data of this DNDarray is distributed across multiple processes.

__key_is_singular(key: any, axis: int, self_proxy: torch.Tensor) → bool

__key_adds_dimension(key: any, axis: int, self_proxy: torch.Tensor) → bool

item()

Returns the only element of a 1-element DNDarray. Mirror of the pytorch command by the same name. If size of DNDarray is >1 element, then a ValueError is raised (by pytorch)

Examples

>>> import heat as ht
>>> x = ht.zeros((1))
>>> x.item()
0.0

__len__() → int

The length of the DNDarray, i.e. the number of items in the first dimension.

numpy() → numpy.array

Returns a copy of the DNDarray as numpy ndarray. If the DNDarray resides on the GPU, the underlying data will be copied to the CPU first.

If the DNDarray is distributed, an MPI Allgather operation will be performed before converting to np.ndarray, i.e. each MPI process will end up holding a copy of the entire array in memory. Make sure process memory is sufficient!

Examples

>>> import heat as ht
T1 = ht.random.randn((10,8))
T1.numpy()

_repr_pretty_(p, cycle)

Pretty print for IPython.

__repr__() → str

Returns a printable representation of the passed DNDarray, targeting developers.

ravel()

Flattens the DNDarray.

See also

ravel()

Examples

>>> a = ht.ones((2, 3), split=0)
>>> b = a.ravel()
>>> a[0, 0] = 4
>>> b
DNDarray([4., 1., 1., 1., 1., 1.], dtype=ht.float32, device=cpu:0, split=0)

redistribute_(lshape_map: torch.Tensor | None = None, target_map: torch.Tensor | None = None)

Redistributes the data of the DNDarray along the split axis to match the given target map. This function does not modify the non-split dimensions of the DNDarray. This is an abstraction and extension of the balance function.

Parameters:

• lshape_map (torch.Tensor, optional) – The current lshape of processes. Units are [rank, lshape].

• target_map (torch.Tensor, optional) – The desired distribution across the processes. Units are [rank, target lshape]. Note: the only important parts of the target map are the values along the split axis, values which are not along this axis are there to mimic the shape of the lshape_map.

Examples

>>> st = ht.ones((50, 81, 67), split=2)
>>> target_map = torch.zeros((st.comm.size, 3), dtype=torch.int64)
>>> target_map[0, 2] = 67
>>> print(target_map)
[0/2] tensor([[ 0,  0, 67],
[0/2]         [ 0,  0,  0],
[0/2]         [ 0,  0,  0]], dtype=torch.int32)
[1/2] tensor([[ 0,  0, 67],
[1/2]         [ 0,  0,  0],
[1/2]         [ 0,  0,  0]], dtype=torch.int32)
[2/2] tensor([[ 0,  0, 67],
[2/2]         [ 0,  0,  0],
[2/2]         [ 0,  0,  0]], dtype=torch.int32)
>>> print(st.lshape)
[0/2] (50, 81, 23)
[1/2] (50, 81, 22)
[2/2] (50, 81, 22)
>>> st.redistribute_(target_map=target_map)
>>> print(st.lshape)
[0/2] (50, 81, 67)
[1/2] (50, 81, 0)
[2/2] (50, 81, 0)

__redistribute_shuffle(snd_pr: int | torch.Tensor, send_amt: int | torch.Tensor, rcv_pr: int | torch.Tensor, snd_dtype: torch.dtype)

Function to abstract the function used during redistribute for shuffling data between processes along the split axis

Parameters:

• snd_pr (int or torch.Tensor) – Sending process

• send_amt (int or torch.Tensor) – Amount of data to be sent by the sending process

• rcv_pr (int or torch.Tensor) – Receiving process

• snd_dtype (torch.dtype) – Torch type of the data in question

resplit_(axis: int = None)

In-place option for resplitting a DNDarray.

Parameters:

axis (int) – The new split axis, None denotes gathering, an int will set the new split axis

Examples

>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, None)
>>> a.split
None
>>> a.lshape
(0/2) (4, 5)
(1/2) (4, 5)
>>> a = ht.zeros(
...     (
...         4,
...         5,
...     ),
...     split=0,
... )
>>> a.lshape
(0/2) (2, 5)
(1/2) (2, 5)
>>> ht.resplit_(a, 1)
>>> a.split
1
>>> a.lshape
(0/2) (4, 3)
(1/2) (4, 2)

__setitem__(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Global item setter

Parameters:

• key (Union[int, Tuple[int,...], List[int,...]]) – Index/indices to be set

• value (Union[float, DNDarray,torch.Tensor]) – Value to be set to the specified positions in the DNDarray (self)

Notes

If a DNDarray is given as the value to be set then the split axes are assumed to be equal. If they are not, PyTorch will raise an error when the values are attempted to be set on the local array

Examples

>>> a = ht.zeros((4, 5), split=0)
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
(2/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 0., 0., 0., 0.]])
>>> a[1:4, 1] = 1
>>> a
(1/2) >>> tensor([[0., 0., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])
(2/2) >>> tensor([[0., 1., 0., 0., 0.],
                  [0., 1., 0., 0., 0.]])

__setter(key: int | Tuple[int, Ellipsis] | List[int, Ellipsis], value: float | DNDarray | torch.Tensor)

Utility function for checking value and forwarding to :func:__setitem__

Raises:

NotImplementedError – If the type of value ist not supported

__str__() → str

Computes a string representation of the passed DNDarray.

tolist(keepsplit: bool = False) → List

Return a copy of the local array data as a (nested) Python list. For scalars, a standard Python number is returned.

Parameters:

keepsplit (bool) – Whether the list should be returned locally or globally.

Examples

>>> a = ht.array([[0, 1], [2, 3]])
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=0)
>>> a.tolist()
[[0, 1], [2, 3]]

>>> a = ht.array([[0, 1], [2, 3]], split=1)
>>> a.tolist(keepsplit=True)
(1/2) [[0], [2]]
(2/2) [[1], [3]]

__torch_function__(func, types, args=(), kwargs=None)

Supports PyTorch’s dispatch mechanism.

__torch_proxy__() → torch.Tensor

Return a 1-element torch.Tensor strided as the global self shape. Used internally for sanitation purposes.

__xitem_get_key_start_stop(rank: int, actives: list, key_st: int, key_sp: int, step: int, ends: torch.Tensor, og_key_st: int) → Tuple[int, int]

self

Auxiliary single-process functions and base class for batch-parallel k-clustering

_initialize_plus_plus(X, n_clusters, p, random_state=None, weights: torch.tensor = 1, max_samples=2**24 - 1)

Auxiliary function: single-process k-means++/k-medians++ initialization in pytorch p is the norm used for computing distances weights allows to add weights to the distribution function, so that the data points with higher weights are preferred; note that weights must have the same dimension as X[0] The value max_samples=2**24 - 1 is necessary as PyTorchs multinomial currently only supports this number of different categories.

_kmex(X, p, n_clusters, init, max_iter, tol, random_state=None, weights: torch.tensor = 1.0)

Auxiliary function: single-process k-means and k-medians in pytorch p is the norm used for computing distances: p=2 implies k-means, p=1 implies k-medians. p should be 1 (k-medians) or 2 (k-means). For other choice of p, we proceed as for p=2 and hope for the best. (note: kmex stands for kmeans and kmedians)

_parallel_batched_kmex_predict(X, centers, p)

Auxiliary function: predict labels for parallel_batched_kmex

class _BatchParallelKCluster(p: int, n_clusters: int, init: str, max_iter: int, tol: float, random_state: int | None, n_procs_to_merge: int | None)

Bases: heat.ClusteringMixin, heat.BaseEstimator

Base class for batch parallel k-clustering

n_clusters

_init

max_iter

tol

random_state

n_procs_to_merge

_p

_cluster_centers = None

_n_iter = None

_functional_value = None

fit(x: heat.core.dndarray.DNDarray)

Computes the centroid of the clustering algorithm to fit the data x.

Parameters:

• x (DNDarray) – Training instances to cluster. Shape = (n_samples, n_features). It must hold x.split=0.

• weights (torch.tensor) – Add weights to the distribution function used in the clustering algorithm in kmex

predict(x: heat.core.dndarray.DNDarray)

Predict the closest cluster each sample in x belongs to.

In the vector quantization literature, cluster_centers_() is called the code book and each value returned by predict is the index of the closest code in the code book.

Parameters:

x (DNDarray) – New data to predict. Shape = (n_samples, n_features)

class BatchParallelKMeans(n_clusters: int = 8, init: str = 'k-means++', max_iter: int = 300, tol: float = 0.0001, random_state: int = None, n_procs_to_merge: int = None)

Bases: _BatchParallelKCluster

Batch-parallel K-Means clustering algorithm from Ref. [1]. The input must be a DNDarray of shape (n_samples, n_features), with split=0 (i.e. split along the sample axis). This method performs K-Means clustering on each batch (i.e. on each process-local chunk) of data individually and in parallel. After that, all centroids from the local K-Means are gathered and another instance of K-means is performed on them in order to determine the final centroids. To improve scalability of this approach also on a large number of processes, this procedure can be applied in a hierarchical manner using the parameter n_procs_to_merge.

Variables:

• n_clusters (int) – The number of clusters to form as well as the number of centroids to generate.

• init (str) – Method for initialization for local and global k-means: - ‘k-means++’ : selects initial cluster centers for the clustering in a smart way to speed up convergence [2]. - ‘random’: choose k observations (rows) at random from data for the initial centroids. (Not implemented yet)

• max_iter (int) – Maximum number of iterations of the local/global k-means algorithms.

• tol (float) – Relative tolerance with regards to inertia to declare convergence, both for local and global k-means.

• random_state (int) – Determines random number generation for centroid initialization.

• n_procs_to_merge (int) – Number of processes to merge after each iteration of the local k-means. If None, all processes are merged after each iteration.

References

[1] Rasim M. Alguliyev, Ramiz M. Aliguliyev, Lyudmila V. Sukhostat, Parallel batch k-means for Big data clustering, Computers & Industrial Engineering, Volume 152 (2021). https://doi.org/10.1016/j.cie.2020.107023.

init = 'k-means++'

class BatchParallelKMedians(n_clusters: int = 8, init: str = 'k-medians++', max_iter: int = 300, tol: float = 0.0001, random_state: int = None, n_procs_to_merge: int = None)

Bases: _BatchParallelKCluster

Batch-parallel K-Medians clustering algorithm, in analogy to the K-means algorithm from Ref. [1]. This requires data to be given as DNDarray of shape (n_samples, n_features) with split=0 (i.e. split along the sample axis). The idea of the method is to perform the classical K-Medians on each batch of data (i.e. on each process-local chunk of data) individually and in parallel. After that, all centroids from the local K-Medians are gathered and another instance of K-Medians is performed on them in order to determine the final centroids. To improve scalability of this approach also on a range number of processes, this procedure can be applied in a hierarchical manor using the parameter n_procs_to_merge.

Variables:

• n_clusters (int) – The number of clusters to form as well as the number of centroids to generate.

• init (str) – Method for initialization for local and global k-medians: - ‘k-medians++’ : selects initial cluster centers for the clustering in a smart way to speed up convergence [2]. - ‘random’: choose k observations (rows) at random from data for the initial centroids. (Not implemented yet)

• max_iter (int) – Maximum number of iterations of the local/global k-Medians algorithms.

• tol (float) – Relative tolerance with regards to inertia to declare convergence, both for local and global k-Medians.

• random_state (int) – Determines random number generation for centroid initialization.

• n_procs_to_merge (int) – Number of processes to merge after each iteration of the local k-Medians. If None, all processes are merged after each iteration.

References

[1] Rasim M. Alguliyev, Ramiz M. Aliguliyev, Lyudmila V. Sukhostat, Parallel batch k-means for Big data clustering, Computers & Industrial Engineering, Volume 152 (2021). https://doi.org/10.1016/j.cie.2020.107023.

init = 'k-medians++'

_validate_input(X, labels, metric='euclidean')

Input validation for clustering metrics. Converts input to DNDarray if needed.

Parameters:

• X ({DNDarray, list}) – Input data.

• labels ({DNDarray, list}) – Labels.

• metric (str, optional) – The metric to use for validation. Default is “euclidean”.

Returns:

• X (DNDarray) – The converted and validated X.

• labels (DNDarray) – The converted and validated labels.

Examples

>>> import heat as ht
>>> X = ht.array([[1, 2], [3, 4]], dtype=ht.float)
>>> labels = ht.array([0, 1])
>>> _validate_input(X, labels)
(DNDarray([[1., 2.], [3., 4.]], dtype=ht.float32, device=cpu:0, split=None),
 DNDarray([0, 1], dtype=ht.int64, device=cpu:0, split=None))

silhouette_samples(X, labels, *, metric='euclidean')

Compute the Silhouette Coefficient for each sample.

The Silhouette Coefficient is a measure of how close an object is to its own cluster (cohesion) compared to other clusters (separation). The best value is 1 and the worst value is -1. Values near 0 indicate overlapping clusters.

• The score is 0 for clusters with only a single sample.

• The calculation involves computing the mean intra-cluster distance (a) and the mean nearest-cluster distance (b) for each sample.

XDNDarray

An array of pairwise distances between samples, or a feature array. If metric=’precomputed’, X is assumed to be a distance matrix and a feature array otherwise.

labelsDNDarray

Labels for each sample.

metricstr, optional

The metric to use when calculating distance between instances in a feature array. If metric is “precomputed”, X is assumed to be a distance matrix. Default is “euclidean”.

DNDarray

Silhouette value of all individual samples in the clustering

The Silhouette Coefficient $s(i)$ for a single sample is defined as: $$s(i) =

rac{b(i) - a(i)}{max(a(i), b(i))}$$

where $a(i)$ is the mean distance to other samples in the same cluster and $b(i)$ is the mean distance to samples in the nearest neighbor cluster.

ValueError

If metric=’precomputed’ and the diagonal contains non-zero elements.

silhouette_score : Average silhouette coefficient over all samples.

>>> import heat as ht
>>> X = ht.array([[1, 2], [1, 1], [4, 4], [4, 5]], split=0)
>>> labels = ht.array([0, 0, 1, 1], split=0)
>>> ht.cluster.silhouette_samples(X, labels)
DNDarray([0.7452, 0.7836, 0.7452, 0.7836], dtype=ht.float64, device=cpu:0, split=0)

silhouette_score(X, labels, *, metric='euclidean', sample_size=None, random_state=None, **kwargs)

Compute the mean Silhouette Coefficient of all samples.

The Silhouette Coefficient is calculated using the mean intra-cluster distance (a) and the mean nearest-cluster distance (b) for each sample. The Silhouette Coefficient for a sample is $(b - a) / max(a, b)$.

• This function returns the average of silhouette_samples.

• To clarify, $b$ is the distance between a sample and the nearest cluster that the sample is not a part of.

Parameters:

• X (DNDarray) – An array of pairwise distances between samples, or a feature array.

• labels (DNDarray) – Labels for each sample.

• metric (str, optional) – The metric to use when calculating distance between instances in a feature array. If metric is “precomputed”, X is assumed to be a distance matrix. Default is “euclidean”.

• sample_size (int, optional) – The size of the sample to use when computing the Silhouette Coefficient on a random subset of the data. If sample_size is None, no sampling is used.

• random_state (int, optional) – Determines random number generation for selecting a subset of samples. Used when sample_size is not None.

• **kwargs (optional) – Additional keyword arguments passed to silhouette_samples.

Returns:

Silhouette score of the clustering

Return type:

float

Notes

The best value is 1 and the worst value is -1. Values near 0 indicate overlapping clusters.

See also

silhouette_samples

Silhouette Coefficient for each individual sample.

Examples

>>> import heat as ht
>>> X = ht.array([[1, 2], [1, 1], [4, 4], [4, 5]], split=0)
>>> labels = ht.array([0, 0, 1, 1], split=0)
>>> ht.cluster.silhouette_score(X, labels)
0.76439