heat.utils.data.mnist

File for the MNIST dataset definition in heat

Module Contents

class MNISTDataset(root: str, train: bool = True, transform: Callable = None, target_transform: Callable = None, download: bool = True, split: int = 0, ishuffle: bool = False, test_set: bool = False)

Bases: torchvision.datasets.MNIST

Dataset wrapper for torchvision.datasets.MNIST. This implements all of the required functions mentioned in heat.utils.data.Dataset. The __getitem__ and __len__ functions are inherited from torchvision.datasets.MNIST.

Parameters:

• root (str) – Directory containing the MNIST dataset

• train (bool, optional) – If the data is the training dataset or not, default is True

• transform (Callable, optional) – Transform to be applied to the data dataset in the __getitem__ function, default is None

• target_transform (Callable, optional) – Transform to be applied to the target dataset in the __getitem__ function, default is None

• download (bool, optional) – If the data does not exist in the directory, download it if True (default)

• split (int, optional) – On which access to split the data when it is loaded into a DNDarray

• ishuffle (bool, optional) – Flag indicating whether to use non-blocking communications for shuffling the data between epochs Note: if True, the Ishuffle() function must be defined within the class Default: False

• test_set (bool, optional) – If this dataset is the testing set then keep all of the data local Default: False

Variables:

• htdata (DNDarray) – full data

• httargets (DNDarray) – full target data

• comm (communication.MPICommunicator) – heat communicator for sending data between processes

• _cut_slice (slice) – slice to remove the last element if all are not equal in length

• lcl_half (int) – integer value of half of the data on the process

• data (torch.Tensor) – the local data on a process

• targets (torch.Tensor) – the local targets on a process

• ishuffle (bool) – flag indicating if non-blocking communications are used for shuffling the data between epochs

• test_set (bool) – if this dataset is the testing set then keep all of the data local

Notes

For other attributes see torchvision.datasets.MNIST.

test_set = False

partial_dataset = False

comm

htdata

httargets

ishuffle = False

Shuffle()

Uses the datatools.dataset_shuffle() function to shuffle the data between the processes

Ishuffle()

Uses the datatools.dataset_ishuffle() function to shuffle the data between the processes