neodroidvision.utilities.torch_utilities.distributing.distributing_utilities

Created on 01/03/2020

Functions

all_gather_cuda(data)	Run all_gather on arbitrary picklable data (not necessarily tensors) Args: data: any picklable object Returns: list[data]: list of data gathered from each rank
reduce_dict(input_dict[, average])	Args: input_dict (dict): all the values will be reduced average (bool): whether to do average or sum Reduce the values in the dictionary from all processes so that all processes have the averaged results.
setup_for_distributed(is_master)	This function disables printing when not in master process
is_distribution_ready()	return
is_main_process()	return
init_distributed_mode(args)	param args
save_on_master(*args, **kwargs)	param args
global_distribution_rank()	return
global_world_size()	return
set_benchmark_device_dist(distributed, ...)	param distributed
synchronise_torch_barrier()	Helper function to synchronize (barrier) among all processes when using distributed training
setup_distributed_logger(name, distributed_rank)	param name