Collective Allreduce

Kind np time (sec)
Allreduce 1 0.000009
Allreduce 2 0.000232
Allreduce 4 0.000492
Allreduce 8 0.000816