H. Zhang, C. Hsieh, and V. Akella, HogWild++: A New Mechanism for Decentralized Asynchronous Stochastic Gradient Descent, 2016 IEEE 16th International Conference on Data Mining (ICDM), 2016.

R. Adolf, S. Rama, B. Reagen, G. Wei, and D. M. Brooks, Fathom: reference workloads for modern deep learning methods, 2016 IEEE International Symposium on Workload Characterization (IISWC), 2016.

T. Akiba, S. Suzuki, and K. Fukuda, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

S. Andermatt, S. Pezold, and P. Cattin, Multi-dimensional Gated Recurrent Units for the Segmentation of Biomedical 3D-Data, Deep Learning and Data Labeling for Medical Applications, pp.142-151, 2016.

B. B. Avants, N. J. Tustison, G. Song, P. A. Cook, A. Klein et al., A reproducible evaluation of ANTs similarity metric performance in brain image registration, NeuroImage, vol.54, issue.3, pp.2033-2044, 2011.

M. A. Babyak, What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models, Psychosomatic Medicine, vol.66, issue.3, pp.411-421, 2004.

M. Baker, 1,500 scientists lift the lid on reproducibility, Nature, vol.533, issue.7604, pp.452-454, 2016.

D. Balouek, A. C. Amarie, G. Charrier, F. Desprez, E. Jeannot et al., Adding Virtualization Capabilities to the Grid?5000 Testbed, Communications in Computer and Information Science, vol.367, pp.3-20, 2013.
URL : https://hal.archives-ouvertes.fr/hal-00946971

S. Bao and A. C. Chung, Multi-scale structured CNN with label consistency for brain MR image segmentation, Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, vol.6, issue.1, pp.113-117, 2016.

T. Ben-nun and T. Hoefler, Demystifying Parallel and Distributed Deep Learning, ACM Computing Surveys, vol.52, issue.4, pp.1-43, 2019.

Y. Bengio, Learning Deep Architectures for AI, Foundations and Trends® in Machine Learning, vol.2, issue.1, pp.1-127, 2009.

Y. Bengio, Practical Recommendations for Gradient-Based Training of Deep Architectures, Lecture Notes in Computer Science, pp.437-478, 2012.

J. Bergstra and Y. Bengio, Random search for hyper-parameter optimization, J. Mach. Learn. Res, vol.13, pp.281-305, 2012.

J. Bergstra and Y. Bengio, Random search for hyper-parameter optimization, Journal of machine learning research, vol.13, pp.281-305, 2012.

J. Bergstra, D. Yamins, and D. D. Cox, Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms, Proceedings of the 12th Python in Science Conference, pp.13-20, 2013.

J. Beutel, H. L. Kundel, and R. L. Van-metter, Handbook of Medical Imaging, Volume 1. Physics and Psychophysics, vol.1, 2000.

H. K. Bhadeshia, Neural Networks in Materials Science., ISIJ International, vol.39, issue.10, pp.966-979, 1999.

A. Birenbaum and H. Greenspan, Longitudinal Multiple Sclerosis Lesion Segmentation Using Multi-view Convolutional Neural Networks, Deep Learning and Data Labeling for Medical Applications, pp.58-67, 2016.

L. Bottou, Stochastic Gradient Descent Tricks, Lecture Notes in Computer Science, pp.421-436, 2012.

G. Bradski, The OpenCV Library. Dr. Dobb's Journal of Software Tools, 2000.

R. Bramley, K. Chiu, S. Diwan, D. Gannon, M. Govindaraju et al., A component based services architecture for building distributed applications, Proceedings the Ninth International Symposium on High-Performance Distributed Computing, pp.51-59

T. Brosch, L. Y. Tang, Y. Yoo, D. K. Li, A. Traboulsee et al., Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation, IEEE Transactions on Medical Imaging, vol.35, issue.5, pp.1229-1239, 2016.

A. Brown, S. Johnston, and K. Kelly, Using service-oriented architecture and componentbased development to build web service applications, Rational Software Corporation, vol.6, pp.1-16, 2002.

H. Calkins, J. Brugada, D. L. Packer, R. Cappato, S. Chen et al., HRS/EHRA/ECAS Expert Consensus Statement on Catheter and Surgical Ablation of Atrial Fibrillation: Recommendations for Personnel, Policy, Procedures and Follow-Up, Heart Rhythm, vol.4, issue.6, pp.816-861, 2007.

M. J. Cardoso, M. J. Clarkson, M. Modat, G. R. Ridgway, and S. Ourselin, Locally weighted Markov random fields for cortical segmentation, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2010.

K. S. Chahal, M. S. Grover, K. Dey, and R. R. Shah, A Hitchhiker?s Guide On Distributed Training Of Deep Neural Networks, Journal of Parallel and Distributed Computing, vol.137, pp.65-76, 2020.

X. Chen, L. Xu, Y. Yang, and J. Egger, A semi-automatic computer-aided method for surgical template design, Scientific Reports, vol.6, issue.1, p.20280, 2016.

M. Cho, U. Finkler, S. Kumar, D. S. Kung, V. Saxena et al., , 2017.

H. Choi and K. H. Jin, Fast and robust segmentation of the striatum using deep convolutional neural networks, Journal of Neuroscience Methods, vol.274, pp.146-153, 2016.

F. Chollet, Keras

A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. Lecun, The loss surfaces of multilayer networks, Artificial Intelligence and Statistics, pp.192-204, 2015.

L. O. Chua and T. Roska, The CNN paradigm, IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, vol.40, issue.3, pp.147-156, 1993.

D. Ciresan, U. Meier, J. Masci, and J. Schmidhuber, A committee of neural networks for traffic sign classification, The 2011 International Joint Conference on Neural Networks, pp.1237-1242, 2011.

V. Codreanu, D. Podareanu, and V. Saletore, Large Minibatch Training on Supercomputers with Improved Accuracy and Reduced Time to Train, 2018 IEEE/ACM Machine Learning in HPC Environments (MLHPC), 2018.

P. Cook, Y. Bai, S. Nedjati-gilani, K. Seunarine, M. Hall et al., Camino: open-source diffusion-mri reconstruction and processing, 14th scientific meeting of the international society for magnetic resonance in medicine, vol.2759, p.2759, 2006.

D. D. Cox and T. Dean, Neural Networks and Neuroscience-Inspired Computer Vision, Current Biology, vol.24, issue.18, pp.R921-R929, 2014.

Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli et al., Stratification Learning: Detecting Mixed Density and Dimensionality in High Dimensional Point Clouds, Advances in Neural Information Processing Systems 19, pp.2933-2941, 2007.

A. De-brebisson and G. Montana, Deep neural networks for anatomical brain segmentation, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp.20-28, 2015.

J. Dean, G. S. Corrado, R. Monga, K. Chen, M. Devin et al., Large scale distributed deep networks, Proceedings of the 25th International Conference on Neural Information Processing Systems, vol.1, pp.1223-1231, 2012.

S. Dieleman, J. Schluter, C. Raffel, E. Olson, S. K. Sønderby et al., , 2015.

J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan et al., Long-term Recurrent Convolutional Networks for Visual Recognition and Description, 2014.

J. Dowling, LAS VEGAS SANDS CORP., a Nevada corporation, Plaintiff, v. UKNOWN REGISTRANTS OF www.wn0000.com, www.wn1111.com, www.wn2222.com, www.wn3333.com, www.wn4444.com, www.wn5555.com, www.wn6666.com, www.wn7777.com, www.wn8888.com, www.wn9999.com, www.112211.com, www.4456888.com, www.4489888.com, www.001148.com, and www.2289888.com, Defendants., Gaming Law Review and Economics, vol.20, issue.10, pp.859-868, 2016.

J. Duchi, E. Hazan, and Y. Singer, Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey, Optimization for Machine Learning, vol.12, pp.2121-2159, 2011.

M. E. Falagas, E. I. Pitsouni, G. A. Malietzis, and G. Pappas, Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses, The FASEB Journal, vol.22, issue.2, pp.338-342, 2007.

W. Felter, A. Ferreira, R. Rajamony, and J. Rubio, An updated performance comparison of virtual machines and Linux containers, 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp.171-172, 2015.

J. F. Ferreira, J. L. Sobral, and A. J. Proenca, JaSkel: a Java skeleton-based framework for structured cluster and grid computing, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06), vol.1, p.4, 2006.

P. Fingar, Component-based frameworks for e-commerce, Communications of the ACM, vol.43, issue.10, pp.61-67, 2000.

R. A. Fisher, Statistical methods for research workers, 2006.

E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra et al., Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation, Recent Advances in Parallel Virtual Machine and Message Passing Interface, pp.97-104, 2004.

A. Garcia-garcia, S. Orts-escolano, S. Oprea, V. Villena-martinez, P. Martinez-gonzalez et al., A survey on deep learning techniques for image and video semantic segmentation, Applied Soft Computing, vol.70, pp.41-65, 2018.

A. P. George and W. B. Powell, Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming, Machine Learning, vol.65, issue.1, pp.167-198, 2006.

M. Ghafoorian, N. Karssemeijer, T. Heskes, I. W. Van-uden, C. I. Sanchez et al., Location Sensitive Deep Convolutional Neural Networks for Segmentation of White Matter Hyperintensities, Scientific Reports, vol.7, issue.1, p.5110, 2017.

A. Gibiansky, Proceedings of MLHPC 2018: Machine Learning in HPC Environments, 2018 IEEE/ACM Machine Learning in HPC Environments (MLHPC), 2018.

E. Gibson, W. Li, C. Sudre, L. Fidon, D. I. Shakir et al., NiftyNet: a deep-learning platform for medical imaging, Computer Methods and Programs in Biomedicine, vol.158, pp.113-122, 2018.

T. Goldstein and S. Osher, The Split Bregman Method for L1-Regularized Problems, SIAM Journal on Imaging Sciences, vol.2, issue.2, pp.323-343, 2009.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, 2016.

I. Goodfellow, Y. Bengio, and A. Courville, Deep learning, 2016.

I. J. Goodfellow, O. Vinyals, and A. M. Saxe, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

I. J. Goodfellow, D. Warde-farley, P. Lamblin, V. Dumoulin, M. Mirza et al., Pylearn2: a machine learning research library, 2013.

N. Gordillo, E. Montseny, and P. Sobrevilla, State of the art survey on MRI brain tumor segmentation, Magnetic Resonance Imaging, vol.31, issue.8, pp.1426-1438, 2013.

P. Goyal, P. Dollár, R. B. Girshick, P. Noordhuis, L. Wesolowski et al., Accurate, large minibatch SGD: training imagenet in 1 hour. CoRR, abs/1706.02677, 2017.

A. Graves, A. Mohamed, and G. E. Hinton, Speech recognition with deep recurrent neural networks, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, vol.5778, 2013.

A. Graves, A. Mohamed, and G. Hinton, Speech recognition with deep recurrent neural networks, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp.6645-6649, 2013.

H. Greenspan, B. Van-ginneken, and R. M. Summers, Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique, IEEE Transactions on Medical Imaging, vol.35, issue.5, pp.1153-1159, 2016.

S. Guedria, N. De-palma, F. Renard, and N. Vuillerme, R2D2: A scalable deep learning toolkit for medical imaging segmentation, Software: Practice and Experience, vol.50, issue.10, pp.1966-1985, 2020.
URL : https://hal.archives-ouvertes.fr/hal-02915085

S. Guedria, N. De-palma, F. Renard, and N. Vuillerme, Auto-CNNp: a component-based framework for automating CNN parallelism, 2019 IEEE International Conference on Big Data (Big Data), pp.3330-3339, 2019.
URL : https://hal.archives-ouvertes.fr/hal-02894478

S. Guedria, N. De-palma, F. Renard, and N. Vuillerme, Automating CNN Parallelism with Components, 2019 International Conference on Computational Science and Computational Intelligence (CSCI), pp.943-948, 2019.
URL : https://hal.archives-ouvertes.fr/hal-02894479

Y. Guo, G. Wu, L. A. Commander, S. Szary, V. Jewells et al., Segmenting Hippocampus from Infant Brains by Sparse Patch Matching with Deep-Learned Features, Medical Image Computing and Computer-Assisted Intervention ? MICCAI 2014, pp.308-315, 2014.

S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, Deep learning with limited numerical precision, International Conference on Machine Learning, pp.1737-1746, 2015.

S. Gupta, W. Zhang, and F. Wang, Model Accuracy and Runtime Tradeoff in Distributed Deep Learning: A Systematic Study, Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017.

A. Harlap, D. Narayanan, A. Phanishayee, V. Seshadri, N. Devanur et al., Pipedream: Fast and efficient pipeline parallel dnn training, 2018.

M. Havaei, A. Davy, D. Warde-farley, A. Biard, A. Courville et al., Brain tumor segmentation with Deep Neural Networks, Medical Image Analysis, vol.35, pp.18-31, 2017.

M. Havaei, A. Davy, D. Warde-farley, A. Biard, A. Courville et al., Brain tumor segmentation with Deep Neural Networks, Medical Image Analysis, vol.35, pp.18-31, 2017.

M. Havaei, N. Guizard, N. Chapados, and Y. Bengio, HeMIS: Hetero-Modal Image Segmentation, Medical Image Computing and Computer-Assisted Intervention ? MICCAI 2016, pp.469-477, 2016.

K. He, X. Zhang, S. Ren, and J. Sun, Deep Residual Learning for Image Recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

K. He, X. Zhang, S. Ren, and J. Sun, Deep Residual Learning for Image Recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.770-778, 2016.

X. He, K. Zhao, and X. Chu, Automl: A survey of the state-of-the-art, 2019.

G. T. Heineman and W. T. Councill, Component-based software engineering. Putting the pieces together, addison-westley, 2001.

G. E. Hinton, A Practical Guide to Training Restricted Boltzmann Machines, Lecture Notes in Computer Science, vol.7700, pp.599-619, 2012.

G. E. Hinton, S. Osindero, and Y. Teh, A Fast Learning Algorithm for Deep Belief Nets, Neural Computation, vol.18, issue.7, pp.1527-1554, 2006.

Q. Ho, J. Cipar, H. Cui, S. Lee, J. K. Kim et al., More effective distributed ml via a stale synchronous parallel parameter server, Advances in Neural Information Processing Systems, vol.26, pp.1223-1231, 2013.

E. Hoffer, I. Hubara, and D. Soudry, Train longer, generalize better: closing the generalization gap in large batch training of neural networks, Advances in Neural Information Processing Systems, pp.1731-1741, 2017.

K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural Networks, vol.2, issue.5, pp.359-366, 1989.

D. H. Hubel and T. N. Wiesel, Receptive fields of single neurones in the cat's striate cortex, The Journal of Physiology, vol.148, issue.3, pp.574-591, 1959.

J. D. Hunter, Matplotlib: A 2D Graphics Environment, Computing in Science & Engineering, vol.9, issue.3, pp.90-95, 2007.

F. N. Iandola, M. W. Moskewicz, K. Ashraf, and K. Keutzer, FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.2592-2600, 2016.

A. Isin, C. Direkoglu, and M. Sah, Review of mri-based brain tumor image segmentation using deep learning methods, Procedia Comput. Sci, vol.102, issue.C, pp.317-324, 2016.

A. Ivakhnenko, Application of the theory of combination systems to adaptive cybernetic systems, IFAC Proceedings Volumes, vol.1, issue.1, pp.578-586, 1960.

K. Janocha and W. M. Czarnecki, On Loss Functions for Deep Neural Networks in Classification, Schedae Informaticae, vol.1/2016, 2017.

M. Jenkinson, C. F. Beckmann, T. E. Behrens, M. W. Woolrich, and S. M. Smith, FSL, NeuroImage, vol.62, issue.2, pp.782-790, 2012.
URL : https://hal.archives-ouvertes.fr/inserm-01149484

Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long et al., Caffe, Proceedings of the ACM International Conference on Multimedia - MM '14, 2014.

J. Jiang, B. Cui, C. Zhang, and L. Yu, Heterogeneity-aware Distributed Parameter Servers, Proceedings of the 2017 ACM International Conference on Management of Data - SIGMOD '17, pp.463-478, 2017.

P. Joelle, The machine learning reproducibility checklist

S. F. Johnsen, Z. A. Taylor, M. J. Clarkson, J. Hipwell, M. Modat et al., NiftySim: A GPU-based nonlinear finite element package for simulation of soft tissue biomechanics, International Journal of Computer Assisted Radiology and Surgery, vol.10, issue.7, pp.1077-1095, 2014.

E. Jones, T. Oliphant, and P. Peterson, {SciPy}: Open source scientific tools for {Python}, 2014.

P. Julkunen, R. K. Korhonen, M. J. Nissi, and J. S. Jurvelin, Mechanical characterization of articular cartilage by combining magnetic resonance imaging and finite-element analysis?a potential functional imaging technique, Physics in Medicine and Biology, vol.53, issue.9, pp.2425-2438, 2008.

K. Kamnitsas, E. Ferrante, S. Parisot, C. Ledig, A. V. Nori et al., DeepMedic for Brain Tumor Segmentation, Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, pp.138-149, 2016.

K. Kamnitsas, C. Ledig, V. F. Newcombe, J. P. Simpson, A. D. Kane et al., Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, Medical Image Analysis, vol.36, pp.61-78, 2017.

K. Kamnitsas, C. Ledig, V. F. Newcombe, J. P. Simpson, A. D. Kane et al., Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, Medical Image Analysis, vol.36, pp.61-78, 2017.

B. Karlik and V. Olgac, Performance analysis of various activation functions in generalized mlp architectures of neural networks, 2011.

X. Ke, K. Sierszecki, and C. Angelov, COMDES-II: A Component-Based Framework for Generative Development of Distributed Real-Time Control Systems, 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA 2007), pp.199-208, 2007.

Y. Kim, Convolutional Neural Networks for Sentence Classification, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), vol.5882, 2014.

D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, 2014.

J. Kleesiek, G. Urban, A. Hubert, D. Schwarz, K. Maier-hein et al., Deep MRI brain extraction: A 3D convolutional neural network for skull stripping, NeuroImage, vol.129, pp.460-469, 2016.

D. J. Kriegman, P. N. Belhumeur, and A. S. Georghiades, Representations for Recognition Under Variable Illumination, Shape, Contour and Grouping in Computer Vision, pp.95-131, 1999.

A. Krizhevsky, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, Communications of the ACM, vol.60, issue.6, pp.84-90, 2017.

A. Krogh and J. A. Hertz, PAC Generalization Bounds for Co-training, Advances in Neural Information Processing Systems 14, vol.4, pp.950-957, 2002.

R. , How to create a mind: the secret of human thought revealed, Choice Reviews Online, vol.50, issue.11, p.50-6167-50-6167, 2013.

I. Larrabide, P. Omedas, Y. Martelli, X. Planes, M. Nieber et al., GIMIAS: An Open Source Framework for Efficient Development of Research Tools and Clinical Prototypes, Functional Imaging and Modeling of the Heart, pp.417-426, 2009.

S. Lawrence, C. L. Giles, . Ah-chung-tsoi, and A. D. Back, Face recognition: a convolutional neural-network approach, IEEE Transactions on Neural Networks, vol.8, issue.1, pp.98-113, 1997.

Q. V. Le, M. Ranzato, R. Monga, M. Devin, K. Chen et al., Building high-level features using large scale unsupervised learning, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp.507-514, 2013.

Y. Lecun and Y. Bengio, The handbook of brain theory and neural networks. chapter Convolutional Networks for Images, Speech, and Time Series, pp.255-258, 1998.

Y. Lecun, Y. Bengio, and G. E. Hinton, Deep learning, Nature, vol.521, issue.7553, pp.436-444, 2015.

Y. Lecun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard et al., Backpropagation Applied to Handwritten Zip Code Recognition, Neural Computation, vol.1, issue.4, pp.541-551, 1989.

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE, vol.86, issue.11, pp.2278-2324, 1998.

Y. Lecun and C. Cortes, MNIST handwritten digit database, 2010.

Y. A. Lecun, L. Bottou, G. B. Orr, and K. Müller, Efficient BackProp, Lecture Notes in Computer Science, pp.9-48, 2012.

L. Lee and S. Liew, A survey of medical image processing tools, 2015 4th International Conference on Software Engineering and Computer Systems (ICSECS), vol.08, 2015.

S. Lee, J. K. Kim, X. Zheng, Q. Ho, G. A. Gibson et al., On model parallelization and scheduling strategies for distributed machine learning, Advances in neural information processing systems, pp.2834-2842, 2014.

H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein, Causal Categorization with Bayes Nets, Advances in Neural Information Processing Systems 14, pp.6389-6399, 2002.

M. Li, D. G. Andersen, J. W. Park, A. J. Smola, A. Ahmed et al., Proceedings of the second USENIX symposium on Operating systems design and implementation - OSDI '96, Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation, OSDI'14, pp.583-598, 1996.

M. Li, D. G. Andersen, A. J. Smola, and K. Yu, Effects of Stress and Genotype on Meta-parameter Dynamics in Reinforcement Learning, Advances in Neural Information Processing Systems 19, pp.19-27, 2007.

Z. Li and S. Arora, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

M. Lin, Q. Chen, and S. Yan, , 2013.

G. Litjens, T. Kooi, B. E. Bejnordi, A. A. Setio, F. Ciompi et al., A survey on deep learning in medical image analysis, Medical Image Analysis, vol.42, pp.60-88, 2017.

J. Long, E. Shelhamer, and T. Darrell, Fully convolutional networks for semantic segmentation, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.

I. Loshchilov and F. Hutter, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

J. B. Maintz and M. A. Viergever, A survey of medical image registration, Medical Image Analysis, vol.2, issue.1, pp.1-36, 1998.

A. Mansoor, J. J. Cerrolaza, R. Idrees, E. Biggs, M. A. Alsharid et al., Deep Learning Guided Partitioned Shape Model for Anterior Visual Pathway Segmentation, IEEE Transactions on Medical Imaging, vol.35, issue.8, pp.1856-1865, 2016.

D. Maturana and S. Scherer, VoxNet: A 3D Convolutional Neural Network for real-time object recognition, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.922-928, 2015.

D. Mccaughey and N. S. Bruning, Rationality versus reality: the challenges of evidence-based decision making for health policy makers, Implementation Science, vol.5, issue.1, p.39, 2010.

A. Mehrtash, M. Pesteie, J. Hetherington, P. A. Behringer, T. Kapur et al., DeepInfer: open-source deep learning deployment toolkit for image-guided therapy, Medical Imaging 2017: Image-Guided Procedures, Robotic Interventions, and Modeling, vol.10135, 2017.

Q. Meng, W. Chen, Y. Wang, Z. Ma, and T. Liu, Convergence analysis of distributed stochastic gradient descent with shuffling, Neurocomputing, vol.337, pp.46-57, 2019.

B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-cramer, K. Farahani et al., The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS), IEEE Transactions on Medical Imaging, vol.34, issue.10, pp.1993-2024, 2015.
URL : https://hal.archives-ouvertes.fr/hal-00935640

B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-cramer, K. Farahani et al., The multimodal brain tumor image segmentation benchmark (brats), IEEE transactions on medical imaging, vol.34, issue.10, pp.1993-2024, 2014.
URL : https://hal.archives-ouvertes.fr/hal-00935640

F. Milletari, S. Ahmadi, C. Kroll, A. Plate, V. Rozanski et al., Hough-CNN: Deep learning for segmentation of deep brain regions in MRI and ultrasound, Computer Vision and Image Understanding, vol.164, pp.92-102, 2017.

F. Milletari, N. Navab, and S. Ahmadi, V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation, 2016 Fourth International Conference on 3D Vision (3DV), pp.565-571, 2016.

M. Minsky and S. A. Papert, Perceptrons, 2017.

P. Moeskops, M. A. Viergever, A. M. Mendrik, L. S. De-vries, M. J. Benders et al., Automatic Segmentation of MR Brain Images With a Convolutional Neural Network, IEEE Transactions on Medical Imaging, vol.35, issue.5, pp.1252-1261, 2016.

C. More, L. Colaco, and R. Sardinha, Application of Component-Based Software Engineering in Building a Surveillance Robot, Advances in Intelligent Systems and Computing, pp.651-658, 2015.

P. Nagarajan, G. Warnell, and P. Stone, The impact of nondeterminism on reproducibility in deep reinforcement learning, 2018.

V. Nair and G. E. Hinton, Rectified linear units improve restricted boltzmann machines, Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML'10, pp.807-814, 2010.

E. , National Academies of Sciences and Medicine. Reproducibility and Replicability in Science, 2019.

A. Y. Ng, Feature selection, L1 vs. L2 regularization, and rotational invariance, Twenty-first international conference on Machine learning - ICML '04, p.78, 2004.

D. Nie, L. Wang, Y. Gao, and D. Shen, Fully convolutional networks for multi-modality isointense infant brain image segmentation, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp.1342-1345, 2016.

H. Noh, S. Hong, and B. Han, Learning Deconvolution Network for Semantic Segmentation, 2015 IEEE International Conference on Computer Vision (ICCV), 2015.

J. D. Owens, M. Houston, D. Luebke, S. Green, J. E. Stone et al., GPU Computing, Proceedings of the IEEE, vol.96, issue.5, pp.879-899, 2008.

S. J. Pan and Q. Yang, A Survey on Transfer Learning, IEEE Transactions on Knowledge and Data Engineering, vol.22, issue.10, pp.1345-1359, 2010.

P. Patarasuk and X. Yuan, Bandwidth optimal all-reduce algorithms for clusters of workstations, Journal of Parallel and Distributed Computing, vol.69, issue.2, pp.117-124, 2009.

N. Pawlowski, S. I. Ktena, M. C. Lee, B. Kainz, D. Rueckert et al., DLTK: state of the art reference implementations for deep learning on medical images

. Corr, Corr, Philip, Encyclopedia of Personality and Individual Differences, pp.1-2, 2017.

S. Pereira, A. Pinto, V. Alves, and C. A. Silva, Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images, IEEE Transactions on Medical Imaging, vol.35, issue.5, pp.1240-1251, 2016.

L. Perez and J. Wang, The effectiveness of data augmentation in image classification using deep learning, 2017.

N. Pessemier, L. Seinturier, L. Duchien, and T. Coupaye, A component-based and aspect-oriented model for software evolution, International Journal of Computer Applications in Technology, vol.31, issue.1/2, p.94, 2008.
URL : https://hal.archives-ouvertes.fr/inria-00269895

G. Piantadosi, S. Marrone, and C. Sansone, On Reproducibility of Deep Convolutional Neural Networks Approaches, Reproducible Research in Pattern Recognition, pp.104-109, 2019.

S. Pieper, B. Lorensen, W. Schroeder, and R. Kikinis, The NA-MIC Kit: ITK, VTK, Pipelines, Grids and 3D Slicer as An Open Platform for the Medical Image Computing Community, 3rd IEEE International Symposium on Biomedical Imaging: Macro to Nano, 2006., pp.698-701

S. Pieper, B. Lorensen, W. Schroeder, and R. Kikinis, The NA-MIC Kit: ITK, VTK, Pipelines, Grids and 3D Slicer as An Open Platform for the Medical Image Computing Community, 3rd IEEE International Symposium on Biomedical Imaging: Macro to Nano, 2006., pp.698-701

C. E. Rasmussen and C. K. Williams, Gaussian Processes for Machine Learning, Gaussian Processes in Machine Learning, 2005.

R. Reed and R. J. Marks, Neural Smithing, 1999.

F. Renard, S. Guedria, N. D. Palma, and N. Vuillerme, Variability and reproducibility in deep learning for medical image segmentation, Scientific Reports, vol.10, issue.1, pp.1-16, 2020.
URL : https://hal.archives-ouvertes.fr/hal-02917117

O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, Lecture Notes in Computer Science, pp.234-241, 2015.

F. Rosenblatt, The perceptron: A probabilistic model for information storage and organization in the brain., Psychological Review, vol.65, issue.6, pp.386-408, 1958.

H. R. Roth, L. Lu, J. Liu, J. Yao, A. Seff et al., Efficient False Positive Reduction in Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation, Deep Learning and Convolutional Neural Networks for Medical Image Computing, pp.35-48, 2017.

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh et al., ImageNet Large Scale Visual Recognition Challenge, International Journal of Computer Vision, vol.115, issue.3, pp.211-252, 2015.

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh et al., ImageNet Large Scale Visual Recognition Challenge, International Journal of Computer Vision, vol.115, issue.3, pp.211-252, 2015.

R. Salakhutdinov and G. Hinton, An Efficient Learning Procedure for Deep Boltzmann Machines, Neural Computation, vol.24, issue.8, pp.1967-2006, 2012.

J. Sanders and E. Kandrot, CUDA by example: an introduction to general-purpose GPU programming, 2010.

W. S. Sarle, Correction Methods and Other Remedies for Improving Sensory Profile Data, Statistics for Sensory and Consumer Science, pp.39-46, 2010.

J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks, vol.61, pp.85-117, 2015.

J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks, vol.61, pp.85-117, 2015.

A. Sergeev and M. D. Balso, Horovod: fast and easy distributed deep learning in tensorflow, 2018.

M. Shakeri, S. Tsogkas, E. Ferrante, S. Lippe, S. Kadoury et al., Sub-cortical brain structure segmentation using F-CNN'S, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp.269-272, 2016.

N. Sharma, A. K. Ray, K. K. Shukla, S. Sharma, S. Pradhan et al., Automated medical image segmentation techniques, Journal of Medical Physics, vol.35, issue.1, p.3, 2010.

D. Shen, G. Wu, and H. Suk, Deep Learning in Medical Image Analysis, Annual Review of Biomedical Engineering, vol.19, issue.1, pp.221-248, 2017.

J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell, vol.22, issue.8, pp.888-905, 2000.

H. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu et al., Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning, IEEE Transactions on Medical Imaging, vol.35, issue.5, pp.1285-1298, 2016.

G. M. Shipman, T. S. Woodall, R. L. Graham, A. B. Maccabe, and P. G. Bridges, Infiniband scalability in Open MPI, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium, p.10, 2006.

P. E. Shrout and J. L. Fleiss, Intraclass correlations: Uses in assessing rater reliability., Psychological Bulletin, vol.86, issue.2, pp.420-428, 1979.

K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, 2014.

. Skymind, Distributed deep learning, part1 : An introduction to distributed training of neural networks

L. N. Smith, Cyclical Learning Rates for Training Neural Networks, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.464-472, 2017.

L. N. Smith, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

S. L. Smith, P. Kindermans, and Q. V. Le, Don't decay the learning rate, increase the batch size, 2017.

J. Snoek, H. Larochelle, and R. P. Adams, Bayesian Policy Gradient Algorithms, Advances in Neural Information Processing Systems 19, vol.25, pp.2951-2959, 2007.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res, vol.15, issue.1, pp.1929-1958, 2014.

M. F. Stollenga, W. Byeon, M. Liwicki, and J. Schmidhuber, Parallel multi-dimensional lstm, with application to fast biomedical volumetric image segmentation, Advances in neural information processing systems, pp.2998-3006, 2015.

N. Strom, TeraChem Cloud: A High-Performance Computing Service for Scalable Distributed GPU-Accelerated Electronic Structure Calculations, INTERSPEECH

A. Stupple, D. Singerman, and L. A. Celi, The reproducibility crisis in the age of digital medicine, npj Digital Medicine, vol.2, issue.1, 2019.

H. Su and H. Chen, Experiments on parallel training of deep neural network using model averaging, 2015.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, Rethinking the Inception Architecture for Computer Vision, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

C. Szegedy, . Wei-liu, . Yangqing-jia, P. Sermanet, S. Reed et al., Going deeper with convolutions, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, Rethinking the Inception Architecture for Computer Vision, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

A. A. Taha and A. Hanbury, Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool, BMC Medical Imaging, vol.15, issue.1, p.29, 2015.

N. Ketkar, Introduction to Theano, Deep Learning with Python, pp.35-61, 2017.

T. Tieleman and G. Hinton, Using fast weights to improve persistent contrastive divergence, Proceedings of the 26th Annual International Conference on Machine Learning - ICML '09, vol.4, pp.26-31, 2009.

H. Tien-lin and C. Lin, A study on sigmoid kernels for svm and the training of non-psd kernels by smo-type methods, 2003.

C. Tobon-gomez, A. J. Geers, J. Peters, J. Weese, K. Pinto et al., Benchmark for Algorithms Segmenting the Left Atrium From 3D CT and MRI Datasets, IEEE Transactions on Medical Imaging, vol.34, issue.7, pp.1460-1473, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01260607

J. K. Udupa, V. R. Leblanc, Y. Zhuge, C. Imielinska, H. Schmidt et al., A framework for evaluating image segmentation algorithms, Computerized Medical Imaging and Graphics, vol.30, issue.2, pp.75-87, 2006.

A. Vedaldi and K. Lenc, MatConvNet, Proceedings of the 23rd ACM international conference on Multimedia - MM '15, 2015.

P. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol, Extracting and composing robust features with denoising autoencoders, Proceedings of the 25th international conference on Machine learning - ICML '08, pp.1096-1103, 2008.

P. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol, Extracting and composing robust features with denoising autoencoders, Proceedings of the 25th international conference on Machine learning - ICML '08, vol.11, pp.3371-3408, 2008.

L. Wang, C. Xie, and Y. Hu, IDDF2018-ABS-0260 Deep learning for polyp segmentation, Clinical Gastroenterology, 2018.

Y. Wang, G. Wei, and D. Brooks, Adaptive packet processing on CPU-GPU heterogeneous platforms, Many-Core Computing: Hardware and Software, pp.247-269, 2019.

. Wikipedia, Comparison of deep-learning software

I. Wolf, M. Vetter, I. Wegner, T. Böttger, M. Nolden et al., The Medical Imaging Interaction Toolkit, Medical Image Analysis, vol.9, issue.6, pp.594-604, 2005.

Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi et al., Google's neural machine translation system: Bridging the gap between human and machine translation, 2016.

Y. You, I. Gitman, and B. Ginsburg, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

Y. You, Z. Zhang, C. Hsieh, J. Demmel, and K. Keutzer, ImageNet Training in Minutes, Proceedings of the 47th International Conference on Parallel Processing, 2018.

Y. You, Z. Zhang, C. Hsieh, J. Demmel, and K. Keutzer, ImageNet Training in Minutes, Proceedings of the 47th International Conference on Parallel Processing, 2018.

S. R. Young, D. C. Rose, T. P. Karnowski, S. Lim, and R. M. Patton, Optimizing deep learning hyper-parameters through an evolutionary algorithm, Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments - MLHPC '15, pp.1-5, 2015.

L. Yu, S. Wang, and K. K. Lai, An integrated data preparation scheme for neural network data analysis, IEEE Transactions on Knowledge and Data Engineering, vol.18, issue.2, pp.217-230, 2006.

Y. Yu, J. Jiang, and X. Chi, Using Supercomputer to Speed up Neural Network Training, 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS), pp.942-947, 2016.

M. D. Zeiler, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

M. D. Zeiler and R. Fergus, Visualizing and Understanding Convolutional Networks, Computer Vision ? ECCV 2014, pp.818-833, 2014.

S. Zhang, A. E. Choromanska, and Y. Lecun, Neural Information Processing Systems, The Deep Learning Revolution, pp.685-693, 2018.

W. Zhang, S. Gupta, X. Lian, and J. Liu, Staleness-aware async-sgd for distributed deep learning, Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI'16, pp.2350-2356, 2016.

W. Zhang, R. Li, H. Deng, L. Wang, W. Lin et al., Deep convolutional neural networks for multi-modality isointense infant brain image segmentation, NeuroImage, vol.108, pp.214-224, 2015.

Y. J. Zhang, A survey on evaluation methods for image segmentation, Pattern Recognition, vol.29, issue.8, pp.1335-1346, 1996.

L. Zhao and K. Jia, Multiscale CNNs for Brain Tumor Segmentation and Diagnosis, Computational and Mathematical Methods in Medicine, vol.2016, issue.7, pp.1-7, 2016.

L. Zhao and K. Jia, Multiscale CNNs for Brain Tumor Segmentation and Diagnosis, Computational and Mathematical Methods in Medicine, vol.2016, pp.1-7, 2016.

B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, Learning Transferable Architectures for Scalable Image Recognition, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.8697-8710, 2018.