[Submitted on 11 Dec 2019 (v1), last revised 12 Dec 2019 (this version, v2)] Authors:Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry Download PDF Abstract: Standard practice in training neural networks involves initializing the weights in an independent fashion. The results of recent work suggest that feature "diversity" at initialization plays an important role in training the network. However, other initialization schemes with reduced feature diversity have also been shown to be viable.
