Sponsored by
[ Events ]

Activity Search
Sort out
NCTS Seminar on March Events
11:10 - 12:00, March 10, 2020 (Tuesday)
R430, Astronomy-Mathematics Building, NTU
(台灣大學天文數學館 430室)
Unrectify Non-linear Networks
Wen-Liang Hwang (Institute of Information Science , Academia Sinica)


We propose a novel technique to "un-rectify" the nonlinear activations into data-dependent linear equations and constraints, from which we derive explicit expressions for the affine linear operators, their domains and ranges in terms of the network parameters. We show how increasing the depth of the network refines the domain partitioning and derive atomic decompositions for the corresponding affine mappings that process data belonging to the same partitioning region. In each atomic decomposition the connections over all hidden network layers are summarized and interpreted in a single matrix. We apply the decompositions to study the Lipschitz regularity of the networks and give sufficient conditions for network-depth-
independent stability of the representation, drawing a connection to compressible weight distributions. Such analyses may facilitate and promote further theoretical insight and exchange from both the signal processing and machine learning communities.


back to list  
 (C) 2019 National Center for Theoretical Sciences