Fredi: Unitary CNN

I worked on the research project “Unitary CNN”. Since I already had some experience with Machine Learning and Deep Learning and did not have any experience with research, I chose to focus on working on a pure research project.

The goal was to extend the work of “Tunable Efficient Unitary Neural Networks (EUNN) and Their Application to RNNs” to work with CNN networks and by consequence to work as any linear transformation which is a basic building block of neural networks.

The main problem I had during the project was to construct a CNN layer as a linear transformation that preserves the norm of the input vector, which we, my mentor Mladen and I, managed to construct but we did not manage to find the unitary/orthogonal parametrization of NxM matrix needed for such linear transformation. This was mainly due to mistakes in equations and poor explanation of the process used to obtain results in original paper.

Although the project (research) is not yet finished, Mladen and I agreed to continue to work on it (I think we will be successful in finding such parametrization that is computationally efficient). I have learned how research actually looks like and that it is hard work. Also, I have extended my knowledge of linear algebra.