Flavio Martinelli

Rare pic of me well dressed

I am a PhD student in the Laboratory of Computational Neuroscience at EPFL, supervised by Wulfram Gerstner and Johanni Brea.

My main research revolves around understanding weight structures in neural networks.
I apply this knowledge on the following topics:

🕵️ Identifiability: can network parameters be reverse engineered? What types of constraints are needed? What are the implications for neural circuits?

🏋️ Trainability: what weight structures are easier or harder to learn? What are the weight structures of sub-optimal solutions (local minima)?

💡 Interpretability: understanding network symmetries to improve manipulation and interpretation of network models.

news

Mar 01, 2025	🌎 I am currently based in Boston for a 4 months visit in Kanaka Rajan’s lab. Reach out if you want to have a chat!
Sep 02, 2024	I gave a talk on Expand-and-Cluster for the EfficientML reading group – link to video.
Jul 21, 2024	In Vienna for ICML ‘24 and a visit to Tim Vogel’s lab in IST, Austria.
May 20, 2024	In Trieste for the Youth in High Dimensions meeting.
May 02, 2024	📝 Expand-and-Cluster is accepted to ICML 2024! 🔍 We identify the weights of a neural network from simple input-output queries. ⚠️ Symmetries and overparameterised loss landscapes are heavily involved.
Feb 08, 2024	Gave a talk and visited the laboratory of Jakob Macke at the Tübingen AI Center, Germany.
Jan 31, 2024	Gave a talk at the annual Swiss Computational Neuroscience meeting.
Sep 26, 2023	In Berlin for a poster at the Bernstein conference.

selected publications

ICML
Expand-and-Cluster: Parameter Recovery of Neural Networks

Flavio Martinelli, Berfin Simsek, Wulfram Gerstner^*, and Johanni Brea^*

In Forty-first International Conference on Machine Learning (ICML) , 2024

Abs Bib HTML Code

Can we identify the weights of a neural network by probing its input-output mapping? At first glance, this problem seems to have many solutions because of permutation, overparameterisation and activation function symmetries. Yet, we show that the incoming weight vector of each neuron is identifiable up to sign or scaling, depending on the activation function. Our novel method ’Expand-and-Cluster’ can identify layer sizes and weights of a target network for all commonly used activation functions. Expand-and-Cluster consists of two phases: (i) to relax the non-convex optimisation problem, we train multiple overparameterised student networks to best imitate the target function; (ii) to reverse engineer the target network’s weights, we employ an ad-hoc clustering procedure that reveals the learnt weight vectors shared between students – these correspond to the target weight vectors. We demonstrate successful weights and size recovery of trained shallow and deep networks with less than 10% overhead in the layer size and describe an ’ease-of-identifiability’ axis by analysing 150 synthetic problems of variable difficulty.
@inproceedings{martinelli2023expand, title = {Expand-and-Cluster: Parameter Recovery of Neural Networks}, author = {Martinelli, Flavio and Simsek, Berfin and Gerstner, Wulfram and Brea, Johanni}, booktitle = {Forty-first International Conference on Machine Learning (ICML)}, year = {2024}, doi = {}, }
ICASSP
Spiking neural networks trained with backpropagation for low power neuromorphic implementation of voice activity detection

Flavio Martinelli, Giorgia Dellaferrera, Pablo Mainar, and Milos Cernak

In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2020

Abs Bib HTML Code

Recent advances in Voice Activity Detection (VAD) are driven by artificial and Recurrent Neural Networks (RNNs), however, using a VAD system in battery-operated devices requires further power efficiency. This can be achieved by neuromorphic hardware, which enables Spiking Neural Networks (SNNs) to perform inference at very low energy consumption. Spiking networks are characterized by their ability to process information efficiently, in a sparse cascade of binary events in time called spikes. However, a big performance gap separates artificial from spiking networks, mostly due to a lack of powerful SNN training algorithms. To overcome this problem we exploit an SNN model that can be recast into a recurrent network and trained with known deep learning techniques. We describe a training procedure that achieves low spiking activity and apply pruning algorithms to remove up to 85% of the network connections with no performance loss. The model competes with state-of-the-art performance at a fraction of the power consumption comparing to other methods.
@inproceedings{martinelli2020spiking, title = {Spiking neural networks trained with backpropagation for low power neuromorphic implementation of voice activity detection}, author = {Martinelli, Flavio and Dellaferrera, Giorgia and Mainar, Pablo and Cernak, Milos}, booktitle = {ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pages = {8544--8548}, year = {2020}, organization = {IEEE}, doi = {10.1109/ICASSP40776.2020.9053412}, }