Smaller AI Models Suffice For Brain-Language Studies, Says IIIT-H Study
Researchers found models with about 3 billion parameters perform almost as well as those with up to 14 billion, potentially reducing the computing power needed for brain research.
Hyderabad: Bigger artificial intelligence (AI) models may not necessarily be better for understanding how the human brain processes language, according to research from the International Institute of Information Technology Hyderabad (IIIT-H).
Researchers found models with about 3 billion parameters perform almost as well as those with up to 14 billion, potentially reducing the computing power needed for brain research.
In their work ‘Linguistic Properties and Model Scale in Brain Encoding: From Small to Compressed Language Models’ presented at the International Conference on Machine Learning (ICML) in Seoul, Prof. Bapi Raju and PhD researcher Vijay Rowtula tested whether increasing a language model’s size necessarily improves its ability to predict human brain activity.
The researchers examined small language models and made them lighter using quantisation, which reduces the numerical precision used within a model, and pruning, which removes less important parts. Most techniques tested, including quantisation and moderate pruning, reduced model size without substantially affecting predictions of brain activity.
The findings challenge earlier research suggesting that moving from smaller to larger models improved brain prediction accuracy by about 15 per cent.
However, compression did affect performance on some conventional language tasks involving grammar, discourse and morphology, even while brain alignment remained largely unchanged. “The novel finding is the dissociation observed between brain alignment and linguistic competence,” Prof. Raju said.
He also suggested that abilities needed to score well on language benchmarks may differ from the representations useful for modelling how the brain processes language.
The finding could have practical implications for computational neuroscience. Smaller models require less memory and computing resources, making brain-language studies potentially cheaper and faster. Prof. Raju said this could be a “game changer” for brain-decoding workflows used in developing brain-computer interfaces.
For Rowtula, who moved into computational neuroscience after working in computer vision and industry, the research is part of a broader effort to understand whether computational models can imitate human brain functions.
At ICML, the researchers also exchanged ideas with computational neuroscience researchers, including members of the NeuroAI Lab at EPFL, opening possibilities for future collaboration.
