Optimization and sloppy parameter analyses allowed the effect of parameters used by a complex cognitive model of reading aloud (CDP) to be understood.

Optimizing the parameters of CDP does not cause over-fitting and the model generalizes better than hand-picked parameters.

Sloppy parameter analysis shows that only a small number of parameters are responsible for most of CDP’s quantitative performance.

The parameters of CDP show an exponential hierarchy of sensitivity distribution, similar to models in many other areas of science.

The sloppy parameter method used here could be meaningfully applied to other complex cognitive models.

A major strength of computational cognitive models is their capacity to accurately predict empirical data. However, challenges in understanding how complex models work and the risk of overfitting have often been addressed by trading off predictive accuracy with model simplification. Here, we introduce state-of-the-art model analysis techniques to show how a large number of parameters in a cognitive model can be reduced into a smaller set that is simpler to understand and can be used to make more constrained predictions with. As a test case, we created different versions of the Connectionist Dual-Process model (CDP) of reading aloud whose parameters were optimized on seven different databases. The results showed that CDP was not overfit and could predict a large amount of variance across those databases. Indeed, the quantitative performance of CDP was higher than that of previous models in this area. Moreover, sloppy parameter analysis, a mathematical technique used to quantify the effects of different parameters on model performance, revealed that many of the parameters in CDP have very little effect on its performance. This shows that the dynamics of CDP are much simpler than its relatively large number of parameters might suggest. Overall, our study shows that cognitive models with large numbers of parameters do not necessarily overfit the empirical data and that understanding the behavior of complex models is more tractable using appropriate mathematical tools. The same techniques could be applied to many different complex cognitive models whenever appropriate datasets for model optimization exist.

Reading

Optimization

Sloppy parameters

Computational modelling

Any data not already on the osf site for this project (https://osf.io/bkn6j/) is available from CP.

© 2023 The Authors. Published by Elsevier Inc.