Analysis by Synthesis

Modeling in Scientific Research

Modeling is an important tool for verifying and estimating how well a theory explains a certain phenomenon. It often provides immediate intuitive feedback just by listening to the resulting audio or by comparing the spectrograms of the output with a real example. Thus, it is often easy to qualitatively evaluate if the theory is capturing the essential elements of a phenomenon.

In fact, modeling is a convenient method for investigating the perceptual influence of individual sound parameters. For example, using a model for music performance, the individual effect of timing and dynamics variations can be estimated by varying these two parameters independently in a listening experiment. In most practical cases this cannot be obtained using real-world data, such as audio recordings of different music examples. The reason is that in such recordings, a number of parameters typically exhibit a strong covariation. For example, the tempo might covariate with dynamics since they both contribute to perceived energy, or pitch might covariate with dynamics since it is difficult to produce high pitches with soft dynamics on many instruments. This means that in practical work modeling is a good way of determining the effect of a certain parameter in isolation.

[Page 44]A drawback using a modeling approach is that all details of the algorithm need to be specified. A research-based computational model should be based on previous findings. However, all required details have seldom been investigated in previous research. Therefore, there is also a certain ad hoc element built into any model. Although a few of these can be evaluated, it is seldom possible to independently assess all parameters of a model.

Another use of models is obviously in applied research and in commercial settings when there is a need to use the models for specific applications, such as speech or singing synthesis, or synthesis of various musical instruments' sounds or music generation in computer games.

Evaluation

Testing the validity of a computational model can be a rather complex task. The number of parameters is often high, implying that each parameter cannot for practical reasons be evaluated independently. Therefore, only details that clearly affect the final outcome might be evaluated.

...