Section: Research Program

Data-based identification of characteristic scales and automated modeling

Data are often acquired at the highest possible resolution, but that scale is not necessarily the best for modeling and understanding the system from which data was measured. The intrinsic properties of natural processes do not depend on the arbitrary scale at which data is acquired; yet, usual analysis techniques operate at the acquisition resolution. When several processes interact at different scales, the identification of their characteristic scales from empirical data becomes a necessary condition for properly modeling the system. A classical method for identifying characteristic scales is to look at the work done by the physical processes, the energy they dissipate over time. The assumption is that this work matches the most important action of each process on the studied natural system, which is usually a reasonable assumption. In the framework of time-frequency analysis [46], the power of the signal can be easily computed in each frequency band, itself matching a temporal scale.

However, in open and dissipative systems, energy dissipation is a prerequisite and thus not necessarily the most useful metric to investigate. In fact, most natural, physical and industrial systems we deal with fall in this category, while balanced quasi-static assumptions are practical approximation only for scales well below the characteristic scale of the involved processes. Open and dissipative systems are not locally constrained by the inevitable rise in entropy, thus allowing the maintaining through time of mesoscopic ordered structures. And, according to information theory [48], more order and less entropy means that these structures have a higher information content than the rest of the system, which usually gives them a high functional role.

We propose to identify characteristic scales not only with energy dissipation, as usual in signal processing analysis, but most importantly with information content. Information theory can be extended to look at which scales are most informative (e.g. multi-scale entropy [39], $\epsilon$-entropy [38]). Complexity measures quantify the presence of structures in the signal (e.g. statistical complexity [43], MPR [56] and others [45]). With these notions, it is already possible to discriminate between random fluctuations and hidden order, such as in chaotic systems [42], [56]. The theory of how information and structures can be defined through scales is not complete yet, but the state of art is promising [44]. Current research in the team focuses on how informative scales can be found using collections of random paths, assumed to capture local structures as they reach out [37].

Building on these notions, it should also possible to fully automate the modeling of a natural system. Once characteristic scales are found, causal relationships can be established empirically. They are then clustered together in internal states of a special kind of Markov models called $ϵ$-machines [43]. These are known to be the optimal predictors of a system, with the drawback that it is currently quite complicated to build them properly, except for small system [62]. Recent extensions with advanced clustering techniques [36], [47], coupled with the physics of the studied system (e.g. fluid dynamics), have proved that $ϵ$-machines are applicable to large systems, such as global wind patterns in the atmosphere [52]. Current research in the team focuses on the use of reproducing kernels, coupled possibly with sparse operators, in order to design better algorithms for $ϵ$-machines reconstruction. In order to help with this long-term project, a collaboration with J. Crutchfield lab at UC Davis was initiated in 2017.