Disordered Systems and Neural Networks
- [1] arXiv:2405.09535 [pdf, ps, html, other]
-
Title: Restoring balance: principled under/oversampling of data for optimal classificationComments: 9 pages + appendix, 3 figuresSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG)
Class imbalance in real-world data poses a common bottleneck for machine learning tasks, since achieving good generalization on under-represented examples is often challenging. Mitigation strategies, such as under or oversampling the data depending on their abundances, are routinely proposed and tested empirically, but how they should adapt to the data statistics remains poorly understood. In this work, we determine exact analytical expressions of the generalization curves in the high-dimensional regime for linear classifiers (Support Vector Machines). We also provide a sharp prediction of the effects of under/oversampling strategies depending on class imbalance, first and second moments of the data, and the metrics of performance considered. We show that mixed strategies involving under and oversampling of data lead to performance improvement. Through numerical experiments, we show the relevance of our theoretical predictions on real datasets, on deeper architectures and with sampling strategies based on unsupervised probabilistic models.
New submissions for Thursday, 16 May 2024 (showing 1 of 1 entries )
- [2] arXiv:2405.08861 (cross-list from quant-ph) [pdf, ps, html, other]
-
Title: Measurement-induced phase transitions in systems with diffusive dynamicsComments: 7 pages, 4 figures + Supplementary Material (10 pages,10 figures)Subjects: Quantum Physics (quant-ph); Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech)
The competition between scrambling and projective measurements can lead to measurement-induced entanglement phase transitions (MIPT). In this work, we show that the universality class of the MIPT is drastically altered when the system is coupled to a diffusing conserved density. Specifically, we consider a 1+1d random Clifford circuit locally monitored by classically diffusing particles (``measurers''). The resulting diffusive correlations in the measurement density are a relevant perturbation to the usual space-time random MIPT critical point, producing a new universality class for this phase transition. We find ``Griffiths-like'' effects due to rare space-time regions where, e.g., the diffusive measurers have a low or high density, but these are considerably weaker than the Griffiths effects that occur with quenched randomness that produce rare spatial regions with infinite lifetime.
- [3] arXiv:2405.09078 (cross-list from cond-mat.soft) [pdf, ps, html, other]
-
Title: Dynamic constraints predict the relaxation of granular materialsSubjects: Soft Condensed Matter (cond-mat.soft); Disordered Systems and Neural Networks (cond-mat.dis-nn); Materials Science (cond-mat.mtrl-sci); Statistical Mechanics (cond-mat.stat-mech)
Granular materials such as sand, powders, and food grains are ubiquitous in civil engineering, geoscience, agriculture, and medicine. While the influence of friction between the grains on the static structure of these systems is well understood, its impact on the dynamics is an open problem. Here we use particle-based simulations of a granular pack under cyclic shear and discover that the relaxation time of the system is a non-monotonic function of friction. By introducing the concept of dynamic constraints, we reveal that this re-entrant dynamics is due to the competition between increasing frictional coupling and a concurrent change in the structure of the granular pack. Our theoretical approach, which unifies the dynamics of friction-less systems with frictional ones, is applicable to other systems that have a complex free energy landscape and a dynamics which involves time-dependent constraints, thus setting the stage for a description of the dynamic behavior of a large class of complex systems.
- [4] arXiv:2405.09324 (cross-list from math.NA) [pdf, ps, html, other]
-
Title: Learning Coarse-Grained Dynamics on GraphComments: 33 pages, 12 figuresSubjects: Numerical Analysis (math.NA); Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG)
We consider a Graph Neural Network (GNN) non-Markovian modeling framework to identify coarse-grained dynamical systems on graphs. Our main idea is to systematically determine the GNN architecture by inspecting how the leading term of the Mori-Zwanzig memory term depends on the coarse-grained interaction coefficients that encode the graph topology. Based on this analysis, we found that the appropriate GNN architecture that will account for $K$-hop dynamical interactions has to employ a Message Passing (MP) mechanism with at least $2K$ steps. We also deduce that the memory length required for an accurate closure model decreases as a function of the interaction strength under the assumption that the interaction strength exhibits a power law that decays as a function of the hop distance. Supporting numerical demonstrations on two examples, a heterogeneous Kuramoto oscillator model and a power system, suggest that the proposed GNN architecture can predict the coarse-grained dynamics under fixed and time-varying graph topologies.
- [5] arXiv:2405.09542 (cross-list from cs.ET) [pdf, ps, html, other]
-
Title: Hybrid Magnonic Reservoir ComputingSubjects: Emerging Technologies (cs.ET); Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG); Applied Physics (physics.app-ph)
Magnonic systems have been a major area of research interest due to their potential benefits in speed and lower power consumption compared to traditional computing. One particular area that they may be of advantage is as Physical Reservoir Computers in machine learning models. In this work, we build on an established design for using an Auto-Oscillation Ring as a reservoir computer by introducing a simple neural network midstream and introduce an additional design using a spin wave guide with a scattering regime for processing data with different types of inputs. We simulate these designs on the new micro magnetic simulation software, this http URL, and show that the designs are capable of performing on various real world data sets comparably or better than traditional dense neural networks.
Cross submissions for Thursday, 16 May 2024 (showing 4 of 4 entries )
- [6] arXiv:2311.09418 (replaced) [pdf, ps, html, other]
-
Title: Unlearning regularization for Boltzmann MachinesComments: 18 pages, 13 figuresSubjects: Disordered Systems and Neural Networks (cond-mat.dis-nn)
Boltzmann Machines (BMs) are graphical models with interconnected binary units, employed for the unsupervised modeling of data distributions. When trained on real data, BMs show the tendency to behave like critical systems, displaying a high susceptibility of the model under a small rescaling of the inferred parameters. This behaviour is not convenient for the purpose of generating data, because it slows down the sampling process, and induces the model to overfit the training-data. In this study, we introduce a regularization method for BMs to improve the robustness of the model under rescaling of the parameters. The new technique shares formal similarities with the unlearning algorithm, an iterative procedure used to improve memory associativity in Hopfield-like neural networks. We test our unlearning regularization on synthetic data generated by two simple models, the Curie-Weiss ferromagnetic model and the Sherrington-Kirkpatrick spin glass model. We show that it outperforms $L_p$-norm schemes and discuss the role of parameter initialization. Eventually, the method is applied to learn the activity of real neuronal cells, confirming its efficacy at shifting the inferred model away from criticality and coming out as a powerful candidate for actual scientific implementations.
- [7] arXiv:2404.08556 (replaced) [pdf, ps, html, other]
-
Title: Quantum entropy couples matter with geometryComments: (32 pages, 1 figure)Subjects: Disordered Systems and Neural Networks (cond-mat.dis-nn); Statistical Mechanics (cond-mat.stat-mech); General Relativity and Quantum Cosmology (gr-qc); Mathematical Physics (math-ph); Quantum Physics (quant-ph)
We propose a theory for coupling matter fields with discrete geometry on higher-order networks, i.e. cell complexes. The key idea of the approach is to associate to a higher-order network the quantum entropy of its metric. Specifically we propose an action having two contributions. The first contribution is proportional to the logarithm of the volume associated to the higher-order network by the metric. In the vacuum this contribution determines the entropy of the geometry. The second contribution is the quantum relative entropy between the metric of the higher-order network and the metric induced by the matter and gauge fields. The induced metric is defined in terms of the topological spinors and the discrete Dirac operators. The topological spinors, defined on nodes, edges and higher-dimensional cells, encode for the matter fields. The discrete Dirac operators act on topological spinors, and depend on the metric of the higher-order network as well as on the gauge fields via a discrete version of the minimal substitution. We derive the coupled dynamical equations for the metric, the matter and the gauge fields, providing an information theory principle to obtain the field theory equations in discrete curved space.
- [8] arXiv:2402.03044 (replaced) [pdf, ps, html, other]
-
Title: Elementary processes in dilatational plasticity of glassesComments: Slightly revised presentation, no change in content, fixed bibliographical listJournal-ref: Physical Review Research 6, 023167 (2024)Subjects: Soft Condensed Matter (cond-mat.soft); Disordered Systems and Neural Networks (cond-mat.dis-nn); Materials Science (cond-mat.mtrl-sci); Statistical Mechanics (cond-mat.stat-mech)
Materials typically fail under complex stress states, essentially involving dilatational (volumetric) components that eventually lead to material decohesion/separation. It is therefore important to understand dilatational irreversible deformation -- i.e., dilatational plasticity -- en route to failure. In the context of glasses, much focus has been given to shear (volume-preserving) plasticity, both in terms of the stress states considered and the corresponding material response. Here, using a recently-developed methodology and extensive computer simulations, we shed basic light on the elementary processes mediating dilatational plasticity in glasses. We show that plastic instabilities, corresponding to singularities of the glass Hessian, generically feature both dilatational and shear irreversible strain components. The relative magnitude and statistics of the strain components depend both on the symmetry of the driving stress (e.g., shear vs.~hydrostatic tension) and on the cohesive (attractive) part of the interatomic interaction. We further show that the tensorial shear component of the plastic strain is generally non-planar and also extract the characteristic volume of plastic instabilities. Elucidating the fundamental properties of the elementary micro-mechanical building blocks of plasticity in glasses sets the stage for addressing larger-scale, collective phenomena in dilatational plasticity such as topological changes in the form of cavitation and ductile-to-brittle transitions. As a first step in this direction, we show that the elastic moduli markedly soften during dilatational plastic deformation approaching cavitation.
- [9] arXiv:2405.07878 (replaced) [pdf, ps, html, other]
-
Title: Effective medium properties of stealthy hyperuniform photonic structures using multiscale physics-informed neural networksSubjects: Optics (physics.optics); Disordered Systems and Neural Networks (cond-mat.dis-nn); Applied Physics (physics.app-ph)
In this article, we employ multiscale physics-informed neural networks (MscalePINNs) for the inverse retrieval of the effective permittivity and homogenization of finite-size photonic media with stealthy hyperuniform (SHU) disordered geometries. Specifically, we show that MscalePINNs are capable of capturing the fast spatial variations of complex fields scattered by arrays of dielectric nanocylinders arranged according to isotropic SHU point patterns, thus enabling a systematic methodology to inverse retrieve their effective dielectric profiles. Our approach extends the recently developed high-frequency homogenization theory of hyperuniform media and retrieves more general permittivity profiles for applications-relevant finite-size SHU systems, unveiling unique features related to their isotropic nature. In particular, we demonstrate the existence of a transparency region beyond the long-wavelength approximation, enabling effective and isotropic homogenization even without disorder-averaging, in contrast to the case of uncorrelated Poisson random patterns. We believe that the multiscale network approach introduced here enables the efficient inverse design of general effective media and finite-size metamaterials with isotropic electromagnetic responses beyond the limitations of traditional homogenization theories.