Resolving short-range structural distortions that persist within high-symmetry crystal structures remains a central challenge in materials chemistry. These distortions are dynamic, with correlations extending over nanometer length scales and picosecond timescales, making their atomistic modeling computationally demanding. Here, we address this challenge by modeling the parent compound of ruthenium-pnictide superconductors, RuP, in which such distortions and correlations are prominent. We fine-tune an atomistic foundation model (AFM) on ab initio molecular dynamics datasets and use the resulting interatomic potential to perform machine-learning molecular dynamics simulations of RuP. Analysis of the temperature dependence of lattice parameters and structure factors reveals a two-step phase transition, including an isosymmetric monoclinic-to-monoclinic transition near 180 K, followed by a monoclinic-to-orthorhombic transition near 330 K. Local order metrics and correlation functions further show that short-range monoclinic distortions persist above 330 K within the average high-symmetry phase, providing direct evidence of local symmetry breaking. Electronic-structure calculations reproduce the experimentally observed evolution from an insulating low-temperature state to an intermediate pseudogap-like regime and finally to a metallic high-temperature state, connecting the two-stage electronic transition to the underlying structural transformation. Phonon dispersion analysis reveals the emergence of imaginary phonon modes above the second transition, consistent with the dynamical instability of the average orthorhombic structure and the persistence of local lattice distortions. These findings provide an atomistic picture of local symmetry breaking in RuP and demonstrate that fine-tuned AFMs provide a practical approach for quantifying short-range distortions and correlated fluctuations in crystals with complex phase behavior.