While the accurate description of redox reactions remains a challenge for first-principles calculations, it has been shown that extended Hubbard functionals (DFT+U+V) can provide a reliable approach, mitigating self-interaction errors, in materials with strongly localized d or f electrons. Here, we first show that DFT+U+V molecular dynamics is capable of following the adiabatic evolution of oxidation states over time, using representative Li-ion cathode materials. In turn, this allows to develop redox-aware machine-learning potentials. We show that considering atoms with different oxidation states (as accurately predicted by DFT+U+V) as distinct species in the training leads to potentials that are able to identify the correct ground state and pattern of oxidation states for redox elements present. This can be achieved, e.g., through a systematic combinatorial search for the lowest-energy configuration or with stochastic methods. This brings the advantages of machine-learning potentials to key technological applications (e.g., rechargeable batteries), which require an accurate description of the evolution of redox states.