Memory effects are ubiquitous in small-scale systems. They emerge from interactions between accessible and inaccessible degrees of freedom and give rise to evolution equations that are non-local in time. If the characteristic time scales of accessible and inaccessible degrees of freedom are sharply separated, locality can be restored through the standard Markov approximation. Here, we show how this approach can be rigorously extended to a precisely defined weak-memory regime, where the relevant time scales can be of comparable order of magnitude. We derive explicit bounds on the error of the local approximation and a convergent perturbation scheme for its systematic construction. Being applicable to any non-local time evolution equation that is autonomous and linear in the variables of interest, our theory provides a unifying framework for the systematic description of weak but significant memory effects.