As children learn their mother tongues, they make systematic errors. For example, English-speaking children regularly say "mouses" rather than "mice." Because children's errors aren't explicitly corrected, it has been argued that a child could never learn to make the transition to adult language based on the available evidence, and thus that learning even simple aspects of grammar is logically impossible. Here, we examine the effect of prediction-error on plural word learning. Prediction and prediction-error are important and uncontroversial components of animal learning theory, but are overwhelming absent from analyses of language learning. We present a model of the task facing children learning plurals that generates a surprising prediction: that at an appropriate point in learning, exposure to regular plurals alone will cause children's tendency to over-regularize irregular plurals to decrease. Interestingly, the model predicts that the same exposure should have an opposite effect earlier on in learning. Consistent with this, we found that while memory testing for regular plurals increased over-regularization in four-year-olds, it significantly decreased irregular plural over-regularization in six-year-olds. Our modeling and results suggest that children's over-regularization errors arise, and resolve themselves, as a natural consequence of the distribution of error in the linguistic environment, and that far from presenting a logical puzzle for learning, they are inevitable consequences of it. Error-driven learning mechanisms are supported both by animal and neurobiological models, and appear to be a key mechanism through which people acquire information about their environment. The idea that the structure and semantics of language are innate hinges on arguments that language is otherwise unlearnable. These arguments ignore modern learning theory, and the distribution of error in learning. Incorporating a more substantial view of learning into analyses of language and its development can only increase our understanding of them.