What transpires in the mind of a speaker or her audience? We propose that when we communicate, we engage in a process of mutual prediction, exploiting a system of conventionalized cultural and experiential cues. This idea is grounded in the notion that cognition and learning are both fundamentally predictive - that the purpose of a cognitive system is to successfully predict events in the environment, while the purpose of learning is to minimize surprise so this can be achieved. Language is a central aspect of cognition, and framing it in similarly predictive terms allows us to resolve a number of curious linguistic puzzles. With this in mind, we draw upon evidence from computational linguistics, corpus analysis and a series of experiments to advance the idea that grammar is probabilistic rather than deterministic; understanding is predictive rather than absolute; and language, far from being innately specified, is learnable.