Speaking involves the translation of abstract thought into movements of the lips, face, and tongue to produce sound. A large body of research in both psycholinguistics and phonetics has attempted to answer how these movements are planned, essentially probing the relationship between linguistic thought and speech motor production. In many dominant models of speech and word production, the link between these two systems is thought to be the syllable, which serves as both the output of the language planning system and the input of the speech motor control system. Here, I will present a series of experiments using a novel assessment tool—sensorimotor adaptation to altered auditory feedback—that question this consensus. By examining 1) how sensorimotor adaptation of a trained word or syllable transfers to other words and syllables and 2) how different linguistic structures enable speakers to learn opposing auditory perturbations applied to the same vowel in different contexts, the results from these studies suggest a rich link between the linguistic and speech motor systems, including not only syllables but also segments, words, and abstract prosodic structure.