Listeners are highly proficient at adapting to the speech of a novel talker. Various mechanisms may underlie such adaptation, and at differing time courses. These may include phonetic cue calibration (e.g., McMurray & Jongman, 2012), phonetic covariation (Chodroff & Wilson, 2018), or general auditory contrast, particularly at short timescales (e.g., Lotto & Kluender, 1998). Under the cue calibration account, listeners adapt by estimating a talker-specific average for each phonetic cue or dimension; under the cue covariation account, listeners adapt by exploiting consistencies in how the realization of speech sounds varies across talkers; under the auditory contrast account, adaptation is not talker-specific, but results instead from (partial) masking of spectral components shared by adjacent stimuli. To distinguish between these mechanisms, we investigate perceptual adaptation to talker-specific sibilant fricatives at short and long timescales. In rapid adaptation, our findings indicate a strong role of general auditory mechanisms; however, a longer period between talker-specific exposure and test could reveal secondary, phonetic-based learning mechanisms.