We explored the interaction between perceptual learning and statistical learning, two domains of sensory learning that are traditionally investigated separately. Using a standard perceptual learning protocol, we trained observers to improve their sensitivity to orientation of Gabor patches while differentially manipulating task irrelevant context of the training, such as the background color of the training scenes. Overall, we found that irrelevant context not only strongly influenced observed perceptual learning performance, but it also induced highly specific effects in the post-training test determined by the statistical structure of the context modulation. Our results suggest that the task irrelevant statistical structure present in perceptual tasks is automatically and implicitly built in the developing internal representation during learning. Thus perceptual and statistical learning processes are strongly related and create an integrated and complex internal representation even in the simplest perceptual learning tasks.