It is commonly assumed that humans learn generative or discriminative representations of the sensory input depending on task context (e.g. Hsu & Griffiths, 2010). Following our earlier findings (Orban et al., 2008), we propose that humans always form generative models based on the statistics of the input. To test this proposal, we investigated whether a learned internal model of the visual input would automatically incorporate task-irrelevant dimensions, and whether a generative model is formed even when the task requires only a simpler, discriminative representation. Participants (N=30) were presented with circle ensembles of varying mean size and standard deviation (SD). Their task was to estimate one of these parameters throughout the experiment, making the other dimension task-irrelevant. Unbeknown to the participants, the input formed two implicit categories across trials, one with small means and large SDs, and the second with large means and small SDs. Participants showed the same significant regression to the mean bias in either dimension both during the estimation task and after a categorization along the task-irrelevant dimension. Thus, even in a restricted or discriminative context, humans implicitly form a generative model of the distribution of the data, which model automatically influences their subsequent decisions.