Divergence in gene regulation is hypothesized to underlie much of phenotypic evolution, but the role of natural selection in shaping the molecular phenotype of gene expression continues to be debated. To resolve the mode of gene expression, evolution requires accessible theoretical predictions for the effect of selection over long timescales. Evolutionary quantitative genetic models of phenotypic evolution can provide such predictions, yet those predictions depend on the underlying hypotheses about the distributions of mutational and selective effects that are notoriously difficult to disentangle. Here, we draw on diverse genomic data sets including expression profiles of natural genetic variation and mutation accumulation lines, empirical estimates of genomic mutation rates, and inferences of genetic architecture to differentiate contrasting hypotheses for the roles of stabilizing selection and mutation in shaping natural expression variation. Our analysis suggests that gene expression evolves in a domain of phenotype space well fit by the House-of-Cards (HC) model. Although the strength of selection inferred is sensitive to the number of loci controlling gene expression, the model is not. The consistency of these results across evolutionary time from budding yeast through fruit fly implies that this model is general and that mutational effects on gene expression are relatively large. Empirical estimates of the genetic architecture of gene expression traits imply that selection provides modest constraints on gene expression levels for most genes, but that the potential for regulatory evolution is high. Our prediction using data from laboratory environments should encourage the collection of additional data sets allowing for more nuanced parameterizations of HC models for gene expression.