Experiments are increasingly applied as a data generation method in conceptual modeling research. Conceptual models, modeling languages and, in particular, their graphical notations are subject to experimental studies. The present comprehensive survey examines the current state of experimentation in conceptual modeling research. Based on an exhaustive sampling of publications, the study identifies and analyzes 98 publications reporting on experiments in conceptual modeling research published between 2005 and 2018 to develop an organizing overview along essential methodological dimensions. Findings reveal a remarkable variety in theoretical foundations, research designs and experiment procedures, and suggest to rethink hypotheses development and to further develop methodical standards for experimentation in conceptual modeling research.