Introduction Sympathovagal balance measured by heart rate variability is a core component of psychophysiological research. Through the close link of physiological and psychological aspects, often a reduced heart rate variability is associated with impaired cognitive function. A better understanding of the associations between cognitive and cardiovascular dysfunctions is necessary to prevent the manifestation of diseases. Therefore, this study investigated phasic heart rate variability using rest, anticipatory, stress, and recovery periods and the association with high and low cognitive performance in a generally healthy population setting. Methods 114 healthy individuals (40 males, 74 females) aged 20 to 70 participated in the cross-sectional study. The heart rate variability based on standard deviation of NN intervals (SDNN), and the root means square of successive differences (RMSSD), low frequency (LF), high frequency (HF) and LF/HF ratio and its association with high and low cognitive performance measured by the California Verbal Learning Task II were examined. Results The results of this study indicate that the paradigm was successful in producing stress and showed a significant association between phasic heart rate variability (SDNN) and verbal episodic memory performance, irrespective of age and sex. Discussion The results of this study suggest that a reduced heart rate variability is associated with reduced cognitive function regardless of age and sex and seem to be an early indicator of sympathovagal disbalance. Conclusion This leads to the conclusion that differences between high and low cognitive performance might show differences in heart rate variability at an early stage, where no diseases are yet manifest.