We investigated how emotional responses reflected in autonomic nervous system activities and facial muscles activities are related to learning in decision-making. Based on the conventional Q-learning model, we constructed novel learning models that incorporate the trial-to-trial variability in the physiological responses. In our models, the variables reflecting the physiological activities can modulate two important parameters of the model: (1) the learning rate, which determines the degree of update in response to the current choice outcome, and (2) the reward value, which quantifies the valence of the current outcome. We applied the models to the data from two types of decision-making task; one used emotional pictures as decision outcomes, and another used monetary reward. The valence of the outcomes was stochastically contingent on participants' choices. We demonstrated that proposed models that incorporated physiological measures including skin conductance, corrugator muscle activity and orbicular muscle activity, improved the prediction of the model, mainly for the emotional picture task. Our results suggest that some emotional responses are related to the subsequent choice behavior.