1. Introduction

The word “reward” is socially linked to happiness or “hedonic process”, but is defined in affective neuroscience researches as an object or event that generates approach behavior, produces learning of such behavior, and is an outcome of decision making (Schultz, 2007). Consumption of reward either primary (e.g. palatable food or drinks, mating, or drugs) or secondary (e.g. money) produce hedonic experience which itself initiates a process of associative learning to consolidate behaviors and related cues (Arias-Carrion and Poppel, 2007). Animal and human lesion studies revealed specific brain structures implicated in reward processing, including the orbitofrontal, medial prefrontal regions, amygdala, striatum, and dopaminergic midbrain. These regions are highly interconnected to each other and can be considered as an integrated network (O’Doherty, 2004, Wachter et al., 2009).

Integration of the reward into motor behavior occurs where reward-related neural signals meet circuits concerned with motor performance. The striatum receives inputs from various regions of the cerebral cortex, and parts of the thalamus. These excitatory glutamatergic inputs converge with dopamine inputs from the substantia nigra in the striatum. The output of the striatum influences other basal ganglia nuclei, which through direct and indirect pathways reach the thalamus. Finally, those projections go back to the frontal cortex including the primary motor cortex (M1). This anatomical organization provides a favorable substrate in the striatum for integrating dopaminergic reward signals with sensory cues and generating motor commands to motor areas (Wickens et al., 2003, Schultz, 2004, Ikemoto, 2007, Hikosaka et al., 2008). Accordingly, reward-related signals might induce changes in the excitability of M1 which may be an important brain region to be studied in relation to the reward processing.

The midbrain dopaminergic system may have an important role in both analysis of the informational content of reward and also in control of reward-related behavior as a part of the reward network through its connections to other brain regions responsible to reward processing in the brain (Schultz et al., 2000, Schultz, 2004, Ikemoto, 2007, Hikosaka et al., 2008). The former role is associated with the orbitofronatal, prefrontal, anterior cingulate cortices, hippocampus, striatum and amygdala, while the latter is related to the striatum, nucleas accumbans and dorsal anterior cingulate area (Rolls, 2000, Schultz, 2000, Schultz et al., 2000, O’Doherty et al., 2001, Gottfried et al., 2003, Kringelbach, 2005, Oya et al., 2005, Wise, 2005, Murray, 2007, Hikosaka et al., 2008, Kapogiannis et al., 2008). Animal and human studies showed that the dopamine neurons in the midbrain are activated transiently in response to reward-predicting or rewarding stimuli (Schultz et al., 1993, Schultz et al., 1997, Koepp et al., 1998, Schultz, 1998b, Schultz, 2001, Zald et al., 2004, Zink et al., 2004, Nakazato, 2005, Heien and Wightman, 2006, Schultz, 2007, Natori et al., 2009).

Transcranial magnetic stimulation is a very useful tool to study the physiology of the central nervous system in humans. The amplitude of motor evoked potential(MEP) can be taken as a measure of the cortico-spinal excitability (Ziemann et al., 1996b). Short-interval intracortical inhibition (SICI) refers to MEP inhibition induced by conditioning TMS pulse applied to M1 (Kujirai et al., 1993) and is used to mainly study the activity of GABA-A inhibitory cortical neurons (Ziemann, 2004). Short-latency afferent inhibition (SAI) refers to MEP inhibition induced by a conditioning afferent electrical pulse applied to the peripheral nerve (Tokimura et al., 2000) and is partly related to the activity of cholinergicM1 receptors (Di Lazzaro et al., 2000) and is diminished by activation of certain GABA-A neuronal circuits (Di Lazzaro et al., 2005a, Di Lazzaro et al., 2007). Previous studies showed the changes in M1 excitability in response to the reward prediction (Kapogiannis et al., 2008) and the urge to obtain a rewarding stimulus (Gupta and Aron, 2011), indicating that TMS measures can be used to address the reward-related M1 function.

However, there have been no researches which examined the modulatory effects of momentary reward itself on M1 excitability. To test this hypothesis, we investigated the excitatory and inhibitory system within human M1 by using transcranial magnetic stimulation (TMS) during the reward and sensorimotor control tasks.

2. Methods

2.4. Experimental task

(Fig. 1) To measure the changes in M1 function, we designed the experiment so that a cue (four yellow squares) appears on a screen attached to a computer and placed in front of the subject. Only one of those four yellow squares contains the target stimulus. Subjects were instructed to select one of the yellow squares by pressing its corresponding button (buttons, 1, 2, 3, or 4) by the dominant hand. Each experiment contained two similar tasks; reward task and sensorimotor control task. Each task was composed of a total of 135 trials, 54 trials of them contained the target stimulus, and the remaining 81 trials contained a non-target stimulus (white circuit). For the target trial, the target stimulus was always presented to the subjects, irrespective of the button that they selected. The total trial duration was randomized between 7–8 s. Each trial started by presenting the cue (four squares) for 1 s at the maximum. As soon as the subject selected one of them by pressing the button, the cue disappears. Trials with reaction time (RT) slower than 1 s were excluded from the analysis. Two seconds after the onset of the cue, the target/non-target stimuli were presented for 2 s duration as a feedback for the subject.

1. Download high-res image (211KB)
2. Download full-size image

Fig. 1. Experimental design in reward and sensorimotor control tasks. In the reward task, subjects received either reward target or non-target stimuli in a pseudorandom schedule as a feedback of the subjects’ response to the four squares cue. In the sensorimotor control task, non-reward target replaced the reward target. The duration of the cue presentation is equal to the RT of the subject in each individual trial. The trials’ duration was randomized between 7–8 s. Various TMS measures were done 1 s after the start of each stimulus which are presented for 2 s. (RT = reaction time, SOA = stimulus onset asynchrony from the start of one visual stimulus to that of another one, Sec = second, RT M1 = right primary motor cortex).

In the reward task, the target stimulus (reward target) was a picture of 100 Japanese yen coin which had a rewarding value as it represented an actual momentary money reward. In sensorimotor control task, the target stimulus (non-reward target) was a mauve circle containing asterisk sign (*), and this stimulus represents a mere right target selection without rewarding value, to control attention and other sensorimotor effects.

3. Results

The mean RT ± SD was 406 ± 54 ms and 388 ± 53 ms for reward and sensorimotor control tasks, without any statistically significant difference (F = 1.629, P = 0.224). The mean ± SD of rMT and aMT of the APB muscle were 51 ± 12%, and 42 ± 7% of maximum stimulator output. The mean ± SD of the intensities of SI_1mV and the conditioning pulse for SICI were 66 ± 12% and 40 ± 7% of maximal stimulator output. The percentage of delayed or inappropriate trials was 2.1% and 2.5% for reward and sensorimotor control tasks.

The MEP amplitude of both the APB and ADM muscles did not show any significant changes for target vs. non-target responses for both reward and sensorimotor control tasks (APB: 900 ± 77 vs. 939 ± 99 μV and 865 ± 83 vs. 919 ± 81 μV for reward and sensorimotor control tasks, and ADM: 654 ± 136 vs. 647 ± 103 μV and 697 ± 133 vs. 619 ± 122 μV for reward and sensorimotor control tasks) (Fig. 2, Fig. 3). Repeated measures ANOVA with Task (reward and sensorimotor control) and Response (target and non-target) was insignificant for Experiment, Response and Experiment × Response interaction in both muscles.

1. Download high-res image (151KB)
2. Download full-size image

Fig. 2. EMG traces of a representative subject. Single traces of EMG in one representative subject recorded from the non-dominant APB muscle were shown during reward task (top), and sensorimotor control task (bottom).

1. Download high-res image (157KB)
2. Download full-size image

Fig. 3. Effects of reward and sensorimotor control tasks on MEP in APB and ADM. Mean ± SEM for unconditioned MEP amplitudes in the APB and ADM muscles during target vs. non-target stimuli in reward and sensorimotor control tasks. In both tasks, there was no significant change in the unconditioned MEP amplitudes indicating no effect of reward target on MEP amplitude.

The conditioned MEP ratios for SICI were significantly smaller for the target responses in reward task, but not in sensorimotor control task (0.40 ± 0.05 vs. 0.53 ± 0.06 in control task and 0.50 ± 0.06 vs. 0.52 ± 0.05 in sensorimotor control task) (Fig. 2, Fig. 4). Repeated-measures ANOVA for SICI ratio with Experiment and Response as within subject variables was significant for Experiment × Response interaction (F = 7.922, P = 0.015). The main effect of Response was significant (F = 16.820, P = 0.001). Post hoc analysis for the effect of Response was significant in reward (P = 0.002) but insignificant in sensorimotor control tasks.

1. Download high-res image (136KB)
2. Download full-size image

Fig. 4. Effects of reward and sensorimotor control tasks on SICI in APB. Mean ± SEM for conditioned MEP amplitude ratios in the APB muscle for SICI during target vs. non-target stimuli. In reward task, there was a significant decrease in the conditioned MEP ratios induced by momentary reward target. In sensorimotor control task, there was no significant difference. (^∗∗P < 0.01).

For SAI, the conditioned MEP ratios were significantly larger for the target responses in reward task (0.49 ± 0.05 vs. 0.39 ± 0.04 in control task and 0.39 ± 0.04 vs. 0.41 ± 0.05 in sensorimotor control task) (Fig. 2, Fig. 5). Repeated-measures ANOVA for SAI ratio with Experiment and Response was significant for Experiment × Response interaction (F = 7.042, P = 0.02). Post hoc analysis for Response in each experiment revealed the significant effect for reward task (P = 0.024) but the insignificant effect for sensorimotor control task.

1. Download high-res image (137KB)
2. Download full-size image

Fig. 5. Effects of reward and sensorimotor control tasks on SAI in APB. Mean ± SEM for the conditioned MEP amplitude ratio in the APB muscle for SAI during target vs. non-target stimuli. In reward task, there was a significant increase in the conditioned MEP ratios induced by momentary reward target. In sensorimotor control task, there was no significant difference. (^∗P < 0.05).

7. Discussion

We found that the monetary reward task can modulate M1 excitability via inhibitory neural system within M1. There was significantly increased SICI and decreased SAI in response to the momentary reward. This change in M1 excitability was absent in the control study indicating that it can’t be explained by attention and other sensorimotor factors which are known to affect M1 excitability (Maunsell, 2004, Kotb et al., 2005). The general parameters used for measuring the M1 excitability (MEP) showed no significant change in response to both reward and control tasks.

Animal studies showed that dopamine neurons in the midbrain, in addition to its tonic activity, show phasic activation in response to momentary rewards and reward-predicting stimuli (Schultz et al., 1993, Schultz et al., 1997, Schultz, 1998b, Schultz, 2001, Nakazato, 2005, Heien and Wightman, 2006, Schultz, 2007, Natori et al., 2009). In humans, dopamine release in neural targets of the midbrain dopaminergic neurons, namely the striatum was detected in recent imaging studies in response to various primary and secondary rewarding stimuli (Koepp et al., 1998, Zald et al., 2004, Zink et al., 2004). Similar results were obtained in many fMRI studies (Breiter et al., 1997, Breiter and Rosen, 1999, Breiter et al., 2001, Knutson et al., 2001, O’Doherty et al., 2002, Volkow et al., 2002b, Kirsch et al., 2003, Tricomi et al., 2004, Knutson and Cooper, 2005).

Substantial evidences indicate that dopamine neurons of the primate ventral midbrain code reward prediction error which is the discrepancy between the probability of reward and its actual occurrence (Schultz et al., 1993, Schultz, 1998a, Waelti et al., 2001, Fiorillo et al., 2003) rather than the reward value itself. Accordingly, the phasic burst firing of dopamine neurons was found to be higher in response to unpredicted or under-predicted rewards (Schultz, 1998b, Schultz and Dickinson, 2000). This phasic activation of the midbrain dopaminergic neurons causes the rise in the dopamine concentration of the basal ganglia. In primate animal studies, the rise reaches its peak around 1 s after the onset of the reward-related stimulus, and starts to decline after 2 s, reaching the baseline concentration after around 4 s (Schultz, 1998a, Schultz, 2001, Roitman et al., 2004, Schultz, 2007). Taking this time course in consideration, we applied the TMS measures at the expected time of the peak dopamine concentration in the basal ganglia. In our study, since the reward magnitude and timing were held constant, the reward prediction error would have been related to the reward probability (P) and the actual outcome. Since the reward probability in our study was low, the activation of dopamine neurons might be substantial (Fiorillo et al., 2003).

The changes in SICI and SAI induced by momentary reward in our study were consistent with those induced by dopamine. Parkinson’s disease (PD) patients with the drug-off state (Ridding et al., 1995, Strafella et al., 2000, Bares et al., 2003) and cervical dystonia patients (Kanovsky et al., 2003) showed reduced SICI compared to normal subjects. Also, in patients with attention deficit hyperactivity disorder (ADHD), in which there is dysfunction in the dopamine reward pathway (Volkow et al., 2009), SICI was also reduced (Moll et al., 2000, Richter et al., 2007, Schneider et al., 2007). However, dopaminergic drugs significantly increase SICI in normal subjects (Ziemann et al., 1996a, Ziemann et al., 1997, Korchounov et al., 2007), and in PD patients (Ridding et al., 1995, Strafella et al., 2000, Lefaucheur et al., 2004, Bares et al., 2007). Moreover, methylphenidate which blocks dopamine reuptake into presynaptic nerve endings (Volkow et al., 2001, Volkow et al., 2002a), also increased SICI in ADHD patients (Moll et al., 2000, Buchmann et al., 2007). However, it is still possible that the change in SICI is related to the change in the aMT but not to the intracortical inhibition. Since it was not feasible to measure the aMT in an online way during the study, further studies would be necessary to clarify this point.

The striatum is centrally positioned in the functional network controlling motor and cognitive aspects of behavior (Graybiel et al., 1994, Middleton and Strick, 1997, Middleton and Strick, 2000). In addition to its role in reward processing, it is a well established recipient of M1 glutamatergic and midbrain dopaminergic inputs (Aosaki et al., 1994, Kawaguchi et al., 1995, Wickens et al., 2003, Calabresi et al., 2007). Many animal studies showed that the motor areasincluding the M1 are connected to the pallidal output neurons through the thalamus (Nambu et al., 1988, Tokuno et al., 1992, Kayahara and Nakano, 1996).

Human studies showed that the thalamus may play a role in controlling intracortical inhibition in M1, as SICI was defective in a patient with complex movement disorder who suffered thalamic ischemic lesion (Münchau et al., 2002). On the other side, an opposite effect has been shown in epilepticpatients treated with thalamic deep brain stimulation (DBS) (Molnar et al., 2006). Those thalamic projections to M1 are under tonic inhibitory control from the pallidal output of basal ganglia (Groenewegen, 2003, DeLong and Wichmann, 2007), which might explain the results of SICI in the present study.

Changes in SAI were found in many pharmacological and patient studies; SAI was found to be increased PD patients with off-medications (Di Lazzaro et al., 2004, Nardone et al., 2005) and was significantly decreased in patients with on-medications suggesting that the dopaminergic treatment reduces SAI (Sailer et al., 2003). Moreover, SAI was also found to be decreased in diseases that are thought to be related to abnormalities in basal ganglia–thalamo–cortical circuits as dystonia (Di Lazzaro et al., 2009) and Gilles de la Tourette syndrome (GTS) patients, (Orth et al., 2005, Orth, 2009). Accordingly, dopamine release in the striatum may affect SAI in M1 indirectly. Some studies showed that changes in SICI and SAI are inversely related (Di Lazzaro et al., 2005a, Di Lazzaro et al., 2005b, Di Lazzaro et al., 2007, Alle et al., 2009) and recently, a model of two distinct reciprocally connected subtypes of GABA inhibitory interneurons with convergent projections onto the corticospinal neurons, in which SICI is dominant over SAI, was suggested to explain this inverse relationship (Alle et al., 2009). Our results are further supported by this inverse relation.

Unconditioned MEP amplitude showed no significant changes in response to dopaminergic treatment in PD patients (Ridding et al., 1995, Dioszeghy et al., 1999) and in healthy subjects (Ziemann et al., 1996a, Ziemann et al., 1997), which are in agreement with the results of our study.

In addition to the above mentioned pathway, the effect of reward-related activity may affect the excitability of M1 through its connections to other secondary motor and non-motor brain regions which receive reward-related information, such as prefrontal, orbitofrontal, anterior cingulate, supplementary motor areas, amygdala, and nucleus accumbens (Svensson et al., 1995, Schultz et al., 2000, Gottfried et al., 2003, Williams et al., 2004).

We hereby conclude that the M1, as well as other frontal regions implicated in the reward processing, receives reward-related signals. Striatal dopamine may play an important role in reward-related motor learning (Wickens et al., 2003) and in induction of corticostriatal synaptic plasticity (Calabresi et al., 2007). In animal studies, dopamine either in M1 (Molina-Luna et al., 2009) and/or in the striatum (Centonze et al., 2003, Davis et al., 2007) is essential for motor learning. Also in humans, dopamine is important not only for the development of M1 plasticity (Ueki et al., 2006), but also for its enhancement (Nitsche et al., 2006, Lang et al., 2008, Rodrigues et al., 2008, Rizzo et al., 2009). The excitability changes in M1 induced by the momentary reward may be related to the reward-related motor activity at the cortical level or may reflect its occurrence at striatal level.