Hippocampus is crucial for episodic-like memory formation. The dynamics of hippocampal neuronal ensembles during learning has not been well understood. To investigate the hippocampal circuit property during learning, we designed a new spatial learning task and performed multiunit recording from CA1 neurons. Analyses for hippocampal place cells revealed that hippocampus captured not only the structure of environments, but also the structure of behavior task. Further analyses for hippocampal neuronal replays, the neuronal ensemble reactivations followed by sharp-wave ripples, showed rate increases in the frequency of replays with dynamic changes in replay patterns, reflecting the newly learned pathways. Real-time neuro-feedback experiments confirmed that such prioritized experience replay supports efficient learning. These results indicate that hippocampal neurons represent the self-state in the environment with multiple axes, enabling the hippocampus to flexibly emit experience replays to support the reinforcement of a specific behavior pattern.