Neural plasticity occurs in learning and memory. Coordinated plasticity at glutamatergic and GABAergic neurons during memory formation remains elusive, which we investigate in a mouse model of associative learning by cellular imaging and electrophysiology. Paired odor and whisker stimulations lead to whisker-induced olfaction response. In mice that express this cross-modal memory, the neurons in the piriform cortex are recruited to encode newly acquired whisker signal alongside innate odor signal, and their response patterns to these associated signals are different. There are emerged synaptic innervations from barrel cortical neurons to piriform cortical neurons from these mice. These results indicate the recruitment of associative memory cells in the piriform cortex after associative memory. In terms of the structural and functional plasticity at these associative memory cells in the piriform cortex, glutamatergic neurons and synapses are upregulated, GABAergic neurons and synapses are downregulated as well as their mutual innervations are refined in the coordinated manner. Therefore, the associated activations of sensory cortices triggered by their input signals induce the formation of their mutual synapse innervations, the recruitment of associative memory cells and the coordinated plasticity between the GABAergic and glutamatergic neurons, which work for associative memory cells to encode cross-modal associated signals in their integration, associative storage and distinguishable retrieval.