In order to improve the performance of optically multiplexed multicarrier systems with channel spacing equal to the symbol rate per carrier, we propose and systematically investigate an electronic signal processing technique to achieve near-interchannel crosstalk free and intersymbol-interference (ISI) free operation. We theoretically show that achieving perfect orthogonality between channels in these systems, together with ISI free operation as needed in generic communication systems, requires the shaping of the spectral profiles of not only the demultiplexing filter, but also the signal of each channel before demultiplexing. We develop a novel semianalytical method to quantitatively analyze the levels of residual crosstalk and ISI arising from nonideal system response in these systems. We show that by prefiltering the signal to ensure that the system impulse response before channel demultiplexing approaches the targeted condition, the residual crosstalk due to imperfect orthogonality can be significantly mitigated and the necessity for carrier phase control in single-quadrature format-based system can be relaxed. Further combining prefiltering and receiver-side postfiltering to adaptively trim the demultiplexing filter enhances the performance. The use of the combined digital signal processing (DSP) in coherent-detection quadrature phase-shifted keying (QPSK)-based optically multiplexed multicarrier system shows that this method outperforms conventional QPSK-based multicarrier system without DSP or with only receiver-side DSP, especially when the responses of the transmitter and the demultiplexing filter are not precisely designed and the sampling rate of the analogue-to-digital converter is not sufficiently high. In addition, the inclusion of ISI free operation, with this aspect similar to the reshaping method in conventional wavelength-division-multiplexing systems, allows the relaxation of the modulation bandwidth and chromatic dispersion compensation.
