Motivated by recent experiments that observed low-frequency second-order optical responses in doped striped superconductors, here we investigate the nonlinear electrodynamics of systems exhibiting a charge density wave (CDW) order parameter. Due to the Bragg scattering off the CDW order, an incoming spatially homogeneous electric field in addition to zero momentum current generates Umklapp currents that are modulated in space at momenta of the reciprocal CDW lattice. In particular, here we predict and microscopically evaluate the Umklapp shift current, a finite momentum analog of the regular shift current which represents the second-order optical process that downconverts homogeneous AC electric field into low-frequency, zero momentum current. Specifically, we evaluate real-time response functions within mean-field theory via the Keldysh technique and use the Peierls substitution to compute observables at finite momenta in lattice models. We find that systems with certain lattice symmetries (such as inversion symmetry), where the regular shift current is disallowed, may give rise to the Umklapp one. We apply our framework to investigate lattice symmetries in layered materials with helical-like stripes and show that both types of shift currents provide insight into the nature of intertwined phases of matter. Finally, we discuss the relation of our findings to recent experiments in striped superconductors.