This paper provides a comprehensive study of the nonmonotone forward-backward splitting (FBS) method for solving a class of nonsmooth composite problems in Hilbert spaces. The objective function is the sum of a Fr\'echet differentiable (not necessarily convex) function and a proper lower semicontinuous convex (not necessarily smooth) function. These problems appear, for example, frequently in the context of optimal control of nonlinear partial differential equations (PDEs) with nonsmooth sparsity promoting cost functionals. We discuss the convergence and complexity of FBS equipped with the nonmonotone linesearch under different conditions. In particular, R-linear convergence will be derived under quadratic growth-type conditions. We also investigate the applicability of the algorithm to problems governed by PDEs. Numerical experiments are also given that justify our theoretical findings.
