Surface-states of topological insulators are assumed to be robust against non-magnetic defects in the crystal. However, recent theoretical models and experiments indicate that even non-magnetic defects can perturb these states. Our first-principles calculations demonstrate that the presence of Se vacancies in Bi$_2$Se$_3$, has a greater impact than a mere n-doping of the structure, which would just shift the Fermi level relative to the Dirac point. We observe the emergence of a non-linear band pinned near the Fermi level, while the Dirac cone shifts deeper into the valence band. We attribute these features in the bandstructure to the interaction between the surface and defect states, with the resulting hybridization between these states itself depending on the position and symmetry of the Se vacancy relative to the surfaces. Our results bring us a step closer to understanding the exotic physics emerging from defects in Bi$_2$Se$_3$ that remained unexplored in prior studies.