In our work, we examine, for the first time, the possibility of fast and efficient source localization directly from the uvobservations, omitting the recovering of the dirty or clean images. We propose a deep neural network-based framework that takes as its input a low-dimensional vector of sampled uvdata and outputs source positions on the sky. We investigated a representation of the complex-valued input uv-data via the real and imaginary and the magnitude and phase components. We provided a comparison of the efficiency of the proposed framework with the traditional source localization pipeline based on the state-of-the-art Python Blob Detection and Source Finder (PyBDSF) method. The investigation was performed on a data set of 9164 sky models simulated using the Common Astronomy Software Applications (CASA) tool for the Atacama Large Millimeter Array (ALMA) Cycle 5.3 antenna configuration. We investigated two scenarios: (i) noise-free as an ideal case and (ii) sky simulations including noise representative of typical extra-galactic millimeter observations. In the noise-free case, the proposed localization framework demonstrates the same high performance as the state-of-the-art PyBDSF method. For noisy data, however, our new method demonstrates significantly better performance, achieving a completeness level that is three times higher for sources with uniform signal-to-noise (S/N) ratios between 1 and 10, and a high increase in completeness in the low S/N regime. Furthermore, the execution time of the proposed framework is significantly reduced (by factors about 30) as compared to traditional methods that include image reconstructions from the uv-plane and subsequent source detections.