PbCuTe$_2$O$_6$ is considered as one of the rare candidate materials for a three-dimensional quantum spin liquid (QSL). This assessment was based on the results of various magnetic experiments, performed mainly on polycrystalline material. More recent measurements on single crystals revealed an even more exotic behavior, yielding ferroelectric order below $T_{\text{FE}}\approx 1\,\text{K}$, accompanied by distinct lattice distortions, and a somewhat modified magnetic response which is still consistent with a QSL. Here we report on low-temperature measurements of various thermodynamic, magnetic and dielectric properties of single crystalline PbCuTe$_2$O$_6$ in magnetic fields $B\leq 14.5\,\text{T}$. The combination of these various probes allows us to construct a detailed $B$-$T$ phase diagram including a ferroelectric phase for $B \leq$ $8\,\text{T}$ and a $B$-induced magnetic phase at $B \geq$ $11\,\text{T}$. These phases are preceded by or coincide with a structural transition from a cubic high-temperature phase into a distorted non-cubic low-temperature state. The phase diagram discloses two quantum critical points (QCPs) in the accessible field range, a ferroelectric QCP at $B_{c1}$ = $7.9\,\text{T}$ and a magnetic QCP at $B_{c2}$ = $11\,\text{T}$. Field-induced lattice distortions, observed in the state at $T>$ $1\,\text{K}$ and which are assigned to the effect of spin-orbit interaction of the Cu$^{2+}$-ions, are considered as the key mechanism by which the magnetic field couples to the dielectric degrees of freedom in this material.
