Accumulating evidence reveals that adipose tissue is an immunologically active organ that exerts multiple impacts on the regulation of systemic energy metabolism. Adipose tissue immunity is modulated by the interactions between adipocytes and various immune cells. Nevertheless, the underlying mechanisms that control inter-cellular interactions between adipocytes and immune cells in adipose tissue have not been thoroughly elucidated. Recently, it has been demonstrated that adipocytes utilize lipid metabolites as a key mediator to initiate and mediate diverse adipose tissue immune responses. Adipocytes present lipid antigens and secrete lipid metabolites to determine adipose immune tones. In addition, the interactions between adipocytes and adipose immune cells are engaged in the control of adipocyte fate and functions upon metabolic stimuli. In this review, we discuss an integrated view of how adipocytes communicate with adipose immune cells using lipid metabolites. Also, we briefly discuss the newly discovered roles of adipose stem cells in the regulation of adipose tissue immunity.