A 100% ER positivity is not required for an endocrine therapy response. Furthermore, while estrogen typically promotes the progression of hormone-dependent breast cancer via the activation of estrogen receptor (ER)-α, estrogen-induced tumor suppression in ER+ breast cancer has been clinically observed. With the success in establishing estrogen-stimulated (SC31) and estrogen-suppressed (GS3) patient-derived xenograft (PDX) models, single-cell RNA sequencing analysis was performed to determine the impact of estrogen on ESR1+ and ESR1- tumor cells. We found that 17β-estradiol (E2)-induced suppression of GS3 transpired through wild-type and unamplified ERα. E2 upregulated the expression of estrogen-dependent genes in both SC31 and GS3; however, E2 induced cell cycle advance in SC31, while it resulted in cell cycle arrest in GS3. Importantly, these gene expression changes occurred in both ESR1+ and ESR1- cells within the same breast tumors, demonstrating for the first time a differential effect of estrogen on ESR1- cells. E2 also upregulated a tumor-suppressor gene, IL-24, in GS3. The apoptosis gene set was upregulated and the G2M checkpoint gene set was downregulated in most IL-24+ cells after E2 treatment. In summary, estrogen affected pathologically defined ER+ tumors differently, influencing both ESR1+ and ESR1- cells. Our results also suggest IL-24 to be a potential marker of estrogen-suppressed tumors.