Trastuzumab Deruxtecan Alone or in Combination With Anastrozole for the Treatment of Early Stage HER2 Low, Hormone Receptor Positive Breast Cancer

Inclusion Criteria: * Previously untreated operable invasive carcinoma of the breast greater than 2.0 cm (cT2) in size based on physical exam or imaging. Patients with clinical node negative disease or clinical node (cN1/cN2) positive are allowed provided they are deemed to have operable disease at study entry * Participants with clinically involved lymph nodes should not have radiological evidence of distant disease per standard of care staging prior to patient informed consent form (PICF) signature * In the United States * Tumor is HER2-low by immunohistochemistry (IHC), defined as 1+ or 2+, confirmed by central testing (central testing results not required for enrollment, unless no local results available). If HER2 is 2+ by IHC, fluorescence in situ hybridization (FISH) must be performed (per standard of care) and the FISH result must be HER2 non-amplified per 2018 American Society of Clinical Oncology College of American Pathologists (ASCO CAP) guidelines * Tumor is HR positive (HR+) per ASCO CAP guidelines with known estrogen and progesterone receptor status, locally defined * Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1 * Normal cardiac function (left ventricular ejection fraction \[LVEF\] \>= 50%) based on echocardiogram (ECHO) or multigated acquisition (MUGA) scan within 28 days before randomization/enrollment * Platelet count \>= 100 000/mm\^3 (Platelet transfusion is not allowed within 1 week prior to screening assessment) (within 14 days before randomization/enrollment) * Hemoglobin \>= 9.0 g/dL (red blood cell transfusion is not allowed within 1 week prior to screening assessment) (within 14 days before randomization/enrollment) * Absolute neutrophil count (ANC) \>=1500/mm\^3 (Granulocyte colony-stimulating factor (G-CSF) administration is not allowed within 1 week prior to screening assessment) (within 14 days before randomization/enrollment) * Creatinine clearance \>= 30 mL/min as calculated using the Cockcroft-Gault equation or serum creatinine =\< 1.5 x upper limit of normal (ULN) (within 14 days before randomization/enrollment) * Alanine aminotransferase (ALT), aspartate aminotransferase (AST) =\< 3 x ULN (within 14 days before randomization/enrollment) * Total bilirubin =\< 1.5 x ULN (within 14 days of randomization/enrollment). Participants with Gilbert's syndrome with a total bilirubin =\< 2.0 times ULN and direct bilirubin within normal limits are permitted * Serum albumin \>= 2.5 g/dL (within 14 days before randomization/enrollment) * International normalized ratio (INR)/prothrombin time (PT) and activated partial thromboplastin time (aPTT) =\< 1.5 x ULN (within 14 days before randomization/enrollment) * Has adequate treatment washout period before randomization/enrollment, defined as: * Major surgery \>= 4 weeks * Chloroquine/hydroxychloroquine \> 14 days * Negative pregnancy test (serum) for women of child bearing potential (CBP): * Women are considered of CBP unless: they have had 12 months of natural (spontaneous) amenorrhea with an appropriate clinical profile (i.e. age appropriate, history of vasomotor symptoms) or have had surgical bilateral oophorectomy (with or without hysterectomy), total hysterectomy, or tubal ligation at least six weeks prior to randomization. In the case of oophorectomy alone, only when the reproductive status of the woman has been confirmed by follow-up hormone level assessment she is considered not of CBP * Male and female participants of reproductive/childbearing potential must agree to use a highly effective form of contraception or avoid intercourse during and upon completion of the study and for at least 7 months for females and 4 months for males after the last dose of study drug. Highly effective contraception methods include: * Total abstinence (when this is in line with the preferred and usual lifestyle of the patient). Periodic abstinence (e.g. calendar, ovulation, symptothermal, post-ovulation methods) and withdrawal are not acceptable methods of contraception * Female sterilization (have had surgical bilateral oophorectomy with or without hysterectomy), total hysterectomy or tubal ligation at least 6 weeks before taking trial treatment. In case of oophorectomy alone, only when the reproductive status of the woman has been confirmed by follow up hormone level assessment * Male partner sterilization (at least 6 months prior to randomization). For female patients on the trial the vasectomized male partner should be the sole partner for that patient. If vasectomy of the male partner is the highly effective method of contraception chosen, the success of the vasectomy should be medically confirmed according to local practice * Placement of an intrauterine device (IUD) * Male participants must not freeze or donate sperm starting at screening and throughout the study period, and at least 4 months after the final study drug administration. Preservation of sperm should be considered prior to enrollment in this study * Female participants must not donate, or retrieve for their own use, ova from the time of screening and throughout the study treatment period, and for at least 7 months after the final study drug administration * Estradiol level must be in post-menopausal range per local lab interpretation prior to baseline biopsy * Postmenopausal status is defined as: * Patient underwent bilateral oophorectomy, or * Age \>= 60 years, or * Age \< 60 years and amenorrhea for 12 or more months (in the absence of chemotherapy, tamoxifen, toremifene or ovarian suppression) and follicle-stimulating hormone (FSH) and plasma estradiol are in the postmenopausal ranges per local normal ranges * Note: for women with therapy-induced amenorrhea, serial measurements of FSH and/or estradiol per local clinical guidelines are required for determination of postmenopausal status. All women who do not meet the criteria for postmenopausal status are considered premenopausal for the purpose of thi