Adding an Immunotherapy Drug, MEDI4736 (Durvalumab), to the Usual Chemotherapy Treatment (Paclitaxel, Cyclophosphamide, and Doxorubicin) for Stage II-III Breast Cancer

Inclusion Criteria: * STEP 1: REGISTRATION (SCREENING): Participants must have histologically confirmed estrogen receptor (ER) positive and/or progesterone receptor (PR) positive (hormone receptor positive) and HER2 negative breast cancer, as per American Society of Clinical Oncology (ASCO) College of American Pathologists (CAP) guidelines * NOTE: Participants with HER2 positive disease by ASCO CAP guidelines are ineligible. HER2 negative and HER2 low or equivocal cases as per ASCO CAP guidelines that do not receive HER2 targeted therapy are eligible * STEP 1: REGISTRATION (SCREENING): Participants must have clinical stage II or III breast cancer * NOTE: Participants with inflammatory breast cancer are eligible * NOTE: Participants with occult (i.e. undetectable) primary breast cancer with axillary nodal involvement are not eligible, as MammaPrint testing has not been validated on tissue obtained from an axillary lymph node * STEP 1: REGISTRATION (SCREENING): Participants must not have metastatic disease (i.e., must be clinically M0 or Mx) Systemic staging studies with imaging should follow routine practice as per National Comprehensive Cancer Network (NCCN) and ASCO guidelines * STEP 1: REGISTRATION (SCREENING): Participants must not have locally recurrent breast cancer * STEP 1: REGISTRATION (SCREENING): Participants with multifocal disease in the same breast or synchronous bilateral primary tumors are eligible, however, all tumors that are biopsied must be hormone receptor positive and HER2 negative per ASCO CAP guidelines and at least one of the tumors must be MammaPrint High-2. MammaPrint can be performed sequentially on biopsies as it is sufficient to have MammaPrint High 2 status on at least one of the lesions * NOTE: Biopsy of multiple lesions in the same breast is not required if the clinical presentation is consistent with a single disease process that is multifocal in nature. However, if there is clinical suspicion of two distinct primary breast malignancies, additional biopsies should be pursued * STEP 1: REGISTRATION (SCREENING): Participants must have either adequate tissue available to submit on-study or a prior known MammaPrint Index Score that is MP2 status * Submitting tissue for on-study MammaPrint testing: * Participants must have a minimum of ten, unstained formalin-fixed paraffin-embedded (FFPE) slides (4-5 micron thickness) available from initial tumor biopsy for MammaPrint assessment * NOTE: Participants must agree to have this tissue submitted to Agendia for MammaPrint Index Scoring and to have subsequent results disclosed to Southwest Oncology Group (SWOG) Cancer Research Network OR * Submitting prior known MammaPrint Index Score: * If a MammaPrint Index Score report from within the last 12 weeks is already known and is MP2 status, the participant must be registered to Step 2 immediately following Step 1 registration provided they meet all other criteria. MP2 status is defined as a MammaPrint Index score between negative 1.0 and negative 0.57 (-1.0 to -0.57, including negative 1.0 and negative 0.57) tested from initial tumor biopsy * NOTE: Participants must agree to have their commercial MammaPrint Index Score disclosed to SWOG Cancer Research Network * NOTE: Participants with prior known MammaPrint result that is not MP2 status should not be enrolled to either step of this study * NOTE: Participants enrolling with known MP2 status (i.e. MP already obtained as routine care) must only sign the treatment informed consent form. Screening consent is not required when MP2 status is known prior to study enrollment * STEP 1: REGISTRATION (SCREENING): Participants must not have received any prior treatment for their current breast cancer, including chemotherapy, immunotherapy, biologic or hormonal therapy, and must be candidates for doxorubicin, paclitaxel, and durvalumab therapy * STEP 1: REGISTRATION (SCREENING): Participants must be \>= 18 years old at the time of registration * STEP 1: REGISTRATION (SCREENING): Participants must have body weight \> 30 kg * STEP 1: REGISTRATION (SCREENING): Participants must have Zubrod Performance Status of 0-2 * STEP 1: REGISTRATION (SCREENING): Participants with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * STEP 1: REGISTRATION (SCREENING): Participant must not have medical contraindications to receiving immunotherapy, including history of non-infectious pneumonitis that required steroids or active autoimmune disease that has required systemic treatment with disease modifying agents, corticosteroids or immunosuppressive drugs in the past two years. Replacement therapy (e.g. thyroxine for pre-existing hypothyroidism, insulin for type I diabetes mellitus, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment. Intra-articular steroid injections are allowed * STEP 1: REGISTRATION (SCREENING): NOTE: As a part of the Oncology Patient Enrollment Network (OPEN) registration process the treating institution's identity is provided in order to ensure that the current (within 365 days) date of institutional review board approval for this study has been entered in the system * Participants must be informed of the investigational nature of this study and must sign and give informed consent in accordance with institutional and federal guidelines * For participants with impaired decision-making capabilities, legally authorized representatives may sign and give informed consent on behalf of study participants in accordance with applicable federal, local, and Central Institutional Review Board (CIRB) regulations * STEP 2: RANDOMIZATION: Participants must have met all eligibility criteria for Step 1 Registration * STEP 2: RANDOMIZATION: Particip