A Trial Evaluating the Efficacy of Ivonescimab in Metastatic Endocrine Refractory HR-positive HER2-negative or Triple Negative Invasive Lobular Carcinoma

Inclusion Criteria: * 18 years of age or older. * Historically confirmed invasive lobular cancer with negative E-cadherin staining by IHC. * Estrogen receptor (ER) positive (\>1%) or negative, progesterone receptor (PgR) positive or negative, and HER2-negative according to HER2 testing guidelines from the American Society of Clinical Oncology/College of American Pathologists. * Participants must be willing to undergo biopsy as required by the study if the tumor is safely accessible. * If ER+, participant must be endocrine refractory as per the treating oncologist assessment and has been exposed to at least one line of endocrine therapy prior to enrollment. * Participants who received prior chemotherapy, antibody-drug conjugates (ADCs), mTOR inhibitor and/or PI3K are eligible. o Participants should not have received more than 2 chemotherapeutic agents and/or ADCs in the metastatic setting. * Eastern Cooperative Oncology Group Performance status ≤ 1. * Participant has either measurable disease per response evaluation criteria in solid tumors (RECIST) v1.1 criteria OR at least one predominantly lytic bone lesion must be present. * Adequate Organ Function: * Hematology (no blood transfusions or growth factor therapy used within 7 days of the screening CBC): i. Absolute neutrophil count (ANC) ≥ 1.0 × 109/L ii. Platelet count ≥ 100 × 109/L iii. Hemoglobin ≥ 9.0 g/dL * Kidneys: * Creatinine clearance (CrCL) ≥ 60 mL/min using the Cockcroft-Gault formula or estimated glomerular filtration rate (eGFR) value ≥60 mL/min using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation (adjustment by BSA is not required for eGFR). CrCL or eGFR can be determined using the calculator from the National Kidney Foundation website (www.kidney.org). * Urine protein \< 2+ or 24 hour urine protein quantification \< 1.0 g * Liver: i. Serum total bilirubin (TBIL) ≤ 1.5 × upper limit of normal (ULN); For patients with liver metastases or confirmed/suspected Gilbert syndrome, TBIL ≤3 × ULN ii. Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) ≤ 2.5 × ULN; For participants with liver metastases, AST and ALT ≤ 5 × ULN * Coagulation: prothrombin time (PT) or international normalized ratio (INR) ≤ 1.5 × ULN, and partial prothrombin time (PTT) or activated partial thromboplastin time (aPTT) ≤ 1.5 × ULN (unless abnormalities are unrelated to coagulopathy or prophylactic coagulation). This applies only to patients who are not on therapeutic anti-coagulation. Participants receiving therapeutic anti-coagulation should be on a stable dose. * Participants with active hepatitis B are required to have stable or declining levels of hepatitis B DNA by polymerase chain reaction (PCR) on appropriate anti-viral therapy with acceptable tolerability for one month prior to enrolment on study. * All participants with active hepatitis C (hepatitis C virus \[HCV\] antibody positive with HCV RNA levels above the lower limit of detection) are excluded. * Participants with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression and must have stopped corticosteroids or be on physiologic corticosteroid replacement therapy (prednisone ≤ 10 mg daily or equivalent) * Participants with a prior or concurrent malignancy whose natural history or treatment does not interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial. * The effects of ivonescimab on the developing human fetus are unknown. For this reason, women of child-bearing potential (WOCBP) and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation. (Refer to Pregnancy Assessment Policy MD Anderson Institutional Policy # CLN1114). This includes all female patients, between the onset of menses (as early as 8 years of age) and 55 years unless the participant presents with an applicable exclusionary factor which may be one of the following: * Postmenopausal (no menses in greater than or equal to 12 consecutive months). * History of hysterectomy or bilateral salpingo-oophorectomy. * Ovarian failure (Follicle Stimulating Hormone and Estradiol in menopausal range, who have received Whole Pelvic Radiation Therapy). * History of bilateral tubal ligation or another surgical sterilization procedure. * Approved methods of birth control are as follows: Hormonal contraception (i.e. birth control pills, injection, implant, transdermal patch, vaginal ring), Intrauterine device (IUD), Tubal Ligation or hysterectomy, Participant/Partner post vasectomy, Implantable or injectable. contraceptives, and condoms plus spermicide. Not engaging in sexual activity for the total duration of the trial and the drug washout period is an acceptable practice; however periodic abstinence, the rhythm method, and the withdrawal method are not acceptable methods of birth control. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. * WOCBP must have a negative urine pregnancy test on the day of first dose prior to dosing. * WOCBP must agree to use adequate contraception for the duration of study treatment and 30 days after the last dose of study treatment. * Unsterilized male participants having sex with a female partner of childbearing potential, or a pregnant or breastfeeding partner must agree to use barrier contraception (male condom) for the duration of the treatment period until 30 days after the last dose of ivonescimab. Male patients with female partners of childbearing potential must have the female partner agree to use at least 1 form of highly effective contraception for the duration of the treatment period until 30 days after the last dose of ivonescimab * Ability to understand and the