Durvalumab With/Without Tremelimumab After Palliative Hypofractionated Radiotherapy for Hepatocellular Carcinoma

Inclusion Criteria: 1. Histologically-diagnosed HCC with progression during or after prior PD-(L)1 checkpoint inhibitor immunotherapy (e.g., nivolumab and/or pembrolizumab or atezolizumab; prior durvalumab excluded), or without prior PD-(L)1 checkpoint inhibitor immunotherapy. a. For patients without prior histologic or cytologic diagnosis, radiographic diagnosis is allowed provided patients meet American Association for the Study of Liver Diseases (AASLD) criteria for radiographic diagnosis. 2. At least 1 Response Evaluation Criteria in Solid Tumors (RECIST) 1.1-measurable tumor present which has not received RT or other local therapy prior to enrollment. 3. Clinical indication for RT to any site (e.g. painful primary or metastatic tumor, local complication risk such as impending biliary or vascular obstruction). 4. Child Pugh score of A, B7, or B8 provided other liver function criteria are met. 5. Eastern Cooperative Oncology Group (ECOG) 0 or 1 6. Appropriate antiviral therapy for hepatitis B virus (HBV) according to institutional standard of care with HBV deoxyribonucleic acid (DNA) polymerase chain reaction (PCR) \< 2000 IU/mL. 7. Adequate organ function as defined below: * Hemoglobin \>= 9.0 g/dL * Absolute neutrophil count \>= 1,200/microliter (mcL) * Platelet count \>= 60,000/mcL * Serum bilirubin =\< 1.5 x institutional upper limit of normal. This will not apply to patients with confirmed Gilbert's syndrome (persistent or recurrent hyperbilirubinemia that is predominantly unconjugated in the absence of hemolysis or hepatic pathology), who will be allowed only in consultation with their physician * Aspartate aminotransferase (AST) =\< 2.5 x institutional upper limit of normal unless liver metastases are present in which case it can be =\< 5 x upper limit of normal (ULN) * International normalized ratio (INR) \< 1.5 * Creatinine clearance \> 30 mL/min by Cockcroft Gault formula. 8. No contraindication to immune checkpoint inhibitor immunotherapy. 9. No contraindication to RT. 10. Age \>=18 years at time of study entry. 11. Life expectancy of \>= 12 weeks. 12. Body weight \> 30 kg (66.1 pounds). 13. Evidence of post-menopausal status or negative urinary or serum pregnancy test for female pre-menopausal patients. Women will be considered post-menopausal if they have been amenorrheic for 12 months without an alternative medical cause. The following age-specific requirements apply: * Women \< 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of exogenous hormonal treatments and if they have luteinizing hormone and follicle-stimulating hormone levels in the post-menopausal range for the institution or underwent surgical sterilization (bilateral oophorectomy or hysterectomy) * Women \>= 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of all exogenous hormonal treatments, had radiation-induced menopause with last menses \> 1 year ago, had chemotherapy-induced menopause with last menses \> 1 year ago, or underwent surgical sterilization (bilateral oophorectomy, bilateral salpingectomy or hysterectomy) 14. Women of childbearing potential and men must agree to use adequate contraception from the time of screening through the duration of study participation and for at least 6 months after receiving combination of durvalumab plus tremelimumab and 3 months after last dose of durvalumab. 15. Patient is willing and able to comply with the protocol for the duration of the study including undergoing treatment and scheduled visits and examinations including follow up. 16. Capable of giving signed informed consent, which includes compliance with the requirements and restrictions listed in the informed consent form (ICF) and in this protocol. Written informed consent and any locally required authorization (e.g., Health Insurance Portability and Accountability Act in the United States (US) obtained from the patient/legal representative prior to performing any protocol-related procedures, including screening evaluations. Exclusion Criteria: 1. Prior radiotherapy to tumor sites requiring RT which could compromise safety of additional treatments. 2. Prior radiotherapy to more than 30% of bone marrow or to a wide field within 4 weeks of the first study treatment. 3. Prior treatment with cytotoxic T-lymphocyte-associated protein 4 (CTLA-4) or PD-L1 inhibitor. 4. History of allogenic organ transplantation. 5. On prior PD-1 inhibitor immunotherapy: * Any immune-related adverse events with National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) version (v) 5 grade \>= 3 on any prior immunotherapy or toxicity that led to permanent discontinuation of prior immunotherapy. * Any AEs while receiving prior immunotherapy not resolved to grade =\< 1 or resolved to baseline, with the exception of patients with endocrine AE of grade =\< 2, who are permitted to enroll if they are stably maintained on appropriate replacement therapy and are asymptomatic * Required the use of additional immunosuppression other than corticosteroids for the management of an AE, experienced recurrence of a grade \>= 3 AE if previously re-challenged, and currently require maintenance doses of \> 10 mg prednisone or equivalent per day 6. Major surgery, liver-directed therapy, or any other anticancer therapy (e.g. chemotherapy, immunotherapy, endocrine therapy, targeted therapy, biologic therapy, tumor embolization, monoclonal antibodies) less than 4 weeks prior to enrollment 7. Any other unresolved toxicity NCI CTCAE grade \>= 2 from previous anticancer therapy with the exception of alopecia, vitiligo, and the laboratory values defined in the inclusion criteria. * Patients with grade \>= 2 neuropathy will be evaluated on a case-by-case basis after consultation with the study physician * Patients with irreversibl