Pembrolizumab Plus 177Lu-PSMA-617 in Patients With Castration Resistant Prostate Cancer

Inclusion Criteria: 1. Male participants who are at least 18 years of age on the day of signing informed consent. 2. Histologically confirmed prostate adenocarcinoma that is progressive metastatic castration-resistant prostate cancer by Prostate Cancer Clinical Trials Working Group 3 (PCWG3) criteria at the time of study entry. 3. Castrate level of serum testosterone at study entry (\< 50 ng/dL). Note: Participants without prior bilateral orchiectomy are required to remain on Luteinizing hormone-releasing hormone (LHRH) analogue treatment for duration of study. 4. At least one Prostate-Specific Membrane Antigen (PSMA) Positron Emission Tomography (PET) (PSMA PET) avid lesion on screening PSMA PET. A positive lesion is defined as uptake above background liver. 5. Prior progression on at least one second generation androgen signaling inhibitor including abiraterone, apalutamide, darolutamide, and/or enzalutamide. 6. Adverse events related to prior anti-cancer treatment (excluding LHRH analogs) must have recovered to Grade \<= 1 (except for any grade alopecia and grade \<= 2 neuropathy). 7. Prior radiotherapy is allowed if the last radiotherapy treatment was greater than 2 weeks from start of study treatment on cycle 1, day 1 (C1D1). Note- Participants must have recovered from all radiation-related toxicities, not require corticosteroids, and not have had radiation pneumonitis. A 1-week washout is permitted for palliative radiation (\<=2 weeks of radiotherapy) to non-central nervous system (CNS) disease. 8. Eastern Cooperative Oncology Group (ECOG) performance status \<= 1 (Karnofsky \>= 70%). 9. Demonstrates adequate organ function as defined below: 1. Adequate bone marrow function: * absolute neutrophil count \>=1,500/microliter (mcL) * platelets \>=100,000/mcL * hemoglobin \> 9.0 g/dL 2. Adequate hepatic function: * total bilirubin \<= 1.5 x upper limit of normal (ULN). In patients with known or suspected Gilbert's disease, direct bilirubin \<= ULN * aspartate aminotransferase (AST)/serum glutamic-oxaloacetic transaminase (SGOT) \<= 2.5 x institutional ULN (\<= 5 x ULN in patients with liver metastases) * alanine aminotransferase (ALT)/serum glutamic-pyruvic transaminase (SGPT) \<= 2.5 x institutional upper limit of normal (\<= 5 x ULN in patients with liver metastases) 3. Adequate renal function: * creatinine \<= 1.5 x within institutional upper limit of normal OR * creatinine clearance Glomerular filtration rate (GFR) \>= 50 mL/min/1.73 m\^2, calculated using the Cockcroft-Gault equation or 24 hour urine collection. 10. Participants must use appropriate methods of contraception during study treatment and for at least 6 months after last study treatment. Patients who are sexually active should consider their female partner to be of childbearing potential if she has experienced menarche and is not postmenopausal (defined as amenorrhea \> 24 consecutive months) or has not undergone successful surgical sterilization. Even women who use contraceptive hormones (oral, implanted, or injected), an intrauterine device, or barrier methods (diaphragms, condoms, spermicide) should be considered to be of childbearing potential. Patients who have undergone vasectomy themselves should also be considered to be of childbearing potential. Acceptable methods of contraception include continuous total abstinence, or double-barrier method of birth control (e.g., condoms used with spermicide, or condoms used with oral contraceptives). Periodic abstinence and withdrawal are not acceptable methods of contraception. 11. Participants must provide consent to comply to recommended radioprotection precautions during study. 12. Participants willing to undergo a tumor biopsy. Bone or soft tissue lesion is allowed. Note: The biopsy can be waived if there is no safely accessible lesion in the judgement of the treating investigator. 13. Participants with previously treated brain metastases are eligible provided the following criteria are all met: 1. Last treatment was \> 28 days prior to C1D1. 2. No evidence of new/progressive brain metastases is observed on magnetic resonance imaging (MRI) obtained during screening window 3. Patient is clinically stable without requirement of steroid treatment for at least 14 days prior to first dose of study treatment on C1D1. 14. Individuals with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial. 15. Participants living with HIV must have well-controlled HIV on ART, defined as: 1. Participants on ART must have a CD4+ T-cell count ≥350 cells/mm3 at the time of screening 2. Participants on ART must have achieved and maintained virologic suppression defined as confirmed HIV RNA level below 50 or the LLOQ (below the limit of detection) using the locally available assay at the time of screening and for at least 12 weeks before screening 3. It is advised that participants must not have had any AIDS-defining opportunistic infections within the past 12 months. 4. Participants on ART must have been on a stable regimen, without changes in drugs or dose modification, for at least 4 weeks before study entry (Day 1) and agree to continue ART throughout the study 16. Ability to understand and the willingness to sign a written informed consent document. Exclusion Criteria: 1. De novo small cell neuroendocrine prostate cancer will not be allowed due to putative lower PSMA expression in this tumor subtype. Note-Treatment-emergent small cell neuroendocrine prostate cancer detected in metastatic tumor biopsy is not excluded. 2. Soft tissue lesions (lymph nodes \> 1.5 centimeter (cm) in short axis, visceral/soft tissue lesions \> 1 cm) on screening Computerized tomography (CT) that are negative on PSMA PET. Note: Negative lesions on PSMA PET are defined as t