Testing the Safety and Efficacy of the Combination of Two Anti-cancer Drugs, ZEN003694 and Abemaciclib, for Adult and Pediatric Patients (12-17 Years) With Metastatic or Unresectable NUT Carcinoma, Breast Cancer and Other Solid Tumors

Inclusion Criteria: * Participants must have histologically confirmed malignancy that is metastatic or unresectable and for which standard curative or palliative measures do not exist or are no longer effective * Dose Escalation Cohort Only: Participants must have evaluable disease or measurable disease per RECIST 1.1 criteria * Dose Expansion Cohort Only: * Participants must have a diagnosis of NUT carcinoma (NC) based on standard criteria for the disease, with diagnostic testing performed in a Clinical Laboratory Improvement Act (CLIA) certified laboratory: * Ectopic expression of NUT protein per World Health Organization (WHO) criteria as determined by immunohistochemistry (IHC) testing, OR * Detection of the NUT gene translocation as determined by fluorescence in situ hybridization (FISH) testing, OR * Detection of the NUT gene translocation as determined by either deoxyribonucleic acid (DNA) next-generation sequencing (NGS) or ribonucleic acid (RNA) sequencing. * Participants must have measurable disease per RECIST 1.1 criteria * Any number of prior lines of therapy in the metastatic setting are allowed, including prior BET inhibitor therapy and prior CDK4/6 inhibitor therapy * Patients who received chemotherapy must have recovered (Common Terminology Criteria for Adverse Events \[CTCAE\] grade =\< 1) from the acute effects of chemotherapy except for residual alopecia or grade 2 peripheral neuropathy * Patients who received radiotherapy must have completed and fully recovered from the acute effects of radiotherapy. A washout period of at least 14 days is required between end of radiotherapy * Participants may have previously undergone surgical resection * Age \>= 12 years. Patients 12-17 years of age must be \> 40 kg at enrollment. Patients 12-17 years of age will not participate in the mandatory tumor biopsies. Since there is no data on patients less than 18 years of age, this population may require lower doses and additional safety precautions and should be closely monitored. Because no dosing or adverse event data are currently available on the use of ZEN003694 in combination with abemaciclib in patients \< 12 years of age, younger children are excluded from this study * Eastern Cooperative Oncology Group (ECOG) performance status =\< 2 for participants \>= 16 years of age, Lansky \>= 50% if \< 16 years of age * Hemoglobin \>= 8 g/dL; Patients may receive erythrocyte transfusions to achieve this hemoglobin level at the discretion of the investigator. Initial treatment must not begin earlier than the day after the erythrocyte transfusion * Absolute neutrophil count \>= 1.5 x 10\^9/L * Platelets \>= 1 x 10\^11/L * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) for age. Patients with Gilbert's syndrome with a total bilirubin =\< 2.0 times ULN and direct bilirubin within normal limits are permitted * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional ULN for age * Serum or plasma creatinine =\< 1.5 x institutional ULN OR calculated creatinine clearance \>= 50 mL/min (via the chronic kidney disease epidemiology \[CKD-EPI\] glomerular filtration rate estimation) for participants \>= 18 years old, or 60 mL/min/1.73m\^2 for patients 12-17 years as calculated based on bedside Schwartz formula * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load. Hepatitis C (HepC antibody) testing is required. Hepatitis C RNA is optional; however, a confirmatory negative Hepatitis C RNA test must be obtained to be able to enroll participants with positive Hepatitis C antibody due to prior resolved disease * Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression and has been clinically stable for at least 1 month. Patients must meet the following criteria: * Disease outside the CNS is present * Recovery from acute toxicity associated with the treatment to =\< CTCAE grade 1 or baseline (with the exception of alopecia), with no requirement for escalating doses of corticosteroids over the past 7 days * Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * Patients should be New York Heart Association Functional Classification of class 2B or better * Ability to swallow and retain oral medications * The effects of ZEN00364 and abemaciclib on the developing human fetus are unknown. For this reason and because BETi and CDKi-inhibiting agents are known to be teratogenic, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation. Women of child-bearing potential must have a negative pregnancy test prior to study entry. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men and women treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 3 weeks after completion of ZEN003694 and abemaciclib administration * For female subjects of chi