I think I our issues are related if not the same. I bisected and that brought me to this bug report. It seems like a "fix" for this bug caused my issues. https://bugzilla.kernel.org/show_bug.cgi?id=103271