Mojave was installed about 3 years ago and has been working truly flawlessly with this same RX560 card in that time. Firmware is the required 144.0.0.0.0
GREAT! That eliminates that as an issue.
and yes, I've definitely gone the Carbon Copy Cloner route (so that could certainly be a/the issue as well. I can try a clean install too, but Big Sur is functional on the old ATI card).
newer versions of macos can be hacked to work with the old ATI card! But occasionally the screen will look jumbled, some software or feature may have a graphics problem when it tries to use Metal that isn't there. I would focus on using a proper Metal card. Also everything runs MUCH faster and better on metal.
I have made sure to read as much as I could stomach on the Macrumours forum about the process and that's where I came across the tailored Martin Lo install for cMPs.
hehe yea, I hear you. It's the wild west.
I uninstalled all needless harware (Caldigit USB3/Sata card, PCI-E SSDs) and removed all but one 16GB stick of RAM, as well as disconnected everything but Ethernet and monitor. I tried 3 different monitors to exclude compatibility issues there.
Good idea to break it down and re-introduce things later to see. Just for the record, i am using vanilla Open Core with generic settings, several ssd's on a pci card, a Lynx pci audio card, RX580(a specific one named by apple as compatible), updated bluetooth card, 96GB of ram same as before, all data drive bays full of hard drives or ssd's. Haven't seen a gpu boot screen in years and it's been running great for years. Admittedly while doing opencore updates, there can be some times when you have to look at the blackscreen and wait...or know to press cmd-R and wait...etc..instead of seeing apple's boot menu. But it works fine. At one point I was looking into getting it flashed but certain people I trust advised against it, so I never did. And I didn't really need to I was just being OCD at the time.
I would not expect your USB3 card to be a problem but never know. same with your ssd's on pci card. Use all your ram too.
In retrospect, the dosdude Catalina install was likely hampered by the Samsung 870 EVO incompatibility issue, which I only noticed once I had moved on to trying with Big Sur. I now have a Graphics Card on order to try again. The fact that Big Sur boots fine with my old ATI but not the RX560, for me definitely points to the latter being an issue.
Boots fine with what? its that a martin lo hack? Well anyway, nothing wrong with duplicating Martin's exact hardware specifications and using his configuration. In theory that should work. That's exactly why I bought an RX580 way back when. I know they got to be expensive though. But the RX580 definitely works.
Like I said, newer macos can be hacked to run on the ATI...but being non-metal, anything after Mojave is just rolling the dice as to when you'll have some kind of metal incompatibility issue.
I think I lucked out being able to buy the RX580 when I did. A lot of people are using various Nvidia cards and other things and having to figure out how to hack opencore/macos to make them work as apple drops support. that's where OCLP and martin lo can be helpful.
So I'm going to sit tight and ignore Catalina for the time being. If Big Sur with a flashed card doesn't work out, I'll try dosdude Catalina one more time with my RX560 and the new, flashed card.
flashed card won't help. If it doesn't work, its something else. But it's possible that a different card that matches martin lo's favored configuration may work with his setup out of the box.
I will also say that at the time I found Catalina quite usable...it was a worthwhile upgrade for sure...but then Big Sur I skipped because of problem reports at the time. Monterey, however, has been extremely good for me. Fast and solid. it's the last you can realistically do also. If you're going OpenCore, try to get to Monterey.