Wayland happened so badly compared to e.g. the Systemd transition or the Pipewire transition because it was developed by Xorg developers who had spent years dealing with its complexity and countless regressions caused by bugfixes breaking obscure behavior that clients depended on. As a result, they swung too far in the other direction. They came out with a minimal protocol where all the heavy lifting was done in each compositor rather than having a single common implementation. Most importantly, the base protocol was completely unsuitable for nontrivial programs. Features such as screen sharing, remote desktop, non-integer scaling, screen tearing, or even placing a window in a certain position were not supported.
Because Wayland's base protocol was unable to replace X11, virtually nobody adopted it when it was initially released in 2008. The subsequent 16 years have been taken up by a grueling process of contributors from various desktop environments proposing protocol extensions to make Wayland a workable solution for their users. This has resulted in endless bikeshedding and requires compromise between different stakeholders who have fundamentally different viewpoints on what the Linux desktop should even be (GNOME vs everyone else). As an example, the process for allowing a client to position its own windows has been ongoing for over two years, and has led to multiple proposed protocol extensions with hundreds of comments on each.
The transition would have been much smoother if Wayland had the same "mechanism over policy" philosophy as X11 and allowed for software to easily be ported to it, but long-term that could have caused the same issues that the Xorg maintainers were facing when they created Wayland.
Was Pipewire a shitshow? PulseAudio was, and it had a bunch of issues (idk about design issues - I haven't really dug into the details, but it certainly had a fair number of unpleasant bugs), but I think Pipewire transition was fairly uneventful, in a good way.
And systemd was (and still is, I guess) controversial, and there are certainly some design decisions that are questionable - but even in its early days, at the very least it worked for a number of basic scenarios and the majority of issues I've had or heard about was either in its limitations (I remember having issues with journald) or because folks didn't want to change from their preferred rc system as it already worked well for them and systemd was radically different. So, I think, it was less of a shitshow than Wayland. Or maybe I'm just forgetting things - it was a long while ago.
Because Wayland's base protocol was unable to replace X11, virtually nobody adopted it when it was initially released in 2008. The subsequent 16 years have been taken up by a grueling process of contributors from various desktop environments proposing protocol extensions to make Wayland a workable solution for their users. This has resulted in endless bikeshedding and requires compromise between different stakeholders who have fundamentally different viewpoints on what the Linux desktop should even be (GNOME vs everyone else). As an example, the process for allowing a client to position its own windows has been ongoing for over two years, and has led to multiple proposed protocol extensions with hundreds of comments on each.
The transition would have been much smoother if Wayland had the same "mechanism over policy" philosophy as X11 and allowed for software to easily be ported to it, but long-term that could have caused the same issues that the Xorg maintainers were facing when they created Wayland.