Adding more levels of cache is a time-honored tradition, and it works.
For example, with the M1, Apple tied the DRAM interface to the CPU. That got them more bandwidth, but then you can't upgrade the memory, which is weak.
But why not do both?
Have e.g. 8GB of high-bandwidth DRAM soldered to the board, essentially that amount of L4 cache, but keep the DIMM slots where you can add arbitrarily more memory with less bandwidth.
It adds latency. You have to probe all the earlier levels of the cache before you can access the next one, if you want the caches to help with bandwidth. The cost of probing more levels adds up.
- Remove all or almost all L3 from the CPU die built with the best available logic process, like N3E
- Stack a last level cache die with relatively cheaper process that has similar or better SRAM density, like N5