The problem is more complex than "just" hardware. If I may paraphrase a recent paper out of Google Brain: the current systems (hardware and software) we use for for numerical computing are "stuck in a local basin of performance and programmability."[a]

[a] https://dl.acm.org/doi/pdf/10.1145/3317550.3321441?download=...