I hope you don't mean object in the OOP sense, but in a more general way, because that would be the slowest thing ever, no matter how smart your compiler/framework.
You'd probably be better off to just have each group as a copy of the original one.
> not practical with frameworks like PyTorch and TensorFlow
The problem is not the frameworks, the problem is hardware, and physics.
We've hit the wall with clock speeds, we've hit the wall with pipeline depth, we've hit the wall with branch prediction.
The only thing we have left is specialised hardware, parallel hardware, and bigger caches. But all of these only work with SIMD, because it makes data fetches predictable, it makes the cores simpler and it makes adding more compute cheaper (no separate fetch decode silicon required).
For the memory hierarchy to work you want stuff packed like in ECS systems for games, or Matrices for ML.
Even with specialised hardware, changing your neural architecture on the fly would be ridiculously expensive because of all the communication overhead.
The problem is more complex than "just" hardware. If I may paraphrase a recent paper out of Google Brain: the current systems (hardware and software) we use for for numerical computing are "stuck in a local basin of performance and programmability."[a]
Using a sparse adjacency matrix in combination with regular matrix operations solves the problem of fitting any graph topology into matrices (used in graph neural nets). You can put multiple graphs together in the same adjacency matrix and batch them up for efficient computation.
There's a big "impedance mismatch" between (1) "programmability" (by which I mean, being able to write high-level code that under-the-hood requires dynamic modification, reshaping, and/or combination-optimization of those adjacency matrices you mention, without worrying about performance) and (2) existing infrastructure (frameworks that rely on highly optimized computation "kernels" that cleverly exploit the memory layout and other features of accelerated hardware, i.e., GPUs/TPUs).
I hope you don't mean object in the OOP sense, but in a more general way, because that would be the slowest thing ever, no matter how smart your compiler/framework.
You'd probably be better off to just have each group as a copy of the original one.
> not practical with frameworks like PyTorch and TensorFlow
The problem is not the frameworks, the problem is hardware, and physics.
We've hit the wall with clock speeds, we've hit the wall with pipeline depth, we've hit the wall with branch prediction.
The only thing we have left is specialised hardware, parallel hardware, and bigger caches. But all of these only work with SIMD, because it makes data fetches predictable, it makes the cores simpler and it makes adding more compute cheaper (no separate fetch decode silicon required).
For the memory hierarchy to work you want stuff packed like in ECS systems for games, or Matrices for ML.
Even with specialised hardware, changing your neural architecture on the fly would be ridiculously expensive because of all the communication overhead.