It is much better to not use any swap in Linux, so you will never have this probem, or any other problem caused by swap.
I stopped using swap on Linux about a quarter of century ago, when it was a great improvement, and since then I have never seen a case when swap would have been useful, and I use Linux on a variety of laptops, desktops and servers.
Even if you do not use swap, you can have memory-mapped files that are much bigger than your physical memory. The LLM weights files must be mapped as read-only. In this case, the pages that have not been used recently will be freed when memory is needed to load other pages from the files.
The weights files must be mapped using huge pages, otherwise an excessive amount of physical memory would be wasted and reading new pages would be very slow.
It is likely that it is not possible to reach a good enough performance with a memory-mapped file without using carefully "madvise", with which it is possible to force the reading of the pages that you know that they will be needed in the future and also the freeing of the pages that you know that they will not be needed soon.
On Linux, it is possible to execute "madvise" asynchronously (with liburing). An alternative to liburing is to execute "madvise" from a concurrent thread, which synchronizes with requests to do "madvise" from the orchestrating thread.
I stopped using swap on Linux about a quarter of century ago, when it was a great improvement, and since then I have never seen a case when swap would have been useful, and I use Linux on a variety of laptops, desktops and servers.
Even if you do not use swap, you can have memory-mapped files that are much bigger than your physical memory. The LLM weights files must be mapped as read-only. In this case, the pages that have not been used recently will be freed when memory is needed to load other pages from the files.
The weights files must be mapped using huge pages, otherwise an excessive amount of physical memory would be wasted and reading new pages would be very slow.
It is likely that it is not possible to reach a good enough performance with a memory-mapped file without using carefully "madvise", with which it is possible to force the reading of the pages that you know that they will be needed in the future and also the freeing of the pages that you know that they will not be needed soon.
On Linux, it is possible to execute "madvise" asynchronously (with liburing). An alternative to liburing is to execute "madvise" from a concurrent thread, which synchronizes with requests to do "madvise" from the orchestrating thread.