Post ABeiFDbRRLF3Ph2m6C by loziniak@quitter.pl
 (DIR) More posts by loziniak@quitter.pl
 (DIR) Post #ABadhJWfjczrUMYDEO by wolf480pl@mstdn.io
       2021-09-21T15:10:16Z
       
       0 likes, 0 repeats
       
       Is there a cp -r that uses direct IO?I don't like my whole RAM filled with big files I'm copying...
       
 (DIR) Post #ABaqm1l8W8YXEptVAW by ashwinvis@mastodon.acc.sunet.se
       2021-09-21T17:36:47Z
       
       0 likes, 0 repeats
       
       @wolf480plIs that even possible to bypasd RAM?  :OI would be curious.
       
 (DIR) Post #ABawpBHy8tuTO58e0G by wolf480pl@mstdn.io
       2021-09-21T18:44:35Z
       
       0 likes, 0 repeats
       
       @ashwinvis no, but you can reuse one page over and over instead of filling all the pages with disk data
       
 (DIR) Post #ABcn8Z37PoW7M7iSMi by michal@toot.kottman.xyz
       2021-09-22T16:05:20Z
       
       0 likes, 0 repeats
       
       @wolf480pl @ashwinvis I was in the same camp, but came to realize that unused memory is wasted memory. And disk cache will be made available to allocations anyway - it's not "hogging up memory", `free` shows enough "available" memory.Why is it a problem to keep data around in memory pages? Alternative could perhaps be `dd` with the `nocache` flag but I'm not entirely sure of the implications.
       
 (DIR) Post #ABczBbdjRHxdi13ghs by wolf480pl@mstdn.io
       2021-09-22T18:20:28Z
       
       0 likes, 0 repeats
       
       @michal @ashwinvis Cache only makes sense for things you access more than once. When I have multiple things running, including web browser, shell, text editors, email client, and other things, I'd rather have their data and binaries cached, than some huge file that I'm copying from one disk to another, never to see it again.
       
 (DIR) Post #ABd3XFeqFSOLii9dTc by michal@toot.kottman.xyz
       2021-09-22T19:09:05Z
       
       0 likes, 0 repeats
       
       @wolf480pl @ashwinvis I sometimes wish there was a `mlockall` [1] launcher, something like `setsid`, that would prevent a process memory being swapped out - great for preventing things like sshd freeze pre-oom.One can also use `vmtouch` to make sure stuff is in virtual memory, and TIL that it can run in daemon mode and supports `mlock` too.[1] https://linux.die.net/man/2/mlockall[2] https://linux.die.net/man/8/vmtouch
       
 (DIR) Post #ABd5emKxLV2l0aZPBA by michal@toot.kottman.xyz
       2021-09-22T19:32:52Z
       
       0 likes, 0 repeats
       
       @wolf480pl Small experiment confirms that `dd` with `direct` does not fill up memory (+ https://unix.stackexchange.com/a/597493/3377): # echo 3 > /proc/sys/vm/drop_caches$ free -h # before cp... buff/cache...      1.0Gi$ cp large_file large_file2$ free -h # after cp... buff/cache...      3.4Gi# echo 3 > /proc/sys/vm/drop_caches$ free -h # before dd... buff/cache...     1.0Gi $ dd if=large_file of=large_file2 bs=1M iflag=direct oflag=direct$ free -h # after dd... buff/cache...      1.0Gi
       
 (DIR) Post #ABd6B7sGo21l4WZoy8 by wolf480pl@mstdn.io
       2021-09-22T19:38:49Z
       
       0 likes, 0 repeats
       
       @michal yes, I knew that. It's just that dd with direct doesn't recurse a directory tree.
       
 (DIR) Post #ABd6C7riKCQDKknFi4 by wolf480pl@mstdn.io
       2021-09-22T19:39:01Z
       
       0 likes, 0 repeats
       
       @michal guess I should write a shell wrapper
       
 (DIR) Post #ABeiFDbRRLF3Ph2m6C by loziniak@quitter.pl
       2021-09-23T14:19:19Z
       
       0 likes, 1 repeats
       
       I fixed the problem once, I think it was this solution:https://lwn.net/Articles/572921/