add option --iobuf (file r/w buffersize):

the default (256 KiB) appears optimal in the most popular scenario
(linux host with storage on local physical disk, usually NVMe)

was previously a mix of 64 and 512 KiB;
now the same value is enforced everywhere

download-as-tar is now 20% faster with the default value
This commit is contained in:
ed 2024-03-23 16:17:40 +00:00
parent d30ae8453d
commit 2b24c50eb7
11 changed files with 37 additions and 23 deletions

View file

@ -234,8 +234,9 @@ def u8(gen):
def yieldfile(fn):
with open(fn, "rb") as f:
for block in iter(lambda: f.read(64 * 1024), b""):
s = 64 * 1024
with open(fn, "rb", s * 4) as f:
for block in iter(lambda: f.read(s), b""):
yield block