What is the fastest way to increase the size of a file in linux on a ext4 filesystem from a C executable without creating holes in the file? -
the fastest way increase file size know of ftruncate() or lseek() desired size , write single byte. doesn't fit needs in case because resulting hole in file doesn't reserve space in file system.
is best alternative use calloc() , write()?
int increase_file_size_(int fd, int pages) { int pagesize = 4096; void* data = calloc(pagesize, 1); for(int = 0; < pages; ++i) { // in real world program handle partial writes , interruptions if (write(fd, data, pagesize) != pagesize) { return -1; } return 0; }
perhaps can made faster using writev. next version should faster since calloc has 0 initialize less memory, , more of data fits in cpu cache.
int increase_file_size_(int fd, int pages) { int pagesize = 4096/16; void* data = calloc(pagesize, 1); struct iovec iov[16]; for(int = 0; < 16; ++i) { iov[i].iov_base = data; iov[i].iov_len = pagesize ; } for(int = 0; < pages; ++i) { // in real world program handle partial writes , interruptions if (writev(fd, data, pagesize) != pagesize * 16) { return -1; } return 0; }
i can experiment , see of these approaches , page size faster. however, there approach considered normal best practice extending file? there other approaches should performance test?
thank you.
take @ posix_fallocate()
function: reserves space file without writing data occupy space. allocated space works sort of sparse file in can read though haven't explicitly written it, unlike sparse file, reduces amount of free space in filesystem. you're assured can write region of file later without running out of space.
note posix_fallocate()
doesn't seem make guarantees content of allocated space if read before writing it. think linux implementation return 0 bytes, similar sparse file, shouldn't rely on that. treat garbage before write real it.
also note not filesystem drivers support preallocation feature posix_fallocate()
takes advantage of, , think it'll fall on writing data file (the normal way) if preallocation isn't supported. typical linux filesystems ext4 , xfs ok, if try on fat or ntfs, program end blocking on i/o awhile.
Comments
Post a Comment