Minimizing microSD block rewrites and erasures when datalogging

[/RaspberryPi]

Is there any consensus about good ways to frequently append small chunks of data (say, 30-50 bytes) to a file on the same microSD card holding the OS (though not necessarily the same partition/filesystem) in a way that minimizes having the controller/kernel/filesystem/library/whatever decide to totally rewrite the block and erase the old one?

For example... let's suppose a program in C++ opens a file for appending. I'm guessing that explicitly closing a file is almost guaranteed to trigger a block rewrite and erasure... but what about flushing the buffer while leaving the file open?

How about if a file gets opened for appending, then never gets deliberately flushed or closed? Does that tend to trigger writes only when enough data has accumulated to neatly fill a block, or at least implicitly produce the same outcome as buffering loggable values to ram & batching their writes at less-frequent intervals?

Digging deeper... where is the decision to rewrite-and-discard/erase a block, vs just keep appending, actually made? The SDcard's controller? The Linux kernel? The filesystem? The library?

Is there any remotely straightforward way (using C++) to even discover the underlying native block size in bytes? Or, if that's not something a program running as non-root is allowed to do... is there some utility you can run as root to discover the native block size for a particular microSD card?

If the decision is at least partially up to the kernel or filesystem, are there any alternate filesystems that are explicitly optimized for the use case of, "periodically appending to a file on SD card media, in ways that bend over backwards to avoid triggering block-rewrites and erasures (say, the way niche experimental filesystems like YAFFS and others in Android-land do)?

Do Pi Linux kernels for the Pi Zero (and Pi 1 B+) generally/always use the microSD card as a proper multi-bit-interface "SD Card", or they do use single-bit SPI/MMC mode? From the research I've done so far, I've gotten the impression that block-handling behavior isn't necessarily identical between SPI/MMC mode and "official SD Card with multi-bit interface" mode (or at least, it historically wasn't).
submitted by /u/PantherkittySoftware
[link] [comments]

Source: https://www.reddit.com/r/raspberry_pi/c ... _erasures/