Our application either opens files with O_DSYNC so that writes are not supposed to complete until after the data has been committed to the filesystem or it opens files for direct I/O in which case it uses fdatasync/fsync() calls at points where the data needs to be persistent. On Linux these operations involve asking the disk controller to flush data to disk and while its possible that a disk might not implement these properly, being careful in the hardware selection can avoid that issue. In any case, the point is that the application is careful about writing things to disk in a particular order so that the data on disk is in a consistent state if there is a failure and by choosing the right hardware we can get these guarantees in a deployed system. This logic gets extended to our replication technology that can create other copies of the data.
I could not find anything in the Longhorn documentation describing guarantees related to either writing the active copy or generating the replicas to understand whether we can be certain that we can avoid data integrity issues when using this software. Is there something available which discusses how to craft software to deal with these issues? Is this just so basic that it goes without saying that it works properly?