Thursday, July 30, 2009

[dzmieeiy] Network write journal

Network file systems are finicky and fickle. We need a mechanism such that writes to the filesystem, which if unsuccessful, are "cached" or journaled in memory until the network filesystem is available again, at which point the journal is played recording the writes. FUSE might be a good way to implement this overlay filesystem.

Optionally do in-memory compression of the journal to conserve memory.

This is inspired by zephyr loggers being the principal consumers of memory on Linerva: simply write to AFS, but your tokens may have expired, or it might be the Sunday morning restart.

We can't do anything about reads in the event of network filesystem unavailability, unless it's in cache, which is already done.

No comments :