lsd

LSD

LSD, or Lykso's Sync Daemon, creates and handles messages passed between Lykso's devices. It's part of their ideal datastore project, which, in this incarnation, also relies on Syncthing and a custom union filesystem.

This is a fairly naive implementation for personal and research use. A better implementation would likely replace Syncthing with a P2P sync network with better awareness about this particular use case. Edge cases certainly exist right now due to fact that Syncthing is unaware that the multiple directory trees this utility uses are meant to be treated in many respects as a single directory tree.

Usage

All "path" arguments are relative to the mountpoint, not the current working directory. This may or may not be fixed in a later version, depending on how much time and energy I have for this project going forward.

lsd exclude [--config=<config-path>] --nodes=<node-name>,...  <path>
lsd include [--config=<config-path>] --nodes=<node-name>,... <path>
lsd restore [--config=<config-path>] --nodes=<node-name>,... <path>
lsd reduce [--config=<config-path>] --node=<node-name>,... <path>
lsd info [--config=<config-path>] [<path>]
lsd mount [--config=<config-path>] <path>
lsd nodes [--config=<config-path>]

Config options

The default config file path on Linux boxes is ~/.config/lsd/lsd.yaml

The XDG standard is followed for config and data files.

dataDir: (optional, ~/.local/share/lsd by default)
syncthing:
    url: (optional, "http://localhost:8384" by default)
    key: (required)
    nodes: (required)
        - guid: (required)
          alias: (required)

Node sets

For each unique combination of nodes, there exists:

• A data directory
• A face directory tree for "reduced" file metadata

Face files have an extended attribute, user.lsd.data-path, which points to a path relative to the data directory

Failure modes

This is not an exhaustive list. This is meant to point out failure modes specific to this utility that may not be obvious. E.g., a filesystem operation may fail due to corruption or permissions issues; these are not listed because they are not specific to LSD.

• If a path exists in multiple data directory trees on a single node, any operation on that path will return a non-zero exit code and fail. LSD assumes one canonical path for any given file. If there is not one canonical path, this may be indicative of synchronization or other problems.

• If a path exists in multiple directory trees on a single node, any operation on that path except for the "status" operation will fail and exit with a non-zero code. The "status" operation may display a message indicating that a "reduce" request has been issued but has not yet been served for that path.

Messages

Four message types at the moment:

• Exclude: Represents a request from a node to have a path moved to a directory tree representing the set of nodes the path is currently synced between, less itself.

• Restore: The inverse of exclude. Returns the requested path to the directory tree representing the set of currently syncing nodes, with the addition of the node making the request. Fails if the node is not listed under the "excluded-by" extended attribute.

• Error: Represents the failure of a request.

• Success: Represents the successful completion of a request.

Messages are placed in the "outgoing" directory of the node making the request, thus providing de facto message authentication and replay protection.