Silo vault format, version 1

This document is the published definition of a Silo vault on disk. It is written for someone who has a copy of a vault folder and its recovery phrase (12, 6 or 2 words) and wants to get the files (photos, videos and any other file) out without the app. It is complete on its own: everything needed to read a vault is here.

silo-decrypt is a reference reader that follows it, and a test checks that it decrypts vaults written by the app byte for byte. Where any other description of Silo and this document disagree about bytes, this document is right and the other is a bug.

1. Conventions

• All integers are big-endian and unsigned. u8, u32 and u64 are 1, 4 and 8 bytes.
• a ‖ b is concatenation. bytes[x..y] is the bytes from offset x up to, not including, y.
• hex(x) is the lowercase ASCII hexadecimal text of the bytes x, as bytes. For example hex(0x0aff) is the 4 bytes 30 61 66 66 ("0aff"). Every AEAD’s additional data (AD) is passed this way: hex of the bytes, never the raw bytes.
• Times are milliseconds since 1970-01-01 UTC. Object and album ids are 16 random bytes, written as 32 lowercase hex characters.
• Text is UTF-8. JSON is RFC 8259; trailing ASCII spaces after a JSON value are padding.

2. Primitives

Everything is libsodium’s, and each has a common equivalent:

Name here	Definition	libsodium	Python
AEAD(k, n, m, ad)	XChaCha20-Poly1305 IETF; 32-byte key, 24-byte nonce, output = ciphertext ‖ 16-byte tag	crypto_aead_xchacha20poly1305_ietf_*	PyNaCl nacl.bindings
H(m)	BLAKE2b, 32-byte output, no key	crypto_generichash	hashlib.blake2b(m, digest_size=32)
HK(k, m)	BLAKE2b, 32-byte output, keyed with k	crypto_generichash	hashlib.blake2b(m, digest_size=32, key=k)
HKDF(salt, ikm, info)	HKDF-SHA256 (RFC 5869), 32-byte output; extract = HMAC-SHA256(key = salt, ikm)	crypto_kdf_hkdf_sha256_*	hmac
Subkey(mk, id)	BLAKE2b, 32-byte output, key = mk, empty message, salt = u64 little-endian(id) ‖ 8 zero bytes, personalization = silo_mk1 ‖ 8 zero bytes	crypto_kdf_derive_from_key with context silo_mk1	hashlib.blake2b(b"", digest_size=32, key=mk, salt=…, person=…)

Decryption that fails authentication means the data is damaged or the key is wrong; a reader never uses output from a failed decryption.

3. Folder layout

3.1 The four vault folders

The app keeps its vaults in one folder, vault/, that always holds four folders named v0, v1, v2 and v3. Each is a complete v1 vault, laid out as in 3.2 with paths relative to that folder, and the format of what is inside one does not change. A phone with one vault has it in one of the four, picked at random when the vault was made. Every other folder is unused: it holds a keys.slot and nothing else.

• The keys.slot of an unused folder is a filler. It is 952 bytes, with the same magic, version, kind, entry count, Argon2id salt and Argon2id parameters as a real file (4.2), a random recovery salt, 8 random entries and a random map block. It parses, and without a key nothing tells it from a real file.
• Every keys.slot on the phone has the same Argon2id salt and parameters, so one run of Argon2id per attempt serves all four files. The app tries the resulting key on all 8 entries of all four files, and the recovery phrase on all four files (a phrase of 6 or 2 words costs one more Argon2id run per file, each with its own recovery salt).
• A change to one vault rewrites only that folder’s keys.slot.
• vault/ may also hold, for a short while after an Emergency PIN reset, one directory named trash- and 16 hex digits. It holds the old folders, whose keys were destroyed, and the app deletes it. Readers look only at v0 to v3 and ignore everything else.

What a person looking at the files can learn: that the app is Silo (the layout is published), how many vaults are in use (a folder with an index/ folder is in use, and an unused folder has none), and the size of each. They cannot learn which entries of a keys.slot are in use, or whether a filler is a filler, without a key. What the layout does not hide is exactly this: the folder names, the number of folders in use, and the size of their files.

To read a vault you need a copy of its folder (v0 to v3) and its recovery phrase. A copy of the whole vault/ folder also works: try the phrase on the keys.slot of each of the four folders, and read the one it opens (the reference tool in tools/silo-decrypt does this). A phrase that opens none belongs to no vault there.

Each folder is self-contained: nothing in one refers to another, so a copy of one folder and its phrase is enough.

3.2 Inside one vault folder

A vault is one of those folders. Paths below are relative to it.

Path	Content	Needed to read files
keys.slot	Key slots (section 4)	Yes
objects/SS/ID.silo	One original file (a photo, a video or any other file), a container of kind item (section 5). ID = its object id, SS = the first 2 characters of ID. The short video of a motion photo (6.1) is also a file here	Yes
thumbs/SS/ID.silo	A thumbnail, a container of kind thumb	No
index/snap.NNNNNN.silo	Index snapshot, a container of kind snapshot (section 6). N = generation, 6 or more decimal digits	For albums and deletions
index/wal.NNNNNN.log	Index journal (section 6)	For albums and deletions
index/wal.NNNNNN.damaged.log	A journal the app set aside after finding damage in it	No
health.silo	The app’s integrity-check state, a container of kind snapshot	No
setup.pending	App state only: present until the user has confirmed the recovery phrase after setup. Empty of secrets	No
add.pending, delete.pending	App state only: present while the app adds or deletes a vault in this folder, and removed at the next start (add.pending holds a copy of the new keys.slot). Ignore them	No
tmp/, lost/	Files being written, and files set aside by a repair	No

Any other file can be ignored. If a folder has keys.slot.next and no keys.slot, the former is its key slot file (an interrupted save).

4. From the recovery phrase to the master key

4.1 Phrase to entropy

The phrase is 12, 6 or 2 words from the standard 2048-word English BIP-0039 list. Words are compared after trimming and lowercasing; they are separated by any whitespace. The vault does not record how many words its phrase has: the number of words given decides how the key is derived (4.3), and a wrong count simply opens nothing.

Words	Bits (11 per word)	Entropy bits	Entropy E	Checksum bits
12	132	128	16 bytes	4
6	66	64	8 bytes	2
2	22	22	3 bytes	0

1. Replace each word by its index in the list (0–2047), as 11 bits, most significant first, and join them.
2. The first entropy bits are the entropy E; the remaining bits are the checksum. For 12 and 6 words E is those bits as bytes. For 2 words the 22 bits are the top 22 bits of E (3 bytes), so the last 2 bits of its third byte are 0.
3. For 12 and 6 words the checksum must equal the top 4 (12 words) or 2 (6 words) bits of H("silo-phrase-v1" ‖ E)[0]. This differs from BIP-0039, which uses SHA-256. A mismatch means a word is wrong. A 2-word phrase has no checksum: any two words are a phrase.
4. Any other number of words is not a phrase.

4.2 keys.slot

Exactly 952 bytes:

Offset	Length	Field
0	4	Magic SILO
4	1	Version = 1
5	1	Kind = 4
6	1	Entry count = 8
7	1	Reserved, 0
8	16	Argon2id salt (not needed for recovery)
24	4	Argon2id passes (needed only for a 6- or 2-word phrase, 4.3)
28	4	Argon2id memory in bytes (needed only for a 6- or 2-word phrase, 4.3)
32	16	Recovery salt
48	8 × 104	Entries 0–7
880	72	Map block (not needed for recovery)

Entry i (at offset 48 + 104·i) = nonce (24) ‖ sealed payload (80), with AD = hex(bytes[0..48] ‖ u8(i)). The payload is 64 bytes: kind u8 ‖ 7 zero bytes ‖ MK (32) ‖ 24 zero bytes. Kind 4 is the recovery entry. Unused entries are random bytes and never open.

4.3 The master key

1. RecoveryKey depends on the number of words:
   • 12 words: RecoveryKey = HKDF(salt = recovery salt, ikm = E, info = silo recovery v1).
   • 6 or 2 words: E is too short to be used directly, so it is first stretched with Argon2id: S = Argon2id(password = E (8 or 3 bytes), salt = recovery salt, passes and memory = the values in the keys.slot header, 32-byte output, version 1.3; libsodium crypto_pwhash with ALG_ARGON2ID13, PyNaCl nacl.pwhash.argon2id.kdf; a reader refuses passes outside 3–10 or memory outside 64–256 MiB). Then RecoveryKey = HKDF(salt = recovery salt, ikm = S, info = silo recovery short v1).
2. Try to open each of the 8 entries with RecoveryKey. The one that opens, has kind 4 and has zero reserved bytes holds the master key MK. If none opens, the phrase does not belong to this vault (or was given with the wrong number of words).

4.4 Subkeys

Subkey	Id	Used for
fileWrap	1	Unwraps the FileKey of item and thumb containers
index	2	Unwraps the FileKey of snapshot containers
dedup	3	Keys the plaintext hash
journal	6	Opens journal records

Each is Subkey(MK, id). Ids 4 and 5 exist but are not needed to read a vault.

5. Containers

Every original file, thumbnail and index snapshot is a container: a header, then the content in encrypted chunks.

5.1 Header

Offset	Length	Field
0	4	Magic SILO
4	1	Version = 1
5	1	Kind: 1 item (original), 2 thumb, 3 snapshot
6	2	Reserved, 0
8	4	Header length H
12	1	Chunk size exponent e, 10–24. Chunk size C = 2^e
13	3	Reserved, 0
16	16	Object id. For item and thumb it equals the file name
32	16	Chunk nonce prefix
48	72	Key block: nonce (24) ‖ AEAD(wrap key, FileKey) (48), AD = hex(bytes[0..48])
120	24	Metadata nonce
144	M + 16	AEAD(FileKey, metadata), AD = hex(bytes[0..144])

• H = 160 + M, where M is a multiple of 256 from 256 to 65,536. Reject anything else, and reject non-zero reserved bytes (the same holds for keys.slot).
• The wrap key is the fileWrap subkey for item and thumb, and the index subkey for snapshot.
• Header hash = H(bytes[0..H]).

5.2 Metadata

UTF-8 JSON padded with spaces to M bytes. Ignore unknown fields.

Field	Type	Meaning
size	integer, required	Content length in bytes
chunkCount	integer, required	Number of chunks n
plaintextHash	64 hex, required	Keyed hash of the content (5.4)
name, mime	string	Original file name and media type
takenAt, addedAt	number	Capture time, time added
width, height	number	Pixels
duration	number	Seconds
of	32 hex	Thumbnails only: object id of the original

5.3 Body

The body starts at offset H and is n chunks. Chunk i (0-based) is AEAD(FileKey, nonce, plaintext chunk, AD):

• nonce = nonce prefix (16) ‖ u64(i)
• AD = hex(header hash ‖ final), where final is the single byte 1 for the last chunk (i = n − 1) and 0 otherwise
• Every chunk except the last holds exactly C bytes of plaintext, so it is C + 16 bytes on disk. The last holds the rest (0 to C bytes; at least 1 when n > 1).

Concatenating the decrypted chunks gives the padded content P. The content is the first size bytes of P; the rest is zero-byte padding. A file that ends early, has bytes after the last chunk, or has a chunk that fails authentication is damaged.

A reader does not need the padding rule, because the file length gives the last chunk’s length: it is the body length minus (n − 1)(C + 16), minus 16. For reference, the writer pads size to:

• item and snapshot: Padmé. For L ≥ 2, E = floor(log2 L), S = floor(log2 E) + 1, and L is rounded up to a multiple of 2^(E−S). L < 2 is unchanged.
• thumb: a multiple of 16,384, at least 16,384.

Then n = max(1, ceil(P / C)).

5.4 Plaintext hash

Split the content (not the padding) into pieces of C bytes; k = max(1, ceil(size / C)) pieces (empty content is one empty piece). Then plaintextHash = HK(dedup subkey, "silo-tree-v1" ‖ u64(k) ‖ H(piece 0) ‖ … ‖ H(piece k−1)). A reader that checks it detects a container whose content was swapped or altered by someone holding the keys; the chunk tags already detect any other change.

6. Index

The index holds the album structure, favourites and trash. Without it a reader still gets every original with its file name from its metadata; with it, the reader knows the albums and which items were deleted.

6.1 State

{ "v": 1, "generation": N,
  "items":  [ { "id", "thumb", "name", "mime", "size", "plaintextHash", "addedAt",
                "takenAt"?, "width"?, "height"?, "duration"?,
                "motion"?: { "id", "size", "mime" },
                "albums": [albumId…], "favorite": bool, "trashedAt": ms|null } … ],
  "albums": [ { "id", "name", "createdAt", "order" } … ],
  "tombstones": [ [objectId, purgedAt] … ] }

An item’s id is the object id of its original; thumb is its thumbnail’s object id or null. motion, if present, marks a motion photo: the original is the still image and motion.id is the object id of a second container of kind item in objects/ that holds a short video of the moment; motion.size is that video’s content length and motion.mime its media type. The video belongs to the still: it is not an item of its own. It is deleted and tombstoned with it. Indexes written before the field existed have no motion and stay valid. order is the album’s position in the user’s list. Tombstones are ids of purged originals and thumbnails.

A state is valid only if every object has exactly the keys shown (? = optional) with these types: ids are 32 lowercase hex characters, plaintextHash is 64; name, mime are strings; size, width, height, order and motion.size are non-negative integers; motion has exactly the keys id, size and mime; times and duration are non-negative numbers; favorite is a boolean. Also: item ids, album ids and tombstone ids are each unique; every id in an item’s albums is an album; and no item’s id, thumb or motion.id is a tombstone. An item’s motion.id is different from its id and thumb. A snapshot whose state is not valid does not open.

6.2 Snapshot

index/snap.N.silo is a container of kind snapshot whose content is the state JSON, with generation = N.

6.3 Journal

index/wal.N.log = header ‖ record ‖ record …

• Header, 16 bytes: SILJ ‖ u8 version = 1 ‖ 3 zero bytes ‖ u64 generation (= N).
• Record: u32 L ‖ nonce (24) ‖ AEAD(journal subkey, payload) (L − 24 bytes), with AD = hex(u64(N) ‖ u64(seq)), where seq is the record’s position in this file, from 0. L is from 104 to 65,600.
• Payload: one change as UTF-8 JSON, padded with spaces to a multiple of 64 bytes. The field t names it:

t	Fields	Effect on the state
add	item	Adds the item
update	id, any of name, favorite, takenAt	Sets those fields; takenAt: null removes it
setAlbums	id, albums	Replaces the item’s album list
trash	id, at	Sets trashedAt = at
restore	id	Sets trashedAt = null
purge	id, at	Removes the item; adds tombstones [id, at] and, if it had them, [thumb, at] and [motion.id, at], each only if that id is not already a tombstone
albumCreate	album	Adds the album
albumRename	id, name	Renames it
albumDelete	id	Removes it, and removes its id from every item’s albums
albumOrder	ids	Sets each album’s order to its position in ids

A change that does not apply is skipped: an add whose id already exists or whose id, thumb or motion id is tombstoned; an unknown item or album id anywhere; trash of a trashed item; restore of an item that isn’t trashed; update with none of its fields; albumCreate with an existing id; albumOrder that is not a permutation of every album id. A record with unknown fields or wrong types is skipped too.

6.4 Reading the index

1. List index/. Snapshots are snap.N.silo, journals wal.N.log (ignore .damaged.log and .tmp).
2. Take the snapshot with the highest N that opens, whose JSON is valid and states generation N. That N is G. If none opens, there is no index: read the objects alone (7.2).
3. For each journal with N ≥ G, in ascending order, read records from the start and apply each in turn. Stop reading that file at the first record that is cut short, has a length outside 104–65,600, or fails authentication, keeping what was applied before it, and continue with the next file. A file with a wrong header contributes nothing.

7. Reading a vault

7.1 With the index

1. Get MK (section 4) and the subkeys.
2. Read the index (6.4).
3. For each item: open objects/SS/ID.silo where ID is the item’s id. Check its kind is item and its object id is ID, decrypt the chunks, keep size bytes, and check plaintextHash against the container metadata and the index. Its albums are the names of the albums in its albums; it is deleted if trashedAt is not null. If the item has motion, read the video the same way from objects/SS/motion.id.silo (its size must be motion.size; the index holds no hash for it, so the container’s own plaintextHash is the check) and give it to the user together with the still, for example as name.ext and name.mov.
4. Files in objects/ that no item names (as id or motion.id) and that aren’t tombstoned are originals the index does not know (an import interrupted by a crash, say). Read them as in 7.2. A motion video whose still is not in the index is such a file: it is read as a video of its own, and its still as another original, so nothing is lost but the pairing.

7.2 Without the index

Every file objects/SS/ID.silo is an original. Open it as in 7.1 step 3; its name, type and dates come from the container metadata.

8. Test vectors

The test vectors download holds, in fixtures/format-v1/, single containers and a keys.slot, with fixtures.json describing each (MK, subkeys, plaintexts). fixtures/vault-v1/ holds complete vaults written by the app’s code, each with an expected.json listing what a reader must produce. The recovery phrase of every fixture is abandon amount liar amount expire adjust cage candy arch gather drum business, and its MK is the bytes 00 01 02 … 1f.

9. Versions

A reader must reject a version byte it does not know rather than guess. New optional metadata fields and new journal record types may be added within version 1; readers ignore unknown metadata fields and skip unknown records.