ZFS Native Encryption by Tom Caputi
OpenZFS2 HN recommenders · recommended in 2 months, 2016–2022
talk · advanced · 2016–2022 on HN
Tom Caputi presents the design of native encryption for ZFS at the OpenZFS Developer Summit, including design decisions and how authentication uses checksum fields.
Recommended whenever ZFS encryption comes up as the authoritative explanation of how it was added and its trade-offs at scale.
Up next
12 mo on HNLexical Scanning in Go - Rob Pike
GoogleFOSSSydney
13 HN recommender · 2012–2025
11 mo on HN"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
12 HN recommender · 2014–2024
11 mo on HNThe Value of Values with Rich Hickey
InfoQ
10 HN recommender · 2014–2023
9 mo on HNAWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401)
Amazon Web Services
12 HN recommender · 2019–2025
9 mo on HN"Turning the database inside out with Apache Samza" by Martin Kleppmann
Strange Loop Conference
9 HN recommender · 2015–2025
7 mo on HNConflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016
GOTO Conferences
3 HN recommender · 2017–2020
6 mo on HN01 - Course Introduction & Relational Model (CMU Databases Systems / Fall 2019)
CMU Database Group
6 HN recommender · 2020–2023
6 mo on HNIntroduction to NoSQL • Martin Fowler • GOTO 2012
GOTO Conferences
6 HN recommender · 2015–2021
5 mo on HNRich Hickey rants about HttpServletRequest in 1080p: Death By Specificity from Clojure Made Simple
Fred Overflow
5 HN recommender · 2020–2026
6 mo on HNTigerStyle! (Or How To Design Safer Systems in Less Time) by Joran Dirk Greef
TigerBeetle
2 HN recommender · 2023–2025
5 mo on HNBryan Cantrill from Joyent on Manta: internet-facing object storage facility that features compute
AI Council
4 HN recommender · 2014–2026
5 mo on HNMichael Stonebraker | Big Data is (at least) Four Different Problems
Michigan Engineering
5 HN recommender · 2015–2024
5 comments from Hacker News
The problem is that a stream cipher is going to have some per-object uniqueness (a salt, IV, etc.), so by design even if you feed it related input blocks you will get different output blocks. This is, of course, antithetical to deduplication: so you need to check/store the hash of the input before it goes through the cipher. The presentation about ZFS' native encryption[1] covers many of these sorts of trade-offs necessary to do full-disk encryption at scale. [1]: ▶ this video
On the contrary, it makes it suitable for file systems! File systems are not block devices, they are data structures on top of block devices — it's the job of these data structures to keep stuff, such as data, inodes, and... keys, and IVs, and MACs, checksums, etc. TFS uses Speck in XEX mode (edit: I just realized you're the author, so you already know that :). tptacek wrote a nice post about it: link and I quote: "If you’re encrypting a filesystem and not disk blocks, still don’t use XTS! Filesystems have format-awareness and flexibility. Filesystems can do a much better job of encrypting a disk than simulated hardware encryption can." Edit: check out this presentation on how encryption was bolted on ZFS: ▶ this video (slides: link ) It's not perfect, but it provides data authentication by reusing checksum fields for storing MACs. Edit 2: also check out bcachefs encryption design doc: link (also not perfect, but uses proper AEAD — ChaCha20-Poly1305. I sent some questions and suggestions to the author, but received no reply :/)
I work with Tom at Datto. He presented all of his design decisions at the OpenZFS developer conference here: link Slides: link Video: ▶ this video
Native ZFS encryption may become available soon. It is pending code review: link There was a presentation of this by Tom Caputi at the most recent OpenZFS Developer Summit. Slides: link Video: ▶ this video Conference: link
This is still geli+zfs, and not native zfs encryption that was recently presented in OpenZFS Developer Summit [0]. [0] ▶ this video