Bringing the Unix Philosophy to Big Data by Bryan Cantrill: FutureStack13
New Relic2 HN recommenders · recommended in 2 months, 2014–2022
talk · intermediate · 2014–2022 on HN
Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.
HN recommends it as an elegant argument for simple Unix tools replacing heavyweight Hadoop-style pipelines.
Up next
3 mo on HNMoneyball - Firing people
The BigTime Group
3 HN recommender · 2019–2022
11 mo on HN"Testing Distributed Systems w/ Deterministic Simulation" by Will Wilson
Strange Loop Conference
12 HN recommender · 2014–2024
11 mo on HNThe Value of Values with Rich Hickey
InfoQ
10 HN recommender · 2014–2023
9 mo on HNAWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401)
Amazon Web Services
12 HN recommender · 2019–2025
9 mo on HN"Turning the database inside out with Apache Samza" by Martin Kleppmann
Strange Loop Conference
9 HN recommender · 2015–2025
7 mo on HNConflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016
GOTO Conferences
3 HN recommender · 2017–2020
6 mo on HN01 - Course Introduction & Relational Model (CMU Databases Systems / Fall 2019)
CMU Database Group
6 HN recommender · 2020–2023
6 mo on HNIntroduction to NoSQL • Martin Fowler • GOTO 2012
GOTO Conferences
6 HN recommender · 2015–2021
5 mo on HNRich Hickey rants about HttpServletRequest in 1080p: Death By Specificity from Clojure Made Simple
Fred Overflow
5 HN recommender · 2020–2026
6 mo on HNTigerStyle! (Or How To Design Safer Systems in Less Time) by Joran Dirk Greef
TigerBeetle
2 HN recommender · 2023–2025
5 mo on HNBryan Cantrill from Joyent on Manta: internet-facing object storage facility that features compute
AI Council
4 HN recommender · 2014–2026
5 mo on HNMichael Stonebraker | Big Data is (at least) Four Different Problems
Michigan Engineering
5 HN recommender · 2015–2024
3 comments from Hacker News
This article echoes a talk Bryan Cantrill gave two years ago: ▶ this video It's about how Joyent took the concept of a UNIX pipeline as a true powertool and built a distributed version atop an object filesystem with some little map/reduce syntactic sugar to replace Hadoop jobs with pipelines. The Bryan Cantrill talk is definitely worth your time, but you can get an understanding of Manta with their 3m screencast: link
They're good points, but from the philosophical point The Unix Way , systemd leaves some things to desire. Is there a modern Unix way? I don't see a lot of exploring in this area/side, but I see a lot of this is the way it's gonna be from now on on the other side. I'm not on either side, but I appreciate the way some complexity is dealt with simple little things. So elegantly solving problems: ▶ this video
1.) Be curious. Stay curious. Experiment, play with things. Your time during university is likely the best time for this and it will likely get harder later on when you're on the job and there's much more emphasis on finished work than on learning experiences. Also, I'm quite sure that it will be beneficial for you if you familiarize yourself with different ways of thinking (obviously programming languages, environments and paradigms, but that will likely apply to anything non-technical, as well). I gained this perspective from dealing with people who didn't really do that, but also from knowing that I personally also did this way too late (and from the benefits it brought me nonetheless). If you're only familiar with Windows, try Linux. Try BSD. Try OS X, if you want to. Try working in the terminal. Familiarize yourself with different editors (especially Vim and Emacs, maybe also newer ones like Kakoune). Learn bash, have a look at AWK and sed. Maybe learn Perl. Try out functional programming, logic programming (like Prolog) and so on and so forth. If you're feeling adventurous, play with niche systems like Haiku[1] or TempleOS[2], just to gain new insights on how things could work. Maybe try the ACME editor[3]. [1]: link [2]: link [3]: link Of course, there might be courses that will nudge towards some of those tools. If you visit some of those: Cool! But if there are no such courses (I didn't encounter many of them myself), try to explore these things nonetheless. Ev