Today I Learned
This is the second part of the Expanse series.
Now I understand the name of Rocinante for the spaceship. The guys are like Don Quijote trying to beat giants (or space monsters)
The book is engaging and I devour it in a couple of days but I felt it was the same setup as the first one.
I will read the next one, it is in my to-read list.
Dating summary: Really good all points, most of them I feel them or experienced.
Backwards Law: Emotions are weather, you can’t control them. You can control what you do with the emotions
You never know who you are until you have adversity in life.
SR-IOV (Single Root I/O Virtualization) is an extension to the PCI Express specification that lets a single physical device, like a network adapter, present itself as multiple separate virtual devices. Each virtual machine gets its own direct access to the hardware, bypassing the hypervisor’s software layer and delivering network performance close to what you’d get on bare metal. It was designed by the PCI-SIG standards body specifically to solve the performance bottleneck that virtualization creates when many VMs share one network card.
From this article about Meta Superintelligence, came across this link about Meta backbone designs at scale for EBB
opensm: I can’t find many blog posts with real hands-on about IB things. And this is a good one. It doesnt just show you the configuration, but the problems you can find. It seems it is better to use the Mellanox’s opensm
IB cable EEPROM: Interesting how critical can be having the correct cable from the EEPROM.
The label on a cable is marketing. The EEPROM inside it is a contract, and InfiniBand only honors contracts.
Veritasium: 1 billion heartbeats
Huawei AI Cloud: Power hungry, all optics, etc. Interesting take from China to NVIDIA. And even more interesting, how to fence off all the tariffs and restrictions…
Google Ironwood TPUv7: “It scales up to 9,216 liquid cooled chips linked with breakthrough Inter-Chip Interconnect (ICI) networking spanning nearly 10 MW” I wonder how is the network… but doesnt give low level details, just marketing.
TV garden: TVs from around the world…. just in case you want to learn a language?
Hacker Laws: So many I dont know
Daylight: Looks so nice!!! And it seems it can read kindle books. Tempting
NVIDIA Photonics: I read about co-packaged from some Sherada post’s… but I didn’t see it coming so fast in production. With my network operations hat on…. how is the troubleshooting done? It the part where the fiber breaks, you replace the whole device? I guess this has been thought very deeply.
The power consumed by optics in the network is enormous and so is the capital expense. Anecdotally, we have heard it said many times that the majority of the cost in a datacenter-scale cluster is in the optical transceivers at both ends of a link and the fiber optic cable between them. Some the pieces that link switches to network interface cards is 75 percent to 80 percent of the cost of a network, with the switches and the NICs making up the other 20 percent to 25 percent.
Xsight: Another network silicon vendor. The article mentions Tofino P4.. I hope doesn’t end that way. I didn’t know anything about Avigdor Willenz
In part, that expectation for big change comes from the fact that Avigdor Willenz is the company’s founding investor. Willenz founded Galileo Technology, a maker of Ethernet switch ASICs that sold to Marvell in 2001 for $2.7 billion, and that wealth has been spread around. Willenz invested in Annapurna Labs, which sold to Amazon Web Services in 2015 for $350 million and which has created its Nitro DPUs, Graviton CPUs, and Trainium and Inferentia AI engines. He was president (now chairman) and first investor in distributed flash block storage maker Lightbits Labs. Willenz was a co-founder of AI chip maker Habana Labs, which sold to Intel in 2019 for $2 billion and is the foundation of its Gaudi compute engine line.
Finger Strength: “I’ve never seen strength like this before” true story
TFCC wrist injury: part of life…
Google TPUv6 Analysis: “… cloud infrastructure and which also is being tuned up by Google and Nvidia to run Google’s preferred JAX framework (written in Python) and its XLA cross-platform compiler, which speaks both TPU and GPU fluently.” So I guess this is a cross-compiler for CUDA?
“The A3 Ultra instances will be coming out “later this year,” and they will include Google’s own “Titanium” offload engine paired with Nvidia ConnectX-7 SmartNICs, which will have 3.2 Tb/sec of bandwidth interconnecting GPUs in the cluster using Google’s switching tweaks to RoCE Ethernet.” So again custom ethernet tweaks for RoCE, I hope it makes to the UEC? Not sure I understand having a Titanium offload and a connectx-7, are they not the same?
Alphafold: It is open to be used. Haven’t read properly the license.
Open Interpreter: The next step in LLMs is to control/interact with your system.
In my laptop fails because I have the free version 🙁 need to try a different one, but looks promising!
open-interpreter main$ interpreter --model gpt-3.5-turbo
●
Welcome to Open Interpreter.
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
▌ OpenAI API key not found
To use gpt-4o (recommended) please provide an OpenAI API key.
To use another language model, run interpreter --local or consult the documentation at docs.openinterpreter.com.
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
OpenAI API key: ********************************************************************************************************************************************************************
Tip: To save this key for later, run one of the following and then restart your terminal.
MacOS: echo 'export OPENAI_API_KEY=your_api_key' >> ~/.zshrc
Linux: echo 'export OPENAI_API_KEY=your_api_key' >> ~/.bashrc
Windows: setx OPENAI_API_KEY your_api_key
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
▌ Model set to gpt-3.5-turbo
Open Interpreter will require approval before running code.
Use interpreter -y to bypass this.
Press CTRL-C to exit.
> what is my os?
Traceback (most recent call last):
Walkie-Talkies: Out of James Bond world.
Zero Trust SSH. From Cloudflare. And this video I watched some months ago (and it is already 4y).
Finger Strength: I follow similar protocol, although not everyday, for warm up and I think it works. I am not getting that super results but at least my fingers are stronger…. and I am not getting injuries!!!! \o/
Cisco AI/ML DC Infra Challenges: I am not quiet fan of Cisco products but this is a good overview.
Key points:
Videos:
Social Connection: Long video but worth it. Listen to it in several parts. Send a text to your close friends, connections… daily. Oh, so hard.
Some other notes (copied too)
Stress: boosts immune system short term, crashes brain+body long term.
Sleep. Water. Movement. Sun, Meditation. Rest. Rhythm. Love. Gratitude.
Porn and dopamine: scary
“If I go to my phone, I’m a consumer. If I go to my journal, I’m a creator.”
Optical Protocols for Studying and learning: This is a current video and some months ago I read something similar, but I dont mind to repeat it. It is very good. And a summary
Good sleep, dedicated time, focus, meditations to improve focus, similar time of day, teach it, test it frequently, create engagement. Reduce knowledge decay a.k.a = learning
Cloudflare backbone 2024: Everything very high level. 500% backbone capacity increase since 2021. Use of MPLS + SR-TE. Would be interesting to see how the operate/automate those many PoPs.
Cisco AI: “three of the top four hyperscalers deploying our Ethernet AI fabric” I assume it is Google, Microsoft and Meta? AWS is the forth and biggest.
Huawei Cloud Monitor: Haven’t read the paper RD-Probe. I would expect a git repo with the code 🙂 And refers to AWS pdf and video.
Automated Leetcode: One day, I should have time to use it a learn more programming, although AI can solve them quicker than me 🙂
Alibaba Cloud HPN: linkedin, paper, AIDC material
LLM Traffic Pattern: periodically burst flows, few flows (LB harder)
Sensitive to failures: GPU, link, switch, etc
Limitations of Traditional Clos: ECMP (hash polarization) and SPOF in TORs
HPN goals:
-Scalability: up to 100k GPU
-Performance: low latency (minimum amount of hops) and maximum network utilization
-Reliability: Use two TORs with LACP from the host.
Tier1
– Use single-chip switch 51.2Tbps. They are more reliable. Dual TOR
– 1k GPUs in a segment (like nv-link) Rail-optimized network
Tier2: Eliminating load imbalance: Using dual plane. It has oversubscription
Tier3: connects several pod. Can reach 100k GPUs. Independent front-end network
Altman Universal Base Income Study: It doesnt fixt all problems, but in my opinion, it helps, and it is a good direction.
xAI 100k GPU cluster: 100k liquid-cooled H100s on single RDMA fabric. Looks like Supermicro involved for servers and Juniper only front-end network. NVIDIA provides all ethernet switches with Spectrum-4. Very interesting. Confirmation from NVIDIA (Spectrum used = Ethernet). More details with a video.
Github access deleted data: Didn’t know about it. Interesting and scary.
Nvidia DGX SuperPod: reference architecture. video. 1 pod is 16 racks with 4 DGX each (128×8=1024 GPU per pod), 2xIB fabric: compute + storage, fat tree, rail-optimized, liquid cooling. 32k GPU fills a DC.
How SSH works: So powerful, and I am still so clueless about it
Chips and Cheese GH200: Nice analysis for Nvidia Grace CPU (ARM Neoverse) and Hopper H100 GPU
Origins of deep learning: interesting post. At the beginning all was Matlab and CPU bounded. repo
A16: 1.6nm process for 2026. More frequency, less power.
xz vulnerability repo: Something I need to check in the VP
Turing Award 2024: zero-knowledge-proof.
Cloudflare and Let’s Encrypt’s certificate change: I haven’t heard of this until recently. I use Let’s Encrypt so as far as I can read, makes sense what they are doing. But didnt know 2% Cloudflare customer were using the “cert”
ChatDev: Communicate agents for software development. I am a not a developer but I would use this just as a starting point If I have any idea for a project. I would remove the C-suite agents, at least for low level projects.
IB vs Ethernet: A bit of bias here (the author is from Broadcom -> Ethernet). I have no hands-on experience with IB, but I have read the cables are not cheap… Let’s see when UltraEthernet gets into the market. Another view.
Slingshot and Juniper: A bit of bias again as HP bought Juniper. So how will these interconnects fade inside the company? As far as I know, most supercomputers use some “special” interconnect so not much ethernet there. But the money nowadays is in AI infra… Paper for slingshot (haven’t read it)
Tailscale SSH, wireguard throughput: These are things I should a spend a bit of time one day and consider if I should use them (I dont like it is not opensource though). This netmaker?
Videos:
Jocko Willink: Discipline = Freedom. Remember but not dwell. Good leader, delegate. Be a man -> take action, bonding (pick your activity)
Jimmy Carr: Imposter syndrome each 18 months, so you have to stand-up. People crave the success not the journey. Teaching comedy good for communicating.
Sam Altman – Stanford 2024: First time I see him talking. It has some funny moments. More powerful computers. I missed a question about opensource LLM and closed ones.
Find a girlfriend: I know just a little bit about the person (I want to read one of his books) from other books and videos. I would think he would have already a girlfriend or family. From the three methods, definitely, the face to face approach in the street looks so much better (and that’s what I would like to do)
Jordan Peterson original 42 rules
CNI performance: I have used kubernetes since I studied for CKAD but still I am interested in the networks side. I didn’t know about Kube-router and it did great! I am bit surprised with Calico as I have read more and more about Cilium.
Cilium for network engineers. I have to read this fully (worried that Cisco bought it…)
rsync go: Interesting talk about rsync, as it explains how it works and it is something I didnt know. But then, all other things/projects mentioned are cool and related. I need to try to install rsync go in my vm. ccc slides and repo
NASA to the moon: This is an engaging and provocative video regarding the Artemis III (project back to the moon II). He makes some hard questions to the people in charge (I have no clue about physics) and it seems he has a point. Not sure it this will get any effect but again, looks “smart”. When he mention the NASA SP287 (What made Apollo a success) document as the grial for going back to the moon, I wanted to get a copy (here) so I could read it one day.
Git options: Nice post about popular git config options. I am a very basic git user (and still sometimes I screw up) but the options to improve diff looks interesting so I will give it a go at work.
Undersea cable failures in Africa: It is clear that Africa relays heavily in submarine cables (it doesnt look like there are many cable systems intra continent). And the Red Sea is becoming a hot area due to different conflicts…
Quotes: I like the ones regarding simplicity:
A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. (John Gall)
In programming, simplicity and clarity are a crucial matter that decides between success and failure. (Edsger Dijktra)
Log4j: This is old news but when it came out I tried to run the PoC but I failed 🙁 This is just a reminder. It was annoying because I manged to install all tools but never managed to exploit it.
Done List: I feel totally identified. The to-do list is never done and you feel guilty. Done-list, much healthier.
Dan Lynch: He passed away, and as usual on my ignorance, it seems he is one of the unsung heroes of Internet, migrating ARPANET to TCP/IP.
Systems-Based Productivity: TEMPO refers to five dimensions of productivity: T (Time Management), E (Energy Management), M (Mindset), P (Proficiency) and O (Organization).
Run Africa: very jealous.
InfraOps challenge: A bit beyond me, but interesting If you could try without applying for the job.
Devika: Agent AI. Another thing I would like to have time to play with it. If you have API keys for some LLMs, looks like it shouldn’t be difficult to run and you dont need a powerful laptop (?)
Daytona: My development environment is a joke, just python envs. But I guess for more serious devs, could be interesting
NTP and year 2038: Agree, when it is not DNS, it is likely NTP (seen this with VPNs and SSL certs in boxes with NTP unsync), or something blocking UDP.
Linux crisis tools: I haven’t got my hands dirty with BPF but I am surprised with so many tools. I would add nc, netstat, lsof, traceroute, ping, vim, openssl etc but because I do pure networks.
Jim Kwik: How to improve your reading speed. One improvement is you use your finger or a ruler. Need to watch again.
Rich Roll: The guy is super chill. I would like to be able to do some ultra at some point in life… Very personal conversation.
Ferran Adria: I didnt know much about the person apart from being one of the best Chefs in history. I like how he starts the interview and take over for 15 minutes. Haven’t watched till the end. But just the beginning is priceless.
Mark Manson: I have read all his books and his emails. Interesting his story.
Chocolonely: I didnt know it was a dutch company and interesting history behind. I want to try one day, but I haven’t found a dark choco version.
LLM in 1000 lines puce C: I was always crap at C. But interesting this project as something educational and intro in LLM.
Visual intro to transformers: The easy joke, unfortunately, this is not about Optimus Prime.
Indonesia Heavy Metal Girls: Unexpected. Respect.
Enforce-first-as: I dint know about this until last week. Cisco defined by default. Juniper disabled by default. And this makes sense with Route Servers.
Love Languages: I read this book in 2018. The conclusion I took at that time (and a bit late…) it is that you have to F*! communicate…
Interesting story about imposter syndrome:
We’d like to believe that if we only had the adulation, market success, and fan support of superstars like these, then we’d finally be comfortable and able to do our best.
In fact, it seems the opposite is true. Imposter syndrome shows up because we are imposters, imposters acting ‘as if’ in search of making something better.
Perhaps the best plan is to show up and not walk out.
Self-compassion: Something I have learnt the hard way, and I think at the beginning works but long term doesn’t. I practice it often while climbing and honestly, I feel the difference, and sometimes is mindblowing. Nobody is going to cheer me up so I better off doing it myself.
GTC-2024: Like last year, I registered to watch some conferences. As a network engineer, I haven’t been able to see any (good) recording, just pdfs…. so quite disappointing. This is a summary from somebody that was on site and says it was great. And some other notes that they look interesting: keynote (nvlink and infiniband at 800G), nvdia dgx gb200 (indeed we need nuclear energy to feed all this…)
Juniper Express 5: Looks quite an interesting ASIC. But as far as I can see most ASICs for DC and AI/ML come from Broadcom and the main players are Cisco/Arista. I like the feature of deep buffers.. this is still a bit of a religious dilema… deep vs shallow buffers. And looks like it was announced in HotChips 2022.. so it is not very new? And only in PTX platform. What is the future of QFX?