AI will save the world: Positive view of the AI development. Interesting the attack to China/Karl Marx at the end. In general I feel confident this will be good.
Nutanix kernel upgrade story: This is a bit hardcore for me (and looks a bit old from 2021) but still quite interesting how they did the troubleshooting.
GPU programming: I have never read about how to code for a GPU and this looks interesting and quite different from what I would do in a CPU. From the “Execution Model of the GPU” I started to lose track. Still is nice to see a summary at the end and resources/books.
Meta GenAI infra: link. Interesting they have built two cluster one Ethernet and the other Infiniband, both without bottlenecks. I don’t understand if Gran Teton is where they install the NVIDIA GPUs? And for storage, I would expect something based on ZFS or similar. For performance, “We also optimized our network routing strategy”. And it is critical the “debuggability” for a system of this size. How quick you can detect a faulty cable, port, gpu, etc?
Oracle RDMA: This is an ethernet deployment with RDMA. The interesting part is the development DC-QCN (some ECN enhancement)
Cerebras WSE-3: Looks like outside NVIDIA and AMD, this is the only other option. I wonder how much you need to change your code to work in this setup? They say it is easier… I like the pictures about the cooling and racks.
Co-packaged optics: Interesting to see if this becomes a new “normal”. No more flapping links anybody? It is the fiber or replace the whole switch….
I have been watching several videos lately and I would like to be able to get a tool to give a quick summary of the video so I can have notes (and check if the tool is good). Some tools: summarize.tech, sumtubeai
I hit rock bottom this week. I hope I finally closed one door in my life so I give myself the chance to open others. Made the wrong decision? It is easy when you look back. Do I regret it? The most annoying thing is these are failures so you can’t go back and recover. But I was so bloody newbie!!!…. At least after 5 years…
“For every reason it’s not possible, there are hundreds of people who have faced the same circumstances and succeeded.” Jack Canfield
Head down, crying, cursing, whatever, but forwards. As it has always been.
—-
Somehow managed to list to long videos, something I normally can’t manage (because lack of time, etc)
Negative Beliefs, avoid bitterness, aim for greatness (remarkable things), scape the darkness: Jordan B Peterson with Modern Wisdom: video, podcast.
Find and keep Love: video. 1st Get your shit together. Communication is critical. Be careful with your shopping list….
Using gNOI capabilities to simplify software upgrade use case: video – I had to idea about gNOI so looks interesting. It is crazy that still in XXI, automating a network device is so painful. Thanks to all vendors to make your life miserable.
Go lang for network engineers: videoslides– I always thought that Golang had a massive potential for network automation but there was always lack of support and python is the king. So nice to see that Arista has things to offer.
There are more things, but havent had the chance to review them.
—-
It looks there is new chatbot that is not using the standard NVIDIA GPU. Groq uses LPU (Language Processing Unit). And they say it is better than a GPU. They have this paper but I can’t really see feature of that LPU.
Slurp’it: Show this blog, and the product looks interesting but although is free, it is not opensource and at the end of they you dont want a new vendor-lockin
Container lab in kubernetes: Clabernetes. I would like to play with this one day.
NetDev0x17: videos and sessions. link This is quite low details and most of the time beyond my knowledge. Again, something to take a look at some point.
LLM from scratch: repo. Looks very interesting. But the book it is going to take a long time to hit the market.
AWS Reinvent Intent-Driven Network Infra: Interesting video about Intent-driven networking in AWS. This is the paper he shows in the presentation. Same note as last year, leaf-spine, pizza boxes, all home made. The development of the SIDR as the control plane for scale. And somehow the talk about UltraCluster for AI (20k+ GPU). Maybe that is related to this collaboration NVIDIA-AWS. Interesting that there is no mention to QoS, he said no oversubscription. In general, everything is high level, and done in-house, and very likely they facing problems that very few companies in the world are facing. Still would be nice to open all those techs (like Google has done – but never for network infra). As well, I think he hits the nail on the head how he defines himself from Network Engineer to Technologist, as at the end of the day, you touch all topics.
Google view after 18 years: Very nice read about the culture shift in the company, from do not evil, to make lots of many at any cost.
GTP-Crawler: Negative thing, you need the pay version of chatgpt. I wonder, If I crawke cisco, juniper and arista, what would be nearly all network knowledge in the planet? If that crawler can get ALL that date.
VimGTP: Very interesting project. I haven’t used it. But thinking aloud, you could use it to interact with sites that dont have API (couriers)? I think with Selenium you can do things like that?
Maia AI: CLoud providers like to be masters of their own destiny so try to build as many things by themselves as possible. So now MS has developed its GPU for AI. It is quite interesting the custom rack they had to built with the sidekick for cooling down the new chips. There are no many figures about the chip (5nm, 105b transistors) to compare with other things in the market.
Reptar: new Intel CPU vulnerability. It looks like is a feature from Ice Lake architecture. It looks like you can crash the cores but no yet take over. Still interesting.
I am not affected 🙂
$ grep fsrm /proc/cpuinfo $
Mellanox with Debian: Interesting how you can install a nearly standard Debian into a Mellanox SN2700 switch.
RISC-V into datacenter: Happy to see RISC-V chips in the datacenter. But not clear who is going to use them?
Mirai history: I think most of wired articles read like a holywood movie 🙂 Although 2016 security issues are “old” school, still interesting how teenagers got that far.
Mojo: Interesting because of the people behind of it… really impressive.
Moor’s law analysis: I liked the part about networks, that is not very common mentioned in these type of analysis.
From the AlphaSignal email list, that most of the times go over my lame knowledge, I found this piece of info, quite interesting:
FP8-LM: Training FP8 Large Language Models
Goal: Optimize LLM training with FP8 low-bit data formats. Issue: High cost of LLM computational resources. Solution: FP8 automatic mixed-precision framework for LLMs. Results: Reduced memory by 42%, increased speed by 64%. Insight: FP8 maintains accuracy, optimizes training efficiency.
This is something I want to really understand at one point. FP (Floating-Point) instructions can be from several sizes (8, 16, 32, 64). So the bigger, the better precision. I guess for some scientific tasks that is important. But looks like for AI, with FP8 could be good enough.
Reading across this blog, I came to this statement:
What limits computer performance today is predictability, and the two big ones are instruction/branch predictability, and data locality.
That is from this interview. I dont kown Jim Keller but it is a long and interesting conversation. I liked it when he says he was the laziest person at Tesla!
I didnt know anything about this conference until the last two months started to read news about it from different blogs. I am surprised the webpage doesnt link to the videos. This one is quite interesting but after the 15 minute becomes very hardcode for me.
Good article about LLM from the hardware/networks perspective. I liked it wasnt a show-off from Juniper products, as I haven’t seen any mention of Juniper kit in deployments of LLM in cloud providers, hyperscalers, etc. The points about Infiniband (the comment at the end about the misconceptions of IB is funny) and ethernet were not new but I liked the VOQ reference.
Still as a network engineer, I feel I am missing something about how to make the best network deployment for training LLM.
So NVIDIA has an AI supercomputer via this. Meta, Google and MS making comments about it. And based on this, it is a 24 racks setup using 900GBps NVLink-C2C interface, so no ethernet and no infiniband. Here, there is a bit more info about NVLink:
NVLink Switch System forms a two-level, non-blocking, fat-tree NVLink fabric to fully connect 256 Grace Hopper Superchips in a DGX GH200 system. Every GPU in DGX GH200 can access the memory of other GPUs and extended GPU memory of all NVIDIA Grace CPUs at 900 GBps.
This is the official page for NVlink but only with the above I understood this is like a “new” switching infrastructure.
But looks like if you want to connect up those supercomputers, you need to use infiniband. And again power/cooling is a important subject.
