rsync go, NASA SP287, git options, Undersea cable failures in Africa, Quotes, Log4j, done list, Dan Lynch, Systems-based Productivity, Run Africa

rsync go: Interesting talk about rsync, as it explains how it works and it is something I didnt know. But then, all other things/projects mentioned are cool and related. I need to try to install rsync go in my vm. ccc slides and repo

NASA to the moon: This is an engaging and provocative video regarding the Artemis III (project back to the moon II). He makes some hard questions to the people in charge (I have no clue about physics) and it seems he has a point. Not sure it this will get any effect but again, looks “smart”. When he mention the NASA SP287 (What made Apollo a success) document as the grial for going back to the moon, I wanted to get a copy (here) so I could read it one day.

Git options: Nice post about popular git config options. I am a very basic git user (and still sometimes I screw up) but the options to improve diff looks interesting so I will give it a go at work.

Undersea cable failures in Africa: It is clear that Africa relays heavily in submarine cables (it doesnt look like there are many cable systems intra continent). And the Red Sea is becoming a hot area due to different conflicts…

Quotes: I like the ones regarding simplicity:

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. (John Gall)

In programming, simplicity and clarity are a crucial matter that decides between success and failure. (Edsger Dijktra)

Log4j: This is old news but when it came out I tried to run the PoC but I failed 🙁 This is just a reminder. It was annoying because I manged to install all tools but never managed to exploit it.

Done List: I feel totally identified. The to-do list is never done and you feel guilty. Done-list, much healthier.

Dan Lynch: He passed away, and as usual on my ignorance, it seems he is one of the unsung heroes of Internet, migrating ARPANET to TCP/IP.

Systems-Based Productivity: TEMPO refers to five dimensions of productivity: T (Time Management), E (Energy Management), M (Mindset), P (Proficiency) and O (Organization).

Run Africa: very jealous.

Infraops challenge, Devika, Daytona, NTP 2038, Linux Crisis Tools, videos, Chocolonely, LLM, Transformers, Enforce-first

InfraOps challenge: A bit beyond me, but interesting If you could try without applying for the job.

Devika: Agent AI. Another thing I would like to have time to play with it. If you have API keys for some LLMs, looks like it shouldn’t be difficult to run and you dont need a powerful laptop (?)

Daytona: My development environment is a joke, just python envs. But I guess for more serious devs, could be interesting

NTP and year 2038: Agree, when it is not DNS, it is likely NTP (seen this with VPNs and SSL certs in boxes with NTP unsync), or something blocking UDP.

Linux crisis tools: I haven’t got my hands dirty with BPF but I am surprised with so many tools. I would add nc, netstat, lsof, traceroute, ping, vim, openssl etc but because I do pure networks.

Jim Kwik: How to improve your reading speed. One improvement is you use your finger or a ruler. Need to watch again.

Rich Roll: The guy is super chill. I would like to be able to do some ultra at some point in life… Very personal conversation.

Ferran Adria: I didnt know much about the person apart from being one of the best Chefs in history. I like how he starts the interview and take over for 15 minutes. Haven’t watched till the end. But just the beginning is priceless.

Mark Manson: I have read all his books and his emails. Interesting his story.

Chocolonely: I didnt know it was a dutch company and interesting history behind. I want to try one day, but I haven’t found a dark choco version.

LLM in 1000 lines puce C: I was always crap at C. But interesting this project as something educational and intro in LLM.

Visual intro to transformers: The easy joke, unfortunately, this is not about Optimus Prime.

Indonesia Heavy Metal Girls: Unexpected. Respect.

Enforce-first-as: I dint know about this until last week. Cisco defined by default. Juniper disabled by default. And this makes sense with Route Servers.

Feta and Spinach Filo Pie

I haven’t done this for a long time so it was due. I used to use his book quite often. Recipe:

Ingredients:

100g nuts (I used sunflower and pumpkin nuts)
5 large eggs
300g feta cheese – crumbled
50g grated cheese
oregano
1 lemon zest
olive oil
1 knob of butter
500g fresh spinach
1 x 270g pack of filo pastry
cayenne pepper
nutmeg for grating

Process:

  • Preheat the oven to 200°C
  • Toast the nuts in a large ovenproof frying pan over a medium heat until golden.
  • Crack the eggs into a large mixing bowl add feta, grated cheese, oregano and pepper
  • Add the toasted nuts and mix.
  • In the same frying pan, add a bit of olive oil and add half the spinach. Stir until wilted.
  • Add the lemon zest, grated nutmeg and piece of butter to the spinach.
  • Add the rest of spinach. Stir until all is wilted.
  • Then add the spinach to the egg mix. And clean the frying pan.
  • Take a piece of ovenproof paper roughly 1.5 bigger than your pan. Pass it through water so it is wet.
  • Lay it out on a clean work surface, rub lightly with oil and flatten out again.
  • Keep adding sheets of file on top of the ovenproof paper. Add a bit of oil and other spices in each layer. The idea is to cover the frying pan later. I used 6 layers in total.
  • Move the ovenproof paper and filo to the frying pan. Pour the egg/spinach mix. Add a bit more grated cheese.
  • Close the pie folding the edges (as it should be 1.5 bigger than the pan).
  • Fry at medium heat the pie so the bottom is crunchy. 1-2 minutes. Dont burn it!
  • Add some spices and olive oil on top.
  • Then move the pan to the oven for 20 minutes aprox or golden and crisp.

Quite happy with the result:

Narconomics

Very interesting book. It explains the mechanics of a drug cartel from the point of view of economics. I didnt think issues like supply chain, HR/PR, competition/merges, offshoring, R&D, online business, diversification, etc were part of drug cartel, as you only think of those as part of a licit business. There were many things that I didn’t know like the birth of “legal highs” in NZ (and Matt Bowden)

The goal is to fully understand the “business” because the current laws/actions, etc against drugs are clearly not working. So this way you can really offer a different approach to tackle the issue. You are not going to destroy them 100%. Most of the actions are at the source of the drug business: growing the plant (decrease in growing area causes minimum increase in retail price). But the book shows that is not effective and prevention (done in the consumer’s land: like rehab, education in jails, etc) is much more productive (for the same investment). As well as legalization (ie marijuana) as that brings control (“safer drugs”, tax revenue, etc) and put out of the market the dealers/cartels.

This is a difficult pill to swallow (punt intended) for governments and citizens but the writing is in the wall.

GPU Fabrics, Optimizations, Network Acceleration, Learning Cambridge, British Library

Several posts worth reading. There are plenty of things go over my knowledge. I already posted this, it is a good refresher.

GPU Fabrics: The first of the article is the one I am more lost as it about training and the communications between the GPU depending on the take to handle the models. There are several references to improvements as the use of FP8 and different topologies. As well, a bit more clear about NVLink (as internal switch for connecting GPUs inside the same server or rack)

When it moved to the inter-server traffic, I started to understand a bit more things like “rail-optimized” (it is like having a “plane” for my old job where the leaf only connects to a spine instead of all spines, in this case each GPU connects to just one leaf. If you cluster is bigger then you need spines). I am not keen of modular chassis from operations point of view but it is mentioned as an option. Fat-tree CLOS, Dragon-Fly: reminds me to Infiniband. Like all RDMA.

And Fabric congestion it is a big topic with many different approaches: adaptive LB (IB again), several congestion control protocols and mention to Google (CSIG) and Amazon (SDR) implementations.

In general I liked the article because I dont really feel any bias (she works for Juniper) and it is very open with the solutions from different players.

LLM Inference – HW/SW Optimizations: It is interesting the explanation about LLM inferencing (doubt I can’t explain it though) and all different optimizations. The hw optimization (different custom hw solutions vs GPU) section was a bit more familiar. My summary is you dont need the same infrastructure (and cost) for doing inference and there is an interest for companies to own that as it should be better and cheaper than hosting with somebody else.

Network Acceleration for AI/ML workloads: Nice to have a summary of the different “collectives”. “collectives” refer to a set of operations involving communication among a group of processing nodes (like GPUs) to perform coordinated tasks. For example, NCCL (Nvidia Collective Communication Library) efficiently implements the collective operations designed for their GPU architecture. When a model is partitioned across a set of GPUs, NCCL manages all communication between them. Network switches can help offload some or all of the collective operations. Nvidia supports this in their InfiniBand and NVLink switches using SHARP (Scalable Hierarchical Aggregation and Reduction Protocol – proprietary). This is call “in-network computing”. For Ethernet, there are no standards yet. The Ultra Ethernet Consortium is working on it but will take years until something is seen in production. And Juniper has the programmable architecture Trio (MX routers – paper) that can do this offloading (You need to program it though – language similar to C). Still this is not a perfect solution (using a switches). The usage of collectives in inference is less common than their extensive use during the training phase of deep learning models. This is primarily because inference tasks can often be executed on a single GPU

From a different topics:

Learning at Cambridge: Spend less hours studying, dont take notes (that’s hard for me), go wild with active learning (work in exercises until you fully understand them)

British Library CyberAttack: blog and public learning lesson. I know this is happening to often for many different institutions but this one caught my eye 🙁 I think is a recurrent theme in most government institutions were upgrading is expensive (because it is not done often), tight budgets and IT experts.

“Our major software systems cannot be brought back in their pre-attack form, either because they are no longer supported by the vendor or because they will not function on the new secure infrastructure that is currently being rolled out”

However, the first detected unauthorised access to our network was identified at the Terminal Services server. Likely a compromised account.

Personally, I wonder what you can get from “stealing” in a library ???

Google Networking, AI Cooling, MATx

OpenFlow at Google – 2012: Openflow to manage to network, to simulate your network. 2 backbones: first for customer traffic and second for inter-DC traffic

UKNOF32 – Google Datacenter networking 2015: Evolution until Jupiter. Moving from chassis based solutions to pizza boxes. Smaller blast radius than a chassis. This switches have small buffers but Google uses ECN (QoS) for dealing with it.

Google DC Network via Optical Circuit 2022: (other video paper google post) Adding optical circuit switches, no more Clos network !!! Full mesh connection of aggregation blocks. Spines are expensive and bottlenecks. Traffic flows are predictable at large scale. Not building for worse scenario. Drawback: complex topology and routing control! Shortest path routing is insufficient. TE: variable hedging allows operation on different points along the continuum to tradeoff optimality under correct prediction vs robustness under misprediction -> no more spikes. Hitless topology reconfig. It seems it has been running already for 5y…. To be honest, It goes a bit… beyond my knowledge.

Google TPUv4 + Optical reconfigurable AI Network 2023: Based on the above but for AI at scale. Although there is already TPUv5. From this page, the pictures help to get a view of the connectivity. Still complex though.

Open Computer Project 2023: AI Datacenter – Mainly about how to cool down the AI infra with some much requirement of GPU/power.

MATx: A new company to design hw for AI models

AI will save the world, Nutanix kernel upgrade, GPU Programming

AI will save the world: Positive view of the AI development. Interesting the attack to China/Karl Marx at the end. In general I feel confident this will be good.

Nutanix kernel upgrade story: This is a bit hardcore for me (and looks a bit old from 2021) but still quite interesting how they did the troubleshooting.

GPU programming: I have never read about how to code for a GPU and this looks interesting and quite different from what I would do in a CPU. From the “Execution Model of the GPU” I started to lose track. Still is nice to see a summary at the end and resources/books.

LaVague, S3, Stratego

LaVague: There are web services that dont have API so this could help me to automate the interaction with them? I need to test. Another question, i am not sure if lavague has an API itself!

S3: I had this in my to-read list for a long time… and I after reading today I was a bit surprised because it wasn’t really technical as I expected. The takeouts are: Durability reviews, lightweight formal verification and ownership.

Stratego: I have never played this game but I was surprised that is more “complex” than chess and go. And how DeepNash can bluff and do unexpected things.

Banana Bread v2

I have already a banana bread recipe that I like quite a lot but decided to try a new one. video. recipe. I adapted it to my kitchen because sometimes I think some vegan recipes use ingredients that too difficult and expensive to find… I didnt do it vegan, just used normal low-fat cow milk.

Ingredients:

3 ripe bananas, mashed
125ml coconut oil, melted (pretty sure you can use any non-flavour oil)
125ml milk
6 tbs dried coconut
6 tbs dried cranberries
6 tbs almond flakes
Half handful of pumpkin and sunflower seeds
240g white flour
1 tsp baking powder
125g coconut sugar (or any brown sugar)
1 tsp ground cinnamon
1 tsp ground ginger
1 pinch of fresh grated nutmeg
1/4 tsp ground cardamon
40g dark chocolate: chopped as chips

Topping
1 Banana, sliced length-ways
4 tbs maple syrup

Process:

  • Pre heat your at 180C and line a loaf tin with greaseproof paper.
  • Mix your mashed banana with milk, cranberries, seeds, coconut and melted coconut oil.
  • In a separate bowl, mix together the dry ingredients: flour, baking powder, coconut sugar, spices and chocolate chips.
  • Mix the wet and dry ingredients together until well incorporated. Do not over mix.
  • Pour the batter into the tin. Then lay the sliced banana on top and drizzle over a little maple.
  • Bake for 45 minutes aprox.
  • Check it after 35 minutes, if you can see it getting too caramelised on top, cover it over with foil and leave to cook for the remaining time.
  • Once golden on top leave it to cool down.

Not bad result! I am not sure the coconut oil or the coconut sugar gives any flavour although it smelled a bit of coconut while baking. Go soft with the cardamon, it can overpower the rest of flavours!

Love Languages, imposter syndrome, self-compasion, GTC-2024, Juniper Express 5

Love Languages: I read this book in 2018. The conclusion I took at that time (and a bit late…) it is that you have to F*! communicate…

Interesting story about imposter syndrome:

We’d like to believe that if we only had the adulation, market success, and fan support of superstars like these, then we’d finally be comfortable and able to do our best.

In fact, it seems the opposite is true. Imposter syndrome shows up because we are imposters, imposters acting ‘as if’ in search of making something better.

Perhaps the best plan is to show up and not walk out.

Self-compassion: Something I have learnt the hard way, and I think at the beginning works but long term doesn’t. I practice it often while climbing and honestly, I feel the difference, and sometimes is mindblowing. Nobody is going to cheer me up so I better off doing it myself.

GTC-2024: Like last year, I registered to watch some conferences. As a network engineer, I haven’t been able to see any (good) recording, just pdfs…. so quite disappointing. This is a summary from somebody that was on site and says it was great. And some other notes that they look interesting: keynote (nvlink and infiniband at 800G), nvdia dgx gb200 (indeed we need nuclear energy to feed all this…)

Juniper Express 5: Looks quite an interesting ASIC. But as far as I can see most ASICs for DC and AI/ML come from Broadcom and the main players are Cisco/Arista. I like the feature of deep buffers.. this is still a bit of a religious dilema… deep vs shallow buffers. And looks like it was announced in HotChips 2022.. so it is not very new? And only in PTX platform. What is the future of QFX?