Today I Learned
Good article about LLM from the hardware/networks perspective. I liked it wasnt a show-off from Juniper products, as I haven’t seen any mention of Juniper kit in deployments of LLM in cloud providers, hyperscalers, etc. The points about Infiniband (the comment at the end about the misconceptions of IB is funny) and ethernet were not new but I liked the VOQ reference.
Still as a network engineer, I feel I am missing something about how to make the best network deployment for training LLM.
Original Wiener Apfelstrudel: (ok, this is Austrain) recommended recipe
Franzbrotchen: typical sweet from Hamburg. This recipe?
Off topic: Deutsches Sprichwort
Bei drei Dingen lernt man einen Menschen am besten kennen: In der Liebe, beim Spiel und beim Wein.
There are three things you get to know a person best: love, play and wine
I haven’t watched this video, but looks like the holly book of curl!!!
I'd recommend starting at ~34 minutes.
·You can specify multiple URLS with multiple output options in a single command. Doing this or using globbing (see below) to the same host will use persistent connections and greatly improve performance because the same L5 session is used
·trurl is also made by the project and allows you to programmatically manipulate URLs (change server, path, query parameters, etc.). Pretty neat: https://github.com/curl/trurl
·curl supports URL globbing: curl https://{ftp,www,test}.example.com/img[1-22].jpg -o "foo_#2_#1.jpg"
·By default, curl will resolve requests serially when multiple URLS or globbing is specified, but curl is capable of doing parallel transfers with the -Z or --parallel option. And can do anywhere from 2-300 transfers in parallel. This also has the potential to parallel-ize HTTP/3 transfers even from single URLs.
·You can do curl --help category to get a list of help categories for narrowing down options by categories like http or output
· Long commands for curl can be specified in a file and given to curl either via stdin or -K / --config - These files are essentially just command lines in a file
·You can use the --trace option to provide tcpdump type output from curl. Saving the need to to start tcpdump in the background if you just want to see what's happening from curl
·You can use --connect-to to specify a different DNS name to go to (instead of the one specified in the URL) which is similar to the --resolve option, but doesn't require the user to lookup the IP address ahead of time
·You can override the DNS server that you use to resolve URLs via --dns-ipv4-addr 8.8.8.8 for example
·You can add --libcurl to any curl command and it will spit out C source-code that implements the same command line in C via the library libcurl
·You can set the environment variable SSLKEYLOGFILE to a file name and it will save the runtime TLS secrets to that file, and use that file in WireShark along with a dump of the traffic from tcpdump to see the contents of encrypted HTTP streams
·You can choose to only download files that have changed since the last time they were downloaded with curl via --etag-save <etag_file> and --etag-compare <etag_file>
·You can skip adding the extra -H "Content-Type: application/json" when getting or posting JSON data (with -d), by specifying --json instead of just -d
·You can create JSON easily from the command line with the tool jo: https://github.com/jpmens/jo (basically a reverse jq)
Rant about yaml. And something I learned about yaml some months ago and forgot about it: scalars for making multiline work in yaml.
Elixir: a programming language based on Erlang. Really impressive reports! But still I would like to learn golang (if I ever learn properly python 🙂
git stash: I didnt know about this git command until last week, very handy.
A friend of mine asked to go on picnic to a nearby lake: Teufelssee. The place was really nice, I could swim and the water was perfect! Although I was murdered by mosquitos… The lake is inside a forest and is huge, really impressive thinking that I was so close to the city. As well, we visited Teufelsberg, there were nice graffiti and good views from the top.
Totally agree!!
Of course!
There are worse thins in life 🙂
As well, the local neighbourhood Grunewald was interesting. While cycling could amazing town houses.
This is a recipe from my friend M that is super easy and tasty!
Ingredients:
Process
Ready to eat!
Before putting into the oven and before adding the water:
After taking it out from the oven:
Very tasty! Although I originally used more than 2 tsp of paprika. So for next time, I will stick with just 2 and will see
I finished this book last Sunday. I have heard about some interesting achievements like getting a full MIT degree in one year so I wanted to know more about how was that achieved and what I could learn and apply to myself (if possible). I liked the challenge of learning a language each months… with a total exposure and not able to talk English. Honestly, I want to learn German and I want to do it quickly but properly so I want to take some pieces of advice from the book and apply to my case.
I think this can ge a good technique to master specific subjects. But at the end of day, in my opinion, it is all about focus. And that’s my problem, I want to do too many things.
So in summary, the ultralearning process is based on the following principles:
1- Metalearning: It is the research part. You need to know what you want to learn and how to do it.
2- Focus: It is about getting into flow and efficiency. If you dont have the luxury of dedicating 10h a day to your projects. Make the most of your time. And if you have issues, build the habit and strength bit by bit. You want to learn something challenging and difficult, keep it in mind. I liked the example about Mary Somerville. She was a housewife and mother. Still managed to be a top Mathematician!
3- Directness: Currently in schools, university, etc, most of the things we learn dont have a direct application to the practical world. This is knowledge transference. So it you want to learn python, well, use python. So go direct to want you want to learn. If you want to move your career to the AI/ML network infrastructure, you will have to find the technologies (infiniband, GPU, etc) used and learn from them so your CV can be taken into account.
4- Drill: Find your weakness and work on them. Here I learned about the experiences from Benjamin Franklin in his autobiography.
5- Retrieval: trying to recall facts and concepts from memory. The example here is Srinivasa Ramanujan. Free recall tests, in which students need to recall as much as they can remember without a cue, perform better than getting a cue.
6- Feedback: I liked the intro: “Everybody has a plan until they get punched in the mouth”. Myke Tyson. So testing if we are actually learning, improving it is very important. And how we get that feedback as well.
7- Retention: Another good intro. “Memory is the residue of though” Daniel Willingham. And the example is the World Champion of French Scrabble who didnt speak French, Nigel Richards. So it is about how not forget things. In this kind of challenges, where you are getting a lof of data, it is easy to start forgetting. So the goal is to keep that knowledge, that is with practice (different methods mentioned).
8- Intuition: here the example is mainly Richard Feynman. Main points are: 1) Dont give up on hard problems easily. 2) Prove things to understand them. 3) Start with a concrete example. 3) Be deeply skeptical about your own understanding.
9- Experimentation: It is trying things outside the specific subject you are studying. The example here is how Vincent Van Gogh learnt to paint.
I have read about the Laszlo Polgar experiment about raising his daughters as Chess prodigies but the book gives more details and actually found it even more interesting! I was surprised by the initial “macho” comments from Kasparov… More info about Judit.
In general, nice book, I want/need to put things into practice.
A couple of weeks ago I had to remove the pedals from my bike and I struggled big time. I followed this video for instructions and it was very good. As I didnt have enough time, I only managed to buy the standard spanner key that fitted the screw but that key didnt offer me enough leverage to actually remove the screw and no mutter how hard (or I am not strong enough), I failed. I decided to go to a nearby bike shop and they were nice enough to unscrew the pedals for me in 3 sec… using a pedal spanner like the video. So maybe I need to get one of those for the future (and remember how to actually remove the pedals).
I really enjoyed this article about how LLM works. I was surprised that works are really stored as vectors of many dimensions. I need to read it a couple of times.
…An antidote to chaos. This book was a gift and last week managed to finish it as I havent had much time lately for reading 🙁
I have seen this author books in airports and book shops, but never managed to take a look to see what was all about as it was a best seller, etc etc.
Something that surprised me was the deep analysis of several parts of the old and new testament to show his rules.
Another interesting topic, it was his strong anticommunist stance, focused on Stalin, Mao and Cambodia. And not sure I understood properly but he thinks that there is far-left in top education positions in USA. I find it hardly to believe that in USA apart from people like Noah Chomsky.
His rules are:
1- Stand-up straight with your shoulders back: Dont be like a defeated lobster.
2- Treat yourself like someone you are responsible for helping: This is based on the fact that people are more willing to look after their pets or other than for example taking their own medicines. Here there was a big strong connection to the Genesis with Adam and Eve being expelled from paradise.
3- Make friends with people who want the best for you: Like it is said in Spanish, “se pega todo menos la hermosura”. So if you have people around that are positive influence, you will move forwards, if it is not that case, you are in a bad place.
4- Compare yourself to who you were yesterday, not to who someone else is today: Typical in the instagram world…. and a source of depression and anxiety. Be the best version of yourself, better than yesterday but worse than tomorrow.
5- Do not let your children do anything that makes you dislike them: This is about kids education by parents. Very interesting as he describes cases based on his own kids and experiences. Not the typical pieces of advice you would fine nowadays.
6- Set your house in perfect order before your criticize the world: Get your shit together before criticizing others.
7- Pursue what is meaningful (not what is expedicit): He puts examples from the temptations from Christ in the desert. He shows that Christ could have asked for Gods help and all offers from the Devil were materialistic. And in general, the more materialistic goals we have, the more empty we feel and that makes us miserable and pushing for more stuff.
8- Tell the truth, or at least don’t lie: The basic of a society is trust. Telling lies destroys that fundation. And the same happens on yourself.
9- Assume that the person you are listening to might know something you dont: Learn to listen, you can learn stuff from other.
10- Be precise in your speech: Face the problems, think clearly on them and be specific about them. Communicate clearly and dont hide them.
11- Do not bother children when they are skateboarding: This is about children being able to play, take their own risks, learn and grown as humans beings and not be overprotected. There is a big focus in masculinity and how manhood is getting lost in our society. I was bit surprised in this topic as you read very frequently the contrary as we need more feminism in our world. In one side, I get his point, as I feel it myself.
12- Pet a cat when you encounter one on the street: This is not really about pets but about pain and suffering. How to cope in those situations and how to enjoy the moment.
In general, the rules look very general but then the author can go very deep and you got lost.
There are many references to Dostoevsky and Nietzsche. I really want to read at some point books from them but I am pretty sure I will struggle with deep philosophical inputs.
A friend of mine showed me this cards game and really liked it! Although they way we played, the player who “canta” can give the points to any other player, so it is even more fun!