Ring Memory

Reading this news I was surprised by the mentioned paper where LLM can take up to millions of tokens. My knowledge of LLM infrastucture is very little (and the paper is a bit beyond me…) but I thought the implementation of this models followed kind of “chain/waterfall” where the output of some GPUs fed other GPUs.

Argentina

Patagonia: This page was amazing for organizing the treks in El Chalten. Included even the GPS maps that worked perfect with GPX.

And this is one of the best views in many years. It was hard to get through the snow without proper kit but that view…. I wish I could have stay a long time but the temperature and mainly the wind didn’t help. “Lo bueno, si es breve, dos veces bueno”

Buenos Aires Bakeries: Something that I was really surprised about BA was the insane amount of bakeries (confiterias) and butchers in the city. I followed this link to find some good ones. But you can find some gems in any borough.

This is a random bakery around “La Boca”. Really tempted to apply for a job there.

MediaLunas: It is kind of a small croissant. video, recipe

Empanadas de Acelga: I need to learn how to make them

Futbol: I went to a game of San Lorenzo. I was really impressed with the passion. San Lorenzo lost but the supporters didnt stop singing during the whole game, didnt repeat a song and never stop cheering their players. Not sure if that happens very often in EU. And a lot of families, and a lot of kids playing futbol inside the stadium.

Re-Read-1 and Classics

I re-read this book during holidays without remembering that I completed it over 1 year ago, and for the life of me, I didnt notice it. So doesnt leave me or my memory in a good place 馃檪 Still I enjoyed (again) and it is something I wanted to do, re-read some good books.

And following the trend, I re-read this one (ebook) as I think I needed it. Again, really good read, it grounds me and put me in my place. And another reason was I tried to read an ebook from Kant and Seneca, and couldnt understand anything… It is annoying that I read a lot of references for them but then you go to the source, and I can’t digest them. Too overcomplicated language. Anyway, I will try other “classics” just in case.

How to make the world add up

This is another ebook I read during holidays. I reminded me to “Calling Bullshit” as it is about finding the “truth” among the noise.

The autor leads a radio program “More or less” in BBC. Unfortunately, I dont find time for listening radio or podcast.

The start was very interesting about how an expert of a famous Dutch painter failed to spot a fake picture because he was obsessed with one details from the painter. And how the fake painter exploited that weakness (and from the whole Netherlands….)

To be honest, I didnt take notes so I am not going to remember many things.

I liked a lot the personal details about John Maynard Keynes life that I didnt know (I tried to read one of his books in the past and I failed miserably):

When my information changes, I alter my conclusions. What do you do, sir? (JMK)

The ten rules from the book:

1- Search your feelings (Johannes Vermeer fake paint and Nazis)

2- Ponder your personal experience (London trains are not busy….)

3- Avoid premature enumeration

4- Step back and enjoy the view

5- Get the back story

6- Ask who is missing:

The power to NOT collect data is one of the most important and little-understood sources of power that governments have… By refusing to amass knowledge in the first pace, decision-makers exert power over the rest of us. (Anna Powell-Smith)

7- Demand transparency when the computes says no: Google’s quest to find flu outbreaks via searches.

8- Don’t take statistical bedrock for granted: The need of an independent Statistics body for the country to help the governments with decisions. Examples in USA (Wilbur Mills, then Congressional Budget Office aka CBO), UK (Office for Budget Responsibility – OBR) and Greece (Andreas Gerogiou troubles)/Argentina (Graciela Bevacqua troubles).

9- Remember that misinformation can be beautiful too (Florence Nightingale)

10- Keep an open mind (John Maynard Keynes)

Summary:

Be curious.

The Almanack of Naval Ravikant

This is the first ebook I finished while on holidays. I have read recommendations about Naval before but I didnt know what to expect. In general I liked it, many things resonate with me and i would like to read it again at some point. It is something I need to do, be able to re-read books and drop books that are not “worth” it. That’s something Naval does/says.

The books is about wealth (and not just money) in all senses. But at the end of the day, I wonder if this is for everybody. Can everybody have equity? be its own boss? But again, there are many nice things you can take from the book. I would like to have in paper so could take more notes. So this is kind of the thinks I noted:

  • Equity
  • Do things for its own sake
  • You are never going to get rich renting out your time
  • Leverage: min input, max output
  • Deathbed: the hard things you did
  • Ego
  • Time To Think!
  • Relaxation is who you are
  • Fuck yeah or default no (Need to learn this)
  • Short term pain, long term gain
  • Calm mind, fit body, house full of love. It is earned, not bought. (amen)
  • Economy basics: Adam Smith
  • Fundamental delusion: there is something out there that will make me happy (I feel this…)
  • Desire is a contract with yourself to be unhappy until you get it…
  • Jealousy (I feel this…)
  • Death: Enjoy the moment
  • Take responsibility
  • Physical health, diet, sugar. Vegs and a bit of meat
  • Mental health, meditation, breathing, cold shower.
  • Most of our suffering comes from avoidance.
  • Meditate in bed. Noting. GRATEFUL!
  • Drugs, addictions, thrills, etc: People trying to get away from the voice of their minds -> overdeveloped sense of self
  • Habits
  • Reading habits: Re-read good books, dont need to read full books and be able to drop a book it is not good.

FAQ me

I finished this book last night. It was a recommendation from a friend and I downloaded it some time ago into my kindle. I dont read very often in kindle as i prefer the paper touch but I didnt have anything else at hand so I gave it a go.

I didnt have a clue what was all about, I didnt know the person. And I only checked his history this morning.

In summary, the book is interesting, I liked to read about his personality and treats. Every person is different and it was good to read about his weakness, errors, fears, etc. As he says several times, you have to “bleed” in your writing.

Something that is repeated in the book is the daily practice and his four bodies theory:

  • Physical
  • Emotional: get rid of negative influences, surround yourself with positive ones
  • Mental: write ideas
  • Spiritual: be grateful for what you have, focus on the present, surrender.

He talks about everything and anything. From dating, something important to me, he is clear that there is no magic bullet, you have to go meet people (he insists in tango and cooking classes :), and use all tools available. So there is no short cut.

He talks about being an idea-generating machine. And that takes practice. Write ten ideas, tomorrow another ten, etc. Analyze them, cross them, redefine them, etc.

He is very clear that working for enterprise in the corporate America is the worse you can do, he pushes to be en entrepreneur. And he recommends these books about economics.

I should have taken more notes though. Interesting read, although checking his blog, he hasnt written for a couple of years but he does youtube now.

Anyway, I think it was worth reading it.

LLM: hardware connection

Good article about LLM from the hardware/networks perspective. I liked it wasnt a show-off from Juniper products, as I haven’t seen any mention of Juniper kit in deployments of LLM in cloud providers, hyperscalers, etc. The points about Infiniband (the comment at the end about the misconceptions of IB is funny) and ethernet were not new but I liked the VOQ reference.

Still as a network engineer, I feel I am missing something about how to make the best network deployment for training LLM.

Curl, Yaml, scalars, Elixir, git stash

I haven’t watched this video, but looks like the holly book of curl!!!

I'd recommend starting at ~34 minutes.

路You can specify multiple URLS with multiple output options in a single command. Doing this or using globbing (see below) to the same host will use persistent connections and greatly improve performance because the same L5 session is used

路trurl is also made by the project and allows you to programmatically manipulate URLs (change server, path, query parameters, etc.). Pretty neat: https://github.com/curl/trurl

路curl supports URL globbing: curl https://{ftp,www,test}.example.com/img[1-22].jpg -o "foo_#2_#1.jpg"

路By default, curl will resolve requests serially when multiple URLS or globbing is specified, but curl is capable of doing parallel transfers with the -Z or --parallel option. And can do anywhere from 2-300 transfers in parallel. This also has the potential to parallel-ize HTTP/3 transfers even from single URLs.

路You can do curl --help category to get a list of help categories for narrowing down options by categories like http or output

路 Long commands for curl can be specified in a file and given to curl either via stdin or -K / --config - These files are essentially just command lines in a file

路You can use the --trace option to provide tcpdump type output from curl. Saving the need to to start tcpdump in the background if you just want to see what's happening from curl

路You can use --connect-to to specify a different DNS name to go to (instead of the one specified in the URL) which is similar to the --resolve option, but doesn't require the user to lookup the IP address ahead of time

路You can override the DNS server that you use to resolve URLs via --dns-ipv4-addr 8.8.8.8 for example

路You can add --libcurl to any curl command and it will spit out C source-code that implements the same command line in C via the library libcurl

路You can set the environment variable SSLKEYLOGFILE to a file name and it will save the runtime TLS secrets to that file, and use that file in WireShark along with a dump of the traffic from tcpdump to see the contents of encrypted HTTP streams

路You can choose to only download files that have changed since the last time they were downloaded with curl via --etag-save <etag_file> and --etag-compare <etag_file>

路You can skip adding the extra -H "Content-Type: application/json" when getting or posting JSON data (with -d), by specifying --json instead of just -d

路You can create JSON easily from the command line with the tool jo: https://github.com/jpmens/jo (basically a reverse jq)

Rant about yaml. And something I learned about yaml some months ago and forgot about it: scalars for making multiline work in yaml.

Elixir: a programming language based on Erlang. Really impressive reports! But still I would like to learn golang (if I ever learn properly python 馃檪

git stash: I didnt know about this git command until last week, very handy.

Teufelssee

A friend of mine asked to go on picnic to a nearby lake: Teufelssee. The place was really nice, I could swim and the water was perfect! Although I was murdered by mosquitos… The lake is inside a forest and is huge, really impressive thinking that I was so close to the city. As well, we visited Teufelsberg, there were nice graffiti and good views from the top.

Totally agree!!

Of course!

There are worse thins in life 馃檪

As well, the local neighbourhood Grunewald was interesting. While cycling could amazing town houses.