VIM – Indent multiple lines quickly

This is the thing I have to do from time to time and I never remember how to do:

https://stackoverflow.com/questions/235839/indent-multiple-lines-quickly-in-vi

And this what works for me:

Key presses for more visual people:
1- Enter Command Mode:
Escape

2- Move around to the start of the area to indent:
hjkl↑↓←→

3- Start a block:
v

4- Move around to the end of the area to indent:
hjkl↑↓←→

5- Type the number of indentation levels you want
0..9

6- Execute the indentation on the block:
>

Let’s see if I can remember it…

Backup Blog

I have decided to backup this blog. First time. So following instructions from wordpress.

  • 1- Backup Database:
https://your_domain_or_IP/phpmyadmin
https://wordpress.org/support/article/backing-up-your-database/
  • 2- Backup webserver files
root@vps:/var/www# ls
html wordpress
root@vps:/var/www# cd ..
root@vps:/var# tar zcvf web-backup.tgz www
  • 3- Transfer the tgz and the sql file to a folder in my laptop that is backed-up to external hard drive.
/blog-backup$ tree
.
├── apache-files
│   └── web-backup-20200707.tgz
└── mysql
    └── blog-backup-20200707.sql

Github + ssh-key

There are many links for this in the Internet so I am not going to discover the fire but I struggled a bit so….

The official links from github were ok and other people did a very good job too documenting the process.

https://docs.github.com/en/github/authenticating-to-github/testing-your-ssh-connection

https://docs.github.com/en/github/authenticating-to-github/error-permission-denied-publickey

https://jdblischak.github.io/2014-09-18-chicago/novice/git/05-sshkeys.html

I had already a key that I wanted to use. So adding it to the repo was ok.

Testing it was my challenge. I was missing two things. My key wasn’t following the standard file name so it wasn’t used by my ssh-agent and then, i wasn’t using the “git” user when testing…. I was using my github username.

So add the key and check it is there.

$ ssh-add ~/.ssh/id_ed25519-gh
$ ssh-add -l -E md5
256 MD5:xx:xx:xx:xx:xx:67:xx:6a:73:xx:8a:xx:7f:78:xx:xx user@gh (ED25519)

Check you can ssh to github.

$ ssh -T git@github.com
Hi xxxx! You've successfully authenticated, but GitHub does not provide shell access.
$

Ok, all good now. But this is not a new repo, how I move from the “old” user/pass to the “new” ssh-key process?

You can clone the repo again using ssh:

Or you can change the git config locally in the “url” bit.

/ceos-testing/.git master$ cat config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[remote "origin"]
#url = https://github.com/thomarite/ceos-testing.git
url = git@github.com:thomarite/ceos-testing.git
fetch = +refs/heads/:refs/remotes/origin/
[branch "master"]
remote = origin
merge = refs/heads/master
$

After that you can “git push” using your ssh-key.

2023-01

Looks like I dont learn the lesson….

1- Create Key

$ ssh-keygen -t ed25519 -C "your@email.com"
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/USERNAME/.ssh/id_ed25519): /home/USERNAME/.ssh/id_ed25519.github

2- Upload key to Github

3- Start agent and add key

$ ssh-agent -s
SSH_AUTH_SOCK=/tmp/ssh-XXXXXXjMtZn7/agent.250293; export SSH_AUTH_SOCK;
SSH_AGENT_PID=250294; export SSH_AGENT_PID;
echo Agent pid 250294;
$ ssh-add ~/.ssh/id_ed25519.github
Identity added: /home/USERNAME/.ssh/id_ed25519.github (your@email.com)
$ 

4- Authenticate to git

$ ssh -T git@github.com
Hi USERNAME! You've successfully authenticated, but GitHub does not provide shell access.
$ 

5- Push to git. Be sure your repo is not using https! Change it as showed here.

$ git remote get-url origin
https://github.com/SOMEBODY/scripts.git
$ git remote set-url origin git@github.com:SOMEBODY/scripts.git
$ 
$ git remote get-url origin
git@github.com:SOMEBODY/scripts.git
$ 
$ git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 2.07 KiB | 2.07 MiB/s, done.
Total 4 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To github.com:SOMEBODY/scripts.git
   6a4cb1a..07a4a83  main -> main
$ 

Which SSH keyfile was used to authenticate a login?

I have realised that I had two keys in my VPS and I wasn’t sure which one it was used when I was ssh-ing so I had to search a bit to find out.

These two links cover the process:

https://unix.stackexchange.com/questions/15575/can-i-find-out-which-ssh-key-was-used-to-access-an-account

https://unix.stackexchange.com/questions/147295/how-can-i-determine-which-ssh-keyfile-was-used-to-authenticate-a-login

1- You need to increase the logging of your sshd (destination – server)

server# vim /etc/ssh/sshd_config
LogLevel VERBOSE
server# service sshd restart
server# tail -f /var/log/auth.log

2- From client, just ssh as usual to the server and check auth.log as per above

Jul 3 14:17:55 server sshd[8600]: Connection from IPV6 port 57628 on IPV6::453 port 64022
Jul 3 14:17:55 server sshd[8600]: Postponed publickey for client from IPv6 port 57628 ssh2 [preauth]
Jul 3 14:17:55 server sshd[8600]: Accepted publickey for client from IPv6 port 57628 ssh2: ED25519 SHA256:BtOAX9eVpFJJgJ5HzjKU8E973m+MX+3gDxsm7eT/iEQ
Jul 3 14:17:55 server sshd[8600]: pam_unix(sshd:session): session opened for user client by (uid=0)
Jul 3 14:17:55 server sshd[8600]: User child is on pid 8606
Jul 3 14:17:55 server sshd[8606]: Starting session: shell on pts/7 for client from IPv6 port 57628 id 0

3- So we have the fingertip of the key used by client. Now we need to get the fingertips of our clients keys to find the match:

client $ ssh-keygen -l -f ~/.ssh/id_ed25519.pub
256 SHA256:BtOAX9eVpFJJgJ5HzjKU8E973m+MX+3gDxsm7eT/iEQ client@local (ED25519)

4- So the we can see that I am using my id_ed25519.pub key to connect to the server

ASCII recording

It is something I have watched from several sites and finally noticed a page that mentioned how was done.

So this is the tool: https://asciinema.org/

And this is an example I have recorded. I have just to create a HTML entry and copy the code.

Summary:

$ python -m pip install asciinema
$ asciinema rec  /// or ///  python -m asciiema rec
asciinema: recording asciicast to /tmp/tmpvjba98p7-ascii.cast
asciinema: press or type "exit" when you're done
...
$ exit
exit
asciinema: recording finished
asciinema: press to upload to asciinema.org, to save locally
asciinema: asciicast saved to /tmp/tmphxkq5sp2-ascii.cast
$ asciinema upload /tmp/tmphxkq5sp2-ascii.cast

GCP – Basic Debian VM

Template for getting a basic Debian VM with python virtualenv and pyenv. Run python3 by default

$ sudo aptitude update
$ sudo aptitude dist-upgrade

$ sudo apt-get install -y python3-pip
$ sudo apt install git
$ sudo adduser --home /home/USER --shell /bin/bash USER
$ sudo usermod -a -G sudo USER

$ vim .bashrc

#
Python configuration
#
pyenv
https://github.com/yyuu/pyenv
git clone https://github.com/yyuu/pyenv.git ~/.pyenv
git clone https://github.com/yyuu/pyenv-virtualenvwrapper.git ~/.pyenv/plugins/pyenv-virtualenvwrapper
virtualenvwrapper
http://virtualenvwrapper.readthedocs.org/en/latest/
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
if [ ! -d "$PYENV_ROOT" ];then
git clone https://github.com/pyenv/pyenv.git ~/.pyenv
fi
if [ ! -d "$PYENV_ROOT/plugins/pyenv-virtualenvwrapper" ]; then
mkdir -p $PYENV_ROOT/plugins
git clone https://github.com/yyuu/pyenv-virtualenvwrapper.git $PYENV_ROOT/plugins/pyenv-virtualenvwrapper
fi
if type "pyenv" &> /dev/null; then
eval "$(pyenv init -)"
# TODO: make the prompt work for python and ruby
__pyversion (){
if type "python" > /dev/null; then
pyenv_python_version=$(pyenv version | sed -e 's/ .*//')
printf $pyenv_python_version
fi
}
if pyenv which pip &> /dev/null; then pyenv virtualenvwrapper fi export PS1="py:\$(__pyversion)|$PS1"
fi
export PROJECT_HOME=~/git
export PYTHONDONTWRITEBYTECODE=1
end python

$ bash

$ sudo apt-get install -y --no-install-recommends make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev tcpdump tree

$ pyenv install 3.7.3
$ pyenv global 3.7.3

BPF: basics

I had in my to-try list BPF. After reading this link, I realised that it was quite straightforward to get a vagrant box with all the requirements.

As per link recommends, watch the video. I didnt know Brendan Gregg worked for Netflix and that Facebook was pushing very hard in BPF development.

You can find all the documentation here:

So as per the main link:

git clone https://github.com/codeboten/bpftracing.git && cd bpftracing
vagrant up

Once you have installed the vagrant box (it takes a while) you can “vagrant halt” and start again:

~/storage/technology/linux/bpftracing master$ vagrant status
Current machine states:
default poweroff (virtualbox)
The VM is powered off. To restart the VM, simply run vagrant up
~/storage/technology/linux/bpftracing master$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider…
==> default: Checking if box 'ubuntu/bionic64' version '20200525.0.0' is up to date…
==> default: Clearing any previously set forwarded ports…
==> default: Clearing any previously set network interfaces…
==> default: Preparing network interfaces based on configuration…
default: Adapter 1: nat
==> default: Forwarding ports…
default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations…
==> default: Booting VM…
==> default: Waiting for machine to boot. This may take a few minutes…
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM…
default: The guest additions on this VM do not match the installed version of
default: VirtualBox! In most cases this is fine, but in rare cases it can
default: prevent things such as shared folders from working properly. If you see
default: shared folder errors, please make sure the guest additions within the
default: virtual machine match the version of VirtualBox you have installed on
default: your host and reload your VM.
default:
default: Guest Additions Version: 5.2.34
default: VirtualBox Version: 6.1
==> default: Mounting shared folders…
default: /vagrant => /home/xxx/storage/technology/linux/bpftracing
==> default: Machine already provisioned. Run vagrant provision or use the --provision
==> default: flag to force provisioning. Provisioners marked to run always will still run.
~/storage/technology/linux/bpftracing master$ vagrant ssh
Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-106-generic x86_64)
Documentation: https://help.ubuntu.com
Management: https://landscape.canonical.com
Support: https://ubuntu.com/advantage
System information as of Sun Jun 21 19:25:26 UTC 2020
System load: 0.35 Processes: 99
Usage of /: 32.2% of 9.63GB Users logged in: 0
Memory usage: 12% IP address for enp0s3: 10.0.2.15
Swap usage: 0%
0 packages can be updated.
0 updates are security updates.
Last login: Sun Jun 21 19:22:37 2020 from 10.0.2.2
vagrant@ubuntu-bionic:~$
vagrant@ubuntu-bionic:~$ cd /vagrant/
vagrant@ubuntu-bionic:/vagrant$ ls
Makefile Vagrantfile bpf_program.o monitor-exec
README.md bpf_program.c loader.c ubuntu-bionic-18.04-cloudimg-console.log
vagrant@ubuntu-bionic:/vagrant$

You can find tools (under /usr/sbin)(already compiled and ready to use) or examples (under /usr/share/doc/bpfcc-tools/examples)

These are the tools you can find in the system:

vagrant@ubuntu-bionic:~$ ls -ltr /usr/sbin | grep -i bpfcc
-rwxr-xr-x 1 root root 3496 Nov 29 2017 reset-trace-bpfcc
-rwxr-xr-x 1 root root 7105 Nov 29 2017 deadlock_detector.c-bpfcc
-rwxr-xr-x 1 root root 9029 Mar 27 2018 zfsslower-bpfcc
-rwxr-xr-x 1 root root 5131 Mar 27 2018 zfsdist-bpfcc
-rwxr-xr-x 1 root root 8184 Mar 27 2018 xfsslower-bpfcc
-rwxr-xr-x 1 root root 4431 Mar 27 2018 xfsdist-bpfcc
-rwxr-xr-x 1 root root 6825 Mar 27 2018 wakeuptime-bpfcc
-rwxr-xr-x 1 root root 2636 Mar 27 2018 vfsstat-bpfcc
-rwxr-xr-x 1 root root 1177 Mar 27 2018 vfscount-bpfcc
-rwxr-xr-x 1 root root 2978 Mar 27 2018 ttysnoop-bpfcc
-rwxr-xr-x 1 root root 31977 Mar 27 2018 trace-bpfcc
-rwxr-xr-x 1 root root 4159 Mar 27 2018 tplist-bpfcc
-rwxr-xr-x 1 root root 17766 Mar 27 2018 tcptracer-bpfcc
-rwxr-xr-x 1 root root 9327 Mar 27 2018 tcptop-bpfcc
-rwxr-xr-x 1 root root 5631 Mar 27 2018 tcpretrans-bpfcc
-rwxr-xr-x 1 root root 11996 Mar 27 2018 tcplife-bpfcc
-rwxr-xr-x 1 root root 6858 Mar 27 2018 tcpconnlat-bpfcc
-rwxr-xr-x 1 root root 6963 Mar 27 2018 tcpconnect-bpfcc
-rwxr-xr-x 1 root root 5782 Mar 27 2018 tcpaccept-bpfcc
-rwxr-xr-x 1 root root 12809 Mar 27 2018 syscount-bpfcc
-rwxr-xr-x 1 root root 1231 Mar 27 2018 syncsnoop-bpfcc
-rwxr-xr-x 1 root root 4560 Mar 27 2018 statsnoop-bpfcc
-rwxr-xr-x 1 root root 15860 Mar 27 2018 stackcount-bpfcc
-rwxr-xr-x 1 root root 6244 Mar 27 2018 sslsniff-bpfcc
-rwxr-xr-x 1 root root 6277 Mar 27 2018 solisten-bpfcc
-rwxr-xr-x 1 root root 4048 Mar 27 2018 softirqs-bpfcc
-rwxr-xr-x 1 root root 3409 Mar 27 2018 slabratetop-bpfcc
-rwxr-xr-x 1 root root 5643 Mar 27 2018 runqlen-bpfcc
-rwxr-xr-x 1 root root 5998 Mar 27 2018 runqlat-bpfcc
-rwxr-xr-x 1 root root 58 Mar 27 2018 rubystat-bpfcc
-rwxr-xr-x 1 root root 60 Mar 27 2018 rubyobjnew-bpfcc
-rwxr-xr-x 1 root root 56 Mar 27 2018 rubygc-bpfcc
-rwxr-xr-x 1 root root 58 Mar 27 2018 rubyflow-bpfcc
-rwxr-xr-x 1 root root 59 Mar 27 2018 rubycalls-bpfcc
-rwxr-xr-x 1 root root 60 Mar 27 2018 pythonstat-bpfcc
-rwxr-xr-x 1 root root 58 Mar 27 2018 pythongc-bpfcc
-rwxr-xr-x 1 root root 60 Mar 27 2018 pythonflow-bpfcc
-rwxr-xr-x 1 root root 61 Mar 27 2018 pythoncalls-bpfcc
-rwxr-xr-x 1 root root 9831 Mar 27 2018 profile-bpfcc
-rwxr-xr-x 1 root root 1139 Mar 27 2018 pidpersec-bpfcc
-rwxr-xr-x 1 root root 57 Mar 27 2018 phpstat-bpfcc
-rwxr-xr-x 1 root root 57 Mar 27 2018 phpflow-bpfcc
-rwxr-xr-x 1 root root 58 Mar 27 2018 phpcalls-bpfcc
-rwxr-xr-x 1 root root 4858 Mar 27 2018 opensnoop-bpfcc
-rwxr-xr-x 1 root root 2337 Mar 27 2018 oomkill-bpfcc
-rwxr-xr-x 1 root root 11141 Mar 27 2018 offwaketime-bpfcc
-rwxr-xr-x 1 root root 10464 Mar 27 2018 offcputime-bpfcc
-rwxr-xr-x 1 root root 58 Mar 27 2018 nodestat-bpfcc
-rwxr-xr-x 1 root root 56 Mar 27 2018 nodegc-bpfcc
-rwxr-xr-x 1 root root 9289 Mar 27 2018 nfsslower-bpfcc
-rwxr-xr-x 1 root root 4587 Mar 27 2018 nfsdist-bpfcc
-rwxr-xr-x 1 root root 3221 Mar 27 2018 mysqld_qslower-bpfcc
-rwxr-xr-x 1 root root 12023 Mar 27 2018 mountsnoop-bpfcc
-rwxr-xr-x 1 root root 17963 Mar 27 2018 memleak-bpfcc
-rwxr-xr-x 1 root root 2262 Mar 27 2018 mdflush-bpfcc
-rwxr-xr-x 1 root root 3429 Mar 27 2018 llcstat-bpfcc
-rwxr-xr-x 1 root root 3295 Mar 27 2018 killsnoop-bpfcc
-rwxr-xr-x 1 root root 61 Mar 27 2018 javathreads-bpfcc
-rwxr-xr-x 1 root root 58 Mar 27 2018 javastat-bpfcc
-rwxr-xr-x 1 root root 60 Mar 27 2018 javaobjnew-bpfcc
-rwxr-xr-x 1 root root 56 Mar 27 2018 javagc-bpfcc
-rwxr-xr-x 1 root root 58 Mar 27 2018 javaflow-bpfcc
-rwxr-xr-x 1 root root 59 Mar 27 2018 javacalls-bpfcc
-rwxr-xr-x 1 root root 5154 Mar 27 2018 hardirqs-bpfcc
-rwxr-xr-x 1 root root 3852 Mar 27 2018 gethostlatency-bpfcc
-rwxr-xr-x 1 root root 7124 Mar 27 2018 funcslower-bpfcc
-rwxr-xr-x 1 root root 7442 Mar 27 2018 funclatency-bpfcc
-rwxr-xr-x 1 root root 12448 Mar 27 2018 funccount-bpfcc
-rwxr-xr-x 1 root root 5847 Mar 27 2018 filetop-bpfcc
-rwxr-xr-x 1 root root 7235 Mar 27 2018 fileslower-bpfcc
-rwxr-xr-x 1 root root 3718 Mar 27 2018 filelife-bpfcc
-rwxr-xr-x 1 root root 9605 Mar 27 2018 ext4slower-bpfcc
-rwxr-xr-x 1 root root 5674 Mar 27 2018 ext4dist-bpfcc
-rwxr-xr-x 1 root root 5944 Mar 27 2018 execsnoop-bpfcc
-rwxr-xr-x 1 root root 20036 Mar 27 2018 deadlock_detector-bpfcc
-rwxr-xr-x 1 root root 3920 Mar 27 2018 dcstat-bpfcc
-rwxr-xr-x 1 root root 4009 Mar 27 2018 dcsnoop-bpfcc
-rwxr-xr-x 1 root root 3780 Mar 27 2018 dbstat-bpfcc
-rwxr-xr-x 1 root root 7130 Mar 27 2018 dbslower-bpfcc
-rwxr-xr-x 1 root root 12614 Mar 27 2018 cpuunclaimed-bpfcc
-rwxr-xr-x 1 root root 4975 Mar 27 2018 cpudist-bpfcc
-rwxr-xr-x 1 root root 57 Mar 27 2018 cobjnew-bpfcc
-rwxr-xr-x 1 root root 4142 Mar 27 2018 capable-bpfcc
-rwxr-xr-x 1 root root 6960 Mar 27 2018 cachetop-bpfcc
-rwxr-xr-x 1 root root 4932 Mar 27 2018 cachestat-bpfcc
-rwxr-xr-x 1 root root 9887 Mar 27 2018 btrfsslower-bpfcc
-rwxr-xr-x 1 root root 6214 Mar 27 2018 btrfsdist-bpfcc
-rwxr-xr-x 1 root root 2392 Mar 27 2018 bpflist-bpfcc
-rwxr-xr-x 1 root root 1721 Mar 27 2018 bitesize-bpfcc
-rwxr-xr-x 1 root root 6171 Mar 27 2018 biotop-bpfcc
-rwxr-xr-x 1 root root 4869 Mar 27 2018 biosnoop-bpfcc
-rwxr-xr-x 1 root root 4023 Mar 27 2018 biolatency-bpfcc
-rwxr-xr-x 1 root root 1567 Mar 27 2018 bashreadline-bpfcc
-rwxr-xr-x 1 root root 33534 Mar 27 2018 argdist-bpfcc
vagrant@ubuntu-bionic:~$

You can run them like this:

root@ubuntu-bionic:/# cpudist-bpfcc
Tracing on-CPU time… Hit Ctrl-C to end.
^C
usecs : count distribution
0 -> 1 : 4 | |
2 -> 3 : 0 | |
4 -> 7 : 33 |* |
8 -> 15 : 24 | |
16 -> 31 : 49 |* |
32 -> 63 : 74 |** |
64 -> 127 : 161 |* | 128 -> 255 : 101 | |
256 -> 511 : 76 | |
512 -> 1023 : 116 |* | 1024 -> 2047 : 64 |** | 2048 -> 4095 : 178 |* | 4096 -> 8191 : 558 | |
8192 -> 16383 : 191 | |
16384 -> 32767 : 288 | | 32768 -> 65535 : 338 | |
65536 -> 131071 : 756 |* |
131072 -> 262143 : 1025 || 262144 -> 524287 : 950 |*** |
root@ubuntu-bionic:/#

This link is a bit old (you need to add -bpfcc to all commands) but I think it can give you some more details for some tools.

In general, this is quite beyond my knowledge. But I would like to be able to set it up in Debian and obviously use it.

CI: Basics with Travis

For some time I wanted to learn a bit about CI/CD. Today I have given a go to Travis.

All this is based on Kirk Byers python course and his git repo.

So I just created an empty repo and started working on it:

$ git clone https://github.com/thomarite/test-ci.git

$ cd test-ci
$ pyenv local 3.7.3
$ python -m venv virt_env
$ source virt_env/bin/active

$ python -m pip install pylama
$ python -m pip install black
$ python -m pip install pytest
$ python -m pip install tox

$ mkdir tests

$ vim tests/test_sample.py
def increment(x):
return x + 1


def test_answer():
assert increment(4) == 5

$ vim requirements.txt
pytest==5.4.3
pylama==7.7.1
black==19.10b0

$ vim .travis.yml
language: python
python:
"3.7"
# command to install dependencies
install:
pip install -r requirements.txt
# command to run tests
script:
pylama .
black --check .
py.test -s -v tests/

Then you create an account with Travis-ci.org that is “free” and you link up to your repo. As soon as you commit, you will how the tests run and if they are successful.

As I have now a basic setup, I hope I carry on using it to any new python stuff I try.

FTP Passive

I have a supplier at my employer that requires to use a FTP server to send big files when you open a support ticket. For a long time (a couple of years) whenever I had to upload big files, I had to use my personal VM because my ftp connections failed from the office. I always blamed the super-smart firewall.

One day, I decided to fix the issue and allow the connection in our corporate firewall. I failed. Still couldnt upload files from the office. So keep using my personal VM.

This week I had to upload again a big file. This time I am working from home, so pretty much it is going to work the upload. Wrong! It fails. Ok, I checked a bit and got to the conclusion that it is my ISP or modem at home that is blocking FTP. Most ISP use CGN to stretch as much as possible the limited IPv4. I have IPv6 at home and my VM has IPv6 too… but the ftp server doesnt.

I checked the internet if there was any know issue with my ISP and FTP connections. No luck. I connected to my modem, nothing obvious messing around with FTP.

I decided to give it a proper go to this issue. I knew that it worked from my VM and it didnt from home. I noticed that I was running the same ftp client version in the VM and at home. So let’s debug the ftp client and take a packet capture in both locations.

CLI from the VM:

$ ftp -vd b.b.b.b
ftp: setsockopt: Bad file descriptor
Name: ftp
---> USER ftp
331 Please specify the password.
Password:
---> PASS XXXX
230 Login successful.
---> SYST
215 UNIX Type: L8
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd support
---> CWD support
250 Directory successfully changed.
ftp> cd 211211
---> CWD 211211
250 Directory successfully changed.
ftp> put TEST.txt
local: TEST.txt remote: TEST.txt
---> TYPE I
200 Switching to Binary mode.
ftp: setsockopt (ignored): Permission denied
---> PORT a,a,a,a,162,57
200 PORT command successful. Consider using PASV.
---> STOR TEST.txt
150 Ok to send data.
226 Transfer complete.
28 bytes sent in 0.00 secs (854.4922 kB/s)
ftp> quit
---> QUIT

And this is the packet capture:

After typing “put” in packet 33, I see a “PASV” message from the server and a new connection (initiated by the server!) is established for the data transfer. All good.

So now, make the same from home and compare.

CLI from home without debug:

$ ftp b.b.b.b
Connected to b.b.b.b.
Name: ftp
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd support
250 Directory successfully changed.
ftp> cd 211211
250 Directory successfully changed.
ftp> put TEST.txt
local: TEST.txt remote: TEST.txt
500 Illegal PORT command.
ftp: bind: Address already in use
ftp> quit
221 Goodbye.

CLI from home with debug:

$ ftp -vd b.b.b.b
ftp: setsockopt: Bad file descriptor
Name: ftp
---> USER ftp
331 Please specify the password.
Password:
---> PASS XXXX
230 Login successful.
---> SYST
215 UNIX Type: L8
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd support
---> CWD support
250 Directory successfully changed.
ftp> cd 211211
---> CWD 211211
250 Directory successfully changed.
ftp> put TEST.txt
local: TEST.txt remote: TEST.txt
---> TYPE I
200 Switching to Binary mode.
ftp: setsockopt (ignored): Permission denied
---> PORT 192,168,1,158,202,145
500 Illegal PORT command.
ftp: bind: Address already in use
ftp> quit
---> QUIT
221 Goodbye.

So with and without debug I keep seeing “ftp: bind: Address already in use”…..

And this is the packet capture from home:

So after I type “put” in packet 32, the answer from the server is a “500”.

I wasnt clearly paying attention to the clues. I was still banging my head why the server was sending a “500 Ilegal PORT command”.

I was comparing both captures and both debug outputs… but still didnt it.

I thought I understood FTP. I knew that you use port TCP 21 to establish the control session and the data session / transfer is via new TCP session using a random port. That’s one of the reasons that using NAT or CGN can screw up your FTP sessions.

So I assumed that the issues wasnt my ISP. So it had to be my side (or me).

So finally, I decided to search for “ftp: bind: Address already in use” as it was the message that came up with and without debugging.

Oh boy, first entry in the face!

https://www.linuxquestions.org/questions/linux-distributions-5/problems-with-ftp-server-bind-address-allready-in-use-213509/

An entry from 2004…. it can’t fix my problem for sure…. keep reading and update from 2020… it says it works…. oh boy II

try using a passive connection with "ftp -p" instead, see if it helps...

There we go:

$ ftp -vdp b.b.b.b
ftp: setsockopt: Bad file descriptor
Name: ftp
---> USER ftp
331 Please specify the password.
Password:
---> PASS XXXX
230 Login successful.
---> SYST
215 UNIX Type: L8
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd support
---> CWD support
250 Directory successfully changed.
ftp> cd 211211
---> CWD 211211
250 Directory successfully changed.
ftp> put TEST.txt
local: TEST.txt remote: TEST.txt
---> TYPE I
200 Switching to Binary mode.
ftp: setsockopt (ignored): Permission denied
---> PASV
227 Entering Passive Mode (b,b,b,b,46,248).
---> STOR TEST.txt
150 Ok to send data.
226 Transfer complete.
26 bytes sent in 0.00 secs (12.5386 kB/s)
ftp> quit
---> QUIT
221 Goodbye.

it worked !!!

I felt embarrassed. Time to search for FTP passive vs active…

Really good explanation. I hope I will never forget it.

  • FTP Active: The client issues a PORT command to the server signalling that it will “actively” provide an IP and port number so the server opens the Data Connection back to the client.
  • FTP Passive: The client issues a PASV command to indicate that it will wait “passively” for the server to supply an IP and port number, after which the client opens a Data Connection to the server.

So it worked in my VM because somehow the ftp server sent a PASV command (maybe because it detects there is no NAT as I have a public IP???).

From home, it failed because, by default, the connection is ftp active, so when the server tried to open the new data connection to me(something I couldnt see in the packet capture…) it failed as my ADSL modem wouldnt allow inbound connections.

Once I enabled “-p” in my connection to the server, all worked because it was me who started the new data connection and my firewall allows everything outbound.

Happy to solve the problem after a couple of years, and after a couple of hours of “serious” troubleshooting. It was shocking how blind I was. I had the ftp error message and the PASV from the trace.

Anyway, I learned something new.

Docker MTU + Docker tcpdump

I am troubleshooting an issue in a docker setup with some Arista cEOS where I can’t ping inside a VRF. First I though it was a MTU issue as when you use MPLS, there is an extra tag in the L2 frame.

…But my pings weren’t that big.

Still wanted to increase the MTU because that’s the expected thing to do in your WAN links if you run MPLS and want your users in different VRFs to be able to use the full 1500 bytes.

After some searching, It seems you can change the default value using the config file as per this link:

$ ip link show docker0
9: docker0: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
link/ether 02:42:be:73:8c:d3 brd ff:ff:ff:ff:ff:ff
$ cat /etc/docker/daemon.json
{
"data-root": "/home/somebody/storage/docker",
"mtu": 1600
}
$ sudo service docker restart
..
$ ip link show docker0
9: docker0: mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
link/ether 02:42:fb:c0:cf:a2 brd ff:ff:ff:ff:ff:ff

And restart docker. But still had mtu 1500. Checking another link it seems I actually need to create a container so the bridge come up with the new value

$ docker run -d busybox top
...
9: docker0: mtu 1600 qdisc noqueue state UP mode DEFAULT group default
link/ether 02:42:fb:c0:cf:a2 brd ff:ff:ff:ff:ff:ff

Funny thing, once I started my lab again (using docker-topo) still got MTU 1500!!!

Will have to dig a bit why docker-topo doesnt take the docker mtu 1600 from the config file.

Solution: docker-topo is creating user-defined bridges, so it needs to be told that the mtu is different. The “mtu:1600” in the docker config it is only for the default bridge so when you start the busybox, it is attached to the default bridge and you see 1600.

The other thing I was curious was if I could tcpdump the networks created by docker.

Yes, you can!

# docker network ls

# ifconfig 

# tcpdump -i br-xxxx