I have a supplier at my employer that requires to use a FTP server to send big files when you open a support ticket. For a long time (a couple of years) whenever I had to upload big files, I had to use my personal VM because my ftp connections failed from the office. I always blamed the super-smart firewall.
One day, I decided to fix the issue and allow the connection in our corporate firewall. I failed. Still couldnt upload files from the office. So keep using my personal VM.
This week I had to upload again a big file. This time I am working from home, so pretty much it is going to work the upload. Wrong! It fails. Ok, I checked a bit and got to the conclusion that it is my ISP or modem at home that is blocking FTP. Most ISP use CGN to stretch as much as possible the limited IPv4. I have IPv6 at home and my VM has IPv6 too… but the ftp server doesnt.
I checked the internet if there was any know issue with my ISP and FTP connections. No luck. I connected to my modem, nothing obvious messing around with FTP.
I decided to give it a proper go to this issue. I knew that it worked from my VM and it didnt from home. I noticed that I was running the same ftp client version in the VM and at home. So let’s debug the ftp client and take a packet capture in both locations.
CLI from the VM:
$ ftp -vd b.b.b.b ftp: setsockopt: Bad file descriptor Name: ftp ---> USER ftp 331 Please specify the password. Password: ---> PASS XXXX 230 Login successful. ---> SYST 215 UNIX Type: L8 Remote system type is UNIX. Using binary mode to transfer files. ftp> cd support ---> CWD support 250 Directory successfully changed. ftp> cd 211211 ---> CWD 211211 250 Directory successfully changed. ftp> put TEST.txt local: TEST.txt remote: TEST.txt ---> TYPE I 200 Switching to Binary mode. ftp: setsockopt (ignored): Permission denied ---> PORT a,a,a,a,162,57 200 PORT command successful. Consider using PASV. ---> STOR TEST.txt 150 Ok to send data. 226 Transfer complete. 28 bytes sent in 0.00 secs (854.4922 kB/s) ftp> quit ---> QUIT
And this is the packet capture:
After typing “put” in packet 33, I see a “PASV” message from the server and a new connection (initiated by the server!) is established for the data transfer. All good.
So now, make the same from home and compare.
CLI from home without debug:
$ ftp b.b.b.b Connected to b.b.b.b. Name: ftp 331 Please specify the password. Password: 230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. ftp> cd support 250 Directory successfully changed. ftp> cd 211211 250 Directory successfully changed. ftp> put TEST.txt local: TEST.txt remote: TEST.txt 500 Illegal PORT command. ftp: bind: Address already in use ftp> quit 221 Goodbye.
CLI from home with debug:
$ ftp -vd b.b.b.b ftp: setsockopt: Bad file descriptor Name: ftp ---> USER ftp 331 Please specify the password. Password: ---> PASS XXXX 230 Login successful. ---> SYST 215 UNIX Type: L8 Remote system type is UNIX. Using binary mode to transfer files. ftp> cd support ---> CWD support 250 Directory successfully changed. ftp> cd 211211 ---> CWD 211211 250 Directory successfully changed. ftp> put TEST.txt local: TEST.txt remote: TEST.txt ---> TYPE I 200 Switching to Binary mode. ftp: setsockopt (ignored): Permission denied ---> PORT 192,168,1,158,202,145 500 Illegal PORT command. ftp: bind: Address already in use ftp> quit ---> QUIT 221 Goodbye.
So with and without debug I keep seeing “ftp: bind: Address already in use”…..
And this is the packet capture from home:
So after I type “put” in packet 32, the answer from the server is a “500”.
I wasnt clearly paying attention to the clues. I was still banging my head why the server was sending a “500 Ilegal PORT command”.
I was comparing both captures and both debug outputs… but still didnt it.
I thought I understood FTP. I knew that you use port TCP 21 to establish the control session and the data session / transfer is via new TCP session using a random port. That’s one of the reasons that using NAT or CGN can screw up your FTP sessions.
So I assumed that the issues wasnt my ISP. So it had to be my side (or me).
So finally, I decided to search for “ftp: bind: Address already in use” as it was the message that came up with and without debugging.
Oh boy, first entry in the face!
An entry from 2004…. it can’t fix my problem for sure…. keep reading and update from 2020… it says it works…. oh boy II
try using a passive connection with "ftp -p" instead, see if it helps...
There we go:
$ ftp -vdp b.b.b.b ftp: setsockopt: Bad file descriptor Name: ftp ---> USER ftp 331 Please specify the password. Password: ---> PASS XXXX 230 Login successful. ---> SYST 215 UNIX Type: L8 Remote system type is UNIX. Using binary mode to transfer files. ftp> cd support ---> CWD support 250 Directory successfully changed. ftp> cd 211211 ---> CWD 211211 250 Directory successfully changed. ftp> put TEST.txt local: TEST.txt remote: TEST.txt ---> TYPE I 200 Switching to Binary mode. ftp: setsockopt (ignored): Permission denied ---> PASV 227 Entering Passive Mode (b,b,b,b,46,248). ---> STOR TEST.txt 150 Ok to send data. 226 Transfer complete. 26 bytes sent in 0.00 secs (12.5386 kB/s) ftp> quit ---> QUIT 221 Goodbye.
it worked !!!
I felt embarrassed. Time to search for FTP passive vs active…
Really good explanation. I hope I will never forget it.
- FTP Active: The client issues a PORT command to the server signalling that it will “actively” provide an IP and port number so the server opens the Data Connection back to the client.
- FTP Passive: The client issues a PASV command to indicate that it will wait “passively” for the server to supply an IP and port number, after which the client opens a Data Connection to the server.
So it worked in my VM because somehow the ftp server sent a PASV command (maybe because it detects there is no NAT as I have a public IP???).
From home, it failed because, by default, the connection is ftp active, so when the server tried to open the new data connection to me(something I couldnt see in the packet capture…) it failed as my ADSL modem wouldnt allow inbound connections.
Once I enabled “-p” in my connection to the server, all worked because it was me who started the new data connection and my firewall allows everything outbound.
Happy to solve the problem after a couple of years, and after a couple of hours of “serious” troubleshooting. It was shocking how blind I was. I had the ftp error message and the PASV from the trace.
Anyway, I learned something new.