Reinstalling Debian

Two weeks ago I started having an issue regarding “duplicate file trigger” with some packages related to gtk-2.0 when I was doing my routine “aptitude update/ dist-upgrade”. I use Debian Testing and I have seen in the past weird things with packages but then got fixed. I understand I am in “testing” so these things may happen. Somehow, I tried to take a look If I could understand the problem and fix it. Searching didnt give much info. I found quite old entries about the issue but nothing really clarifying why I ended there. I assume that I am not the only one having these problems in the Debian community so it should be “easier” to find info from more experience people than me. But most of the cases, I rely on old data.

Somehow I “fixed” that issue but I didnt dig enough to understand what it is this thing about “triggers”. So I messed around with “/var/lib/dpkg/triggers/File” but I did a rookie mistake. I didnt do a backup of the file…. (how difficult is to type “cp File File-backup”). I remember that I had to remove several entries and they related to i386 architecture….

Well, then after a couple of days, I had a different issue. Not sure if this was triggered by my “triggers fix”.

Somehow I had some packages failing to install due to missing dependencies. It was mainly related to “python3”. So last Saturday I decided to go deep into it… And I made things worse. When you “aptitude purge” a package, you know the package and dependencies will be removed (if they are not used). While I was trying to solve my “death by dependencies” I deleted many packages that I was sure I needed. Again, I should have stopped or at least take notes of all those packages. At the end, I ended with “just” one issue. I couldnt install python3…. no idea why. In this path of destruction, I removed most of my X11 setup, so no graphic UI for me… But I didnt notice until too late. I tried to reinstall as many packages as possible and still python3 was failing. I have pyenv and I have used python3 so not sure why/how was broken. I reached a point that with so many changes, I had to reboot…. It is not the first time and after spending hours trying to fix something, that you are sure it is fixed, is still not fixed, then your reboot, and it is fixed. So I risked it, I knew that this would be fixed or would blow up big time. The later happened. And again, rookie mistake, I should have refreshed my hard drive backup, just in case you can’t access the laptop. But as my SSD looked healthy, the thought didnt cross my mind…

Reboot, got me stuck in “lightdm can’t start”….. and I couldnt login…. I was impatient. As later I learnt, I should wait a bit (long) for a time out and I would have seen my prompth.

I tried to get into rescue mode / single-user mode, and had to get by root. So I had a bit of access and I tried to run aptitude again…. I didnt have Internet access. Something I give for granted, when you login into your graphic UI, you have your nice network-manager that logs into your wifi. I always thought I was something simple like when you log into wired Internet access… I was totally wrong.

I was lucky that I had Internet in my mobile phone and that helped me a lot to move forwards with each issue. Without that, I would be still writing on paper.

So after a bit of struggling, I learned how to connect to a wifi via CLI. Some I was quite happy with that. I used this link. Thanks to the author!

1) Find out the wireless device name.
# iw dev
phy#0
	Unnamed/non-netdev interface
		wdev 0x5
		addr 60:57:18:00:9d:8a
		type P2P-device
		txpower 0.00 dBm
	Interface wlp2s0
		ifindex 2
.....

2) Check status of wireless interface.
# ip link show wlp2s0
2: wlp2s0: (BROADCAST,MULTICAST) mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether aa:aa:aa:aa:aa:aa brd ff:ff:ff:ff:ff:ff

3) Bring up interface if you dont see "UP"
# sudo ip link set wlp2s0 up  

# ip link show wlp2s0
2: wlp2s0: (NO-CARRIER,BROADCAST,MULTICAST,UP) mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether aa:aa:aa:aa:aa:aa brd ff:ff:ff:ff:ff:ff

4) Check the connection status.
# iw wlp2s0 link
Not connected.

5) Scan to find your local wifi. You need to search for the SSID that you want to connect. This can be a long sentence...
# iw wlp2s0 scan
....
BSS bb:bb:bb:bb:bb:bb(on wlp2s0) -- associated
	last seen: 136826.740s [boottime]
	TSF: 10582137299 usec (0d, 02:56:22)
	freq: 5620
	beacon interval: 100 TUs
	capability: ESS Privacy SpectrumMgmt APSD (0x0911)
	signal: -76.00 dBm
	last seen: 12 ms ago
	Information elements from Probe Response frame:
	SSID: ZZZZZZ PROVIDER SPEED PRODUCT 10Ghz
	Supported rates: 6.0* 9.0 12.0* 18.0 24.0* 36.0 48.0 54.0 
	DS Parameter set: channel 124
	HT capabilities:
		Capabilities: 0x0f
			RX LDPC
			HT20/HT40
			SM Power Save disabled
			No RX STBC
			Max AMSDU length: 3839 bytes
			No DSSS/CCK HT40
		Maximum RX AMPDU length 65535 bytes (exponent: 0x003)
		Minimum RX AMPDU time spacing: 4 usec (0x05)
		HT RX MCS rate indexes supported: 0-15, 32
		HT TX MCS rate indexes are undefined
	HT operation:
		 * primary channel: 124
		 * secondary channel offset: above
		 * STA channel width: any
		 * RIFS: 0
		 * HT protection: no
		 * non-GF present: 1
		 * OBSS non-GF present: 0
		 * dual beacon: 0
		 * dual CTS protection: 0
		 * STBC beacon: 0
		 * L-SIG TXOP Prot: 0
		 * PCO active: 0
		 * PCO phase: 0
	RSN:	 * Version: 1
		 * Group cipher: CCMP
		 * Pairwise ciphers: CCMP
		 * Authentication suites: PSK
		 * Capabilities: 1-PTKSA-RC 1-GTKSA-RC (0x0000)
......
	WPS:	 * Version: 1.0
		 * Wi-Fi Protected Setup State: 2 (Configured)
		 * Response Type: 3 (AP)
		 * UUID: bc329e00-1dd8-11b2-8601-38549b39bdd4
		 * Manufacturer: ABCD
		 * Model: ABCD GateWay
		 * Model Number: V1.0
		 * Serial Number: 12345678
		 * Primary Device Type: ABABABAB
		 * Device name: ABCD AP
		 * Config methods: PBC, Keypad
		 * RF Bands: 0x2
		 * Version2: 2.0
.....

As the blog mentions, the two important things here are SSID and RSN/WPS. I dont think it is common these days to find WEP Wifi. Connecting to a WEP it is far much easier.


6) Connect to WPA/WPA2 WiFi network: Generate wpa_supplicant config file
Generate a configuration file for wpa_supplicant that contains the pre-shared key ("passphrase") for the WiFi network.

# wpa_passphrase "ZZZZZZ PROVIDER SPEED PRODUCT 10Ghz" >> /etc/wpa_supplicant.conf 
...type in the passphrase and hit enter...

wpa_passphrase takes the SSID as the single argument. You must type in the passphrase for the WiFi network after you run the command. Using that information, wpa_passphrase will output the necessary configuration statements to the standard output. Those statements are appended to the wpa_supplicant configuration file located at /etc/wpa_supplicant.conf.

# cat /etc/wpa_supplicant.conf 
# reading passphrase from stdin
network=
	ssid="ZZZZZZ PROVIDER SPEED PRODUCT 10Ghz"
	#psk="mytest123"
	psk=aaaaaasuperlongkeybbbbbbbbbbbbbbbbbbb
}


7) Connect using wpa_supplicant config file.
# sudo wpa_supplicant -B -D wext -i wlp2s0 -c /etc/wpa_supplicant.conf

-B means run wpa_supplicant in the background.
-D specifies the wireless driver. wext is the generic driver.
-c specifies the path for the configuration file.


8) Verify you are attached to your Wifi SSID
# iw wlp2s0 link
Connected to bb:bb:bb:bb:bb:bb (on wlp2s0)
	SSID: ZZZZZZ PROVIDER SPEED PRODUCT 10Ghz
	freq: 2412
	RX: 63825 bytes (471 packets)
	TX: 1344 bytes (12 packets)
	signal: -27 dBm
	tx bitrate: 6.5 MBit/s MCS 0

	bss flags:	short-slot-time
	dtim period:	0
	beacon int:	100

9) Obtain IP address by DHCP and verify IP
# dhclient wlp2s0
#
# ip addr show wlp2s0
2: wlp2s0:  mtu 1500 qdisc mq state UP qlen 1000
    link/ether aa:aa:aa:aa:aa:aa brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.3/24 brd 192.168.1.255 scope global wlp2s0
    inet6 zzz/64 scope link 
       valid_lft forever preferred_lft forever

10) Check routing / Add default if needed:
# ip route show
192.168.1.0/24 dev wlp2s0  proto kernel  scope link  src 192.168.1.3 

The above routing table contains only 1 rule which redirects all traffic destined for the local subnet (192.168.1.x) to the wlp2s0 interface. You may want to add a default routing rule to pass all other traffic through wlp2s0 as well.

# ip route add default via 192.168.1.254 dev wlp2s0
# ip route show
default via 192.168.1.254 dev wlp2s0 
192.168.1.0/24 dev wlp2s0  proto kernel  scope link  src 192.168.1.3 

11) Check external connecitivity
# ping 8.8.8.8

After being able to get Internet access I carried on troubleshooting. I was stuck with python3 issue and now ligthdm not starting up… Using this link, I could see lightdm was complaining about an issue with Xorg. So then I checked the logs of Xorg.

[    34.050] (II) LoadModule: "glx"
[    34.050] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[    34.050] (EE) Failed to load /usr/lib/xorg/modules/extensions/libglx.so: libGL.so.1: cannot open shared object file: No such file or directory
[    34.050] (EE) Failed to load module "glx" (loader failed, 0)

I searched for info about that error. I followed several pages like this and this, I installed an reinstalled the packages mentioned, but not way could fix that “glx”.

After sometime, I decided to work on the “python3” issue. I tried to follow some advice like this. Again I tried to purge as many packets as I could and reinstall everything python related. Same result, I couldn’t install python3. And weird enough, my wifi used to be disconnected whenever removed python so I had reboot and attach to the wifi a couple of times. And as well, I managed to “lock” my wifi card too, so bizarre!!! I think I followed something from here to unlock.

So I reached a point where I was totally stuck. I couldnt fix the Xorg issue and turn up the graphic desktop environment and python3 was very broken. I decided to reinstall, I was getting nowhere. I made full backup of my homedir and most of the system folders and take some logs about the current config of my system and packages currently installed so I could help me to setup the new environment quicker.

So I downloaded a netinstall iso for Debian testing. Thanks to my mobile, I managed to find the link and I “wget” it via my cli. Then I needed to install that iso in a USB so the Debian documentation was quite easy…

# cp debian-testing-amd64-netinst.iso /dev/sdb1
# sync

Ok, it was time to boot up the usb and resintall…. not that easy…. my laptop was not able to see the usb at boot time….. I checked the BIOS and USB was allowed as boot device…. Back to the shell, I could mount the USB, fdisk showed the partition had the flag enabled for booting.

I decided to find another method. I was pretty sure that in the past when creating a boot USB I had to use “dd”… so following this link:

# dd if=debian-testing-amd64-netinst.iso of=/dev/sdb1 bs=4M conv=fdatasync status=progress

The I tried to boot again, and worked!!!

So started the reinstall process. I chose to use the whole disk and the trigger a process to overwrite my whole disk…. I had to leave the laptop overnight because the process was superslow! I didnt know how to skip it….

Something I still haven’t learned is how to scroll up/down in the tty !!! During the whole troubleshooting time, it was quite frustrating because using “less” didnt work always with all my commands… I tried to find some answers like this but no joy.

Next night I follow up. Obviously the installation process didnt detect my wifi card (non-free drivers my love) and I dont have a RJ45 port… so I had to use my usb-rj45 adaptor and long rj45 cable to get back to the internet and complete the installation…

Once the process finished, you end up with a very basic system… no wifi, no Xorg, just pure CLI environment.

So started trying to install some packages I use more often and get my wifi card detected and working. As I had my backup, I could update my apt/sources.list file and get the repositories I needed.

  104  vim /etc/apt/sources.list
  105  aptitude update 
  106  aptitude dist-upgrade 
  108  aptitude install firmware-iwlwifi 
  109  lsmod | grep iw
  110  iw list
  116  reboot

I installed “lightdm” as my display manager. I finally managed to see it again! I logged in and nothing happend. I forgot to install my lightweith desktop environment… lxde.

So finally, I was back to my graphic environment. But again I forgot to install the network-manager to get my Internet connectivity sorted as I was using the CLI technique before.

One final issue was the sound card, I could see videos online but didnt have sound so I was pretty sure I was missing something again, “alsa-utils” and then with “pavucontrol” I noticed the output devices were muted.

So finally got basic functionality. I haven’t moved all my data yet as I need to create my ZFS partition so I will need a bit more tinkering in the coming days.

This is the list of installed packages so far:

aptitude install lightdm
aptitude install sudo
aptitude install gedit
aptitude install tcpdump
aptitude install nmap
aptitude install terminator
aptitude install gkrellm
aptitude install keepass2
aptitude install firefox-esr
aptitude install vagrant
aptitude install mutt
aptitude install screen
aptitude install tmux
aptitude install wpasupplicant 
aptitude install iw
aptitude install minicom
aptitude install lxde
aptitude install pavucontrol
aptitude install libreoffice
aptitude install lstopo
aptitude install apt-file
aptitude install hwloc
aptitude install parted
aptitude install ristretto 
aptitude install network-manager-gnome 
aptitude install net-tools
aptitude install vim
aptitude install git
aptitude install tig
aptitude install firmware-iwlwifi 
aptitude install qemu
aptitude install alsa-tools-gui 
aptitude install alsa-utils 
aptitude install brightnessctl 
aptitude install zfs-dkms 
aptitude install gkrellm-volume 

Things I want to learn:

  • How to setup a “timeshift” (like Apple) for backup in Debian
  • How to scroll in linux terminal tty
  • Send emails with mutt using gmail (and with attachments) so I dont need GUI.

Things I need to setup

  • Have a USB ready with a Debian Testing ISO
  • How to backup my Android mobile phone and erase contents if I lose it.
  • Ideally I should rebuild my laptop more frequently so I should have a PXE-like process. Maybe having some kind of manifest in my git repo. Maybe I overthinking it.