Linux

Commands

  • a command that prints the first n lines of a file
head -n 5 <file.txt>  # prints the first 5 lines of <file.txt>

awk

  • a command that you can use to select particular records in a file and perform operations upon them.
awk '{print $1 " " $2 " " $3 " " $4}' file.csv # prints the first 4 columns of file.csv
awk '{print $1 " " $2 " " $3 " " $4}' file.csv | head -n 5  # prints the first 5 rows and 4 columns of file.csv

tail

  • a command that prints the last n lines of a file.
tail -100 file.txt      # Read last 100 lines, strongly suggest to use tmux so that you can go scroll up
tail -100 file.txt > dumpfile.txt     # Copy last 100 lines to dumpfile.txt
tail -f file.txt      # Not to stop tail command even when it reaches the EOF (end of file). Useful while reading the appending file

more

  • a command that displays the first page of a file.
more <file.txt>

watch

  • a command that monitors a file for changes.
watch -n 1 -d <file.txt>      # Monitor <file.txt> every 1 second and display the changes

ln

  • a command-line utility for creating links between files and directories
  • Hard link: a link to a file or directory that points to the same inode as the original file or directory
  • Soft link: a link to a file or directory that points to a different inode than the original file or directory (like a shortcut)
  • By default, ln creates hard links.
$ ln -s /mnt/<YourPartition> ~/<folder to which you want to symlink>   # To create a symlink folder of mount partition

Note: ~ and $HOME are home directory’s synonyms.

nohup

  • To dump terminal output to a file
nohup <command> > log.out &                      # Replace <command> by command name and dump the terminal output to log file named log.out. You can change the name and path of log file.
nohup <command> > log.out & echo $! > save.pid   # Save the PID of the nohup command to save_pid.txt OR another way is using jobs command

# If you are using tmux and you are in the same terminal session where you ran nohup then, you can use jobs command
jobs -l                                          # To list all the jobs with their PIDs

chmod

  • To change the permission of the file
chmod 755 <executable filename>    # Change the permission so you can execute it.
chmod u+x <executable filename>    # Another method to add execute permission to the file.

top

  • To monitor the CPU and memory usage of the Linux system
top -H -p <pid>   # To view all the threads by program process id.

ssh

  • To connect to a remote Linux system
ssh <username>@<hostname>   # To connect to a remote Linux system
ssh <username>@<hostname> -p <port>   # To connect to a remote Linux system with a specific port
ssh <username>@<hostname> -t <command>   # To execute a command on a remote Linux system.
ssh -tt <username1>@<hostname1> "ssh <username2>@<hostname2>"  # To connect directly to hostname2 with username2.

# If you see ths below error:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
# Then, remove the line where you see the hostname from the `known_hosts` file.
nano ~/.ssh/known_hosts
# Now, in the next time you will connect to that host, you will be asked to generate a new key and you approve it.

scp

  • Copying files via secure copy
  • Be careful while copying files that share the same name and location on both systems, scp will overwrite files without warnings.
scp <filename> <remote username>@<remote ip>:<remote path to copy>  # Copy a local file to a remote system
scp <filename> <remote username>@<remote ip>:./<folder name>  # Copy a local file to remote system of a 1-level folder from home directory
scp <remote username>@<remote ip>:<path of remote file> <local path to copy>  # Copy a remote file to a local system
scp <remote 1 username>@<remote 1 ip>:<path of remote file> <remote 2 username>@<remote 2 ip>:<remote 2 path to copy>  # Copy a file from remote 1 to remote 2

sshfs

  • SSH Filesystem, a filesystem that allows you to mount remote filesystems locally
  • In windows, use sshfs-win. i.e. in Map Network Drive option, add in Folder field to be \\sshfs.r\<remote username>@<remote ip> or \\sshfs.r\<remote username>@<remote ip>\<folder path from home directory>.
  • Note: proxy jump is not supported by Windows network drive. But, there is a tricky way: first you mount the <remote 2> in the <remote 1> via sshfs linux command, and then you mount the <remote 1> in Windows via Map Network Drive.
  • Microsoft’s vscode via Remote - SSH is a good alternative for proxy jump in Windows.
sshfs <remote username>@<remote ip>:<remote path> <local path>   # Mount remote path to local path
fusermount -u <local path>   # Unmount local path

# jumphost or proxy jump
sshfs <remote 2 username>@<remote 2 ip>:<remote path> ~/<local folder path to mount> -o ssh_command='ssh -o PreferredAuthentications=password -J <remote 1 username>@<remote 1 ip>'  # Mount remote path to local path with proxy jump. Here we have: remote 1 is the jumphost and remote 2 is the remote server that we want to mount. (i.e. localhost -> remote 1 -> remote 2)

# You can do the same thing by including below settings in your ~/.ssh/config file:
###--- Start here ---###
Host jumphost
    Hostname <remote 1 ip>
    User <remote 1 username>
    Port <port number, default is 22, optional, you can skip>
    ForwardAgent yes

Host drive
    Hostname <remote 2 ip>
    User <remote 2 username>
    Port <port number, default is 22, optional, you can skip>
    ForwardAgent yes
    ProxyJump jumphost
###--- End here ---###
# Note that ForwardsAgent is optional, if you don't want to forward agent, you can remove it. 'jumphost' and 'drive' is just a random name I gave.

wc

  • To count the number of lines, words and characters in a file
wc -l <file>    # To count the number of lines in a file
ls -1 | wc -l   # To count the number of files in a directory
                # (how?: `ls -1` to list all files in a directory and
                # `wc -l` to count the number of files.
                # Note: | to pipe the output of ls to wc)

ocrmypdf

  • Install ocrmypdf from here
  • Adds an optical character recognition (OCR) text layer to scanned PDF files, allowing them to be searched. It is based on Google’s Tesseract OCR engine.
ocrmypdf <input file> <output file>  # To convert a PDF file to a PDF file with OCR text layer.
ocrmypdf --force-ocr <input file> <output file>  # To convert a PDF file which has already with text layer to a PDF file with OCR text layer
ocrmypdf --force-ocr --oversample 2000 filenamein.pdf filenameout.pdf  # To compress the PDF file. See https://ocrmypdf.readthedocs.io/en/latest/cookbook.html?highlight=--oversample%20#improving-ocr-quality

Extra

  • It is always a wise decision to follow Arch Linux installation from arch wikipage at here.
  • Check arch linux package version: pacman -Qi package-name.
  • Add full name of user in login manager: sudo chfn -f "<first name> <last name>" "<username>".
  • Use lsd pacakge to have icon when we want to list all files & folders inside a directory. Simply, this is an alternative to ls command. You can create an alias as ls for lsd in your shell config.
  • To extend Display/monitors: use the package mons.
  • To hide dotfiles in mc file manager: It’s annoying when I see dot(files | folders) all the time in Midnight Commander file manager (mc for short). So, to get rid off: Use ESC+period (.) or ALT+ period or click F9 -> O -> P -> h -> o.
  • vscode: Tab completion issue with Github Copilot? Goto settings -> Keyboard Shortcuts -> Search -> editor.action.inlineSuggest.commit -> double click “Keybinding” -> Click Tab Tab -> now double tab commit the copilot suggestion.

Extra elaborated

Use git to track the dotfiles

I use folder named .dotfiles to keep track my dotfiles and then, I upload to remote git repository (say, GitHub). Believe me, this is very easy to use. I strongly recommend to use period (.) infront of folder (or file) name, if the folder(or file) is not use very often such that it will be consider as hidden.

mkdir $HOME/.dotfiles
git init --bare $HOME/.dotfiles
echo "alias config='/usr/bin/git --git-dir=$HOME/.dotfiles --work-tree=$HOME'" >> $HOME/.YourShell-rc-File

For example:

echo "alias config='/usr/bin/git --git-dir=$HOME/.dotfiles --work-tree=$HOME'" >> $HOME/.zshrc

OR,

Add this line in the .zshrc file:

alias config='/usr/bin/git --git-dir=$HOME/.dotfiles --work-tree=$HOME'"

I’ve created an alias named config which means instead of using the command git, we use config.

Usage:

First, go to the dotfile path to which you want to track.

config add .filename
config commit -m "+ .filename"

Then, add newly created remote repository in your local machine, after that push it.

config push origin master

Note: I use “+” in my commit message as “Added my”.

Mouse cursor slow issue

Description: Solve mouse cursor moving slowly while copying a large file to USB

Are you facing system freezes/ unresponsive/ unusable while copying the large file to your USB drive in Ubuntu 16.04 LTS. Don’t worry about it. This post will help you to solve this issue.

First of all, we have to open the terminal (ctrl+alt+T). Now, follow the steps below.

sudo su

This command will help you to prevent typing your sudo password again and again. Then, you’ll see;

/home/username#

Now, Type below command to edit the sysctl.conf file.

# gedit /etc/sysctl.conf

Then, a document will instantly open on your desktop. Add the below lines at the end of it. If you’re curious about what are these lines are? Then, Visit the post “Better Linux Disk Catching & Performance with vm.dirty_ratio & vm.dirty_background_ratio”.

vm.dirty_background_ratio = 5
vm.dirty_ratio = 10

Now, save and close it. In your terminal, Type the below command to see whether it is working or not.

# sysctl -p

Then, you’ll see;

# sysctl -p
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10

Hurray, You made it! Now, Type exit in your terminal to exit from sudo su state and again, type exit to close the terminal window. Finally, Copy your large file to test whether it’ll work or not! I hope it’ll work.

Uninstall MATLAB

Description: How to uninstall MATLAB in Ubuntu or other Linux distros?

Most of the software has script to uninstall the whole software but in MATLAB there is no uninstaller. So, in-order to remove MATLAB, you simply need to delete the MATLAB installation by following the instruction below:

Note: If you haven’t activated the license file then, simply escape the step 1 and 2.

  1. If you are running a license manager for MATLAB, shut down the license manager using the lmdown script located in MATLAB’s directory. If this folder doesn’t exist then, remember you’re not running the FlexNet license manager.

For example:

/usr/local/MATLAB/R2010b/etc/lmdown
  1. You should also remove any license manager files in the /var/tmp directory. These files should begin with lm_TMW.

For example:

rm /var/tmp/lm_TMW*
  1. Remove the entire root MATLAB directory using the rm -rf command. The -r argument stands for recursive. It means, it will remove what you ask, as well as all the files and directories underneath it. And, the -f argument stands for force. It means, it will ignore non-existent files and never prompt.

For example:

rm -rf /usr/local/MATLAB/R2010b
  1. Remove any symlinks to MATLAB binaries on the path.

For example:

rm /usr/local/bin/matlab /usr/local/bin/mcc /usr/local/bin/mex /usr/local/bin/mbuild

I hope this will work for you!

Remove ubuntu completely

Description: How to remove Ubuntu 16.04 LTS or even earlier version completely?

Suppose, you want to remove your Ubuntu version completely because of some reason, and format the drive too where the Ubuntu preserved its partition.

First, you have to know, this method will completely wipe your hard drive. So, if you have any important files then, move it to an external storage.

Simplest Steps for you:

  1. Boot from the Ubuntu Installer/Live CD

  2. Choose “Try Ubuntu” and it’ll show you the Live Desktop

  3. Open a terminal by using “ctrl+alt+T”

  4. Now, in your terminal,

Type:

sudo dd if=/dev/zero of=/dev/sda bs=1M count=8 && sync
  1. Shut-down it, by typing “shutdown now” in your terminal.

  2. Hooray! Now, you can install windows.

Last words: In above context, I supposed that you have not that old computer where it uses IDE cables for the hard drives (wide connector). If yes then, you have to use hda instead of “sda” in step 4.

Auto mount the external drives using udiskie

I use “udiskie” package (GUI for udisks2) to auto-mount my external drives, and in i3wm config, I added this line:

exec --no-startup-id udiskie -nas

where flags nas means notification, auto-mount and smart tray respectively.

I also created a symlink folder named “External Devices” in the home directory by doing these:

  1. Create a directory name as /media:
sudo mkdir /media
  1. Open nano by typing this:
sudo nano /etc/udev/rules.d/99-udisks2.rules

Paste this command and save it:

# UDISKS_FILESYSTEM_SHARED
# ==1: mount filesystem to a shared directory (/media/VolumeName)
# ==0: mount filesystem to a private directory (/run/media/$USER/VolumeName)
# See udisks(8)
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{UDISKS_FILESYSTEM_SHARED}="1"
  1. Create symlink as:
ln -s /media ~/External\ Devices

Permalink at https://www.physicslog.com/cs-notes/linux

Published on Nov 1, 2017

Last revised on Jul 12, 2022