MSI UNIX and PBS Cheat Sheet ----------------------------------------------------------------------------- Help on any Unix command. man {command} Type "man ls" to read the manual for the ls command. man {command} > {filename} Redirect help to a file to download. whatis {command} Give short description of command. apropos {keyword} Search for all Unix commands that match keyword, eg "apropos file". ----------------------------------------------------------------------------- List a directory ls {path} It's ok to combine attributes, eg "ls -laF" gets a long listing of all files with types. ls {path_1} {path_2} List both {path_1} and {path_2}. ls -lh {path} Long listing, with date, size (human readable) and permisions. ls -a {path} Show all files, including important .dot files that don't otherwise show. ls -F {path} Show type of each file. "/" = directory, "*" = executable. ls -R {path} Recursive listing, with all subdirs. ls {path} > {filename} Redirect directory to a file. ls {path} | more Show listing one screen at a time. ls -lrt List files in chronological order with the most recently edited files at the bottom. ----------------------------------------------------------------------------- Change to directory cd {dirname} There must be a space between. cd ~ Go back to home directory, useful if you're lost. cd .. Go back one directory. ----------------------------------------------------------------------------- Make a new directory mkdir {dirname} ----------------------------------------------------------------------------- Remove a directory rmdir {dirname} Only works if {dirname} is empty. rm -r {dirname} Remove all files and subdirs. Careful! ----------------------------------------------------------------------------- Print working directory pwd Show where you are as full path. Useful if you're lost or exploring. ----------------------------------------------------------------------------- Copy a file or directory cp {file1} {file2} cp -r {dir1} {dir2} Recursive, copy directory and all subdirs. cat {newfile} >> {oldfile} Append newfile to end of oldfile. ----------------------------------------------------------------------------- Move or rename a file mv {oldfile} {newfile} Moving a file and renaming it are the same thing. mv {oldname} {newname} ----------------------------------------------------------------------------- Delete a file rm {filespec} ? and * wildcards work like DOS should. "?" is any character; "*" is any string of characters. ls {filespec} Good strategy: first list a group to rm {filespec} make sure it's what's you think... then delete it all at once. ----------------------------------------------------------------------------- View a text file more {filename} View file one screen at a time. less {filename} Like more, with extra features. cat {filename} View file, but it scrolls. cat {filename} | more View file one screen at a time. ----------------------------------------------------------------------------- Create a text file. cat > {filename} Enter your text (multiple lines with enter are ok) and press control-d to save. pico {filename} Create some text and save it. ----------------------------------------------------------------------------- Compare two files diff {file1} {file2} Show the differences. sdiff {file1} {file2} Show files side by side. ----------------------------------------------------------------------------- Other text commands grep '{pattern}' {file} Find regular expression in file. sort {file1} > {file2} Sort file1 and save as file2. sort -o {file} {file} Replace file with sorted version. spell {file} Display misspelled words wc {file} Count words in file. ----------------------------------------------------------------------------- Find files on system find {filespec} Works with wildcards. Handy for snooping. find {filespec} > {filename} Redirect find list to file. Can be big! ----------------------------------------------------------------------------- Wildcards and Shortcuts * Match any string of characters, eg "page*" gets page1, page10, and page.txt. ? Match any single character, eg "page?" gets page1 and page2, but not page10. [...] Match any characters in a range, eg "page[1-3]" gets page1, page2, and page3. ~ Short for your home directory, eg "cd ~" will take you home, and "rm -r ~" will destroy it. . The current directory. .. One directory up the tree, eg "ls ..." ----------------------------------------------------------------------------- Pipes and Redirection You pipe a command to another command, and redirect it to a file. {command} > {file} Redirect output to a file, eg "ls > list.txt" writes directory to file. {command} >> {file} Append output to an existing file, eg "cat update >> archive" adds update to end of archive. {command} < {fil1} Get input from a file, eg "sort < file.txt" {command} < {file1} > {file2} Get input from file1, and write to file2, eg "sort < old.txt > new.txt" sorts old.txt and saves as new.txt. {command} | {command} Pipe one command to another, eg "ls | more" gets directory and sends it to more to show it one page at a time. ----------------------------------------------------------------------------- Permissions, important and tricky! Unix permissions concern who can read a file or directory, write to it, and execute it. Permissions are granted or withheld with a magic 3-digit number. The three digits correspond to the owner (you); the group (?); and the world (everyone else). Think of each digit as a sum: execute permission = 1 write permission = 2 write and execute (1+2)= 3 read permission = 4 read and execute (4+1)= 5 read and write (4+2)= 6 read, write and execute (4+2+1)= 7 Add the number value of the permissions you want to grant each group to make a three digit number, one digit each for the owner, the group, and the world. Here are some useful combinations. Try to figure them out! chmod 600 {filespec} You can read and write; the world can't. Good for files. chmod 700 {filespec} You can read, write, and execute; the world can't. Good for scripts. chmod 644 {filespec} You can read and write; the world can only read. Good for web pages. chmod 755 {filespec} You can read, write, and execute; the world can read and execute. Good for programs you want to share, and your public_html directory. ----------------------------------------------------------------------------- Permissions, another way You can also change file permissions with letters: u = user (yourself) g = group a = everyone r = read w = write x = execute chmod u+rw {filespec} Give yourself read and write permission chmod u+x {filespec} Give yourself execute permission. chmod a+rw {filespec} Give read and write permission to everyone. ----------------------------------------------------------------------------- MSI System Info groupquota Used to check how much storage space your group has used. groupquota -h will display the help documentation acctinfo Used to check how man SUs your group has used. acctinfo -h will display the help documentation. ----------------------------------------------------------------------------- Unix Directory Format Long listing ("ls -lh”) format - file d directory, * executable ^ symbolic links (?) file size (human) file name / directory ^ ^ ^ ^ ^ drwxr-xr-x 11 mkummel 2560 Mar 7 23:25 public_html/ -rw-r--r-- 1 mkummel 10297 Mar 8 23:42 index.html ^ ^^^ user permission (rwx) date and time last modified ^^^ group permission (rwx) ^^^ world permission (rwx) ----------------------------------------------------------------------------- Dotfiles (aka Hidden Files) Dotfile names begin with a "." These files and directories don't show up when you list a directory unless you use the -a option, so they are also called hidden files. Type "ls -la" in your home directory to see what you have. Some of these dotfiles are crucial. They initialize your shell and the programs you use, like autoexec.bat in DOS and .ini files in Windows. rc means "run commands". These are all text files that can be edited, but change them at your peril. Make backups first! Here's some of what I get when I type "ls -la": .bashrc startup script for the bash shell. IMPORTANT .history list of past commands. ----------------------------------------------------------------------------- PBS commands Connect to the Lab system and start an “interactive job” that will let you “do stuff” isub default isub settings will get you 2GB of memory and 2 hours of walltime. Your isub session will end when your wall time expires and you will be booted from the lab machine even if you are in the middle of something. isub -m 8GB -w 4:00:00 This isub commands asks for 8GB of memory and 4 hours of time. Submit a job to any of the queues on both the Lab and Itasca systems. qsub Command used to submit a job to the PBS queuing system. man qsub will display all of the qsub commands Most often used qsub flags. -l nodes=1:ppn=8,mem=15GB,walltime=4:00:00 -l Resource list, these commands sets the parameters for the size of the computer that you are reserving. -m abe PBS will send you an email with the job aborts, begins and ends -e trimmomatic.error Names the error file. Important file will contain error messages. -o trimmomatic.out Names the output files. Important files that will contain run info. -N trimmomatic Names the job, will be displayed with qstat is used. Check on the status of a job qstat -u yourUsername This will show you the status of your jobs. Q = queued, R= running, C = canceled. showq Shows the status of the entire queue.