Links

Links

Postby lik » Fri Dec 26, 2008 9:22 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

Re: Links

Postby lik » Thu Apr 09, 2009 2:56 pm

• apropos whatis Show commands pertinent to string. See also threadsafe
• man -t man | ps2pdf - > man.pdf make a pdf of a manual page
which command Show full path name of command
time command See how long a command takes
• time cat Start stopwatch. Ctrl-d to stop. See also sw
• nice info Run a low priority command (The "info" reader in this case)
• renice 19 -p $$ Make shell (script) low priority. Use for non interactive tasks
dir navigation
• cd - Go to previous directory
• cd Go to $HOME directory
(cd dir && command) Go to dir, execute command and return to current dir
• pushd . Put current dir on stack so you can popd back to it
file searching
• alias l='ls -l --color=auto' quick dir listing
• ls -lrt List files by date. See also newest and find_mm_yyyy
• ls /usr/bin | pr -T9 -W$COLUMNS Print in 9 columns to width of terminal
find -name '*.[ch]' | xargs grep -E 'expr' Search 'expr' in this dir and below. See also findrepo
find -type f -print0 | xargs -r0 grep -F 'example' Search all regular files for 'example' in this dir and below
find -maxdepth 1 -type f | xargs grep -F 'example' Search all regular files for 'example' in this dir
find -maxdepth 1 -type d | while read dir; do echo $dir; echo cmd2; done Process each item with multiple commands (in while loop)
• find -type f ! -perm -444 Find files not readable by all (useful for web site)
• find -type d ! -perm -111 Find dirs not accessible by all (useful for web site)
• locate -r 'file[^/]*\.txt' Search cached index for names. This re is like glob *file*.txt
• look reference Quickly search (sorted) dictionary for prefix
• grep --color reference /usr/share/dict/words Highlight occurances of regular expression in dictionary
archives and compression
gpg -c file Encrypt file
gpg file.gpg Decrypt file
tar -c dir/ | bzip2 > dir.tar.bz2 Make compressed archive of dir/
bzip2 -dc dir.tar.bz2 | tar -x Extract archive (use gzip instead of bzip2 for tar.gz files)
tar -c dir/ | gzip | gpg -c | ssh user@remote 'dd of=dir.tar.gz.gpg' Make encrypted archive of dir/ on remote machine
find dir/ -name '*.txt' | tar -c --files-from=- | bzip2 > dir_txt.tar.bz2 Make archive of subset of dir/ and below
find dir/ -name '*.txt' | xargs cp -a --target-directory=dir_txt/ --parents Make copy of subset of dir/ and below
( tar -c /dir/to/copy ) | ( cd /where/to/ && tar -x -p ) Copy (with permissions) copy/ dir to /where/to/ dir
( cd /dir/to/copy && tar -c . ) | ( cd /where/to/ && tar -x -p ) Copy (with permissions) contents of copy/ dir to /where/to/
( tar -c /dir/to/copy ) | ssh -C user@remote 'cd /where/to/ && tar -x -p' Copy (with permissions) copy/ dir to remote:/where/to/ dir
dd bs=1M if=/dev/sda | gzip | ssh user@remote 'dd of=sda.gz' Backup harddisk to remote machine
rsync (Network efficient file copier: Use the --dry-run option for testing)
rsync -P _rsync://rsync.server.com/path/to/file file Only get diffs. Do multiple times for troublesome downloads
rsync --bwlimit=1000 fromfile tofile Locally copy with rate limit. It's like nice for I/O
rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html' Mirror web site (using compression and encryption)
rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/ Synchronize current directory with remote one
ssh (Secure SHell)
ssh $USER@$HOST command Run command on $HOST as $USER (default command=shell)
• ssh -f -Y $USER@$HOSTNAME xeyes Run GUI command on $HOSTNAME as $USER
scp -p -r $USER@$HOST: file dir/ Copy with permissions to $USER's home directory on $HOST
ssh -g -L 8080:localhost:80 root@$HOST Forward connections to $HOSTNAME:8080 out to $HOST:80
ssh -R 1434:imap:143 root@$HOST Forward connections from $HOST:1434 in to imap:143
wget (multi purpose download tool)
• (cd dir/ && wget -nd -pHEKk http://www.pixelbeat.org/cmdline.html) Store local browsable version of a page to the current dir
wget -c http://www.example.com/large.file Continue downloading a partially downloaded file
wget -r -nd -np -l1 -A '*.jpg' http://www.example.com/dir/ Download a set of files to the current directory
wget _ftp://remote/file[1-9].iso/ FTP supports globbing directly
• wget -q -O- http://www.pixelbeat.org/timeline.html | grep 'a href' | head Process output directly
echo 'wget url' | at 01:00 Download url at 1AM to current dir
wget --limit-rate=20k url Do a low priority download (limit to 20KB/s in this case)
wget -nv --spider --force-html -i bookmarks.html Check links in a file
wget --mirror http://www.example.com/ Efficiently update a local copy of a site (handy from cron)
networking (Note ifconfig, route, mii-tool, nslookup commands are obsolete)
ethtool eth0 Show status of ethernet interface eth0
ethtool --change eth0 autoneg off speed 100 duplex full Manually set ethernet interface speed
iwconfig eth1 Show status of wireless interface eth1
iwconfig eth1 rate 1Mb/s fixed Manually set wireless interface speed
• iwlist scan List wireless networks in range
• ip link show List network interfaces
ip link set dev eth0 name wan Rename interface eth0 to wan
ip link set dev eth0 up Bring interface eth0 up (or down)
• ip addr show List addresses for interfaces
ip addr add 1.2.3.4/24 brd + dev eth0 Add (or del) ip and mask (255.255.255.0)
• ip route show List routing table
ip route add default via 1.2.3.254 Set default gateway to 1.2.3.254
• tc qdisc add dev lo root handle 1:0 netem delay 20msec Add 20ms latency to loopback device (for testing)
• tc qdisc del dev lo root Remove latency added above
• host pixelbeat.org Lookup DNS ip address for name or vice versa
• hostname -i Lookup local ip address (equivalent to host `hostname`)
• whois pixelbeat.org Lookup whois info for hostname or ip address
• netstat -tupl List internet services on a system
• netstat -tup List active connections to/from system
windows networking (Note samba is the package that provides all this windows specific networking support)
• smbtree Find windows machines. See also findsmb
nmblookup -A 1.2.3.4 Find the windows (netbios) name associated with ip address
smbclient -L windows_box List shares on windows machine or samba server
mount -t smbfs -o fmask=666,guest //windows_box/share /mnt/share Mount a windows share
echo 'message' | smbclient -M windows_box Send popup to windows machine (off by default in XP sp2)
text manipulation (Note sed uses stdin and stdout. Newer versions support inplace editing with the -i option)
sed 's/string1/string2/g' Replace string1 with string2
sed 's/\(.*\)1/\12/g' Modify anystring1 to anystring2
sed '/ *#/d; /^ *$/d' Remove comments and blank lines
sed ':a; /\\$/N; s/\\\n//; ta' Concatenate lines with trailing \
sed 's/[ \t]*$//' Remove trailing spaces from lines
sed 's/\([\\`\\"$\\\\]\)/\\\1/g' Escape shell metacharacters active within double quotes
• seq 10 | sed "s/^/ /; s/ *\(.\{7,\}\)/\1/" Right align numbers
sed -n '1000p;1000q' Print 1000th line
sed -n '10,20p;20q' Print lines 10 to 20
sed -n 's/.*<title>\(.*\)<\/title>.*/\1/ip;T;q' Extract title from HTML web page
sed -i 42d ~/.ssh/known_hosts Delete a particular line
sort -t. -k1,1n -k2,2n -k3,3n -k4,4n Sort IPV4 ip addresses
• echo 'Test' | tr '[:lower:]' '[:upper:]' Case conversion
• tr -dc '[:print:]' < /dev/urandom Filter non printable characters
• history | wc -l Count lines
set operations (Note you can export LANG=C for speed. Also these assume no duplicate lines within a file)
sort file1 file2 | uniq Union of unsorted files
sort file1 file2 | uniq -d Intersection of unsorted files
sort file1 file1 file2 | uniq -u Difference of unsorted files
sort file1 file2 | uniq -u Symmetric Difference of unsorted files
join -a1 -a2 file1 file2 Union of sorted files
join file1 file2 Intersection of sorted files
join -v2 file1 file2 Difference of sorted files
join -v1 -v2 file1 file2 Symmetric Difference of sorted files
math
• echo '(1 + sqrt(5))/2' | bc -l Quick math (Calculate φ). See also bc
• echo 'pad=20; min=64; (100*10^6)/((pad+min)*8)' | bc More complex (int) e.g. This shows max FastE packet rate
• echo 'pad=20; min=64; print (100E6)/((pad+min)*8)' | python Python handles scientific notation
• echo 'pad=20; plot [64:1518] (100*10**6)/((pad+x)*8)' | gnuplot -persist Plot FastE packet rate vs packet size
• echo 'obase=16; ibase=10; 64206' | bc Base conversion (decimal to hexadecimal)
• echo $((0x2dec)) Base conversion (hex to dec) ((shell arithmetic expansion))
• units -t '100m/9.69s' 'miles/hour' Unit conversion (metric to imperial)
• units -t '500GB' 'GiB' Unit conversion (SI to IEC prefixes)
• units -t '1 googol' Definition lookup
• seq 100 | (tr '\n' +; echo 0) | bc Add a column of numbers. See also add and funcpy
calendar
• cal -3 Display a calendar
• cal 9 1752 Display a calendar for a particular month year
• date -d fri What date is it this friday. See also day
• [ $(date -d "tomorrow" +%d) = "01" ] || exit exit a script unless it's the last day of the month
• date --date='25 Dec' +%A What day does xmas fall on, this year
• date --date='@2147483647' Convert seconds since the epoch (1970-01-01 UTC) to date
• TZ=':America/Los_Angeles' date What time is it on West coast of US (use tzselect to find TZ)
echo "mail -s 'get the train' P@draigBrady.com < /dev/null" | at 17:45 Email reminder
• echo "DISPLAY=$DISPLAY xmessage cooker" | at "NOW + 30 minutes" Popup reminder
locales
• printf "%'d\n" 1234 Print number with thousands grouping appropriate to locale
• BLOCK_SIZE=\'1 ls -l get ls to do thousands grouping appropriate to locale
• echo "I live in `locale territory`" Extract info from locale database
• LANG=en_IE.utf8 locale int_prefix Lookup locale info for specific country. See also ccodes
• locale | cut -d= -f1 | xargs locale -kc | less List fields available in locale database
recode (Obsoletes iconv, dos2unix, unix2dos)
• recode -l | less Show available conversions (aliases on each line)
recode windows-1252.. file_to_change.txt Windows "ansi" to local charset (auto does CRLF conversion)
recode utf-8/CRLF.. file_to_change.txt Windows utf8 to local charset
recode iso-8859-15..utf8 file_to_change.txt Latin9 (western europe) to utf8
recode ../b64 < file.txt > file.b64 Base64 encode
recode /qp.. < file.txt > file.qp Quoted printable decode
recode ..HTML < file.txt > file.html Text to HTML
• recode -lf windows-1252 | grep euro Lookup table of characters
• echo -n 0x80 | recode latin-9/x1..dump Show what a code represents in latin-9 charmap
• echo -n 0x20AC | recode ucs-2/x2..latin-9/x Show latin-9 encoding
• echo -n 0x20AC | recode ucs-2/x2..utf-8/x Show utf-8 encoding
CDs
gzip < /dev/cdrom > cdrom.iso.gz Save copy of data cdrom
mkisofs -V LABEL -r dir | gzip > cdrom.iso.gz Create cdrom image from contents of dir
mount -o loop cdrom.iso /mnt/dir Mount the cdrom image at /mnt/dir (read only)
cdrecord -v dev=/dev/cdrom blank=fast Clear a CDRW
gzip -dc cdrom.iso.gz | cdrecord -v dev=/dev/cdrom - Burn cdrom image (use dev=ATAPI -scanbus to confirm dev)
cdparanoia -B Rip audio tracks from CD to wav files in current dir
cdrecord -v dev=/dev/cdrom -audio *.wav Make audio CD from all wavs in current dir (see also cdrdao)
oggenc --tracknum='track' track.cdda.wav -o 'track.ogg' Make ogg file from wav file
disk space (See also FSlint)
• ls -lSr Show files by size, biggest last
• du -s * | sort -k1,1rn | head Show top disk users in current dir. See also dutop
• df -h Show free space on mounted filesystems
• df -i Show free inodes on mounted filesystems
• fdisk -l Show disks partitions sizes and types (run as root)
• rpm -q -a --qf '%10{SIZE}\t%{NAME}\n' | sort -k1,1n List all packages by installed size (Bytes) on rpm distros
• dpkg-query -W -f='${Installed-Size;10}\t${Package}\n' | sort -k1,1n List all packages by installed size (KBytes) on deb distros
• dd bs=1 seek=2TB if=/dev/null of=ext3.test Create a large test file (taking no space). See also truncate
• > file truncate data of file or create an empty file
monitoring/debugging
• tail -f /var/log/messages Monitor messages in a log file
• strace -c ls >/dev/null Summarise/profile system calls made by command
• strace -f -e open ls >/dev/null List system calls made by command
• ltrace -f -e getenv ls >/dev/null List library calls made by command
• lsof -p $$ List paths that process id has open
• lsof ~ List processes that have specified path open
• tcpdump not port 22 Show network traffic except ssh. See also tcpdump_not_me
• ps -e -o pid,args --forest List processes in a hierarchy
• ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d' List processes by % cpu usage
• ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS List processes by mem usage. See also ps_mem.py
• ps -C firefox-bin -L -o pid,tid,pcpu,state List all threads for a particular process
• ps -p 1,2 List info for particular process IDs
• last reboot Show system reboot history
• free -m Show amount of (remaining) RAM (-m displays in MB)
• watch -n.1 'cat /proc/interrupts' Watch changeable data continuously
system information (see also sysinfo) ('#' means root access is required)
• uname -a Show kernel version and system architecture
• head -n1 /etc/issue Show name and version of distribution
• cat /proc/partitions Show all partitions registered on the system
• grep MemTotal /proc/meminfo Show RAM total seen by the system
• grep "model name" /proc/cpuinfo Show CPU(s) info
• lspci -tv Show PCI info
• lsusb -tv Show USB info
• mount | column -t List mounted filesystems on the system (and align output)
• grep -F capacity: /proc/acpi/battery/BAT0/info Show state of cells in laptop battery
# dmidecode -q | less Display SMBIOS/DMI information
# smartctl -A /dev/sda | grep Power_On_Hours How long has this disk (system) been powered on in total
# hdparm -i /dev/sda Show info about disk sda
# hdparm -tT /dev/sda Do a read speed test on disk sda
# badblocks -s /dev/sda Test for unreadable blocks on disk sda
interactive (see also linux keyboard shortcuts)
• readline Line editor used by bash, python, bc, gnuplot, ...
• screen Virtual terminals with detach capability, ...
• mc Powerful file manager that can browse rpm, tar, ftp, ssh, ...
• gnuplot Interactive/scriptable graphing
• links Web browser
• xdg-open http://www.pixelbeat.org/ open a file or url with the registered desktop application
miscellaneous
• alias hd='od -Ax -tx1z -v' Handy hexdump. (usage e.g.: • hd /proc/self/cmdline | less)
• alias realpath='readlink -f' Canonicalize path. (usage e.g.: • realpath ~/../$USER)
• set | grep $USER Search current environment
touch -c -t 0304050607 file Set file timestamp (YYMMDDhhmm)
• python -m SimpleHTTPServer Serve current directory tree at http://$HOSTNAME:8000/
lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am


http://www.v-nessa.net/scripts-and-tutorials

Postby lik » Sun May 03, 2009 7:38 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

http://desnopass.110mb.com/

Postby lik » Mon May 04, 2009 5:57 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am


http://www.bacula.org

Postby lik » Tue May 12, 2009 5:11 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

cPanel XML API

Postby lik » Sun May 17, 2009 9:11 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

do the next rewrite on the result of this one

Postby lik » Sun May 17, 2009 10:06 am

RewriteEngine on

RewriteMap lowercase int:tolower

# allow CGIs to work
RewriteCond %{REQUEST_URI} !^/cgi-bin/

# check the hostname is right so that the RewriteRule works
RewriteCond ${lowercase:%{SERVER_NAME}} ^www\.[a-z-]+\.isp\.com$

# concatenate the virtual host name onto the start of the URI
# the [C] means do the next rewrite on the result of this one
RewriteRule ^(.+) ${lowercase:%{SERVER_NAME}}$1 [C]

# now create the real file name
RewriteRule ^www\.([a-z-]+)\.isp\.com/(.*) /home/$1/$2

# define the global CGI directory
ScriptAlias /cgi-bin/ /www/std-cgi/


http://httpd.apache.org/docs/2.0/vhosts/mass.html
lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

Rewrite <subdomain>.example.com/<path> to example.com/<subdo

Postby lik » Sun May 17, 2009 10:17 am

The goal is to compare two variables in a RewriteCond. This comparison is a necessary part of solving the "arbitrary subdomain" problem, in order to prevent recursion in mod_rewrite when used in an .htaccess context.

However, RewriteCond constructs of the form

RewriteCond %{server_variableB} (.*)
RewriteCond %{server_variableA} ^%1$
-or-
RewriteCond %{server_variableA} ^%{server_variableB}$

don't work. So it appears that it is impossible to compare two variables in mod_rewrite.

However, we know it is possible to combine two variables, and compare them to a combined CondPattern, such as:

RewriteCond %{HTTP_HOST}%{REQUEST_URI} ^www\.example\.com/index\.html$

and that we can insert an arbitrary character string between the variables to demarcate the boundary between the variables unambiguously:

RewriteCond %{HTTP_HOST}Quux-foo%{REQUEST_URI} ^www\.example\.comQuux-foo/index\.html$

-or, better yet-

RewriteCond %{HTTP_HOST}<->%{REQUEST_URI} ^www\.example\.com<->/index\.html$

where "<->" is just an arbitrary string that has no meaning to regular-expressions, but does imply concatenation when we see it.

However, Andreas' remarkable contribution to this problem is the discovery that a back-reference of the form

RewriteCond %{server_variableA}<->%{server_variableB} (.*)<->\1

does work in mod_rewrite RewriteConds. Here the back-reference is to the first parenthesized CondPattern part on the right, not to the server variables in the TestString on the left, and not to a reference created in a previous RewriteCond.

So, using the idea that if A+B = A+A, then A = B, he created a variable comparison by using this local back-reference technique.

I successfully tested the following code in .htaccess, which may be adaptable to solve MoonStarr's problem:

# Rewrite <subdomain>.example.com/<path> to example.com/<subdomain>/<path>
#
# Skip rewrite if no hostname or if subdomain is www
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\. [NC]
# Extract (required) subdomain (%1), and first path element (%3), discard port number if present (%2)
RewriteCond %{HTTP_HOST}<>%{REQUEST_URI} ^([^.]+)\.example\.com(:80)?<>/([^/]*) [NC]
# Rewrite only when subdomain not equal to first path element (prevents mod_rewrite recursion)
RewriteCond %1<>%3 !^(.*)<>\1$ [NC]
# Rewrite to /subdomain/path
RewriteRule ^(.*) /%1/$1 [L]

I claim no credit for the development of this technique: WebmasterWorld member Andreas Friedrich first posted it in the thread cited above. Hopefully, this explanation of the technique will be useful.



############################
RewriteEngine On

############################
#### FIX TRAILING SLASH ####
############################
# Skip rewrite if no hostname or if subdomain is www
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\. [NC]
# Extract (required) subdomain (%1), and first path element (%3), discard port number if present (%2)
RewriteCond %{HTTP_HOST}<->%{REQUEST_URI} ^([^.]+)\.example\.nl(:80)?<->/([^/]*) [NC]
# Rewrite only when subdomain not equal to first path element (prevents mod_rewrite recursion)
RewriteCond %1<->%3 !^(.*)<->\1$ [NC]
# Check if URI has an trailing slash
RewriteCond %{REQUEST_URI} !/$
# Check if it is an directory and then rewrite it.
RewriteCond %{DOCUMENT_ROOT}/%1%{REQUEST_URI} -d
RewriteRule .* http://%{HTTP_HOST}%{REQUEST_URI}/ [R=301,L]
############################

############################
# Start Subdomain Rewrite
# Rewrite <subdomain>.example.com/<path> to example.com/<subdomain>/<path>
# Skip rewrite if no hostname or if subdomain is www
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\. [NC]
# Extract (required) subdomain (%1), and first path element (%3), discard port number if present (%2)
RewriteCond %{HTTP_HOST}<->%{REQUEST_URI} ^([^.]+)\.example\.nl(:80)?<->/([^/]*) [NC]
# Rewrite only when subdomain not equal to first path element (prevents mod_rewrite recursion)
RewriteCond %1<->%3 !^(.*)<->\1$ [NC]
# Check if subdomain (dir/file) exists, else give an error.
RewriteCond %{DOCUMENT_ROOT}/%1%{REQUEST_URI} -d [OR]
RewriteCond %{DOCUMENT_ROOT}/%1%{REQUEST_URI} -f
# Rewrite to /subdomain/path - And make it the last RewriteRule.
RewriteRule ^(.*) /%1/$1 [L,NS]
# End Subdomain Rewrite.
############################
lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am




hax.tor.hu

Postby lik » Sun Aug 09, 2009 3:06 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

Firewalls/Protocols

Postby lik » Sun Aug 09, 2009 5:57 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

Challenges

Postby lik » Sun Aug 09, 2009 7:06 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

RokSlideshow

Postby lik » Sun Aug 09, 2009 5:20 pm

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

Re: Links

Postby lik » Wed Aug 12, 2009 2:15 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am

Re: Links

Postby lik » Wed Sep 02, 2009 12:41 am

lik
Founder
Founder
 
Posts: 497
Joined: Wed Dec 15, 2010 3:21 am


Return to Interesting links

 


  • Related topics
    Replies
    Views
    Last post
cron