Posts Tagged ‘Linux’

Web app backup simple shell script

Backup Recently, the unofficial discussion board for students of my university department went down because of a misunderstanding with their hosting provider who also reassigned their machine to another customer wiping out all of their data. Now they restore the board but the last backup was dated back to last year so lots of data have been lost. Anyway, when one of the admins announced on facebook the board was up again, i ironically commented writing a small shell script that can be used to backup a simple mysql-based web application such as a discussion board or a CMS installation such as drupal, joomla, wp or whatever… But then i thought this happend more frequently than expected to people i know, so i decided to post that script here :)

#!/bin/bash
#-----------------------------------
# Web App. Dumb Backup Script
# http://www.n0on3.net
#-----------------------------------
$user='your-username-here'
$server='your-domain-name-here'
$appname='your-webapp-name-here'
$apppath='your-webapp-path-here'
$mysqluser='your-mysql-user-here'
$mysqlpassword='your-mysql-pwd-here'
$dbname='your-mysql-db-name-here'
#-----------------------------------
d=`date +'%d-%m-%y'`
ssh $user@$server "tar cjvf backup-$appname-$d-www.tar.bz2 $apppath"
ssh $user@$server "mysqldump -u $mysqluser
                  --password=$mysqlpassword $dbname
                  > backup-$appname-$d-db.sql"
scp $user@$server:$HOME/$user/backup-$appname-$d-www.tar.bz2 \
                  backup-$appname-$d-www.tar.bz2
scp $user@$server:$HOME/$user/backup-$appname-$d-db.sql \
                  backup-$appname-$d-db.sql
ssh $user@$server 'rm backup-$appname-$d-*'

Please notice that here you are using ssh login without password, that means you have to append your client machine public rsa id to your server authorized keys file.
But more important, here you are writing your database password in plaintext because the script must use it, so if you keep such a script on your client machine remember to encrypt it or to take any proper precaution ;)


Download YouTube videos with a shell script

In my last post I wrote about extracting audio from in-streaming-format videos, that people often may get from content streaming services like eg. YouTube. As said, those videos may be downloaded in several ways that may also be different depending on the hosting service. A pretty effective way is using some plugin for your browser that shows the URL of the source the flash/html5 player download the video from, and allow you to get the content from that. Anyway, just for fun, here is a small shell script that can be used to download videos contents from youtube.

#!/bin/bash
#
################################################################
# --- YouTube Downloader ------------------------------------- #
# --- http://www.n0on3.net ----------------------------------- #
################################################################
#
which wget &>/dev/null; 
if [ $? -ne 0 ];then echo "> Please install wget =)!"; exit; fi
if [ $# -ne 1 ];then 
	read -p "> Enter Youtube video ID: " VID
else	VID=$1; fi
read -p "> Do you want me to try to get the HD version ? [y|N]: " HD
case $HD in Y|y|yes) FMT=22;; N|n|*)FMT=18;; esac
OK=1; PAGE=`tempfile`
wget -q -O $PAGE "http://www.youtube.com/watch?v=$VID"
SIG=`cat $PAGE | egrep -o "\"t\": \"[^\"]+\"" | \
     sed 's/\"//g' | awk '{print $2}'`
URL="http://www.youtube.com/get_video?fmt=$FMT&video_id=$VID&t=$SIG"
TITLE=`cat $PAGE | grep VIDEO_TITLE | awk '{$1="";print $0}' `
TITLE=$(echo $TITLE | sed 's/.\(.*\)../\1/' | sed 's/\\//g')
read -p "> Is the title \" $TITLE \" right for output file ? [Y|n]: " TOK
case $TOK in 	n|N|no) read -p "> Enter filename: " TITLE ;;
		Y|y|yes|*) ;; 
esac; echo "> Saving file to $TITLE.mp4 ... "
wget -q -O "$TITLE.mp4" $URL &>/dev/null
if [ $? -ne 0 ];then echo "> Download failed!";
	OK=0; rm -fr "$TITLE.mp4";
	if [ $FMT == 22 ];then 
		echo "> Maybe the HD version isn't available.."
		FMT=18; echo "> Downloading the non-HD version ..."
	URL="http://www.youtube.com/get_video?fmt=$FMT&video_id=$VID&t=$SIG"; 
	wget -q -O "$TITLE.mp4" $URL &>/dev/null; 
		if [ $? -ne 0 ];then 
			echo "> Download failed!"; rm -fr "$TITLE.mp4";
		else OK=1; fi; 
	fi
fi
if [ $OK -eq 1 ]; then echo "> Successfully Downloaded to $TITLE.mp4 !"
fi;
#

To be said, youtube term of service, at point 6.C, states using YouTube you accept not to access the contents with any technology but the YouTube website player or other stuff explicitly authorized by YouTube. ( That means, yeah, you should really not use this u.u” ).


Extract mp3 audio from streaming videos

Sometimes you may just want to get the soundtrack of a video. And you know, today most videos are available in streaming on the web. There are several ways to download those videos, depending on the service that hosts them, but there are also many websites that explains how to do that. To give an example, youtube player retrieve videos requesting an URL with parameters that include the ID of the video and a signature it got when it’s loaded. To emulate this behaviour and get yourself the video to download, you can eg. use one of the appropriate firefox plugins , rather than getting the url from one of the many websites that build it for you. Anyway, streaming videos are in .flv or .mp4 formats. If you want to get the audio from an .flv video, you just need ffmpeg, running

ffmpeg -i "video.flv" -f mp3 -vn -acodec copy "audio.mp3"

If you want to get the .mp3 audio from an .mp4 video instead, you may use faad2 and lame to just decode the .mp4 audio to .wav e reencode it to an .mp3 .

faad -o - "video.mp4" | lame - "audio.mp3"

You can get both faad2 and lame as well as ffmpeg via apt or whatever package manager you use in Linux, and via MacPorts of Fink in Mac Os X.


Physical access to linux boxes

Having physical access to a machine, in many common cases means be able to get a root access to the installed system. This is widely known for some version of MS Windows, I don’t know much about Windows 7 right now, but here we have a video from guys at Offensive Security where windows vista is owned just booting backtrack and replacing C:\Windows\System32\Utilman.exe with cmd.exe, and weak passwords encoded to LM and NTLM hashes can be easily cracked with Ophcrack liveCD or any other live system and large rainbow tables. So people switching to linux may think their systems are more secure now, and bad guys won’t be able to access them if they leave the room for five minutes. Unfortunately that isn’t true for what concern physical access to default-like installations. When linux machines are powered on, a bootloader is launched and it do load the initrd if any and the kernel in memory, with a set of given options. Common bootloader such GRUB allow the user to choose among different entries, that may refer to different kernels or different option sets. By default, nothing prevent those entries from being edited. The following video shows how anyone with physical access to the machine can edit an entry to instruct the kernel in running a shell e.g. instead of the init daemon, without being asked for any password, and gaining this way root access to the machine. It’s also shown how to deny this kind of action, by setting up a password protection for all the entries.



Protecting grub entries anyway is just one little step to decrease risks coming from untrusted physical access. Boot from a LiveCD or external drive will give access to your filesystem easily. You can avoid this by denying the boot from CD-Roms and other devices in the BIOS settings, then setting up a password for the BIOS configuration. But BIOS configuration can be reseted at least removing the battery from the machine motherboard, and anyway someone can still extract your hard drive and mount it on another system. If you care about that, your solution may just be the encryption of your whole partitions, so that a passphrase will be required (every time) when they are mounted.


How to use the "vi" Text Editor

I have been recently asked to deliver a lesson in the base linux course LUG Roma3, which I cooperate with, is keeping at roma3 university on voluntary base.
Here are the slide I sketched out for this occasion.



I delivered a similar lesson on last edition of this course and also wrote a paper from what I presented at that time, but I think this pack is definitely better, and less boring than a long paper, so i’m removing the old post in behalf of this new one =)


OSX – Serial Connections, These strangers

serial-connector When you have to locally manage scrap metals, it doesn’t matter if you are connecting a  microwave oven or a dozen of thousands bucks value brand new router or firewall, you will  probably do that over a serial cable. Devices likely offers a port labeled CONSOLE or  whatever you should connect the cable to. The cable type is device-dependent, but it usually  ends up with a serial connector for a serial port. If your computer doesn’t have that port, you  will need an adapter and its driver, such as to an USB port. While in linux you just modprobe the usbserial module with the appropriate vendor and product code, such as:

modprobe usbserial vendor=0xVID product=0xPID

for osx you may need the adapter drivers from the vendor ( they should be available on the vendor’s site, otherwise change vendor =P ). However, when the cable is plugged, you should find the proper entry for the (adapter) device, named tty.<whatever>

ls -1 /dev/tty.*

Now you will need to attach a terminal emulator to the found serial port to access the connected device console. On Mac Os X, as linux users will like, minicom is one of the best choices. Install it via MacPorts or Fink, then use:

minicom -s

to edit the default /opt/local/etc/minirc.dfl config file or to generate configurations that will be saved to /opt/local/etc/minirc.<your-config-name>. You should at least specify the port, baudrate, data bits, parity, stop bits and flow control. These are often indicated in the machine specs, with strings like the common 9600/8N1.


Return top

About me