Category Archives: Programming

Cisco Router Backup Script With Python and Telnet

We needed a simple way to backup our network settings on a Cisco device at Veriteknik, so I decided to write a script.

You can simple connect to the device via telnet, declaring your username and password. For security reasons, we use IP address restricting as well.

So, it is quite easy to send and recieve telnet commands in Python, simply use the telnetlib library.

Getting the settings of a Cisco device is quite easy, simply enter the “sh run” command and the output is your settings. But normally the device will output the settings in bits and pieces, using a “more” like function. This is set on the terminal length parameter. To check it, simple use the following command on your device,

asr.vp.net.tr#show terminal | in Length
Length: 62 lines, Width: 195 columns

To set this value to 0, which means you’ll get the full output instantly,

terminal length 0

This command will change the option, but that is only for the current session. When you relog to the device, the value will be set to default, which is a good thing cause we want only our Python-Telnet session to get a non-more-like terminal mode. You can read about this on Cisco’s documentation here.

Now using the script below, we can simple get our backups. This script is for Python 2.x, it won’t be that different if you want to use it with 3.x either.

#!/usr/bin/python

import telnetlib
import datetime

now = datetime.datetime.now()

host = "192.168.1.2" # your router ip
username = "administrator" # the username
password = "SuperSecretPassword"
filename_prefix = "cisco-backup"

tn = telnetlib.Telnet(host)
tn.read_until("Username:")
tn.write(username+"\n")
tn.read_until("Password:")
tn.write(password+"\n")
tn.write("terminal length 0"+"\n")
tn.write("sh run"+"\n")
tn.write("exit"+"\n")
output=tn.read_all()

filename = "%s_%.2i-%.2i-%i_%.2i-%.2i-%.2i" % (filename_prefix,now.day,now.month,now.year,now.hour,now.minute,now.second)

fp=open(filename,"w")
fp.write(output)
fp.close()

This script will output a file with a timestamp. This file will contain all the settings (actually the “shell run” output) of your device. Now why not give it a try with a cronjob?

A Backup Script For WordPress

I’ve written a backup script in order to get my WordPress blog backup automatically. The script is only usable on a Linux/Unix box, since it uses default GNU tools.

The script connects to the server via ssh, copies a folder to a location, dumps a database to the same place with the copied folder, creates a tar.gz out of it, then gets the new file via ftp to a prefered location.

The important thing here is that, you should add your ssh public key to the server so that ssh will connect automatically. I also use .my.cnf files to login mysql without specifying password, so you’d better do that. I’ve talked about it in an earlier post here.

Keep in mind that you need an ftp client to connect. If you don’t have it, install it using yum, apt or whatever.

#!/bin/bash

###  START OF EDIT THESE ###
############################
HOST='192.168.1.1' # ip address of your server
SSHUser='root' # user to connect as ssh
FTPUser='myfunkyftpusername' # user to connect as ftp
FTPPass='mysupersecretFTPpassword!' # ftp connection password
MYSQLUser='root' # # user to connect as MySQL

SSHPort=22 # change if different
FTPPort=21 # change if different
DB=wordpress # which database to backup?
DIRECTORY='/home/eaydin/public_html/wp-content' # directory to back up - server side
DIRWRITE='/home/eaydin/' # move the backup here on the server.
DROPBOX='/home/eaydin/Dropbox' # local file path to backup - host side. use your Dropbox folder?
FILENAME='wp-backup' # Filename to use for backups

### END OF EDIT THESE ###
#########################
DIRWRITE=${DIRWRITE%/} # remove trailing / from dir name.
FILENAME=${FILENAME%/} # remove trailing / from filename in case the user types it.
DATE=`eval date +%d%m%Y"-"%H%M` # create date format. (created on the host side, not server. depends on the host time setings.)
FILETAR=$FILENAME-$DATE.tar.gz # name of the tar.gz file (not path!)

ssh -t $SSHUser@$HOST -p $SSHPort "\
cp -R $DIRECTORY $DIRWRITE/$FILENAME-$DATE ;\
mysqldump --add-drop-table -u $MYSQLUser $DB > $DIRWRITE/$FILENAME-$DATE/wordpress.sql ;\
tar -cvzf $DIRWRITE/$FILETAR $DIRWRITE/$FILENAME-$DATE ;\
chown $FTPUser:$FTPser $DIRWRITE/$FILETAR ;\
rm -rf $DIRWRITE/$FILENAME-$DATE
"
ftp -n $HOST $FTPPort <<END_SCRIPT
quote USER $FTPUser
quote PASS $FTPPass
lcd $DROPBOX
cd $DIRWRITE
binary
get $FILETAR
quit
END_SCRIPT
exit 0

The lines between 5 and 17 are the ones you should edit, they’re all self explained in the comments.

It’s a good idea to add the script to your crontab.
In order to do it, especially on Ubuntu systems, just add your current PATH value right below the /bin/sh line. Like this,

eaydin@eaVT:~$ echo $PATH
/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

So you should add this line at the top of the script,

#!/bin/env bash
PATH=/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

Or, just run the script whenever you want. I usually set the download path (defined as the $DROPBOX variable on line 16) to my Dropbox folder, this way my backups get automatically synced on the Dropbox server.

Counting Syllables in the English Language Using Python

Even though it seems like an easy task, counting syllables is very hard in English. After hours of googling I’ve realized that the non-corpus-based algorithms are not perfect, and it’s impossible for them to be. So I wanted to make a better one combining some I’ve read and overcoming the errors I’ve encountered. The goal is to create a step-by-step algorithm using the least amount of dictionary help. Here’s the first time I’m sharing it publicly hoping that it will help someone out.

Even though this is Python based, the important thing is the algorithm. I know it’s not the best way to handle it, but the outcome’s not so bad for a first attempt.

Here are some discussions or algorithms I’ve found on the web.

  1. http://allenporter.tumblr.com/post/9776954743/syllables  simple algorithm
  2. http://www.howmanysyllables.com/howtocountsyllables.html  some pseudo-algorithm
  3. http://www.modulus.com.au/blog/?p=8  a little bit more algorithm
  4. http://www.onebloke.com/2011/06/counting-syllables-accurately-in-python-on-google-app-engine/ this uses the nltk library, which is not what I’m looking for, since that’s not the challenge. (yet, it may be the most clever approach for you)

Reading all these and experimenting, I’ve developed my own set of rules, here they go.

The Algorithm

  1. If number of letters <= 3 : return 1
  2. If doesn’t end with “ted” or “tes” or “ses” or “ied” or “ies”, discard “es” and “ed” at the end. If it has only 1 vowel or 1 set of consecutive vowels, discard. (like “speed”, “fled” etc.)
  3. Discard trailing “e”, except where ending is “le” and isn’t in the le_except array
  4. Check if consecutive vowels exists, triplets or pairs, count them as one.
  5. Count remaining vowels in the word.
  6. Add one if begins with “mc”
  7. Add one if ends with “y” but is not surrouned by vowel. (ex. “mickey”)
  8. Add one if “y” is surrounded by non-vowels and is not in the last word. (ex. “python”)
  9. If begins with “tri-” or “bi-” and is followed by a vowel, add one. (so that “ia” at “triangle” won’t be mistreated by step 4)
  10. If ends with “-ian”, should be counted as two syllables, except for “-tian” and “-cian”. (ex. “indian” and “politician” should be handled differently and shouldn’t be mistreated by step 4)
  11. If begins with “co-” and is followed by a vowel, check if it exists in the double syllable dictionary, if not, check if in single dictionary and act accordingly. (co_one and co_two dictionaries handle it. Ex. “coach” and “coapt” shouldn’t be treated equally by step 4)
  12. If starts with “pre-” and is followed by a vowel, check if exists in the double syllable dictionary, if not, check if in single dictionary and act accordingly. (similar to step 11, but very weak dictionary for the moment)
  13. Check for “-n’t” and cross match with dictionary to add syllable. (ex. “doesn’t”, “couldn’t”)
  14. Handling the exceptional words. (ex. “serious”, “fortunately”)

Like I said earlier, this isn’t perfect, so there are some steps to add or modify, but it works just “fine”. Some exceptions should be added such as “evacuate”, “ambulances”, “shuttled”, “anyone” etc… Also it can’t handle some compund words like “facebook”. Counting only “face” would result correctly “1”, and “book” would also come out correct, but due to the “e” letter not being detected as a “silent e”, “facebook” will return “3 syllables.”

Anyway, here’s the Python (2.x) code, I’ll try and improve it sometime.

import re

def sylco(word) :

    word = word.lower()

    # exception_add are words that need extra syllables
    # exception_del are words that need less syllables

    exception_add = ['serious','crucial']
    exception_del = ['fortunately','unfortunately']

    co_one = ['cool','coach','coat','coal','count','coin','coarse','coup','coif','cook','coign','coiffe','coof','court']
    co_two = ['coapt','coed','coinci']

    pre_one = ['preach']

    syls = 0 #added syllable number
    disc = 0 #discarded syllable number

    #1) if letters < 3 : return 1
    if len(word) <= 3 :
        syls = 1
        return syls

    #2) if doesn't end with "ted" or "tes" or "ses" or "ied" or "ies", discard "es" and "ed" at the end.
    # if it has only 1 vowel or 1 set of consecutive vowels, discard. (like "speed", "fled" etc.)

    if word[-2:] == "es" or word[-2:] == "ed" :
        doubleAndtripple_1 = len(re.findall(r'[eaoui][eaoui]',word))
        if doubleAndtripple_1 > 1 or len(re.findall(r'[eaoui][^eaoui]',word)) > 1 :
            if word[-3:] == "ted" or word[-3:] == "tes" or word[-3:] == "ses" or word[-3:] == "ied" or word[-3:] == "ies" :
                pass
            else :
                disc+=1

    #3) discard trailing "e", except where ending is "le"  

    le_except = ['whole','mobile','pole','male','female','hale','pale','tale','sale','aisle','whale','while']

    if word[-1:] == "e" :
        if word[-2:] == "le" and word not in le_except :
            pass

        else :
            disc+=1

    #4) check if consecutive vowels exists, triplets or pairs, count them as one.

    doubleAndtripple = len(re.findall(r'[eaoui][eaoui]',word))
    tripple = len(re.findall(r'[eaoui][eaoui][eaoui]',word))
    disc+=doubleAndtripple + tripple

    #5) count remaining vowels in word.
    numVowels = len(re.findall(r'[eaoui]',word))

    #6) add one if starts with "mc"
    if word[:2] == "mc" :
        syls+=1

    #7) add one if ends with "y" but is not surrouned by vowel
    if word[-1:] == "y" and word[-2] not in "aeoui" :
        syls +=1

    #8) add one if "y" is surrounded by non-vowels and is not in the last word.

    for i,j in enumerate(word) :
        if j == "y" :
            if (i != 0) and (i != len(word)-1) :
                if word[i-1] not in "aeoui" and word[i+1] not in "aeoui" :
                    syls+=1

    #9) if starts with "tri-" or "bi-" and is followed by a vowel, add one.

    if word[:3] == "tri" and word[3] in "aeoui" :
        syls+=1

    if word[:2] == "bi" and word[2] in "aeoui" :
        syls+=1

    #10) if ends with "-ian", should be counted as two syllables, except for "-tian" and "-cian"

    if word[-3:] == "ian" : 
    #and (word[-4:] != "cian" or word[-4:] != "tian") :
        if word[-4:] == "cian" or word[-4:] == "tian" :
            pass
        else :
            syls+=1

    #11) if starts with "co-" and is followed by a vowel, check if exists in the double syllable dictionary, if not, check if in single dictionary and act accordingly.

    if word[:2] == "co" and word[2] in 'eaoui' :

        if word[:4] in co_two or word[:5] in co_two or word[:6] in co_two :
            syls+=1
        elif word[:4] in co_one or word[:5] in co_one or word[:6] in co_one :
            pass
        else :
            syls+=1

    #12) if starts with "pre-" and is followed by a vowel, check if exists in the double syllable dictionary, if not, check if in single dictionary and act accordingly.

    if word[:3] == "pre" and word[3] in 'eaoui' :
        if word[:6] in pre_one :
            pass
        else :
            syls+=1

    #13) check for "-n't" and cross match with dictionary to add syllable.

    negative = ["doesn't", "isn't", "shouldn't", "couldn't","wouldn't"]

    if word[-3:] == "n't" :
        if word in negative :
            syls+=1
        else :
            pass   

    #14) Handling the exceptional words.

    if word in exception_del :
        disc+=1

    if word in exception_add :
        syls+=1     

    # calculate the output
    return numVowels - disc + syls

Importing Large MySQL Files With phpMyAdmin

It may not be possible to import large SQL files using phpMyAdmin due to it’s uploading limits.
Sometimes this is related to your php.ini settings, but no always.

You can always use the old reliable method of importing sql files from the command line. Or upload to file to server and tell phpMyAdmin to look for that file specifically.

MySQL Command Line Option

Of course, we need to upload the file first. Below, the 1st line is to achieve this. After that we connect to our server via ssh, and then (on the 2nd line) import the sql file.

scp the_sql_file.sql root@myserver.com:/path/to/upload
mysql -u username -p -h localhost DATABASE-NAME < the_sql_file.sql

Here, the MySQL connection is established using a username and password. If you simple create a .my.cnf file in order to get automatic connections, you won’t need all these. Simply create a file to /root/.my.cnf (or any other user directory you want to)

[client]
user=root
pass=mysql_root_password

From now on, you don’t need to use the -u and -p arguments when using mysql, if this file exists at the user’s home directory you’re logged on as, than it will automatically pass them. It may seem insecure, but the /root folder is only accessed by the root anyway, even though it is not a good idea to keep passwords clear text, if your /root folder is compromised, you’re in big trouble anyway. And keep in mind that we usually have to keep MySQL passwords a cleartexts in scripts all the time.

Using phpMyAdmin to Import the SQL File

You can set a default folder for phpMyAdmin to check for uploaded files. This way, if you place a file into this folder, you can easily choose to import it. Find where your config.inc.php file is. If you don’t know where, try the locate command (if it is installed). Open the file and find the $cfg[‘UploadDir’] section. Update it as below

$cfg['UploadDir'] = 'imports';

Now, phpMyAdmin will check the “imports” folder directly. So we need a folder like that. On your terminal, create the folder.

mkdir /path/to/phpmyadmin/imports

Now upload your sql files here.

scp /path/to/the_sql_file.sql root@myserver.com:/path/to/phpmyadmin/imports

Change the owner and group of the imports folder. First check who owns the phpmyadmin folder. Let’s say it is called webapp,

chown -R webapp:webapp /path/to/phpmyadmin/imports

Ok. Now use your browser and access phpmyadmin, at the import db seciton you’ll see a dropdown menu that wasn’t there before. There you’ll see the files inside the “imports” folder. Even though they are larger than the phpmyadmin upload limits, they’ll get imported.

UNIX Process Time Bomb

Here’s a simple script that kills a process if it lives longer than the time specified. It’s written in Python, and is available on github.

Usage : timebomb.py <process-name> <minutes>

Example : $ timebomb.py firefox-bin 20

Outcome : This will kill the process named firefox-bin if it has been running longer than 20 minutes.

Crontab : You should probably add this to your crontab!

Dependency : Standard UNIX tools : Python 2.4.x, pgrep, ps, kill etc.

#!/usr/bin/python
# A Pythonic Time Bomb
# Kills Processes Living Longer than the specified time.
# Don't Forget to add it to your crontab!
# http://github.com/eaydin

import subprocess, sys
if len(sys.argv) != 3 :
    print "Usage : timebomb.py <process-name> <time-in-minutes>"
    print "Takes only and exactly 2 arguments."
    raise SystemExit
    
try : int(sys.argv[2])
except :
    print "%s is not an integer." % sys.argv[2]
    raise SystemExit
    
try :
    a=subprocess.Popen(["pgrep",sys.argv[1]],stdout=subprocess.PIPE).communicate()[0]
    if a == '' :
        raise SystemExit
    else :
        procc = subprocess.Popen(["ps -o pid,bsdtime -p $(pgrep %s)"%(sys.argv)[1]],shell=True,stdout=subprocess.PIPE).communicate()[0]
        procc=procc.strip()
except : raise SystemExit 
for lines in procc.split('\n') :
    if lines != '' :
        l=lines.split()
        if l[0] == 'PID' : pass
        else :
            if int(l[1].split(':')[0]) >= int(sys.argv[2]) :
                try : killer = subprocess.Popen(["kill","-9",l[0]],stdout=subprocess.PIPE).communicate()[0]
                except : pass
            else : pass