My Linux Path

 Political Entry

I have no intellectual issues with paying for a service or product including software. What I have had an issue with is how Microsoft does business. To put it mildly, they are a Monopoly and have shown all of the traits of a bully in their pursuit of having 100 percent control of the software industry. They have bullied competitors into pulling out of the industry, walked all over other organization's legal patents and pretty well ignored the law. Several companies have successfully sued Microsoft for their excesses.

So in the early 90's I was running OS/2 partly because of my dislike of Microsoft and partly because it was an interesting OS  which we studied during a Operating System Theory class  I took at Simpson. It was technically superior to the MS product of the day and even though IBM was a huge Computer and Software Co., I felt they at least behaved like a responsible citizen. MS did manage to keep the software available for the OS/2 platform to a minimum though, but could not completely kill it.

Something New

It was 2000 and I knew I was about to lose a job (due to a merger,) so I broke down and bought an Windows 98 box for a new job search. I won't say I hated it because I didn't. OS/2 was getting weaker all of the time and the word processing program I was using didn't convert well enough to Word in order to send professional looking Resumes.  But on the minus side, viruses bacame a problem for the first time and I had problems with the Registry.  There are benefits to running a Alternative OS.

We started to hear about a new Operating System that was totally free called Linux, and even better yet it was based on a earlier OS called UNIX, which I had studied aforementioned at Simpson. So with my Wife's approval I bought a used p90 to try it out.

Well, first I tried to load RedHat 6.0, which failed. So, I loaded RedHat 5.2 and then ran an upgrade to 6.0. That worked. I named this computer Oldboy, since the computer wasn't that new to begin with. Shortly thereafter I had figured out how to share the dialup internet connection with the two Windows boxes we had in the house and the Oldboy network was born, well it was funny for a little while.

Up to this point, whenever my Wife wanted to get on the internet, I had to get off and vice versa. This was a huge improvement. But dial-up is slow, and so cooperative internet use became the standard. When my Wife was going to download something big, I did light internet use (usually chess on FICS) and when I needed to do something that would use a lot of bandwidth, she would slack off

Next, I started playing with Samba (Samba creates network drives for computer workstations) and within a week or two we had a real home network and then followed learning about MySQL. MySQL became my all time favorite database and still is. I could write Delphi apps that would hit my MySQL server running on my Oldboy Server.

The old p90 did pretty good as long as I didn't log into X, My Wife said she could feel the internet slow down every time I started an X Session to do something. I started to learn to do more and more via telnet. But the p90 was kind of tired and when I inherited a newer computer, I quickly replaced Oldboy with Youngster. But the network was still called the Oldboy Network and my wife was getting tired of that joke. Around this time, I replaced my 98 box with a desktop that ran RedHat alone. Somewhere in there I started to learn a little PHP and Web Programming.

I transferred the Samba and MySQL servers from Youngster to my new desktop. But I left Youngster up running as an IP Gateway. I became a Gnome user, got my Wife to use Linux part of the time at least. She became a Ubuntu user and I continued to use RedHat. I read a lot and as it turns out that is a very good way for me to learn (read some, putter some.) The more I did, the more comfortable I became with the command Prompt.

I still could run most Windows Programs though.  I had purchased a product called Win4Lin, which was an early attempt to run a Virtual Computer.  It mostly worked, but it made updating Linux Versions difficult, so eventually I dropped it.  Fortunately, Wine has become much better and can now run most Win32 apps.  There are still a few programs that don’t run well under Wine, with ChessBase being my biggest headache.  To run ChessBase, i have to run  it under a Virtual Machine or boot up under Windows.  ChessBase if you are listening, please fix this.

Things haven't changed hugely. I find I can do 98% of what I want without touching Windows. My first Linux Desktop has been retired and I have moved on to a second. The current Desktop is also approaching retirement and I would like to replace it with an i5. My current Desktop is running Fedora and I also have a laptop running another Linux Distribution (Mint.) I still run Samba and MySQL on my Desktop. Since the Gnome debacle, my Windows Manager of choice has moved around, I have ran KDE, XFCE and Mate and haven't totally decided on any one. I am slowly replacing my Delphi Programming with Lazarus, which is multi platform. We have retired Youngster and replaced him with a Router and replaced dial-up with a Cable Modem and have wireless through out the house. My Wife uses a combination of Linux Mint and Windows, basically she has certain things she does on Windows and others that she uses Linux.

Backing up Home Computers

-->

I should note that our home network is a mixed network, with the data being saved to Linux, but also being used by several Windows Computers.   The Windows Computers are currently not being backed up, but the network drives that they use is. Backups of our computers at home have always been a little bit haphazard. I have tried to get periodic backups done, but it has always been such a bother. It's probably more important than we like to admit. And since I host the network drives on my desktop, it is doubly important. It's not especially complicated. 

1. Run the backup scripts in cron. That is easy. For my home network, monthly should be quite adequate.
2. Copy the backups to external media.

Remembering to copy the backups to external media has always been the issue.

At first, the media was cd-rom, then dvd-rom. When memory sticks started to climb in capacity and their prices started dropping, I started to use them. But now with terabyte hard disks relativly cheap, I use them. At best, I was probably doing quarterly backups. Also, I have found that the USB hard disks are not always the most reliable media. I have had several of them go bad.

One thing I have always done is make sure I have a good backup before doing any system upgrades, such as upgrading one of my Linux Distros.

With the Rasberry Pi being cheap, low power and readily available, I am starting to use it as a backup server. This way, backups will occur if I remember it or not.

I have set up four separate scripts in Cron:

1. Dump MySQL Databases.
2. Run Dumps of home Directories.
3. Move the dumps to my home account.
4. Use ssh to transfer the Home Account Dumps to the Rasberry Pi.

The MySQL script is just something I found on the internet:

#!/bin/bash

# sonia 16-nov-05

# backup each mysql db into a different file, rather than one big file

# as with --all-databases - will make restores easier

set -x

echo '/**************************************************/'

echo '/**Job Backs up the MySQL Databases to flat **/'

echo '/** Files **/'

echo '/**************************************************/'

date

USER="root"

PASSWORD="MyPass"

OUTPUTDIR="/home/myacct/Documents/MysqlBackups"

MYSQLDUMP="/usr/bin/mysqldump"

MYSQL="/usr/bin/mysql"

# clean up any old backups - save space

rm "$OUTPUTDIR/*bak" > /dev/null 2>&1

# get a list of databases

databases=`$MYSQL --user=$USER --password=$PASSWORD \

-e "SHOW DATABASES;" | tr -d "| " | grep -v Database`

# dump each database in turn

for db in $databases; do

echo $db

$MYSQLDUMP --force --opt --user=$USER --password=$PASSWORD \

--databases $db > "$OUTPUTDIR/$db.bak"

done

date

I added the two date commands, just so I could see how long it runs. I also added rm commands to delete my old backup files to this script. (I have removed those lines from this example for security reasons.) I scheduled this to run at 00:10 on Sundays and it runs less than a minute. There isn't a lot of data out there, but I do rely on MySQL a lot. I would hate to lose my data.

At 00:15 on Sunday Morning, the backups run and the script looks like this:

#!/bin/bash

set -x

date

echo '/**************************************************/'

echo '/**Job Starts Full Backup of home directories **/'

echo '/**************************************************/'

date=$(date +'%Y-%m-%d %H:%M:%S')

read Y M D h m s <<< ${date//[-: ]/ }

if [ $D -lt 8 ] ;then

echo "Run Full Backups"

// who -u | grep terry awk '{printf("kill -9 %d", $2)}' | sh

// ping 1.1.1.1 -n 1 -w 5000 > NUL

at -f /home/myacct/Documents/bash/User1Bkp now

at -f /home/myacct/Documents/bash/User2Bkp now

at -f /home/myacct/Documents/bash/User3Bkp now

at -f /home/myacct/Documents/bash/User4Bkp now

at -f /home/myacct/Documents/bash/User5Bkp now

fi

date

The If Statement checks the day of the month, and if it is less then 8, starts the backups. There is no need to run full backups every week for my home network. The data just doesn't change that fast. Each user's backup is a separate script and is started via the At command, this way they can run concurrently with each other rather than in sequence. Each User Backup Script will be similar to the following:

#!/bin/bash

echo '********************************'

echo '** Backup of User1 Acct **'

echo '********************************'

date

tar -czvf /home/user1bkp /home/user1

date

It is just a basic tar script to back up that specific user home account, which is also their network drive via Samba. I have added date commands at the beginning and end of the script, so I can tell how long they are running. If I start the transfer too soon, the backups would be useless. The output from the scripts are going to be huge and all we really have to look at is begin and end run times. Since I am running this from my Root Cron Account, the output will be sent to the Root Linux/Unix Mail Account and can be read using the mail program. I found though, that reading it with Mutt is even better, since by pressing the '|' key sends the mail message to a shell and then all that would have to be typed in is:

grep 'Sun '

and then something like the following should be listed:

Sun Jan 6 00:15:00 CST 2013

Sun Jan 6 01:58:53 CST 2013

So, in this case the job ran about 1.75 hours. I checked the run time on all of the backups and adjusted the start time of the move and copy scripts respectivly. Currently all of my backups complete before 2:30 am.

The third script runs at 3:10 am and it only runs a few seconds:

#!/bin/bash

echo '********************************'

echo '** Copies backup files to **'

echo '** my home account **'

echo '** and makes me the owner **'

echo '********************************'

date

mv /home/*bkp /home/user1

chown user1 /home/user1/*bkp

date

Finally, the last script runs at 3:20. For this to work, I first had to install the “sshpass” program, which is not included in the ssh utility. It's probably not the most secure. But since this is running on a home network and with only family members having access, I am not too concerned. If this was for a business, I might look for something more secure. I considered installing a ftp server on the Rasberry Pi, but then I found that ssh's Secure Copy was already built in. The -arcfour gives the lowest level of encryption that ssh allows. Again, I don't feel security is a big issue on home networks and if I could have turned encryption off, I would have.

The let "Remainder=$M%3" does a Modulus divide, setting the $Remainder variable to the Remainder of the Current Months number divided by 3. This way, each month will go into a separate directory for 3 months, and if I need to go back to a previous months backup, I can. This job will run for hours, but speed shouldn't be an issue.

#!/bin/bash

set -x

echo '/**********************************************************/'

echo '/**Job Copies Backup Files to my Rasberry PI at: **/'

echo '/** 192.168.0.50 **/'

echo '/**********************************************************/'

date

date=$(date +'%Y-%m-%d %H:%M:%S')

read Y M D h m s <<< ${date//[-: ]/ }

if [ $D -lt 8 ] ;then

let "Remainder=$M%3"

if [ $Remainder = 0 ] ;then

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user1bkp pi@192.168.0.50:/media/usbhdd/Month1

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user2bkp pi@192.168.0.50:/media/usbhdd/Month1

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user3bkp pi@192.168.0.50:/media/usbhdd/Month1

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user4bkp pi@192.168.0.50:/media/usbhdd/Month1

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user5bkp pi@192.168.0.50:/media/usbhdd/Month1

fi

if [ $Remainder = 1 ] ;then

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user1bkp pi@192.168.0.50:/media/usbhdd/Month2

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user2bkp pi@192.168.0.50:/media/usbhdd/Month2

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user3bkp pi@192.168.0.50:/media/usbhdd/Month2

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user4bkp pi@192.168.0.50:/media/usbhdd/Month2

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user5bkp pi@192.168.0.50:/media/usbhdd/Month2

fi

if [ $Remainder = 2 ] ;then

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user1bkp pi@192.168.0.50:/media/usbhdd/Month3

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user2bkp pi@192.168.0.50:/media/usbhdd/Month3

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user3bkp pi@192.168.0.50:/media/usbhdd/Month3

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user4bkp pi@192.168.0.50:/media/usbhdd/Month3

/usr/bin/sshpass -p 'MyPass' /usr/bin/scp -c arcfour /home/user1/user5bkp pi@192.168.0.50:/media/usbhdd/Month3

fi

fi

date

Lastly, the crontab is kind of goofy. I have to look up the format every time I use it. You can read about Cron here. The easiest thing to do is put the runtimes for the root account and the MyUser Account in separate files. The MyUser Cron file is set up as follows:

10 00 * * 0 /home/MyUser/Documents/bash/MysqlBackup

20 03 * * 0 /home/MyUser/Documents/bash/CopyBkp

And then the Root Cron File is set up as:

15 00 * * 0 /home/MyUser/Documents/bash/FullBackUp

10 03 * * 0 /home/MyUser/Documents/bash/RtCmd

In these, the first column is the minute to start, the second column is the hour it is to start and the fifth column is the day of the week.

Then, to change the current crontab with these, just sign into that user command line and type:

crontab FileName

where FileName is the name of that user's crontab file.