Thursday 31 July 2014

grep in all files in all subdirectories

You want to find some line or phrase but you do know exact location?

Use grep!

Syntax is simple!

server#grep -r fraze /dir

This -r option means recursive so depending of your current location this will grep in all files in all subdirectories!


how to fast drop partition table in mysql - EXPLAINED!

So your file system is full (or will be very soon) and there are serious possibility that your mysql database will stop. You have to drop some old partitions.
There are two ways to do it.

First way - slow way


server#mysql
# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
.

.
mysql> use database_name;
Database changed
mysql> alter table table_name drop partition partition_name;


Remember that partiotion_name has to be name of oldest partition so when you drop that partition consistency of data stored in database will be preserved.
In my case this look like this


mysql> alter table SystemEvents drop partition p20140701;
Query OK, 0 rows affected (0.23 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql>

In case that you have more then few partition do drop, you can drop more partitions in one mysql row or my can drop partition one by one. This is up to you. This can be quite boring and time consuming.

Different way - fast way

This way require just list text manipulation skill.
As you know you can execute mysql command by just redirecting conf file in mysql. Syntax goes like this

mysql database_name < conf_file

So how to write that file?

1.List partition that you want to drop.
In my case this look like this

server# ls /var/lib/mysql/Syslog/|grep 201407|grep MYI |awk -F# '{print $3}'|awk -F. '{print $1}' >list.txt
server#cat list
p20140702
p20140703
p20140704
.
.
p20140728
p20140729
p20140730

2.Add mysql command in first line of that file

alter table SystemEvents drop partition p20140702
p20140703
p20140704
.
.
p20140728
p20140729
p20140730

3.And coma (,) as last character in every  line.

server#awk '{print $0","}' line.txt >line_1.txt

4.Close mysql syntax by adding ; at the end of last line.

Your file should look like this

server#cat line_1.txt
alter table SystemEvents drop partition p20140702,
p20140703,
p20140704,
.
.
p20140728,
p20140729,
p20140730;


5. Lets drop partition

server#mysql Syslog <line_1.txt
server#

And that is that!



Wednesday 30 July 2014

Iso repository on Red Hat or Centos - EXPLAINED!

In case that you want to install some new or update already installed packages, you can do it by using rpm command and have possible problems considering dependency issues or you can do it by using yum and avoid dependency issue because yum will do all the work for you!

In case that your server do not have access to internet or you do not have central repository server you can always use iso image and add it to your yum repository.

How to do this?

1. Copy iso image to your server
2. Mount iso image

server#mount -t iso9660 /iso_location /mount_point -o loop

In my case

server#mount -t iso9660 rhel-server-5.9-x86_64-dvd.iso /mnt -o loop

Check if it is mounted

server# mount |grep mnt
/opt/iso/rhel-server-6.3-i386-dvd.iso on /mnt type iso9660 (rw,loop=/dev/loop0)

3. Go to yum configuration folder

server#cd /etc/yum.repos.d

Create new conf file. Extension has to be .repo. 

server# cat dvd.repo
[DVD]
name=RHEL $releasever - DVD
baseurl=file:///mnt/
enabled=1
gpgcheck=0
 

First line in name of repository and you have to have this line.
Second line is just for user to know what repository he is using. This line is not necessary but it is nice to
Third line is location of your mounted iso.
Fourth line is for enabling repository. 1 is for enable,0 is for disable
Fifth line is for gpg signature check. Because I am using iso from vendor I disabled this.

You can add more options to this file if you like but this is I think simplest configuration you can get.

4.  Insert new repository in yum
# yum repolist
Loaded plugins: product-id, refresh-packagekit, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
repo id                        repo name                                  status
DVD                            RHEL 6Server - DVD                         2,797
repolist: 2,797
 


5. Test if it is working.Try to install some new package

server# yum install yp-tools
Loaded plugins: product-id, refresh-packagekit, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package yp-tools.i686 0:2.9-12.el6 will be installed
--> Processing Dependency: ypbind for package: yp-tools-2.9-12.el6.i686
--> Running transaction check
---> Package ypbind.i686 3:1.20.4-29.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package           Arch          Version                     Repository    Size
================================================================================
Installing:
 yp-tools          i686          2.9-12.el6                  DVD           64 k
Installing for dependencies:
 ypbind            i686          3:1.20.4-29.el6             DVD           51 k

Transaction Summary
================================================================================
Install       2 Package(s)

Total download size: 115 k
Installed size: 241 k
Is this ok [y/N]: n
Exiting on user Command

server#



As you can see, you can install new package from DVD repository.

And that is it!


Friday 25 July 2014

custom script to speed up mount iso - EXPLAINED

I read once that good system admin will write a script for something that he has to do more than few times. Especially if you have to write down long command lines for something that is not very important but it is still time consuming.

So...
Very often I have to mount iso files on file system. Command line for this is

server#mount -t iso9660 /location_of_iso /mount_point -o loop

This is not very long but after some time even this can be very annoying.
So I decided to speed things up.

I wrote simple script called mount_iso. I look like this

server# cat mount_iso
#!/bin/bash
ISO="$1"
echo $ISO
mount -t iso9660 $ISO /mnt/ -o loop


Make it executable.
server#chmod +x mount_iso

In is good to move this script in folder from where you call other scripts and programs to run.

server# env |grep PATH
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin


I moved my to /bin.


How it works?

server# mount_iso rhel-server-6.3-i386-dvd.iso
rhel-server-6.3-i386-dvd.iso
server# mount |grep mnt
/opt/iso/rhel-server-6.3-i386-dvd.iso on /mnt type iso9660 (rw,loop=/dev/loop0)

I predefined mount point to be /mnt just because this is convenient for me.

This only saves few seconds of my time but it I find it very useful.



Wednesday 23 July 2014

How to add date to logrotate files- EXPLAINED!

By default, when log files are rotated they are first renamed and after compressed if this option is enabled.
Those moved files get extension in numbers so depending of how many rotation cycles you have it looks something like this

server# pwd
/var/log
server# ls |grep messages
messages
messages.1
messages.2
messages.3
messages.4


In case that you want to change these number to date expression you have to add this line to /etc/logrotate.conf

.
.
dateext
.
.

On next rotation cycle your log files will look like this

.
.
messages-20140209
messages-20140216
messages-20140223
.
.

logrotate change from gzip to bzip2 compression - EXPAINED

Default compression of log files in logrotate is gzip. In case that you want more efficient compression, you can switch gzip compression with bzip2 compression.

How to do this?


In /etc/logrotate.conf file you settings should look like this

.
.
#compress
compresscmd /usr/bin/bzip2
.
.

On next logrotate cycle your files will be compressed with bzip2.

How to compress logs with logrotate?

By default, logs created in /var/log/  are rotated and depending of Linux distro that you use are compressed or not. Depending on how frequent rotation is your log files can be quite big. To avoid file system digestion, it is wise to compress old log files during rotation cycle.

To enable compression of log files in /etc/logrotate.conf uncomment compress statement

.
.
# uncomment this if you want your log files compressed
compress
.
.

There is no need for restart of any service. During next logrotate cycle, new configuration will be executed. If you can not wait execute it manually

server#logrotate /etc/logrotate.conf

 and compressed logs will appear in /var/log folder. 


Wednesday 9 July 2014

Adding new LUN with multipath on Linux

So you have need to add another LUN to your server. Here is procedure how to do it if you use multipath on your server!

First your storage admin has to assign new LUN to you server. You may see this in you log when LUN is added from storage side

kernel: sd 1:0:0:10: [sdbb] Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments.
After this is done, you can proceed.
List your current multipath status

server# multipath -ll
|-+- policy='round-robin 0' prio=4 status=active
| |- 1:0:1:9  sdau 66:224 active ready running
| `- 2:0:1:9  sdan 66:112 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  |- 1:0:0:9  sdk  8:160  active ready running
  `- 2:0:0:9  sdai 66:32  active ready running

Go to /sys/class/fc_host folder.

server# cd /sys/class/fc_host/
server:/sys/class/fc_host # ls
host1  host2 


From multipath -ll you can see that we are using host1 and host2 (this is important in case that you have more than two fc single port card or some dual port card- every fc port will be represented as hostX). In this case we have two single port cards and ports are represented as host1 and host2.

First we would like to restart port on host1!

server#cd host1
server#echo 1 >issue_lip

In you /var/log/messages something like this should appear

kernel: qla2xxx [0000:0a:00.7]-801c:5: Abort command issued xxxxx 
kernel: scsi 1:0:0:10: Direct-Access     DGC      VRAID            0532 PQ: 0 ANSI: 4
kernel: scsi 1:0:0:10: alua: supports implicit and explicit TPGS
This means FC port detected newly added LUN.

Now rescan scsi host host1!

server# cd /sys/class/scsi_host/host1
server# echo "- - -" >scan

This will add that new LUN to the system as scsi devices!
In your log something like this should appear

.
sd 1:0:0:10: [sdbb] Attached SCSI disk
.
.

Now if you check your multipath
server#multipath -ll
|-+- policy='round-robin 0' prio=4 status=active
| |- 1:0:1:9  sdau 66:224 active ready running
| `- 2:0:1:9  sdan 66:112 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  |- 1:0:0:9  sdk  8:160  active ready running
  `- 2:0:0:9  sdai 66:32  active ready running




|-+- policy='round-robin 0' prio=4 status=active
| `- 1:0:1:10  sdbb 67:80 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 1:0:0:10  sdbb 67:16  active ready running



As you can see old LUN has 4 paths and new LUN has only 2 paths. Other two paths will appear when you do rescan on host2.

Don't be confused on my server I see 4 paths for every LUN because I have two FC ports connected to two FC switch. From single FC switch to storage, we have two connection because of redundancy. So in total, we have  4 paths from server to storage.


Tuesday 8 July 2014

crontab deleted accidently - how to restore it EXPLAINED!

Well, accidents do happen! Usually, when they do happened you are not prepared for them.

One of those accidents in Linux sysadmin world is definitely erasing crontab. You did not have to do it. But you have to restore it!

Ok, few things you need to know first!
1.Crontab is scheduler you users. Every user has it's own crontab. One user can not change crontabs of another user (except if root gave him permission)

# whoami
test
# crontab -e -u test1
must be privileged to use -u


But user can delete it's own crontab. How? Well that is not subject of this post but usually reason is lack of knowledge. By this I think of this

#crontab -r

This will remove your crontab. And you can't do nothing about it. Point is - if you do not know what you are doing then DON'T do it! Read first about it or use some test server,etc.

By the time user calls you to say that his crontab is missing, damage is already done. And they will call you.

2. When any user create crontab, file is generated with that crontab. So, when you edit or list your crontab, you actually reading that file. Depending of your Linux distro, end location may wary but
they are all located in /var/spool/cron/ folder ( on Ubuntu location is /var/spool/cron/crontabs, on Sles11 location is /var/spool/cron/tabs, on Centos location is /var/spool/cron). So when you list this folder you will see something like this

# ls
root  test  test1

This means that users root, test and test1 have crontabs! If you read these files you will see that they are same as crontab that you edit!
Let's delete crontab for user test1!

$ whoami
test1
$ crontab -r
$ crontab -l
no crontab for test1

List
# ls
root  test

And now.... restore of crontab! If you don't have backup of this folder/files/OS, they you can't restore crontab for that perticular user (in are case test1)! If this is the case, then your user has to write down crontab again. This can be a very big problem because at this point all of your scripts that were running throught crontab will not run on that particular time. This is just small example why you (if you are serious about administration) need to have backup!
 
If you do have backup, just restore file to crontab location!

# cp /root/cron123 test1
root@ubuntu:/var/spool/cron/crontabs# ls
root  test  test1
root@ubuntu:/var/spool/cron/crontabs# crontab -l -u test1
* * * * * date


In case that user accuse you that you deleted his crontab file(because root can edit/delete other user crontabs), depending of log level in your system log, you will have notification of crontab actions!

Jul  8 09:20:26 ubuntu crontab[22165]: (test1) LIST (test1)
Jul  8 09:20:36 ubuntu crontab[22167]: (test1) DELETE (test1)
Jul  8 09:20:45 ubuntu crontab[22168]: (test1) LIST (test1)





Thursday 3 July 2014

How to show adsence ads in archive posts!

I already explained how to insert adsence ads before post text but after post title!

My friend recently told me that I do not have ads in archive article! When I looked in to it, that was really true!

Hm...
Ads were showned normally on home page! When go to archive and click on month, posts that are showned have ads! But when I click on single post in archive, ads were missing!

So what was the problem?

Problem is itemprop! Because I put my adsence code in section where itemprop = articleBody
and this only shows that code in home page!

<data:post.body/>
        <div style='clear: both;'/> <!-- clear for photos floats -->
      </div>
    <b:else/>
      <div class='post-body entry-content' expr:id='&quot;post-body-&quot; + data:post.id' itemprop='articleBody'>
        <div style='float: left; margin: 10px 10px 10px 0;'>
    &lt;script async src=&quot;//pagead2.googlesyndication.com ...
.
.
</div>

I insert same code in section where itemprop='description articleBody' and now I have ads showned in archive post!

<div class='post-body entry-content' expr:id='&quot;post-body-&quot; + data:post.id' itemprop='description articleBody'>
           <div style='float: left; margin: 10px 10px 10px 0;'>
    &lt;script async src=&quot;//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js&quot;&gt;&lt;/script&gt;
.
.
</div>


As you can see, only difference is description!

 

Tuesday 1 July 2014

How to run a Linux script every few seconds under cron - EXPLAINDED and SOLVED!

Crontab is a scheduler for Linux that run scripts on certain time of day, week or month!

It has 5 fields for setting that!

_  _  _  _  _  script.sh

First field is for minutes.(00-59)
Second field is for hours.(00-23)
Third field is for day in a month(1-31)
Forth field is for month in a year (1-12)
Fifth field is for day in a week (0-6, where 0 is sunday)

If you have star(it is joker sigh after all) is some field that means that all numbers is included!
So

* * * * * script.sh

means that script.sh will execute every minute, every hour, every day in a month, every day in a week and every month in a year.
 
Minimum time resolution that you can use for cron is one minute!  Script will be executed in 00 seconds on particular time. But what if you had do execute your script on 20th second of every minute or every 20 seconds?
From cron you cannot resolve this, because as we said - minimum time resolution is one minute. (Resolution is smallest step between two neighbour points)

In case that you that "/number of seconds" will resolve your issue, you are wrong. This / does not mean divide like in math. It means repeat every number of minutes, hours,etc. depending in what field it is standing.

If you have something like this

*/5 * * * * script.sh

this will execute script.sh every 5 minutes.

or

* */5 * * * script.sh

will execute script every 5 hours!
You get the point!

Ok, are problem needs different approach!

If you want script.sh to be executed every 15 seconds, do following thing!

server#cp script.sh script15.sh
server#cp script.sh script30.sh
server#cp script.sh script45.sh 

In script15.sh enter following line

#!/bin/bash
sleep 15
.
.
 

before commands in script!
For script30.sh enter sleep 30, for script 45.sh enter sleep 45.
Sleep command delay everything for certain amount of time in seconds!

sleep - delay for a specified amount of time

Now in crontab

* * * * * script.sh
* * * * * script15.sh
* * * * * script30.sh
* * * * * script45.sh
So, all scripts will be executed in same time but because of sleep command inside of then they will be delayed for specific amount of seconds!