sftp user action logs - EXPLAINED and SOLVED

I already wrote how to setup and why to use SFTP.Time has passed and someone ask you to see did someone using sftp add/rename/download/upload/delete some file or folder. You go to /var/log and search for logs about sftp actions. And all that you can find is ...

ubuntu sshd[3510]: subsystem request for sftp by user boris
ubuntu sshd[3510]: pam_unix(sshd:session): session closed for user boris



So as you can see there is no log about what user connected by using sftp is doing. At some point in time someone will tell you that this is a security/where are these files/who delete files issue. So how to do this? It is quite simple if you understand how logs are created, chroot and off course if you are reading this!

Check ssh version first!

 First and most important thing to know. On older version of openssh there is very good chance that this will not work. Why, sftp-server dont have these options for older versions.
I tested this on
root@ubuntu:/var/log# cat /etc/issue
Ubuntu 12.04.4 LTS \n \l
root@ubuntu:/var/log# uname -a
Linux ubuntu 3.5.0-23-generic #35~precise1-Ubuntu SMP Fri Jan 25 17:15:33 UTC 2013 i686 i686 i386 GNU/Linux
root@ubuntu:/var/log# dpkg -l openssh*
ii  openssh-client 1:5.9p1-5ubunt secure shell (SSH) client, for secure access
ii  openssh-server 1:5.9p1-5ubunt secure shell (SSH) server, for secure access

But on CentOs 5.8 this do not work
# cat /etc/issue
CentOS release 5.8 (Final)
# rpm -qa |grep ssh
openssh-4.3p2-82.el5
openssh-server-4.3p2-82.el5

Few thing to know about logs and chroot

As you know sftp users(in my case user boris) are all chrooted to directory configured in /etc/ssh/sshd_config with ChrootDirectory directive. In case that you do not know what this means, it means that once user is connected he cannot live this location. This user can't do anything outside this this folder. Read/write operations are only limited to his sftp folder. If you are wondering why I wrote 3 sentences about that use cannot leave chroot dir folder, wait just a bit more. Rsyslog or syslog capture events things by using socket /dev/log. This is important because sftp of features of ssh and ssh use rsyslog for logs storing then in /var/log. Permissions on /dev/log is  

# ls -la /dev/log
srw-rw-rw- 1 root root 0 Oct 31 12:54 /dev/log
So anybody that can approach to /dev/log can make logs by using rsyslog. But can sftp user approach to /dev/log? NO! Why? Because he is captured inside of his chrooted directory!!! So, idea is to still have chrooted user but that he can write in /dev/log.

Configuration in sshd_config

To enable logs for sftp-server we must fist enable it in sshd_config.  Change line

ForceCommand internal-sftp 
to
ForceCommand internal-sftp -l INFO -f AUTH

Option -l(small letter L) is for log level and option -f is for location of log. Do not for -f add location of file, this is done in rsyslog.conf.
After you make neccesery changes, restart ssh service.

#/etc/init.d/sshd restart

Accessing /dev/log from chrooted folder 

How to do this? My sftp folder is defined in sshd_config in ChrootDirectory /opt/sftp_test/%u directive and my sftp user is boris. Follow these steps!

#cd /opt/sftp_test/boris
#mkdir dev
#touch dev/log

#chmod 511 dev
#chattr +i dev
# mount --bind /dev/log dev/log

And this is it!!!

How to test if this is working?
Go to your AUTH log. Depending of Linux distro this can be in /var/log/secure or /var/log/auth.log.
In this test server I am using Ubuntu 12.04.
#tail -f /var/log/auth.log
Oct 31 15:28:30 ubuntu internal-sftp[2460]: session opened for local user boris from [x.x.x.x]
Oct 31 15:28:31 ubuntu internal-sftp[2460]: opendir "/"
Oct 31 15:28:31 ubuntu internal-sftp[2460]: closedir "/"
Oct 31 15:28:47 ubuntu internal-sftp[2460]: opendir "/boris"
Oct 31 15:28:47 ubuntu internal-sftp[2460]: closedir "/boris"
Oct 31 15:28:53 ubuntu internal-sftp[2460]: mkdir name "/boris/12/123" mode 0777
Oct 31 15:28:58 ubuntu internal-sftp[2460]: opendir "/boris/12/123"
Oct 31 15:28:58 ubuntu internal-sftp[2460]: closedir "/boris/12/123"
Oct 31 15:29:05 ubuntu internal-sftp[2460]: open "/boris/12/123/analy.jpg" flags WRITE,CREATE,TRUNCATE mode 0700
Oct 31 15:29:05 ubuntu internal-sftp[2460]: close "/boris/12/123/analy.jpg" bytes read 0 written 28925
Oct 31 15:29:19 ubuntu internal-sftp[2460]: session closed for local user boris from [x.x.x.x]
Oct 31 15:29:19 ubuntu sshd[2459]: Received disconnect from x.x.x.x: 11: disconnected by user
Oct 31 15:29:19 ubuntu sshd[2287]: pam_unix(sshd:session): session closed for user boris

As you can see  this is much better log!
If you have more then one user that is using sftp, you have to do this for every one of them!

In case that you want this to work after reboot. read this post fstab mount bind!

fstab mount bind error - SOLVED and EXPLAINED!

In case that you are using mount --bind option for something(in my case this was a must for sftp user action logs), you want this mount --bind survive reboot. So you will put this mount point in /etc/fstab.

First I mount it "by hand"

server:#mount --binb /dev/log /opt/sftp_test/boris/dev/log

This worked so I need it put make it mount automaticly after reboot.  

My fstab options was like this

/dev/log  /opt/sftp_test/boris/dev/log    none    bind

So, I test it if this works!

"False" test pass. Why I called it false test? I umount /opt/sftp_test/boris/dev/log and automaticly mounted it.

server:#umount /opt/sftp_test/boris/dev/log

server:#mount -a

server:#mount
/dev/mapper/ubuntu-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sda1 on /boot type ext2 (rw)
rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw)
/dev/log on /opt/sftp_test/boris/dev/log type none (rw,bind) 

 

Fstab mount bind error

Ok, now only true and real test! Reboot! As I was waiting for ping to start again so that I can ssh log to my test server, think that nothing can go wrong... Ping started to pass but I could not ssh connect. Ok,ssh service did not started yet.10 seconds, 20 seconds, 30 seconds... Ok, now I know that something is wrong! I looked at my VirtualBox server and there is was black stale screen.

Error was:

The disk drive for /opt/sftp_test/boris/dev/log is not ready yet or not present.

So I just pressed S and boot process continued. This is why reboot is real test! :)

What can be reason for this? I suspect that reason is that mounting partitions from fstab is one of the first thing that OS during boot process do. There is still no /dev/log location at this point. Error message The disk drive for...  is pretty clear.

How to resolve this? In /etc/rc.local insert following line

mount --bind /dev/log /opt/sftp_test/boris/dev/log

Reboot!!!

root@ubuntu:/opt/sftp_test/boris/dev# reboot
root@ubuntu:/opt/sftp_test/boris/dev#
Broadcast message from root@ubuntu
        (/dev/pts/0) at 12:53 ...

The system is going down for reboot NOW!
login as: root
root@x.x.x.x's password:
Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.5.0-23-generic i686)
Last login: Fri Oct 31 12:47:43 2014 from .x.x.x.x

root@ubuntu:~# mount
/dev/mapper/ubuntu-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sda1 on /boot type ext2 (rw)
rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw)
/dev/log on /opt/sftp_test/boris/dev/log type none (rw,bind)

Problem solved!

Huawei U8650 sim unlock and cyanogenmod issue

A few days ago I wanted to insert another operator SIM card into my old HUAWEI U8650. I already wrote about rooting this phone and installing custom rom. I installed Cyanogenmod 7.2! I have to say that this is one of the best custom roms out there for this phone and generaly for older andoroid phones.

SIM Unlock - what does this means?

When you buy phone from telecom operator, SIM slot is locked to work only with that operator SIM cards. This way operator protect his investment because when you sigh contract for some time period that includes phone you usually get phone very cheap or for free. If you want to unlock SIM by yourself you would loose warranty, if you go to telecom operator they will tell you that they cannot SIM unlock your phone until contract expires! When you insert SIM card from other telecom operator, phone will report that SIM is not inserted and then ask for NETWORK UNLOCK CODE. You will be prompted to insert this code. By inserting SIM unlock code, you will unlock your phone to SIM cards from other telecom operators. You will not be able to use it until you unlock SIM protection.

How to get SIM unlock code?

Depending of policies of your telecom operator, sim unlock code you can get from your telecom operator for free or not when contract expires. If you lost your contract papers you can buy sim unlock code from ebay. SIM unlock code is generated by using phone IMEI and telecom operator "key". How to do this, I do not know but those information is only thing (besides money) that you need to get SIM unlock code! I takes a few days to receive code.

Cyanogenmod SIM unlock issue

In case you have already installed Cyanogenmod ROM, there is a problem.
This is from one forum:
"I had this problem. Cyanogenmod does not prompt you for an unlock code when putting in a different sim.

You have to flash back to a stock rom, and then put in your unlock code. Then flash back to Cyanogenmod."

What to do?


1. Find stock rom for your telephone! For Huawei U8650 here is stock custom rom!
2. http://sysadmin-tricks.blogspot.com/2013/12/huawei-u8650-rooting-and-downgrade-from.html
    part about rooting and downgrade
3.This will install, stock custom rom. When you insert SIM card from another telecom operator and turn on the phone, you will prompted with this

NETWORK UNLOCK CODE

Insert code.

You will be prompted that network is unlocked!

That is it! You phone is now SIM unlocked! 

Only bad thing about stock rom installing is that wipes up CWM recovery so if you want to install some custom rom, you have to first install CWM!