User Tools

Site Tools


bdnog5:track2agenda:log-lab

Rsyslog and logrotate/log archiving

By default, there is an instance of rsyslog that runs inside every new Ubuntu installation. The rsyslog tool takes care of receiving all the log message from the kernel and operating system applications and distributing them over files in /var/log.

However, rsyslog can do much more than that which includes logging into a remote server. This can be extremely useful for aggregating logs across a large fleet of servers or when it is not possible to write logs on disk.

Installing rsyslog on Remote Server

You will need a copy of rsyslog running on a remote machine which will be receiving the logs from your existing server. It’s best that you have this in a remote location. The reason being that if this server crashes at the same time as your server crashes, you won’t be able to get any logs to troubleshoot any issues.

Assuming that you’re using Ubuntu on the remote machine, you’ll already be running rsyslog. If not, you can install it by following the instructions provided inside the rsyslog website.

Once it’s installed, you will need to make sure that it listens on a port which we will send logs to. The default port is 514 which we’ll keep. You will need to edit the file /etc/rsyslog.conf and uncomment the following lines:

# provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514

After uncommenting the lines, it should look like the following:

# provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514

Once that’s done, you can now restart the rsyslog service by running the command service rsyslog restart. Your rsyslog instance is now ready to receive logs from remote hosts.

Setting Up rsyslog on Local Server

This process is extremely easy. All you need to do is tell the existing rsyslog instance on your server to ship logs to your remote server. It’s as easy as creating a file inside the /etc/rsyslog.d folder called 10-rsyslog.conf. Inside that file, all you need to put is

*.*   @192.168.30.<your Group Mate's IP>:514

Once you write that file, restart the rsyslog service by running service rsyslog restart and your logs will now start being shipped to your remote server. You can verify that by logging into the remote server and checking the log folder, you’ll find that you now have logs labeled with the hostname of the client server.

Check out the contents of the /var/log folder.

# ls -la /var/log

Customizing More

Create new directory in /var/log

Create new directory in /var/log called rsyslog_custom. so that we will keep all Servers log in this directory.

mkdir -p /var/log/rsyslog_custom

Log Rotating without logrotate

Rather than logrotate we will try to customize rsyslog as a log archive with it's builtin features. Templatiing Engine.

Add the following lines at the end of the file /etc/rsyslog.conf:

# this line creates a template that will store the messages for each host in a seperate file.
# a new file will ben created daily because of the date in the filename.
$template DailyPerHostLogs,"/var/log/rsyslog_custom/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/%PROGRAMNAME%.log"
*.* -?DailyPerHostLogs;TraditionalFormat

If you have any specific services which creates so many logs feel free to modify with hourly log. Like for a heavy duty mail site use the following

$template HourlyMailLog,"/var/log/rsyslog_custom/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/mail.%$HOUR%.log"
# Log all the mail messages in one place.
mail.*                                                  -?HourlyMailLog

Log Compression

Logrotate also can compress log files. But unfortunately rsyslog cannot. However we can tweak cron to check for files which haven't been modified over a day and compress them. Lets modify cron by editing the file /etc/cron.hourly/syslog-bzip2:

# Compress *.log-files not changed in more than 24 hours:
find /var/log/rsyslog_custom -type f -mtime +1  -name "*.log" -exec bzip2 '{}' \;

Rsyslog can be more customized based on requirements. Check the man pages for a full list of possibilities.

bdnog5/track2agenda/log-lab.txt · Last modified: 2016/04/10 02:06 by Muhammad Moinur Rahman