Tips to analyse logs using unix commands

I have seen many developer struggling to analyze the logs on unix. They download the files and then open file in editors which is not efficient way to analyze the logs. Task can be achieved with much easier way using commands. First thing I advise to use bash command before starting, This was all commands will be saved in history. You can access the history of commands using “history” command.

  • Display first 100 lines of log file
    head -100 /logs/error.log
  • Display last 100 lines of log files
    tail -100 /logs/error.log
  • View growing log file in real time using tail command
    tail -f /logs/error.log
  • View growing log file in real time with last 100 lines
    tail -100f /logs/error.log
  • Display complete log file
    cat /logs/error.log
  • Display specific lines (based on line number) of a file using head and tail command

The example below will display line numbers 101 – 110 of /var/log/anaconda.log file
M – Starting line number
N – Ending line number
Syntax: cat file | tail -n +N | head -n (M-N+1)
$ cat /logs/error.log | tail -n +101 | head -n 10
cat : prints the whole file to the stdout.

tail -n +101 : ignores lines upto the given line number, and then start printing lines after the given number.
head -n 10 : prints the first 10 line, that is 101 to 110 and ignores the remaining lines.

  • Display lines matching a pattern, and few lines following the match.
    grep “SocketTimeOutExcdption” /logs/error.log
  • Using grep command in combination of other commands
    cat /logs/error.log | grep “SocketTimeOutException”
    tail -f /logs/error.log | grep “SocketTimeOutException”
  • Display lines which not matching the pattern
    grep -v “httpStatus=200” /logs/error.log
    cat /logs/error.log | grep -v “httpStatus=200”
    tail -f /logs/error.log | grep -v “httpStatus=200”
  • How to count specific errors
    grep -c “SocketTimeOutExcdption” /logs/error.log
    cat /logs/error.log | grep -c “SocketTimeOutException”
  • How to match regular expressions in file
    grep “RegularExpression” File_name
    grep “Socket*Exception” /logs/error.log

    A regular expression may be followed by one of several repetition operators:
  1. ? The preceding item is optional and matched at most once.
  2. The preceding item will be matched zero or more times.
  3. + The preceding item will be matched one or more times.
  4. {n} The preceding item is matched exactly n times.
  5. {n,} The preceding item is matched n or more times.
  6. {,m} The preceding item is matched at most m times.
  7. {n,m} The preceding item is matched at least n times, but not more than m times.
  • Match case insensitive strings in file
    grep -I “sockettimeoutexcdption” /logs/error.log
  • How to view compressed file
    zcat /logs/error.log.gz
  • You can user other various commands in combination like grep, head, tail
    zcat /logs/error.log.gz | grep “SocketTimeOutException”
    zcat /logs/error.log.gz | head -100
  • How to grep in compressed file
    zgrep “SocketTimeOutException” /logs/error.log
  • Use awk/nawk for column specific processing. Follow my next article on this awk/nawk. 

Leave a Reply

Your email address will not be published. Required fields are marked *