If your copying files using cp or rsync or whatever method. Sometimes you want to see the new files or changed files in the past N seconds.

You can do that like so

Now you can see the difference using “diff” or even “comm”

Diff will show you the before and after. When looking at different files and what changed, we are just interested in seeing whats new, what changed. We are not interested in the old value (if you are then keep using “diff”). comm can help us isolate only whats new.

If you look at what comm does, it seperates whats unique to each file into different columns. So for example: all of the data unique to file1 (/tmp/OLD) will go in first column. all of the data unique to file2 (/tmp/NEW) will go in 2nd column. All of the data unique to both files (that appears in both /tmp/OLD and /tmp/NEW) will go in the 3rd column. using -1 (unique to file1),-2 (unique to file2), or -3 (unique to file3), we can suppress different outputs. For example from the above we only want to see whats new, so thats what is unique to /tmp/NEW (what lines only exist in /tmp/NEW in other words). So that means we only want the 2nd column. With comm, we dont tell it what we want, we tell it what we dont want. So we tell it to supress 1 and 3, leaving us with 2 – which is what is unique to file2 (/tmp/NEW).

Help output of comm might help

TIP: its best to sort the data for both files prior to running diff or comm. comm is more picky on it though.

Cool thats pretty cool. We can add some tweaks, like making ls output better.

ls -lisa -B1  shows entire contents of folders, so we dont want to run that on folders, we only run it on files. Either way we are only curious in seeing what files are new and get bigger. folders dont really make a difference unless there is something in them.

Now we can add loop

WATCH LOOP

watch is an interesting program that basically gives you the ability to do an infinite loop that nicely shows you data at the same time. I will use that to its advantage here.

This loops the following: ls all of the files with find and save that to /tmp/NEW, now diff /tmp/NEW and /tmp/OLD (first run of this fails because /tmp/OLD doesnt exist yet), now make a copy of /tmp/NEW to /tmp/OLD (move/rename would work as well), wait a few seconds, repeat… the second time around /tmp/OLD is the old find output, and /tmp/NEW is the new find output, so we can diff & comm them.

If ls freaks out you can just use the simpler find.

WHILE LOOP

Not all systems support “watch”. However all shells should have a while loop.

Here is one with the intricate better looking ls output

Here is one with the generic find  included ls output

The end

Leave a Reply

Your email address will not be published. Required fields are marked *