On 2009-10-28 18:09, Truejack wrote: > Need a scripting help to sort out a list and list all the duplicate lines. > > My data looks somethings like this > > host6:dev406mum.dd.mum.test.com:22:11:11:no > host7:dev258mum.dd.mum.test.com:36:17:19:no > host7:dev258mum.dd.mum.test.com:36:17:19:no > host17:dev258mum.dd.mum.test.com:31:17:19:no > host12:dev258mum.dd.mum.test.com:41:17:19:no > host2:dev258mum.dd.mum.test.com:36:17:19:no > host4:dev258mum.dd.mum.test.com:41:17:19:no > host4:dev258mum.dd.mum.test.com:45:17:19:no > host4:dev258mum.dd.mum.test.com:36:17:19:no > > I need to sort this list and print all the lines where column 3 has a > duplicate entry. > > I need to print the whole line, if a duplicate entry exists in column 3. > > I tried using a combination of "sort" and "uniq" but was not successful. Long time ago (when I was still young and beautiful) and encountering also the limitations of "uniq", I wrote a small program in C to do these kinds of things. It is designed to handle record oriented stuff in groups similar to uniq. The primary purpose was as prepocessor to awk/perl, but simple things like this are builtin. You find it here: ftp://ftp.xplanation.com/utils/by-src.zip Unpack; make; and copy the program "by" somehwere in your PATH. Then, to solve your problem do: sort -t: -k 3 InputFile | by -F: -f3 -D This sorts the input on field 3, fields separated by colon, and outputs all lines that are duplicate according to field 3 (-D). The program can do more as well, and a little tutorial is included in the zip. -- Paul Bijnens, Xplanation Technology Services Tel +32 16 397.525 Interleuvenlaan 86, B-3001 Leuven, BELGIUM Fax +32 16 397.552 *********************************************************************** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, ~., * * stop, end, ^]c, +++ ATH, disconnect, halt, abort, hangup, KJOB, * * ^X^X, :D::D, kill -9 1, kill -1 $$, shutdown, init 0, Alt-F4, * * Alt-f-e, Ctrl-Alt-Del, Alt-SysRq-reisub, Stop-A, AltGr-NumLock, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***********************************************************************