This simple perl script does the job of sort -u (of course not in windows)
Script:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
#!/usr/bin/perl use strict; use warnings; #File reading code open(my $in,"<:utf8",$ARGV[0]) or die "Cannot open $ARGV[0]:$!n"; my @in =<$in>; close($in); #declare hash my %hash=(); #store content in hash foreach my $in (@in) { chomp($in); $hash{$in} = 1; } #Finally print the hash foreach my $keys(sort keys %hash){ print "$keysn"; } |
How to run:
Lets assume that our script is saved as my_unique.pl
$ perl my_unique.pl <filename>
* Without angular brackets.
Explanation:
Here in Perl hash the duplicate keys are overridden which means only single instance of each line in the file is stored in the hash. Remember that while storing the lines in hash each single line is treated as key so as to remove duplicates.
Note: Always use strict and use warnings when dealing with Perl code so as to write bug free code.