Rearranging BT Meridian csv reports

This is a pretty specific-use script, but I'll stick it here anyway since I keep losing it. It's to rearrange the output from a BT Meridian switch reporting on the usage of its DNs. What it does is to space the DNs such that each DN appears in a row with a number equal to its own. DN 123 appears on row 123, DN 498 appears on row 498 etc. I don't know why this is particularly useful, but I'm assured it is.

There are enough comments, hopefully, to make it reasonably easy to modify this script for similar uses in future. In short, it reads the csv file as plain-text line-by-line and uses a regexp to pick out the DN. It writes the whole line to the array, at the element numbered the same as the DN. (if the same DN appears twice, the last one encountered survives, the rest don't). It then prints _every_ line of the array to a CSV file, including null ones, such that even if $array[285] is the first non-null element, it is still the 285th line printed to the file.

To use it, stick the csv file you want processed and the script in the same directory, and run the script. It will produce 'out.csv' which is the processed script. It always operates on the alphabetically first *.csv file it finds (except out.csv) and will silently overwrite out.csv if it exists. If it has problems, it writes 'errors.txt' which contains the output. It's mostly run on Windows boxen outside of a terminal

  1. #! /usr/bin/perl
  3. use strict;
  5. ## Most Recent Changes:
  6. ##
  7. ## Added dupe checking. Before writing to array, checks whether that element exists or not.
  8. ## If it does, writes message to error array and carries on. At the end of the run, if the
  9. ## error array contains messages it dumps them to a new errors file, and exits. It does not
  10. ## create the output csv.
  11. ##
  12. ## Also writes more general errors to the file, including if it finds no csv files.
  13. ##
  14. ## Might support filenames with spaces.
  15. ##
  16. ##
  18. my $output_file = './out.csv';
  20. my $error_file = "./errors.txt";
  21. my @error_array = ();
  23. # Open the current directory and read its full contents to a new array called @files.
  24. # Then go through @files and, if the file does not begin with a dot, ends with `csv`
  25. # and is not `out.csv` add it to an array. We then sort the array and pick the first
  26. # file and presume that is our input file.
  27. opendir(DIR, ".") || die("Error opening working dir");
  28. my @files=readdir(DIR);
  29. my @csvs;
  30. foreach my $f (@files){
  31. if (($f !~ /^\./) && ($f =~ /csv$/) && ($f !~ /^out\.csv$/)){
  32. push( @csvs, $f);
  33. }
  34. }
  35. if (@csvs < 1){
  36. print "no csvs";
  37. push (@error_array, "Can find no csv files. Check it's not saved as xls");
  38. }
  40. @files=sort(@csvs);
  41. my $input_file = "./$files[0]";
  43. # Handy info for the user
  44. print $input_file."\n";
  46. # Initialise an array for the output, and one for error messages.
  47. my @output_array = ();
  49. # Go through each element of the array (and therefore line of the file) and,
  50. # if it contains the string 'LO' but not 'Unknown', process it
  53. open INPUT_FILE , "< $input_file" || die ("Error opening input file at $input_file");
  55. foreach (<INPUT_FILE>){
  56. my $line = $_;
  57. ## If the line begins with two or more decimal digits, it's probably one of
  58. ## those funny bungay ones
  59. if ($line =~ /^\d{2}/){
  60. my $line_number = (split /,/, $_)[0];
  61. $line_number--;
  62. ## If the line we want to write to is non-empty (i.e. it has
  63. ## digits in it), then error
  64. if ($output_array[$line_number] =~ /\d/){
  65. my $repeat_count = 0;
  66. foreach (<input_file>){
  67. if ((split /,/, $_)[0] =~ m/$line_number/){
  68. $repeat_count++;
  69. }
  70. }
  71. $error_array[$line_number] = $line_number." exists about ".$repeat_count." times.";
  72. }
  75. $output_array[$line_number]=$line;
  76. ## If it doesn't, carry on as normal.
  77. }elsif ($line =~ /LO/ ){
  78. if ($line !~ /Unknown/){
  79. # Pick out where the TN is in the line (in the third /-separated
  80. # group of the third comma-separated group).
  81. # Then write the entire line to the element of the array that is one
  82. # less than the TN. Chop() gets rid of the last character of the string
  83. # which here is a "
  84. # (remember that the first element of an array is 0, but the first
  85. # line of a file is 1; element 0 will become line 1)
  86. my @cur_line = split (/,/, $_);
  87. my @tn_string = split (/\//, $cur_line[2]);
  88. my $tn = $tn_string[2];
  89. $tn--;
  90. $output_array[$tn] = $line;
  91. }
  92. }
  93. }
  94. close INPUT_FILE;
  96. foreach (@error_array){print;}
  98. # If the error array contains records (i.e. its length is greater than zero),
  99. # create an error log file
  100. if (@error_array > 0){
  101. open ERROR_FILE , "> $error_file" || die ("Error opening error log at $error_file. You're fucked.");
  102. print ERROR_FILE "Errors encountered processing ".$input_file.":\n";
  103. print STDERR "Errors encountered processing ".$input_file.":\n";
  104. foreach my $error (@error_array){
  105. print ERROR_FILE $error;
  106. print STDERR "\t".$error;
  107. }
  108. close ERROR_FILE;
  109. print "\n";
  110. }
  113. # Open the output file, and write the contents of the output array to file.
  114. # Remember that Perl will append a \n to each non-null element of the array, but we want one
  115. # on the end of every element, including null. So we chomp each line to remove the /n from
  116. # those that have it, and re-add it to _every_ array.
  118. open OUTPUT_FILE, "> $output_file" || die ("Error opening output file at $output_file");
  119. my $output_line;
  120. foreach $output_line (@output_array){
  121. chomp $output_line;
  122. print OUTPUT_FILE $output_line. "\n";
  123. #print $_, "\n";
  124. }
  125. close OUTPUT_FILE;
  126. print "\n";
  127. </input_file>

Stop OSX writing .DS_Store and ._ files on network shares

All our fileservers are Windows. Most of the files are manipulated in OSX. This isn't likely to change soon since, irrespective of who is accountable for most of the bytes, Windows are far and away the most populous in terms of hosts.

.DS_Store files are apple's way of storing the finder preferences for a particular directory. They contain things like background image, sort order, window size and position, and are written when any value for the current directory deviates from that of the parent, which includes every time it is moved.
._ files are Apple's way of storing attributes supported by HFS and HFS+ on less featureful filesystems (like NTFS and FAT).

To prevent OSX writing .DS_Store files to SMB shares, run the following:

  1. defaults write DSDontWriteNetworkStores true

This sets a per-profile value, so must be run as each user.
Preventing ._ files is a bit more tricky, but it can be done in/for the shell with:


To prevent Samba accepting these files, enter the following into smb.conf

  1. veto files = /.DS_Store/

And read LavaLite's bit on it