Archive for category: Scripts

Merge CSV files keeping one header

Merge CSV files keeping one header

 
8 Kudos
Don't
move!

There are typical situations in life that require lots of engineering to be addressed. Other situations don’t. This is exactly one of those commands you may want to learn by heart but, for those who can’t, that’s why blogs exist.

Suppose you have collected statistics for several days and you have lots of files having the same structure, e.g.:

cola,colb,colc
1,2,3
1,3,5
4,3,1
...

and you want to create one big file that keeps the header from just the first one and appends all the files so that you can quickly load it into your favourite tool for further analysis.
Here is the trick:

Explanation:
FNR is the number of records read by awk (per file), NR is the number of lines read overall. Therefore, the condition FNR==1 && NR>1 evaluates to true only if the line being evaluated is the first line of the file and we already read at least one line overall (so it is not the first line we read). What happens when it is true? {next;}, hence the line gets ignored.

Warning
I am pretty sure that some of you just came here, copied&paste the line above, tested and found out that the command never ends. Well, if this is the case the file you generated is probably taking all of your free disk space. Yummie. Why? Probably because the output file already existed before and now it matches the wildcard condition of awk. So, make sure that the output file doesn’t exist when executing the command or, at least, doesn’t end in your shell expansion.

May 19, 2015 1 comment Read More
[HOW-TO] Homebrew: remove a formula and all its dependencies

[HOW-TO] Homebrew: remove a formula and all its dependencies

 
27 Kudos
Don't
move!

Since homebrew does not officially support an automated way to do that, I created a small zsh function to remove a formula and all its dependencies.

I use zsh on the Mac so the function works with it. Feel free to adjust it if you are using bash or something else.

brew-remove-with-deps() {
  formula="$*"

  if [ "x$formula" = "x" ]
  then
     echo "Invalid empty parameter"
  else
     echo "Removing" "$formula" "and all its deps.."
     brew rm $formula
     brew rm $(join <(brew leaves) <(brew deps "$formula"))
  fi
}

And, just for testing purposes:

[email protected] ~ % brew-remove-with-deps sloccount
Removing sloccount and all its deps..
Uninstalling /usr/local/Cellar/sloccount/2.26...
Uninstalling /usr/local/Cellar/md5sha1sum/0.9.5...

Et voilà.

References:
[1] – Stackoverflow – Uninstall / remove a Homebrew package including all its dependencies

April 16, 2013 0 comments Read More
[HOW-TO] Automatically download your favourite TV Series

[HOW-TO] Automatically download your favourite TV Series

 
29 Kudos
Don't
move!

This blogpost explains how to setup an environment to automatically download TV series when they come up in the net. Probably it’s not the most legal thing in the world but I’m a scientist, you know.

It’s not easy, not trivial, not error-free and it requires you to have some *nix/scripting skills (besides you having a server, or generally speaking, a computer that stays up 24/7).

October 15, 2012 1 comment Read More
R build system for Sublime Text

R build system for Sublime Text

 
34 Kudos
Don't
move!

Today I’ve been played with R.
Since I’m using Sublime Text as my first choice editor, I needed a build system script to quickly run the code.

It’s very trivial:


{
"cmd": ["R", "--no-save", "-f $file"]
}

February 28, 2012 2 comments Read More
Migrate from Typo blog to WordPress

Migrate from Typo blog to WordPress

 
7 Kudos
Don't
move!

Those of you that follow my blog have already noticed that in the past days a major change to the website has happened.

In fact, I moved from Typo to WordPress.
The most difficult part is not to find a theme, fix the CSS, play with the sidebar or anything like that; the mess comes when you have to migrate the data (posts, categories, tags, comments, …).
In order to do so I readapted a Ruby script to work with the latest version of the two blog engines (Typo 6.0.5 vs WordPress 3.1.2). You can find my version here.

To use it, just copy it inside your $TYPO_ROOT/script directory. Then simply issue

RAILS_ENV=production ruby script/wp_export.rb > out.wxr

and wait for the magic to happen.

Once the scripts ends, out.wxr file will contain all the content you had on your previous Typo installation.
In order to proceed, you have to install WordPress Importer, a plugin that allows you to import a WordPress blog from file. Install it, activate it, then import the file generated in the step above and you’re all set.

August 4, 2011 7 comments Read More