Thursday, August 13, 2009

Data scrubbing with sed

This is just s quickie, I recently had to do some clean up work on a database
that had irregular columns separators. There were single tabs, multiple tabs
mixed with single white spaces and multiple white space. Here's a quick one liner
in sed that will clean those up and leave you with just single white space.


#!/bin/bash
#first we strip off the tabs and replace with white spaces
sed -e 's/\t/ /g' -e 's/ */ /g' $1

No comments: