Trying my wings with Awk

Last week I described my goal of changing my hledger account names to lowercase (except for proper nouns).

For example, instead of

Expenses:Groceries

I wished to have

expenses:groceries

In short, the process I described was to

• use the hledger “accounts” command to generate a list of accounts
• use Excel to build a list of sed instructions
• create a file with the above sed instructions
• process the above file with sed

One of my reasons for using Excel was that for my purposes I didn’t see an easy way for sed to do the lowercasing. Excel, on the other hand, has a function to make text lowercase. (Note, though, that some versions of sed reportedly have simple ways to do the lowercasing. Alas, mine did not.) However, in my research, I noticed that awk had a function to do the job, and it looked relatively painless to use. I had almost zero experience using awk, but awk has always sounded like something fun. Time to try awk!

It’s worth stopping and asking: Isn’t this a blog about hledger? How come the discussion of sed and now awk? Here is why: hledger is a plaintext accounting tool, meaning that the data files are text files. Both sed and awk are tools that manipulate text. So, sed and awk have the potential to make me a more proficient hledger user.

As a side note, if you are new to sed or awk, you might find the O’Reilly book sed & awk, 2nd Edition to be very helpful. At least I did.

Turning back to awk, here is the command I found that worked for my needs. Note that it uses an input file, accounts.txt, which is a list of account names, currently that have uppercase letters that need to be changed to lowercase.

awk '{ print "s/"$0"/"tolower($0)"/g" }' accounts.txt > lower.sed

The above generates a list of sed commands, saved to a file named lower.sed. For example, one of the sed commands in this file would be:

s/Groceries/groceries/g

I hand edited the lower.sed file to correct instances where I have a proper name that I want to remain with a capital letter. There aren’t many, so it’s just a couple of minutes of work. Then, similar to last week’s blog, with one command I tell sed to process the list of commands in the lower.sed file, running them on my hledger data file.

While I am very new to awk, it seems that awk must have a way to do the entire job without needing sed. That is, why use both awk and sed if awk alone will do the trick? Something new for me to explore! I will report back next week what I have found.