Noulakaz

The blog of Avinash, Christina, Anya and Kyan Meetoo.

LUGM

Splitting a large text file on every blank line

By Leave a Comment

I had a file of about a thousand lines containing the full description of multiple products (audio plugins from Plugin Alliance in my case). The file starts like this:

249
19.99
Mixland
Vac Attack★★★★★
A warm harmonically-rich optical limiter for compression with tube saturation that’s great on vocals, drums, basses, and your stereo bus.
Hardware EmulationsHidden GemsPA MembersSale

89.99
29.99
Woodlands Studio
VOXILLION★★★★★
A stunning and sophisticated, and complete vocal chain in one streamlined workflow. Featuring a high-end blend of a tube-driven preamp, two types of compressors, Nasal Dynamic EQs, Harmonics and more
Hidden GemsLimited Time OnlyPA_EXTSale

279
39.99
ADPTR AUDIO
Metric AB★★★★★
The mastering engineer’s best friend: Compare your mix to your favorite reference tracks. See,hear and learn even more with new expanded features.
Hidden GemsMasteringSale

and ends like this:

FreeDownload
PA FREE
bx_shredspread★★★★★
Intelligent M/S width for doubled riff guitars. Auto-avoid common phase problems, and sound extra-wide and tight!
FREEGuitar & BassHidden GemsM/S InsideMade by BX

FreeDownload
PA FREE
bx_tuner★★★★☆
Accurate tuning for guitar & bass with useful features like volume dimming. Tune up right before you hit the record button.
FREEGuitar & BassMade by BXSale

FreeDownload
PA FREE
bx_yellowdrive★★★★★
Warm to crunch to shred with this classic “Yellow” pedal in plugin form.
Creative FXFREEGuitar & BassHardware EmulationsMade by BXSale

What I wanted to do was to split the long file of a thousand lines into smaller files, each containing the lines corresponding to one plugin only.

Fortunately, the plugins were separated by an empty line. After some searching, I stumbled upon a Stackoverflow post of someone having the same issue and I settled on this solution enhanced by this comment:

gawk -v RS= '{ print > ("plugin-" NR ".txt")}' plugins.txt

As explained in the solution, “setting RS to null tells (g)awk to use one or more blank lines as the record separator. Then one can simply use NR to set the name of the file corresponding to each new record”. As explained in the comment, this will fail with the basic version of awk as it cannot handle too many open files at the same time (only 252 files were created with awk). Switching to gawk works great and now I have 277 files which I can further process.

Getting unknown terminal type warnings when using SSH?

By Leave a Comment

I regularly SSH to my computers and servers and, on some of them, I was getting the following warning message:

'ansi+tabs': unknown terminal type.

This was just a warning and everything worked perfectly. But, still, having a warning message was not very, ahem, professional…

I have just found the cure: you need to have additional terminal definitions installed on the remote computer. On Red Hat / CentOS / Fedora, do:

dnf install ncurses-term

and on Debian, CentOS and derivatives:

apt-get install ncurses-term

and everything will be fine.

Convert multiple lines into one tab-separated line using paste

By Leave a Comment

I generally have a lot of text files with such content:

1
Play track
Turn Around
Unlove this track
Panic Attack
Avinash Meetoo
110 scrobbles
2
Play track
Playing the Angel
Unlove this track
Precious
Depeche Mode
101
3
Play track
The Singles 86>98
Unlove this track
Personal Jesus
Depeche Mode
98

These are the three tracks I have listened to most since I started using Last.fm way back in 2005.

What I want to achieve is to easily convert these 21 lines (3 x 7) into 3 tab-separated lines which can then be easily imported into, say, Libreoffice Calc:

1<TAB>Play track<TAB>Turn Around<TAB>Unlove this track<TAB>Panic Attack<TAB>Avinash Meetoo<TAB>110 scrobbles
2<TAB>Play track<TAB>Playing the Angel<TAB>Unlove this track<TAB>Precious<TAB>Depeche Mode<TAB>101
3<TAB>Play track<TAB>The Singles 86>98<TAB>Unlove this track<TAB>Personal Jesus<TAB>Depeche Mode<TAB>98

For many years, each time I had to do that, I wrote a small Awk script. But, today, thanks to RudiC on the Unix Stack Exchange, I have the perfect recipe using paste:

paste -s -d "\t\t\t\t\t\t\n" file-containing-all-the-lines.txt

Naturally, you can adjust the number of “\t” if you have fewer or more fields.

Enjoy :-)