Sed notes
author: Paul Kim
categories: sed, bash
tags: sed, bash
Sed notes
Sed is a stream editor used to perform basic text transformations on an input stream.
Basic Sed commands
Supposed you have a file example.txt:
AAAAAAAAAA
BBBBBBBBBB
CCCCCCCCCC
DDDDDDDDDD
EEEEEEEEEE
HHHHHHHHHH
IIIIIIIIIIWe can remove lines with Sed
# remove 2nd line from example.txt
sed 2d example.txt
AAAAAAAAAA
CCCCCCCCCC
DDDDDDDDDD
EEEEEEEEEE
HHHHHHHHHH
IIIIIIIIII
# remove lines 2-4 from example.txt
sed 2,4d example.txt
AAAAAAAAAA
EEEEEEEEEE
HHHHHHHHHH
IIIIIIIIII
# remove empty lines
sed '/^$/d' example.txt
AAAAAAAAAA
BBBBBBBBBB
CCCCCCCCCC
DDDDDDDDDD
EEEEEEEEEE
HHHHHHHHHH
IIIIIIIIII
# remove the first line
sed 1d example.txt
BBBBBBBBBB
CCCCCCCCCC
DDDDDDDDDD
EEEEEEEEEE
HHHHHHHHHH
IIIIIIIIII
# remove the last line
sed '$d' example.txt
AAAAAAAAAA
BBBBBBBBBB
CCCCCCCCCC
DDDDDDDDDD
EEEEEEEEEE
HHHHHHHHHH
# remove all lines which contain 'C' character
sed '/C/'d example.txt
AAAAAAAAAA
BBBBBBBBBB
DDDDDDDDDD
EEEEEEEEEE
HHHHHHHHHH
IIIIIIIIII
# remove all lines which contain 'C' character and edit file in place
sed -i '/C/'d example.txt
# remove all lines which contain 'C' character and redirect to new file
sed '/C/'d example.txt > example2.txtAdvanced Sed commands
Supposed you have a file example.tsv:
1193782372 Lips Like Sugar (12" Mix) Echo & the Bunnymen 80's/12": The Extended Collection a76d9b04-51d9-4672-801f-356ab36dbae7 ccd4879c-5e88-4385-b131-bf65296bf245 1abb270a-e791-407f-a989-ff3ad6f8401cWe want to escape any double quotes in each field and wrap each field with double quotes like the following:
"1193782372" "Lips Like Sugar (12"" Mix)" "Echo & the Bunnymen" "80's/12"": The Extended Collection" "a76d9b04-51d9-4672-801f-356ab36dbae7" "ccd4879c-5e88-4385-b131-bf65296bf245" "1abb270a-e791-407f-a989-ff3ad6f8401c"We do this with Sed
# escape any double quotes in each tab delimited field
# wrap each tab delimited field with double quotes
sed $'s/"/""/g;s/[^\t]*/"&"/g' example.tsv
# note: \t does not work on mac so we prepend $