sed命令详解

1. sed命令简介

sed是非交互式的编辑器。它不会修改文件,除非使用shell重定向来保存结果。默认情况下,所有的输出行都被打印到屏幕上。
sed编辑器逐行处理文件(或输入),并将结果发送到屏幕。具体过程如下:首先sed把当前正在处理的行保存在一个临时缓存区中(也称为模式空间),然后处理临时缓冲区中的行,完成后把该行发送到屏幕上。sed每处理完一行就将其从临时缓冲区删除,然后将下一行读入,进行处理和显示。处理完输入文件的最后一行后,sed便结束运行。sed把每一行都存在临时缓冲区中,对这个副本进行编辑,所以不会修改原文件。

2. sed命令格式

sed命令的一般格式为:sed [选项] “脚本” 文件

(下文中“命令”都是指sed的命令部分。)

选项:

sed的选项形如-n, -e, -f等,都有个’-‘。

-n    忽略默认输出(默认输出就是整个文件的内容)。

-e    执行多个编辑任务

-f     执行文件中的命令集合

脚本:

sed的脚本有p, d, i\等,文中会重点介绍。

如果脚本部分为空,则可以用来打印一个文件的内容:

[root@ubuntu]sed_test:$ sed "" sed_test.txt
1 It's been a long day without you my friend
2 And I'll tell you all about it when I seeyou again
3 We've come a long way from where we began
4 Oh I'll tell you all about it when I seeyou again
5 When I see you again

sed支持正则表达式:

元字符

 功能

 ^

 行首定位符

 $

 行尾定位符

 .

 匹配除换行符以外的单个字符

 *

 匹配零个或多个前导字符

\?

匹配零个或1个前导字符

\+

匹配1个或多个前导字符

\|

或操作

\s

匹配单个空白字符(如\t)

\S

匹配单个非空白字符

\w

匹配单个单词

\W

匹配单个非单词

 []

 匹配指定字符组内的任一字符

 [^]

 匹配不在指定字符组内的任一字符

[-]

匹配字符集的字符范围

 (..)

 保存已匹配的字符(括号前面要加反斜杠\,csdn没显示出来)

 &

 保存查找串以便在替换串中引用

 \<

 词首定位符

 \>

 词尾定位符

 x\{m\}

 连续m个x

 x\{m,\}

 至少连续m个x

 x\{m,n\}

 至少连续m个,但不超过n个x

(str)

正则表达式中对字符串的引用(括号前面要加反斜杠\,csdn没显示出来)

中括号表达式

如[:alnum:],[:alpha:],[:digit:],[:lower:],

[:upper:],[:punct:],[:space:]

3. sed选项和命令介绍

选项和(脚本中的)命令我会穿插在一起介绍,大家注意区分带“-”的是选项,否则是命令。

p     打印行

例:上面的sed_test.txt文件为例。

打印第3行的内容:

[root@ubuntu]sed_test:$sed -n "3p" sed_test.txt
3 We've come a long way from where we began

把3和p分开也可以,并且更能分辨出p是个命令。

注意加-n选项是为了不显示默认输出,如果这里不加-n,效果是这样的:

[root@ubuntu]sed_test:$sed "3p" sed_test.txt
1 It's been a long day without you my friend
2 And I'll tell you all about it when I see you again
3 We've come a longway from where we began
3 We've come a longway from where we began
4 Oh I'll tell you all about it when I see you again
5 When I see you again

即既打印出所有内容,又打印出了红色部分的内容。由红色部分的位置可知,sed是进行行操作的。

如下,addr1,addr2addr1,+Naddr1~stepaddr1,~N可以表示寻址范围。

打印1-3行的内容(注意行数是从1开始的):

[root@ubuntu]sed_test:$sed -n "1,3p" sed_test.txt
1 It's been a long day without you my friend
2 And I'll tell you all about it when I see you again
3 We've come a long way from where we began

打印第三行及后续2行:

[root@ubuntu]sed_test:$sed -n "3,+2p" sed_test.txt
3 We've come a long way from where we began
4 Oh I'll tell you all about it when I see you again
5 When I see you again

打印第2行以及4-10行:

[root@ubuntu]sed_test:$sed -n "2p;4,10p" sed_test.txt
2 And I'll tell you all about it when I see you again
4 Oh I'll tell you all about it when I see you again
5 When I see you again

“first~step”形式表示从first行作为0计数起点,打印step倍数行的内容。

打印第3行开始2的倍数行,即3,5,7 …行的内容:

[root@ubuntu]sed_test:$sed -n "3~2p" sed_test.txt

那么打印所有奇数行:

[root@ubuntu]sed_test:$sed -n "1~2p" sed_test.txt

“addr1,~N”形式表示从addr1行开始,到找到N的倍数行结束。

从第13行开始,到5的倍数行结束,这里即第15行结束:

[root@ubuntu]sed_test:$sed -n "13,~5p" sed_test.txt

/regexp/、\{5749fe182deba6f703e69800a8cc3afb9894ad400f350437bd2be724fa41f418}regexp{5749fe182deba6f703e69800a8cc3afb9894ad400f350437bd2be724fa41f418} 匹配行

用来寻找匹配正则表达式regexp的行。

例如,打印包含“again”的行:

[root@ubuntu]sed_test:$ sed -n "/again/p" see_you_again.lrc
And I'll tell you all about it when I see you again
Oh I'll tell you all about it when I see you again
When I see you again

如果需要匹配斜杠“/”,则需要用反斜杠转义“\/”。

{5749fe182deba6f703e69800a8cc3afb9894ad400f350437bd2be724fa41f418}regexp{5749fe182deba6f703e69800a8cc3afb9894ad400f350437bd2be724fa41f418}的形式仍然是匹配行,其中{5749fe182deba6f703e69800a8cc3afb9894ad400f350437bd2be724fa41f418}可以替换为任意字符。这种形式可以用在regexp里包含斜杠“/”的情况,这样就不需要用反斜杠转义。

/regexp/I、\{5749fe182deba6f703e69800a8cc3afb9894ad400f350437bd2be724fa41f418}regexp{5749fe182deba6f703e69800a8cc3afb9894ad400f350437bd2be724fa41f418}I  匹配行(忽略大小写)

后面添加大写的I,表示匹配时忽略大小写。

d     删除行

删除2-4行的内容:

[root@ubuntu]sed_test:$sed "2,4d" sed_test.txt
1 It's been a long day without you my friend
5 When I see you again

注意-n的作用就体现出来了,这里不能加-n。注意,这并没有修改文件本身的内容。

 

移除文件中的空行:

[root@ubuntu]sed_test:$  sed "/^$/d" sed_test.txt

删除包含“We”和“When”的行以及他们之间的行:

[root@ubuntu]sed_test:$sed "/We/,/When/d" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again

正则表达式

举例:see_you_again.lrc的内容如下:

[root@ubuntu]sed_test:$sed '' see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again
We'vecome a long way from where we began
Oh I'll tell you all about it when I see you again
When I see you again

打印包含“again”的行:

[root@ubuntu]sed_test:$sed -n "/again/p" see_you_again.lrc
And I'll tell you all about it when I see you again
Oh I'll tell you all about it when I see you again
When I see you again

打印不以“We”开头的行:

[root@ubuntu]sed_test:$sed -n "/^We/! p" see_you_again.lrc
It'sbeen a long day without you my friend
And I'll tell you all about it when I see you again
Oh I'll tell you all about it when I see you again
When I see you again

在kallsyms文件中查找包含“[gmac]”的行,并在这些行中找到不以83aa开头的行:

 

打印以“again”结尾的行:

[root@ubuntu]sed_test:$sed -n "/again$/p" see_you_again.lrc
And I'll tell you all about it when I see you again
Oh I'll tell you all about it when I see you again
When I see you again

打印包含“I” “you”,中间为任意多个字符的行:

[root@ubuntu]sed_test:$sed -n "/I.*you/p" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again
Oh I'll tell you all about it when I see you again
When I see you again

打印包含“I”,但后面不跟字母的行:

[root@ubuntu]sed_test:$sed -n "/I[^a-zA-Z]/p" see_you_again.lrc
And I'll tell you all about it when I see you again
Oh I'll tell you all about it when I see you again
When I see you again

打印包含2个连续的“e”字母的行:

[root@ubuntu]sed_test:$sed -n "/e\{2\}/p" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again
Oh I'll tell you all about it when I see you again
When I see you again

            注意,上面都是针对单个字符的匹配,如果要匹配一个字符串,要加小括号:

打印包含1到2个连续“bang”的行:

另外需要注意的是,中括号[]里都是匹配单个字符的,无法匹配一个字符串。

 

打印包含以“wh”开头的单词的行:

[root@ubuntu]sed_test:$sed -n "/\

s     用一个字符串替换另一个字符串

g     在行内进行全局替换

i      替换时不区分大小写

(…)      对字符串分组

将包含“you”的行中的“you”替换为“her”:

[root@ubuntu]sed_test:$sed -n "s/you/her/gp" see_you_again.lrc
It's been a long day without her my friend
And I'll tell her all about it when I see her again
Oh I'll tell her all about it when I see her again
When I see her again

将包含“you”的行中的“you”替换为“her”,只替换每行的第一处:

[root@ubuntu]sed_test:$sed -n "s/you/her/p" see_you_again.lrc
It's been a long day without her my friend
And I'll tell her all about it when I see you again
Oh I'll tell her all about it when I see you again
When I see her again

将包含“long”的行中的“long”替换为“tough”,不区分大小写:

[root@ubuntu]sed_test:$sed -n "s/LOng/tough/pi" see_you_again.lrc
It's been a tough day without you my friend
We've come a tough way from where we began

可以用(…)对字符串进行分组,并且在后面使用1,2,3…依次代替匹配列表,例如,将“long”和“way”中间加一个“-”:

 

将4-5行的“you”替换为“her”:

[root@ubuntu]sed_test:$sed -n "4,5s/you/her/gp" see_you_again.lrc
Oh I'll tell her all about it when I see her again
When I see her again

将包含“long”的行中的空格替换为逗号:

[root@ubuntu]sed_test:$sed -n "/long/s/ /,/gp" see_you_again.lrc
It's,been,a,long,day,without,you,my,friend
We've,come,a,long,way,from,where,we,began

匹配包含“friend”以及“Oh”的行,并同时打印出他们之间的所有行(应用所有匹配行):

[root@ubuntu]sed_test:$sed -n "/friend/,/Oh/ p" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again
We've come a long way from where we began
Oh I'll tell you all about it when I see you again

如果逗号前面的模式没有匹配行,则从第1行开始打印;如果逗号后面的模式没有匹配行,则打印到最后一行。

打印包含“friend”的行,并同时打印所有匹配行的后续2行:

[root@ubuntu]sed_test:$sed -n "/friend/,+2 p" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again
We've come a long way from where we began

可见“,”号和“+”号和前面讲p命令时有着相同的意义。

分隔符默认为正斜杠,但可以改变。无论什么字符(换行符除外),只要紧跟s命令,就成了新的串分隔符。如下面的字符z:

[root@ubuntu]sed_test:$sed -n "4,5szyouzherzgp" see_you_again.lrc
Oh I'll tell her all about it when I see her again
When I see her again

将Makefile文件中前3行注释掉:

[root@ubuntu]sed_test:$  sed '1,3s/^/#/' Makefile

将空格改为“,0x”,然后将每行最后两个字符删掉,然后将每行开头加上“0x”:

sed -n "s/ /,0x/g;s/..$//g;s/^/0x/p" spcontent.txt

将每行的第三个空格改为逗号:

sed -n "s/ /,/3p" spcontent.txt

将每行的第三个和第四个空格改为逗号(注意第一次改完之后第四个空格对于第二次来讲是第三个空格):

sed -n "s/ /,/3;s/ /,/3p" spcontent.txt

将每行的第三个以及之后所有的空格改为逗号:

sed -n "s/ /,/3gp" spcontent.txt

另外,我们知道vim中进行字符串替换的命令也是s和g,例如,将文件中的nihao改为hello,命令为“:s/nihao/hello/g”,而“:m,ns/nihao/hello/g”为将m-n行的nihao改为hello。和sed中的替换命令类似。

-e    用于执行多个编辑任务

将“you”变成“her”,并且将“again”删掉:

[root@ubuntu]sed_test:$sed -n -e "s/you/her/g" -e "s/again//gp" see_you_again.lrc
And I'll tell her all about it when I see her
Oh I'll tell her all about it when I see her
When I see her

使用分号可以分隔两个命令,这样,两个命令可以写在一行,所以上面还可以这样写:

[root@ubuntu]sed_test:$sed -n "s/you/her/g;s/again//gp" see_you_again.lrc
And I'll tell her all about it when I see her
Oh I'll tell her all about it when I see her
When I see her

 

r,  R  读命令

在see_you_again.lrc中查找包含“long”的行,并将文件newone.txt的内容追加到该行后面:

[root@ubuntu]sed_test:$cat newone.txt
Damn,who knew all the planes we flew
[root@ubuntu]sed_test:$sed "/long/r newone.txt" see_you_again.lrc
It's been a long day without you my friend
Damn, who knew all the planes we flew
And I'll tell you all about it when I see you again
We've come a long way from where we began
Damn, who knew all the planes we flew
Oh I'll tell you all about it when I see you again
When I see you again

如果使用R命令,只在第一个匹配行后面追加newone.txt的内容。

w,  W  写命令

在see_you_again.lrc中查找包含“long”的行,并将该行内容写到newone.txt文件中:

[root@ubuntu]sed_test:$sed -n "/long/w newone.txt" see_you_again.lrc

可以看到newone.txt中的内容被清空并且写入了新的内容。

将see_you_again.lrc中的奇数行写到newone.txt文件中:

[root@ubuntu]sed_test:$  sed -n "1~2w newone.txt" see_you_again.lrc

大写W命令则只将模式空间中第一行的内容写到newone.txt文件中。

a \   追加新行

在包含“see you again”的行后面添加一行,内容为“your pretty face”:

[root@ubuntu]sed_test:$sed "/see you again/a \your pretty face" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again
your pretty face
We've come a long way from where we began
Oh I'll tell you all about it when I see you again
your pretty face
When I see you again
your pretty face

该命令不加反斜杠也可以,即aa\a \都可以。

在第2行后面追加your pretty face:

[root@ubuntu]sed_test:$sed "2a \your pretty face" see_you_again.lrc
It's been a long daywithout you my friend
And I'll tell you all about it when I see you again
your pretty face
We've come a long way from where we began
Oh I'll tell you all about it when I see you again
When I see you again

将文件末尾插入your pretty face:

[root@ubuntu]sed_test:$sed "$ a \your pretty face" see_you_again.lrc
It'sbeen a long day without you my friend
And I'll tell you all about it when I see you again
We've come a long way from where we began
Oh I'll tell you all about it when I see you again
When I see you again
your pretty face

 

i \    插入新行

和a\命令的区别是,a\是在行的后面追加新行,而i\是在行的前面插入新行。

在文件开头插入your pretty face:

[root@ubuntu]sed_test:$sed "1i \your pretty face" see_you_again.lrc
your pretty face
It's been a long day without you my friend
And I'll tell you all about it when I see you again
We've come a long way from where we began
Oh I'll tell you all about it when I see you again
When I see you again

 

c \   替换行

将包含“see you again”的行内容替换为“your pretty face”:

[root@ubuntu]sed_test:$sed "/see you again/c\your pretty face" see_you_again.lrc
It's been a long day without you my friend
your pretty face
We've come a long way from where we began
your pretty face
your pretty face

 

y     字符替换

将文件中所有的y换成Y,o换成O,u换成U,空格换成逗号:

[root@ubuntu]sed_test:$sed "y/you /YOU,/" see_you_again.lrc
It's,been,a,lOng,daY,withOUt,YOU,mY,friend
And,I'll,tell,YOU,all,abOUt,it,when,I,see,YOU,again
We've,cOme,a,lOng,waY,frOm,where,we,began
Oh,I'll,tell,YOU,all,abOUt,it,when,I,see,YOU,again
When,I,see,YOU,again

 

正则表达式元字符对y命令不起作用。由于是一一对应的替换,所以y命令中替换和被替换的字母串中字母数目必须相同。

和s命令一样,斜杠可以换成其他字符。

 

将第8行中的“/”换成“\”:

[root@zhenfg sed_test]# sed "8y#/#\\\#" sed_test.txt

反斜杠是特殊字符,所以要处理一下子。又如,双引号也是特殊字符,所以要换转双引号也要加反斜杠转义:

将第8行的双引号改为单引号:

[root@zhenfg sed_test]# sed "8y#\"#'#" sed_test.txt

 

-f 执行文件中的命令集合

sed还支持以下格式,可以从scriptfile里读取命令部分:

sed [-n] -fscriptfile files

例如:

[root@ubuntu]sed_test:$cat commands.txt
s/you/her/
s/ /,/g
[root@ubuntu]sed_test:$  sed -f commands.txt see_you_again.lrc
It's,been,a,long,day,without,her,my,friend
And,I'll,tell,her,all,about,it,when,I,see,you,again
We've,come,a,long,way,from,where,we,began
Oh,I'll,tell,her,all,about,it,when,I,see,you,again
When,I,see,her,again

我们看到,可以把多条命令写到文件里一起执行,相当于-e选项。

l 显示隐藏字符

举例略。

q [exit-code]、Q [exit-code] 停止执行流

打印3行就停止:

[root@ubuntu]sed_test:$sed "3q" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again
We've come a long way from where we began

匹配到“again”就停止:

[root@ubuntu]sed_test:$  sed "/again/q" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again

q后面可以接数字,可作为sed命令的返回值:

[root@ubuntu]sed_test:$ sed "/again/q 14" see_you_again.lrc
It's been a long day without you my friend
And I'll tell you all about it when I see you again
[root@ubuntu]sed_test:$ echo $?

大写Q的效果如下,即匹配的那一行不会打印出来:

[root@ubuntu]sed_test:$ sed "/again/Q" see_you_again.lrc
It's been a long day without you my friend

 

e 执行外部命令

在第1行前面添加date命令的结果:

[root@ubuntu]sed_test:$  sed "1 e date" see_you_again.lrc
2017年 01月 01日 星期日 02:36:26 CST
It's been a long day without you my friend
And I'll tell you all about it when I see you again
We've come a long way from where we began
Oh I'll tell you all about it when I see you again
When I see you again

也可以直接执行文件中的命令:

[root@ubuntu]sed_test:$  sed 'e' shell.sh

 

n 单行next,移动到文本的下一行

找到包含“friend”的行,然后删除它的下一行:

[root@ubuntu]sed_test:$  sed "/friend/{n;d}" see_you_again.lrc
It's been a long day without you my friend
We've come a long way from where we began
Oh I'll tell you all about it when I see you again
When I see you again

 

= 输出行号

例如,显示文件的行号信息:

[root@ubuntu]sed_test:$  sed "=" see_you_again.lrc
1
It's been a long day without you my friend
2
And I'll tell you all about it when I see you again
3
We've come a long way from where we began
4
Oh I'll tell you all about it when I see you again
5
When I see you again

输出包含“We”的行的行号:

[root@ubuntu]sed_test:$  sed -n "/We/=" see_you_again.lrc
3

输出文件的行数:

[root@ubuntu]sed_test:$  sed -n "$=" see_you_again.lrc
5

 

& 用于存储模式匹配的内容

&命令通常与s命令配合使用,例如,在每一行的末尾追加字符串“ end”:

[root@ubuntu]sed_test:$  sed "s/$/& end/g" see_you_again.lrc
It's been a long day without you my friend end
And I'll tell you all about it when I see you again end
We've come a long way from where we began end
Oh I'll tell you all about it when I see you again end
When I see you again end

这里&就指代前面的模式匹配到的内容。

-i[suffix]  修改文件

使用-i选项可以修改原文件,例如,删除script.sh中以#开头的行:

[root@ubuntu]sed_test:$ sed -i "/^#/d" script.sh

在-i后面加一个后缀,就会产生以“原文件名+suffix”命名的备份文件。

sed -i.bak "/^#/d" script.sh

 

#  注释

可以在sed的脚本部分中添加注释,注释的内容以一个新行结束,或者在一个-e片段的末尾结束。