Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to remove unwanted characters in a file (using a shell script)

Tags:

linux

shell

sed

awk

I have a file which looks like this (file.txt)

AYOnanl3knsgv2StRr44  CRITICAL","component  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr45  CRITICAL","component  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr46  CRITICAL","component  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr47  CRITICAL","component  MMP-FileService  [email protected]     CODE_SMELL
AYOnanmeknsgv2StRr48  MAJOR",               MMP-FileService  [email protected]  CODE_SMELL
AYOnanm-knsgv2StRr4-  BLOCKER",             MMP-FileService  [email protected]      CODE_SMELL
AYOnanm6knsgv2StRr49  MAJOR",               MMP-FileService  [email protected]      CODE_SMELL
AYOnannKknsgv2StRr4_  BLOCKER",             MMP-FileService  [email protected]   BUG
AYODwmuBknsgv2StRqkr  MINOR",               MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqkt  MINOR",               MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqks  MINOR",               MMP-component    [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqku  MINOR",               MMP-component    [email protected]   CODE_SMELL
AYODsI7Fknsgv2StRqac  MAJOR",               MMP-component    [email protected]   CODE_SMELL
AYODsI7Nknsgv2StRqad  MAJOR",               MMP-component    [email protected]   CODE_SMELL
AYODsI-Qknsgv2StRqai  MAJOR",               MMP-component    [email protected]   CODE_SMELL

I have to remove unwanted characters in 2nd column ","component and ",

then expected output

AYOnanl3knsgv2StRr44  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr45  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr46  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr47  CRITICAL  MMP-FileService  [email protected]    CODE_SMELL
AYOnanmeknsgv2StRr48  MAJOR     MMP-FileService  [email protected]  CODE_SMELL
AYOnanm-knsgv2StRr4-  BLOCKER   MMP-FileService  [email protected]    CODE_SMELL
AYOnanm6knsgv2StRr49  MAJOR     MMP-FileService  [email protected]    CODE_SMELL
AYOnannKknsgv2StRr4_  BLOCKER   MMP-FileService  [email protected]    BUG
AYODwmuBknsgv2StRqkr  MINOR     MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqkt  MINOR     MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqks  MINOR     MMP-component  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqku  MINOR     MMP-component  [email protected]   CODE_SMELL
AYODsI7Fknsgv2StRqac  MAJOR     MMP-component  [email protected]   CODE_SMELL
AYODsI7Nknsgv2StRqad  MAJOR     MMP-component  [email protected]   CODE_SMELL
AYODsI-Qknsgv2StRqai  MAJOR     MMP-component  [email protected]   CODE_SMELL

This is what I tried

cat file.txt | tr -d '",' | sed 's/component//'

then output I got

YOnanl3knsgv2StRr44  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr45  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr46  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr47  CRITICAL  MMP-FileService  [email protected]     CODE_SMELL
AYOnanmeknsgv2StRr48  MAJOR               MMP-FileService  [email protected]  CODE_SMELL
AYOnanm-knsgv2StRr4-  BLOCKER             MMP-FileService  [email protected]      CODE_SMELL
AYOnanm6knsgv2StRr49  MAJOR               MMP-FileService  [email protected]      CODE_SMELL
AYOnannKknsgv2StRr4_  BLOCKER             MMP-FileService  [email protected]   BUG
AYODwmuBknsgv2StRqkr  MINOR               MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqkt  MINOR               MMP-FileService  [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqks  MINOR               MMP-    [email protected]   CODE_SMELL
AYODwmuBknsgv2StRqku  MINOR               MMP-    [email protected]   CODE_SMELL
AYODsI7Fknsgv2StRqac  MAJOR               MMP-    [email protected]   CODE_SMELL
AYODsI7Nknsgv2StRqad  MAJOR               MMP-    [email protected]   CODE_SMELL
AYODsI-Qknsgv2StRqai  MAJOR               MMP-    [email protected]   CODE_SMELL

my executed shell command is applying to the other columns as well (in this case it has applied to 3rd column too) that is the problem I am having. Is there any way to apply command only for 2nd column?

Can someone help me to figure out this? Thanks in advance!

Note: I am not allowed to use jq or other scripting languages as JavaScript, Python etc.

like image 492
Zero5 Avatar asked Nov 19 '25 15:11

Zero5


1 Answers

It can be done in a single sub:

awk '{sub(/"[^[:blank:]]*$/, "", $2)} 1' file | column -t

AYOnanl3knsgv2StRr44  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr45  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr46  CRITICAL  MMP-FileService  [email protected]  CODE_SMELL
AYOnanl3knsgv2StRr47  CRITICAL  MMP-FileService  [email protected]    CODE_SMELL
AYOnanmeknsgv2StRr48  MAJOR     MMP-FileService  [email protected]   CODE_SMELL
AYOnanm-knsgv2StRr4-  BLOCKER   MMP-FileService  [email protected]    CODE_SMELL
AYOnanm6knsgv2StRr49  MAJOR     MMP-FileService  [email protected]    CODE_SMELL
AYOnannKknsgv2StRr4_  BLOCKER   MMP-FileService  [email protected]    BUG
AYODwmuBknsgv2StRqkr  MINOR     MMP-FileService  [email protected]    CODE_SMELL
AYODwmuBknsgv2StRqkt  MINOR     MMP-FileService  [email protected]    CODE_SMELL
AYODwmuBknsgv2StRqks  MINOR     MMP-component    [email protected]    CODE_SMELL
AYODwmuBknsgv2StRqku  MINOR     MMP-component    [email protected]    CODE_SMELL
AYODsI7Fknsgv2StRqac  MAJOR     MMP-component    [email protected]    CODE_SMELL
AYODsI7Nknsgv2StRqad  MAJOR     MMP-component    [email protected]    CODE_SMELL
AYODsI-Qknsgv2StRqai  MAJOR     MMP-component    [email protected]    CODE_SMELL

Here:

  • "[^[:blank:]]*$: Matches text starting with " in input ($2) and we replace it with an empty string.
  • column -t is used for tabular output only that you can remove if you don't want.
like image 164
anubhava Avatar answered Nov 21 '25 08:11

anubhava



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!