Category: shell

Shell command to remove `(1)` from filename

Posted on by Bian Xi Leave a Comment on Shell command to remove `(1)` from filename

Shell command to remove (1) from filename

To compare massive number of files with (1) in file name, with the original files without (1), such as ABCD(1).txt and ABCD.txt, following commands can be used. Beware, they are not steps but commands.

Use bash substring

  • Find out all *(1)* files and check whether have original file in same folder.
find . -name "*\(1\)*" | while read line
do
    if test -e "${line/(1)/}"; then
        echo "$line"
    fi
done

Then can clean up them one by one.

  • Move them to another directory

  • Rename them to be the same as original file in same folder

find . -name "*\(1\)*" | while read line
do
    if test ! -e "${line/(1)/}"; then
        mv "$line" "${line/(1)/}"
    fi
done
  • Compare them with original files in same folder

Note: This method only work with the original filename has no (1) string.

Use sed

Following sample script can be used for same task.

#!/bin/bash

find . -name "*" -type f | while read line
do
        dname="`dirname -- \"$line\"`"
        bname="`basename -- \"$line\"`"
        # pattern='s/\(([0-9])\)\./\1/'         # remove "." if match "(1).", \1 == ([0-9])
        # pattern='s/(\([0-9]\))\./\1/'         # remove "(", ")" and "." if match "(1).", \1 == [0-9]
        # pattern='s/([0-9]).//'                # remove "(1)"+any_char
        # pattern='s/[0-9]\.//'                 # remove "(1)."
        # pattern='s/([0-9])\././'              # remove "(1)"
        pattern='s/\s*([0-9])\././'           # remove any_space+"(1)"
        # pattern='s/\s*\././'                  # remove any_space before "."
        # pattern='s/^\./11./'                  # add "11" in front if start with "."
        # pattern='s/^01\./10./'                        # replace starting "01." to "10."
        # pattern='s/^0\([2-9]\)\./1\1./'               # replace starting "01." to "10."
        nname="`echo \"$bname\" | sed -e "$pattern"`"
        # echo "$bname"; echo "$nname"

        if [ "$nname" != "$bname" -a ! -e "$nname" ] ; then
                pushd "$dname"
                echo "$bname"; echo "$nname"
                mv "$bname" "$nname"
                popd
        fi
done

Use vim

  • Use following command to get the list of file name
find . -name "*(1).*" -exec echo mv ~{}~ ~{}~ \; > list
  • Use vim to edit the file
vi list
  • Use lookahead to replace the last (1)
%s/.*\zs(1)//
  • Replace ~ to ", then save it
%s/\~/"/g
  • Run the script
sh list

References

How to change last occurrence of the string in the line?
Regex lookahead and lookbehind

Only pipe STDERR output in `bash` with timestamp

Posted on by Bian Xi Leave a Comment on Only pipe STDERR output in `bash` with timestamp

Only pipe STDERR output in bash with timestamp

bash

In order to discard standard output and only log the standard error, following command can be used. The second part of command is to prefix the current timestamp in the output

sh monitor 2>&1>/dev/null | ts '[%Y-%m-%d %H:%M:%S]'

dash

If need to run in dash, such as running in crontab, above syntax is wrong, use following command instead

sh monitor 3>&1 1>/dev/null 2>&3 3>&- | ts '[%Y-%m-%d %H:%M:%S]'

Note: This command can be run in bash too

References

Prepending a timestamp to each line of output from a command
Pipe only STDERR through a filter

Apply filter to STDERR in Linux

Posted on by Bian Xi Leave a Comment on Apply filter to STDERR in Linux

Apply filter to STDERR in Linux

STDOUT ────────────────┐
                       ├─────> terminal/file/whatever
STDERR ── [ filter ] ──┘

Method

If ./a.out outputs as below

In STDERR:

stderr output

In STDOUT:

more regular

Then the following command will output as below.

# ./a.out 3>&1 1>&2 2>&3 3>&- | sed 's/e/E/g'
more regular
stdErr output

Explanation

    First save stdout as &3 (&1 is duped into 3)
    Next send stdout to stderr (&2 is duped into 1)
    Send stderr to &3 (stdout) (&3 is duped into 2)
    close &3 (&- is duped into 3)

References

Pipe only STDERR through a filter

The `sed` command uncommon behaviors

Posted on by Bian Xi Leave a Comment on The `sed` command uncommon behaviors

The sed command uncommon behaviors

The sed command is used in Unix, some strange behaviors can let time waste.

Escape char in regex

Normally, the \ is escape character, but it wasn't in some cases.

For example, . is to match any character, it needs to have \ as escape character if need it to be a dot character.

$ echo "test (111) help . 1" | sed -e "s/.//"
est (111) help . 1
$ echo "test (111) help . 1" | sed -e "s/\.//"
test (111) help  1

But this is not for (), without \, they are (), with \, they are indicating subpattern.

$ echo "test (111) help . 1" | sed -e 's/(111)//'
test  help . 1
$ echo "test (111) help . 1" | sed -e 's/\(111\)//'
test () help . 1
$

Same for {}

echo "test (111) help . 1" | sed -e 's/hel{1}p//'
test (111) help . 1
$ echo "test (111) help . 1" | sed -e 's/hel\{1\}p//'
test (111)  . 1

and ?

$ echo "test (111) help . 1" | sed -e 's/he?lp//'
test (111) help . 1
$ echo "test (111) help . 1" | sed -e 's/h?lp//'
test (111) help . 1
$ echo "test (111) help . 1" | sed -e 's/he\?lp//'
test (111)  . 1

* and +

The sed understands the meaning of *, but not for +.

$ echo "test (111) help . 1" | sed -e 's/hel*p//'
test (111)  . 1
$ echo "test (111) help . 1" | sed -e 's/hel+p//'
test (111) help . 1

\* and \+

The sed understands the meaning of \+, but not for \*.

$ echo "test (111) help . 1" | sed -e 's/hel\*p//'
test (111) help . 1
$ echo "test (111) help . 1" | sed -e 's/hel\+p//'
test (111)  . 1