×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

delete 376K files from a list
3

delete 376K files from a list

delete 376K files from a list

(OP)
I have a txt file with filenames going straight down. I need to delete them.

rm -f $(echo $(cat files_to_del.txt))

gives me arg list too long.

also tried

xargs rm -rf < files_to_del.txt

it doesn't do anything.

any help would be greatly appreciated.

RE: delete 376K files from a list

if the names are one per line:

CODE -->

cat files_to_del.txt | while read fn; do rm -f ${f}; done 

Before you do this, test what commands will run by adding echo, as in:

CODE -->

cat files_to_del.txt | while read fn; do echo rm -f ${f}; done 

RE: delete 376K files from a list

You can use "xargs" if you add a command line switch...

CODE

xargs -n20 rm -f < files_to_del.txt 

That would make it do 20 at a time. That could be quicker with 376K files to delete. Deleting 376K files one at a time means "rm" is called 376K times. Doing 20 at a time means "rm" is only called 18,800 times. If "rm" is an internal command, it probably won't matter, but if your shell is actually calling "/bin/rm", then it could.

If the command line is still too long, you can lower the number (i.e. "-n10" to do 10 at a time). Depending on how large each file name in the file is (including any path), you might be able to even bump it up to do more per "rm".

RE: delete 376K files from a list

(OP)
rm -f
rm -f
rm -f
rm -f
rm -f
rm -f
rm -f
rm -f
rm -f
rm -f
rm -f
rm -f
rm -f
etc....

RE: delete 376K files from a list

Small typo, try this...

CODE

cat files_to_del.txt | while read fn; do echo rm -f ${fn}; done 

RE: delete 376K files from a list

(OP)
LOL, I'm sorry I didn't catch it either!

RE: delete 376K files from a list

Good catch Sambones, Thank you. It's all my keyboard's fault.

RE: delete 376K files from a list

(OP)
so...I ran the echo first and got the correct output:
rm -rf PXPROD_7csoh8si_64_1
rm -rf PXPROD_7dsoh8si_101_1
rm -rf PXPROD_7hsoh8sj_101_1
rm -rf PXPROD_7lsoh8sk_101_1
rm -rf PXPROD_jks8868l_101_1
cat files_to_del.txt | while read fn; do echo rm -f ${fn}; done

but when I took the echo out....nothing was deleted. I tried doing a few manually to make sure it wasn't a priv issue and I was able to delete them manually.

Any ideas or things to check?

RE: delete 376K files from a list

check that that you are in the directory where the files reside, or use full paths in that list.

RE: delete 376K files from a list

(OP)
Yeah, I checked that too. I am in the directory of the files. and the file looks like this:
PXPROD_bqsj2c03_77_1
PXPROD_bqsj2c03_78_1
PXPROD_bqsj2c03_79_1
PXPROD_bqsj2c03_80_1
PXPROD_bqsj2c03_81_1
PXPROD_bqsj2c03_82_1

how would I spit the output to a log file to try and look for any error(s)?

RE: delete 376K files from a list

There is no output from the script? normally you would add to the end of the line:

CODE

>output.log 2>&1 
I just noticed you are doing
rm -rf 
where 'r' is for recursive. Is it what you need? I added 'f' for forcing, no confirm deletions. So to avoid unintended consequences remove 'r'. To see if there are prompts, also remove the 'f' so the line would read:

CODE

cat files_to_del.txt | while read fn; do rm ${fn}; done >/tmp/output.log 2>&1 

RE: delete 376K files from a list

2
Try this...

CODE

# Using head instead of cat to only try 20 while testing
head -20 files_to_del.txt | while read fn; do ls ${fn}; done 

If the file exists, it will just print its name. If the file isn't there where you expect it, it will say "ls: cannot access PXPROD_bqsj2c03_77_1: no such file or directory". Or whatever each filename is. This is just to see if they are there.

Have you done a few spot checks to make sure the files in your "files_to_del.txt" file are really there? Pick a couple filenames and grep the file for them. Your file could have become out of sync with what's actually in the directory. It may look correct with a quick glance, but it could be off enough to miss them all.

Do you have permission to delete the files. You will not only need permission on the file itself, but you will also need 'w' amd 'x' on the directory they are in (it needs to be updated that the file is gone). If it's your current default directory, then look at the perms for the directory '.' (that's one dot).

Also, where did the "files_to_del.txt" originate? If it came from a Windows system, you probably have a carriage-return/line-feed pair ending each line. This always messes up things like this. Look for a utility called "dos2unix", or something like that. That will "fix" a Windows file.

Also, the files could have invisible characters in their names. Try adding a '-b' switch to an 'ls' to see any non-printable characters in the names.

Lastly, if you know you're in the right directory, and you've checked all those things, the server is haunted. Burn it. Then take the remains and bury in among the roots of a banyan tree under a full moon. It's best to just cut your losses and move on in cases like this.

RE: delete 376K files from a list

(OP)
dos2unix fixed it

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close