×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Newbie Question

Newbie Question

Newbie Question

(OP)
Hi,

I have a folder with a list of files - 1.dat, 2.dat, 3.dat etc
The file structure is:

rndsid = |11026|1342296563|
lastactive = |1342296563|
lastvisit = |1340862364|
sitename = |Free PC Help|
email = |admin@mysite.com|
pmpopup = |1|
lastpost = |1340736611|
password = |******|
posts = |2|
registered = |1339327069|
dateformat = |0|
timezone = |0|
timeformat = |0|
sn = |Administrator|
siteurl = |http://www.pcfixzone.co.uk|

What I need to do is read the content of each .dat file and remove the entries for siteurl and sitename if they exist.

Is this possible?

Many thanks

RE: Newbie Question

Hi

Quote (martinjburgess)

remove the entries for siteurl and sitename if they exist.
You mean, remove the entire such line ?

This does in-place modification, so better make backups before trying it :

CODE --> command-line

perl -pi -e 'undef$_ if/^site(url|name)\s*=/' *.dat 

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
Thanks for the reply but that just gives a 500 error

What I need is a small script that when run will:

open each .dat file in the Members folder
Read each line
If siteurl or sitename exists either delete the line or set it to ''

RE: Newbie Question

Hi

Quote (martinjburgess)

Thanks for the reply but that just gives a 500 error
What is "a 500 error" ? Show us how exactly you tried it and what exactly you got.

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
I pasted it at the top of another script and when run it just gives a HTTP 500 error.

I think I will need something more on the lines of:

open(FILE,"$members/$_.dat");
while(<FILE>) {
code
code
code
close(FILE);
}

or something along those lines.

Sorry for being so vague.

RE: Newbie Question

Hi

Quote (martinjburgess)

I pasted it at the top of another script

Quote (Feherke)

vvvvvvvvvvv--- execute it from the command line

CODE --> command-line

perl -pi -e 'undef$_ if/^site(url|name)\s*=/' *.dat 

Quote (martinjburgess)

just gives a HTTP 500 error.
How gets HTTP involved in all this ?

Anyway.

CODE --> Perl

use File::Temp qw{tempfile};

foreach $file (<*.dat>) {
  open FIN,"<$file";
  ($fout,$name)=tempfile;
  while ($str=<FIN>) {
    print $fout $str unless $str=~m/^site(url|name)\s*=/;
  }
  close $fout;
  close FIN;

  rename $name,$file;
} 
Next time please specify all requirements at the beginning.

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
Sorry but no joy.

I did change this:

foreach $file (<*.dat>) {

to this:

foreach $file (<$members/*.dat>) {

as this is the folder the .dat files are in.

RE: Newbie Question

Hi

Quote (Feherke)

Show us how exactly you tried it and what exactly you got.

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
Thanks for your patience.
This is the code and nothing happens:

#!/usr/bin/perl

use CGI::Carp fatalsToBrowser;

# Global information:
$theblahver = 16;
$version = $versioncr = '10.3.6'; # Said Version; Copyright version
$blanktarget = " target='_new'";

# Uncomment this for better time precision
use Time::HiRes qw(time);

# Filename information
$scriptname = $scriptname || 'Blah.pl'; # Change name of Blah.pl
$modrewrite = $modrewrite || '?'; # Setting, mod_rewrite: on = '' | off = '?'

use Fcntl ':flock';

# Default language
$languagep = "English";
$languages = "./Languages";

require('Settings.pl');

require("$code/QuickCore.pl");
UFS();
CheckCookies();
GetThemes();

$language = "$languages/$languagep";
require("$language.lng");
require("$code/Routines.pl");
require("$code/Load.pl");


# Remove the theme variable for guests/search engines
redirect() if($URL{'theme'} && $username eq 'Guest');

# Load basic features we can use later
CreateGroups();
BoardCheck();
ClickLog();
AL();

use File::Temp qw{tempfile};

foreach $file (<$members/*.dat>) {
open FIN,"<$file";
($fout,$name)=tempfile;
while ($str=<FIN>) {
print $fout $str unless $str=~m/^site(url|name)\s*=/;
}
close $fout;
close FIN;

rename $name,$file;
}
1;

RE: Newbie Question

Hi

So it is a CGI script. That makes the debugging abit complicated. The steps I would do :
  • Check if the user with who's permission the web server runs, is allowed to write those files.
  • Add warningsToBrowser :

    CODE --> (fragment)

    # change this
    use CGI::Carp qw{fatalsToBrowser warningsToBrowser};
    
    # ...
    
    # add this after the HTTP headers were sent
    warningsToBrowser(1); 
  • Check your web server's error log.
  • Add some print calls to see what gets executed :

    CODE --> (fragment)

    # ...
    
    print "start dat editing <br>\n";
    use File::Temp qw{tempfile};
    
    print "directory is $members<br>\n";
    print "files to process are ",<$members/*.dat>,"<br>\n";
    foreach $file (<$members/*.dat>) {
      print "start file $file<br>\n";
      open FIN,"<$file";
      ($fout,$name)=tempfile;
      print "write temp file $name<br>\n";
      while ($str=<FIN>) {
        print $fout $str unless $str=~m/^site(url|name)\s*=/;
      }
      close $fout;
      close FIN;
    
      print "rename file $name to $file<br>\n";
      rename $name,$file;
      print "end file $file<br>\n";
    }
    print "end dat editing<br>\n";
    1; 

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
It looks like it is functioning but it doesn't actually delete the lines siteurl and sitename.


start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat./Members/5.dat
start file ./Members/1.dat
write temp file /tmp/WFTaSsK1ga
rename file /tmp/WFTaSsK1ga to ./Members/1.dat
end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/b5adpSuXC5
rename file /tmp/b5adpSuXC5 to ./Members/2.dat
end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/TSvi8giLoH
rename file /tmp/TSvi8giLoH to ./Members/3.dat
end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/CBfa12yMke
rename file /tmp/CBfa12yMke to ./Members/4.dat
end file ./Members/4.dat
start file ./Members/5.dat
write temp file /tmp/cOaQsiwo82
rename file /tmp/cOaQsiwo82 to ./Members/5.dat
end file ./Members/5.dat
end dat editing
Content-type: text/html; charset=UTF-8

RE: Newbie Question

Hi

Seems my first point was right :

Quote (Feherke)

  • Check if the user with who's permission the web server runs, is allowed to write those files.

To be sure, you can add one more print :

CODE --> (fragment)

print rename $name,$file; 

It will output 1 on success and 0 on failure. Probably will be all 0s. This may be caused by more things, but most probably is the lack of write permission.

To correct this give write permission on Members/ directory. So from the command line first become superuser, then execute chmod a+x /path/to/Members/. ( Of course, in place of a ( all ) use g ( group ) in case the web server's configuration sets the group to the same group as Members/ is owned by. )

If still not works, you will have to add it to the files themselves with chmod a+x /path/to/Members/*.dat. But in this case you will have the same problem after creating/generating newer dat files. ( I suppose there will be more of them later. )

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
Thanks, almost there.
It is now removing the line sitename but not the line siteurl

There are full write permissions on the folder.

RE: Newbie Question

Hi

Strange. Are you sure the sample in your opening post is correct ? I tested with that one and worked correctly. ( Given that siteurl is the last line, I double-checked both with and without trailing newline. )

Maybe there is a leading space in that line ? If yes, you can get ride of it with this :

CODE --> (fragment)

print $fout $str unless $str=~m/^\s*site(url|name)\s*=/; 

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
My mistake, it's not deleting any of the lines, it just reorderd them.

Here's what I now get:

start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat./Members/5.dat
start file ./Members/1.dat
write temp file /tmp/nKijKk93AQ
rename file /tmp/nKijKk93AQ to ./Members/1.dat
0end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/6ydGcqERS1
rename file /tmp/6ydGcqERS1 to ./Members/2.dat
0end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/Ck7RE8hTNL
rename file /tmp/Ck7RE8hTNL to ./Members/3.dat
0end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/rSsiIt7fF0
rename file /tmp/rSsiIt7fF0 to ./Members/4.dat
0end file ./Members/4.dat
start file ./Members/5.dat
write temp file /tmp/px78Uqg55x
rename file /tmp/px78Uqg55x to ./Members/5.dat
0end file ./Members/5.dat
end dat editing

RE: Newbie Question

Hi

Well, those 0s in the output are clearly showing that rename is still failing.

There is another step to include to see the exact reason of the failure :

CODE --> (fragment)

print "rename file $name to $file<br>\n";
print rename $name,$file;
print "rename error $!<br>\n";
undef $!;
print "end file $file<br>\n"; 

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat
start file ./Members/1.dat
write temp file /tmp/08yw9L8QjX
rename file /tmp/08yw9L8QjX to ./Members/1.dat
0rename error Invalid cross-device link
end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/DXfbMJ81WL
rename file /tmp/DXfbMJ81WL to ./Members/2.dat
0rename error Invalid cross-device link
end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/FKbrf0HXol
rename file /tmp/FKbrf0HXol to ./Members/3.dat
0rename error Invalid cross-device link
end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/iSihH4mK3P
rename file /tmp/iSihH4mK3P to ./Members/4.dat
0rename error Invalid cross-device link
end file ./Members/4.dat
end dat editing

RE: Newbie Question

Hi

Ah. Got it. Your Members/ and /tmp/ directories are on different partitions. Then something stronger than the basic rename is needed :

CODE --> Perl

use File::Temp qw{tempfile};
use File::Copy;

foreach $file (<*.dat>) {
  open FIN,"<$file";
  ($fout,$name)=tempfile;
  while ($str=<FIN>) {
    print $fout $str unless $str=~m/^site(url|name)\s*=/;
  }
  close $fout;
  close FIN;

  move $name,$file;
} 

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
You are a genius Sir, thank you very much, all working like a charm.

RE: Newbie Question

(OP)
One last question.
Is it possible to skip if the user is administrator?

This is the line in the .dat file

sn = |Administrator|


Cheers

RE: Newbie Question

Hi

What do you mean by "skip" ? Remove the siteurl and sitename lines only if the sn's value is not "Administrator" ?

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
That's correct, if sn's value is administrator then leave the siteurl and sitename intact.

RE: Newbie Question

Hi

Well, this can be essentially done in two different ways :
  • Single pass : Process the file as now, but if sn = Administrator line is met, cancel the processing.
  • Two passes : Read the file and check the existence of sn = Administrator line. If non find, process the file as now.
Probably I should have asked this earlier. The dat files will be all like that sample, with 10~20 lines, under 1Kb size ? If yes, we can just slurp in the entire file instead of processing line by line. This way neither temporary file and file moving is involved :

CODE --> Perl

undef $/;

foreach $file (<*.dat>) {
  open FIL,"<$file";
  $data=<FIL>;
  close FIL;

  next if $data=~m/^sn\s*=\s*\|Administrator\|/mi;

  $data=~s/^site(url|name)\s*=.*\r?\n?//gm;

  open FIL,">$file";
  print FIL $data;
  close FIL;
} 

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
That last code completely mucks up the table layout of my test forum for some reason.

These dat files are only small, never seen one larger than 1Kb

RE: Newbie Question

Hi

I assume is because my ugly habit to unset the $/. This should play nicely :

CODE --> Perl

#undef $/; # <-- remove this one

foreach $file (<*.dat>) {
  local $/;

  # ...
} 

Feherke.
http://feherke.github.com/

RE: Newbie Question

(OP)
That's done the trick, I'll do some more testing and let you know.
Thank you, much appreciated.

RE: Newbie Question

(OP)
Sorry it's late but just want to say thanks to feherke for his patience and knowledge. Everything I wanted to achieve I have.
Thank You

RE: Newbie Question

Nothing says "Thank you" better than a star... winky smile

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close