Updating and Collating Array 1

[iggitme]

Hello peoples... This is an aggravating problem for me and any help at all would be greatly appreciated.

I have an array assembled named: @MASTERLIST

Each line is thus:

$titleword1,$catnum11,$valueword1,$catname1

I need to run through the array, find matches of $catnum11 and total the $valueword1 numbers to create on line with the $catnum1 collecting all values of the $valueword1 as addition. So a large list of quite a few duplicate $catnum1's is merged and added into a much smaller list of one entry of each $catnum1 with the corresponsing $valueword1 a total of all of the same $catnum1 values.

Its identifying duplicates in the array and merging the value of those duplicates into one entry for that number of the entry.

I've tried all sorts of useless exercises including writing to tmp files and updating with reopening on each try, brute force ignorance at work..

A solution to this would be forever appreciated.
Thanks.

 

[iggitme]

And I meanst $catnum1 as $catnum11

Sorry

 

[KevinADC]

Post some sample data and what the output should look like. Next time also post the code you have been trying.

------------------------------------------
- Kevin, perl coder unexceptional!

 

[iggitme]

Well the code i have been trying is gone. There have been hours of variations.

All I'm looking for is the same output as the input only collated, combined and totalled by number

example showing data:


array has:

fred,100,35,names
fred,100,20,names
fred,100,45,names
john,101,35,names
john,101,20,names
john,101,45,names
betty,102,35,names
betty,103,20,names
betty,104,45,names

and many others

needs to wind up as

fred,100,100,names
john,100,100,names
betty,100,100,names

and any others that do not have multiples to be left alone

surely hope that helps out a bit

 

[iggitme]

OK obviously i'm not all here:

fred,100,35,names
fred,100,20,names
fred,100,45,names
john,101,35,names
john,101,20,names
john,101,45,names
betty,102,35,names
betty,103,20,names
betty,104,45,names


fred,100,100,names
john,100,100,names
betty,100,100,names

should have been

fred,100,35,names
fred,100,20,names
fred,100,45,names
john,101,35,names
john,101,20,names
john,101,45,names
betty,102,35,names
betty,102,20,names
betty,103,45,names


fred,100,100,names
john,101,100,names
betty,102,100,names


sorry

 

[iggitme]

Good grief why can't i edit one of these posts instead of repeating things:

betty,102,35,names
betty,102,20,names
betty,103,45,names

should be

betty,102,35,names
betty,102,20,names
betty,102,45,names

id numbers stay the same
values are totalled

 

[KevinADC]

Good grief why can't i edit one of these posts instead of repeating things:

Use the "preview" button to preview your posts. If you need to make changes you will see the "edit" button in the preview screen, that will return you to the thread where you can edit the post before submittng it.

What you want to do is use a hash or a hash of hashes, or whatever you think is best. Here is one way using a hash of hashes (not well tested)

[url=http://perldoc.perl.org/functions/use.html][black][b]use[/b][/black][/url] [green]strict[/green][red];[/red]
[black][b]use[/b][/black] [green]warnings[/green][red];[/red]
[url=http://perldoc.perl.org/functions/my.html][black][b]my[/b][/black][/url] [blue]%hash[/blue] = [red]([/red][red])[/red][red];[/red]
[olive][b]while[/b][/olive][red]([/red][black][b]my[/b][/black] [blue]$line[/blue] = <DATA>[red])[/red][red]{[/red]
   [url=http://perldoc.perl.org/functions/chomp.html][black][b]chomp[/b][/black][/url][red]([/red][blue]$line[/blue][red])[/red][red];[/red]
   [black][b]my[/b][/black] [blue]@t[/blue] = [url=http://perldoc.perl.org/functions/split.html][black][b]split[/b][/black][/url][red]([/red][red]'[/red][purple],[/purple][red]'[/red],[blue]$line[/blue][red])[/red][red];[/red]
   [olive][b]if[/b][/olive] [red]([/red] [url=http://perldoc.perl.org/functions/exists.html][black][b]exists[/b][/black][/url] [blue]$hash[/blue][red]{[/red][blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red][red]}[/red] [red])[/red] [red]{[/red]
      [blue]$hash[/blue][red]{[/red][blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red][red]}[/red][red]{[/red][red]'[/red][purple]valueword[/purple][red]'[/red][red]}[/red] += [blue]$t[/blue][red][[/red][fuchsia]2[/fuchsia][red]][/red][red];[/red]
   [red]}[/red]
   [olive][b]else[/b][/olive] [red]{[/red]
      [blue]$hash[/blue][red]{[/red][blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red][red]}[/red] = [red]{[/red][purple]titleword[/purple] => [blue]$t[/blue][red][[/red][fuchsia]0[/fuchsia][red]][/red], [purple]catnum[/purple] => [blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red], [purple]valueword[/purple] => [blue]$t[/blue][red][[/red][fuchsia]2[/fuchsia][red]][/red], [purple]catname[/purple] => [blue]$t[/blue][red][[/red][fuchsia]3[/fuchsia][red]][/red][red]}[/red][red];[/red]
   [red]}[/red]
[red]}[/red]
[olive][b]foreach[/b][/olive] [black][b]my[/b][/black] [blue]$catnum[/blue] [red]([/red][url=http://perldoc.perl.org/functions/sort.html][black][b]sort[/b][/black][/url] [red]{[/red][blue]$a[/blue] <=> [blue]$b[/blue][red]}[/red] [url=http://perldoc.perl.org/functions/keys.html][black][b]keys[/b][/black][/url] [blue]%hash[/blue][red])[/red] [red]{[/red]
   [url=http://perldoc.perl.org/functions/print.html][black][b]print[/b][/black][/url] [url=http://perldoc.perl.org/functions/join.html][black][b]join[/b][/black][/url][red]([/red][red]'[/red][purple],[/purple][red]'[/red],[blue]$hash[/blue][red]{[/red][blue]$catnum[/blue][red]}[/red]->[red]{[/red][red]'[/red][purple]titleword[/purple][red]'[/red][red]}[/red],[blue]$hash[/blue][red]{[/red][blue]$catnum[/blue][red]}[/red]->[red]{[/red][red]'[/red][purple]catnum[/purple][red]'[/red][red]}[/red],[blue]$hash[/blue][red]{[/red][blue]$catnum[/blue][red]}[/red]->[red]{[/red][red]'[/red][purple]valueword[/purple][red]'[/red][red]}[/red],[blue]$hash[/blue][red]{[/red][blue]$catnum[/blue][red]}[/red]->[red]{[/red][red]'[/red][purple]catname[/purple][red]'[/red][red]}[/red][red])[/red],[red]"[/red][purple][purple][b]\n[/b][/purple][/purple][red]"[/red][red];[/red]
[red]}[/red]
[teal]__DATA__[/teal]
[teal]fred,100,35,names[/teal]
[teal]fred,100,20,names[/teal]
[teal]fred,100,44,names[/teal]
[teal]john,101,35,names[/teal]
[teal]john,101,21,names[/teal]
[teal]john,101,41,names[/teal]
[teal]betty,102,8,names[/teal]
[teal]betty,102,22,names[/teal]
[teal]betty,102,49,names[/teal]

[tt]------------------------------------------------------------
Pragmas (perl 5.8.8) used :
[ul]
[li]strict - Perl pragma to restrict unsafe constructs[/li]
[li]warnings - Perl pragma to control optional warnings[/li]
[/ul]
[/tt]

------------------------------------------
- Kevin, perl coder unexceptional!

 

[iggitme]

I am very sorry and thank you for your help, but the code provided will not run without a 500 error when being used alone, and when being used inside the script i have, it
takes a long time to result in no output.
Thanks for the shot, though. This is really in need of being accomplished, if there are any other potentials please let me know.

 

[KevinADC]

Lets see your code and some real data.

------------------------------------------
- Kevin, perl coder unexceptional!

 

[iggitme]

Thank you for your continued interest!!!

Here is real data and the form used to get there:

Each line of the array is pipe separated:
This array is @MASTERLIST1 which is already sorted

line structure is:
$titleword|$catnum1|$rankcount|$catname|\n

A previous routine is assigning a rankcount for each instance of titleword...

I am getting this list now:

word 30 500.000 aids testing & information
word 523 500.000 drug & alcohol detection & testing
word 822 500.000 laboratories testing
word 1013 500.000 paternity testing
word 30 313.750 aids testing & information
word 822 313.750 laboratories testing
word 1013 313.750 paternity testing
word 822 220.625 laboratories testing

first column is the word used to acquire the rank score
second column is the id number
third column is the 'ranking number'

It needs to wind up being:

word 822 1034.375 laboratories testing
word 30 813.750 aids testing & information
word 1013 813.750 paternity testing
word 523 500.000 drug & alcohol detection & testing

where identical id numbers are added together
to come to the final ranking order, sorted in order
of the $rankcount.

At that point the new array containing the properly collated and sorted data is presented to another section of the script collection for completely different functions.

Its about wits end time here... whomever has the solution will indeed be listed as author of it

Thank you very much AGAIN!!!

 

[KevinADC]

You said @MASTERLIST1 is already sorted but the example of real data you posted does not appear to be sorted in any meaningful way. How is @MASTERLIST1 sorted and does that sorting have to be maintained or will the new list be sorted some other way?

If you script was not written using "strict' originaly that might explain the "500" error mssage from my code. Is this script running as a CGI script or a non-CGI script?

------------------------------------------
- Kevin, perl coder unexceptional!

 

[iggitme]

Running as CGI... third column is the sort contained in the array

500.000 aids testing & information
500.000 drug & alcohol detection & testing
500.000 laboratories testing
500.000 paternity testing
313.750 aids testing & information
313.750 laboratories testing
313.750 paternity testing
220.625 laboratories testing


masterlist1 does not need to be sorted at that point, its sorting after the results of this topic that matters

 

[KevinADC]

If your script is not complied with "strict" it's up to you to determine if the variable names will not clash with any existing ones in your script:

[url=http://perldoc.perl.org/functions/my.html][black][b]my[/b][/black][/url] [blue]%hash[/blue] = [red]([/red][red])[/red][red];[/red]
[olive][b]foreach[/b][/olive] [black][b]my[/b][/black] [blue]$line[/blue] [red]([/red][blue]@MASTERLIST1[/blue][red])[/red][red]{[/red]
   [url=http://perldoc.perl.org/functions/print.html][black][b]print[/b][/black][/url] [red]"[/red][purple][blue]$line[/blue][purple][b]\n[/b][/purple][/purple][red]"[/red][red];[/red]
   [black][b]my[/b][/black] [blue]@t[/blue] = [url=http://perldoc.perl.org/functions/split.html][black][b]split[/b][/black][/url][red]([/red][red]/[/red][purple][purple][b]\|[/b][/purple][/purple][red]/[/red],[blue]$line[/blue][red])[/red][red];[/red]
   [olive][b]if[/b][/olive] [red]([/red] [url=http://perldoc.perl.org/functions/exists.html][black][b]exists[/b][/black][/url] [blue]$hash[/blue][red]{[/red][blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red][red]}[/red] [red])[/red] [red]{[/red]
      [blue]$hash[/blue][red]{[/red][blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red][red]}[/red][red][[/red][fuchsia]2[/fuchsia][red]][/red] += [blue]$t[/blue][red][[/red][fuchsia]2[/fuchsia][red]][/red][red];[/red]
   [red]}[/red]
   [olive][b]else[/b][/olive] [red]{[/red]
      [blue]$hash[/blue][red]{[/red][blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red][red]}[/red] = [red][[/red] [blue]$t[/blue][red][[/red][fuchsia]0[/fuchsia][red]][/red],[blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red],[blue]$t[/blue][red][[/red][fuchsia]2[/fuchsia][red]][/red],[blue]$t[/blue][red][[/red][fuchsia]3[/fuchsia][red]][/red] [red]][/red][red];[/red]
   [red]}[/red]
[red]}[/red]
[blue]@sorted[/blue] = [url=http://perldoc.perl.org/functions/map.html][black][b]map[/b][/black][/url] [red]{[/red][blue]$_[/blue]->[red][[/red][fuchsia]1[/fuchsia][red]][/red][red]}[/red] [url=http://perldoc.perl.org/functions/sort.html][black][b]sort[/b][/black][/url] [red]{[/red][blue]$b[/blue]->[red][[/red][fuchsia]0[/fuchsia][red]][/red]<=>[blue]$a[/blue]->[red][[/red][fuchsia]0[/fuchsia][red]][/red][red]}[/red] [black][b]map[/b][/black] [red]{[/red][red][[/red][blue]$hash[/blue][red]{[/red][blue]$_[/blue][red]}[/red][red][[/red][fuchsia]2[/fuchsia][red]][/red],[url=http://perldoc.perl.org/functions/join.html][black][b]join[/b][/black][/url][red]([/red][red]'[/red][purple]|[/purple][red]'[/red],[blue]@[/blue][red]{[/red][blue]$hash[/blue][red]{[/red][blue]$_[/blue][red]}[/red][red]}[/red][red])[/red][red]][/red][red]}[/red] [url=http://perldoc.perl.org/functions/keys.html][black][b]keys[/b][/black][/url] [blue]%hash[/blue][red];[/red]

This puts the collated data back into a pipe delimited array of strings like @MASTERLIST1. If that is not what you want that can be changed. The above code works with the limited data you posted. The output is:

word|822|1034.375|laboratories testing
word|1013|813.75|paternity testing
word|30|813.75|aids testing & information
word|523|500.000|drug & alcohol detection & testing

which is sorted numericaly descending by the third field.

If you are procesing a large amount of data this can take some time to complete. By large I mean 10s of thousands of lines in @MASTERLIST1. A few thousand lines should process pretty quickly though.

------------------------------------------
- Kevin, perl coder unexceptional!

 

[iggitme]

THAT is beautiful and it works wonderfully. Thank you very very much. Actually it shouldn't be processing more than 800 items at a time, ever.. and normally more like 20... Thank you Again...

 

[KevinADC]

Very good. See how "easy" that was?

------------------------------------------
- Kevin, perl coder unexceptional!