Python - Data Manipulation

Question

Python - Data Manipulation

nvhuser (Programmer)

(OP)

27 May 15 16:58

Hello everyone,

I am getting started with python programming and I am facing some problems regarding data manipulation.
Please consider the following data:

CODE -->

2.9        -4.0   50.0
3.4        -5.0  150.0
2.9        -1.0  350.0
2.4        -4.0   50.0
2.0        -2.0  110.0

The numbers are available in an 8-digit system, it means that the first number should be between the column 1 to 8, while the second one should be between the column 9 and 16, and so on.

I would like to create a script that reads this information stored in a file "file.txt" available in a given directory and evaluate the sum of the three components, saving the file "file2.txt". Thank you for your help.

IPGuru (Vendor) · Answer 1 · 2015-05-28 08:18:43

you could try slicing but with the example data split would suffice

[code]
>>>a= "2.9 -4.0 50.0"
>>>b=a.split()
>>>b
['2.0','-4.0','50.0'
>>>float(b[0]+float(b[1])+float(b[2])
48.9
[code]

A Maintenance contract is essential, not a Luxury.
Do things on the cheap & it will cost you dear

nvhuser (Programmer) · Answer 2 · 2015-05-28 13:40:47

Thank you IPGuru, your commands work fine!
Could you tell me how to read all data and do that for all lines?
Thanks

IPGuru (Vendor) · Answer 3 · 2015-05-28 18:12:26

i suggest you check the online tutorial for open.

in principle you open a file for reading

loop over each line in turn (hint a for loop is ideal here)
process the data

to give you a full solution would not help your learning.

post some code & we well you what you are doing wrong (ant what you are doing right) and po9sibly sugest some imrovements.
if the code fails copy the python traceback as well

A Maintenance contract is essential, not a Luxury.
Do things on the cheap & it will cost you dear

nvhuser (Programmer) · Answer 4 · 2015-05-29 09:12:29

Thank you IPGuru, I will check the tutorial.
After that, I will try to post here the solution for future users :)

nvhuser (Programmer) · Answer 5 · 2015-05-29 17:25:26

As promised, here is the python code:

CODE --> python

#!/bin/python

file_object=open('file1.txt','r')
f=file_object.readlines()

count=len(f)
for i in range(0,len(f)):
        b=f[i].split()
        c=float(b[0])+float(b[1])+float(b[2])
        print c

Now, I just need to give the instruction to save the contents in the file2.txt
Thank you for your help!

IPGuru (Vendor) · Answer 6 · 2015-06-01 09:43:27

your method for iterating over the input, while valid is not very pythonic

you almost never want to create a range to iterate over as an index. the pythonic way to achive this would be:-

CODE

#!/bin/python

file_object=open('file1.txt','r')
for line in file_object:http://www.tek-tips.com/viewthread.cfm?qid=1750241
        b=line.split()
        c=float(b[0])+float(b[1])+float(b[2])
        print c

if you open a 2nd file for writing & replace your print statement with a write to file you project will be complete.
the official python tutorials should give you plenty of examples
(google python write to file & also python with)

A Maintenance contract is essential, not a Luxury.
Do things on the cheap & it will cost you dear

nvhuser (Programmer) · Answer 7 · 2015-06-01 13:04:19

Hi IPGuru, thank you for your Tip!
I think here is the complete solution:

CODE -->

#!/bin/python
file_object=open('file1.txt','r')
file_object2=open('file2.txt','w')
for line in file_object:
        b=line.split()
        c=float(b[0])+float(b[1])+float(b[2])
        file_object2.write(str(c)+"\n")

Suggestions/comments will be very welcome! Thank you for your help

IPGuru (Vendor) · Answer 8 · 2015-06-01 13:57:05

looks fine to me
For robustness in a real world program you should have some error checking.
(what happens if you cant open the input file?, what happens if you cant open the output file, what happens if your routine fills all of the available disk space etc.)

These are things you will learn about as you progress through the tutorials

A Maintenance contract is essential, not a Luxury.
Do things on the cheap & it will cost you dear

nvhuser (Programmer) · Answer 9 · 2015-06-01 16:08:36

Hi IPGuru, thank you for your advices!
In fact, I can add a if loop to check if it the file can open

I have another question:

Imagine that I have a matrix with four rows and two columns, like the following:

CODE -->

and I want to eliminate the duplicated entries in column 2? in order to have this:

CODE -->

2 3
4 4
6 2

My problem is that I get always an extra line like this:

CODE -->

The code that I am using is:

CODE -->

#/bin/python

file_object=open('file1.txt','r')
file_object2=open('file2.txt','w')

read_data=file_object.readlines()
nd=[]

for line in read_data:
        s=line
        if s[2] not in nd:
                nd.append(s[2])
                file_object2.write(str(line)+"\n")

Thanks in advance.

nvhuser (Programmer) · Answer 10 · 2015-06-02 07:45:50

I think one solution would be:

CODE -->

#/bin/python

file_object=open('file1.txt','r')
file_object2=open('file2.txt','w')

read_data=file_object.readlines()
nd=[]

for line in read_data:
        s=line
        if s[2] not in nd:
                nd.append(s[2])
                line = line.strip('\n')
                file_object2.write(str(line)+"\n")

Using line.strip. What do you think?

nvhuser (Programmer) · Answer 11 · 2015-06-02 12:13:09

The previous code only works for 1-digit numbers
this works for two numbers separated by a space.

CODE -->

#/bin/python
file_object=open('sample.txt','r')
file_object2=open('sample2.txt','w')

read_data=file_object.readlines()
nd=[]

for line in read_data:
        s=line
        b=s.split()
        if b[1] not in nd:
                nd.append(b[1])
                line = line.strip('\n')
                file_object2.write(str(line)+"\n")

Any comments?

Come Join Us!

Posting Guidelines

Python - Data Manipulation