How to read config.cfg/ini and store data into data structure/list 1

[chrisdq]

Hi everyone,

How do you read a config.cfg or ini file and use data structure (such as list, etc..) must allow duplicates to store [section] attributes = values??

I have a config file with many [sections] and in each section contains a list of attributes with values.

How do you store all the sections attributes and their values uniquely into list or list of list so they can be searched, added, and removed easily.

Thank you very much in advance to those can help

 

[ericbrunson]

Are you saying you need duplicate section names? Or do you need duplicate attributes within a section? Both cases are ambiguous.

 

[EBGreen]

I have never done this and I am very new to python as well, but experience with other languages makes me think of two possible ways to do this. First let me make sure that I understand the problem. You have an INI that is something like this:

[Section 1]
Key1=Value1
Key2=Value2

[Section 2]
Key1=Value3
Key2=Vlaue4

You want to be able to read the whole thing into a data structure.

The first and IMO preferable method would be to use a dictionary of dictionaries. I'm not positive that this is possible in python, but I would be suprised to find that it wasn't.

The second possibility is a simple dictionary where the keys are comprised of the string formed by concatenating the section name with the key label. so you would have a dictionary storing data structured like this:
Section1Key1=Value1
Section1Key2=Value2
Section2Key1=Value3
Section3Key2=Vlaue4


As I said I've never done either of these in python, but I am learning python and these sort of tasks are common enough that if no one comes along with an answer, then let me know and I will work out the code.


On a side note, If you have total control over the system, then you may consider moving to an XML for configuration control. Then you can use XPath for your configuration changes and avoid having to read all of the data into a data structure at all.

[red]"... isn't sanity really just a one trick pony anyway?! I mean, all you get is one trick, rational thinking, but when you are good and crazy, oooh, oooh, oooh, the sky is the limit!" - The Tick[/red]

 

[chrisdq]

Hi,

Thank you for your responses.

Eric: I need to allow duplicate distributes, do you have any example or code that could help me?


Tom: That's exactly what I"m trying to do.

Your second method:
Section1Key1=Value1
might work but I have to parse out the section string whenever I need to print out the keys+ attribute only in many functions, the section get passed to the class separately.

I like the first method better,
Read all the sections with attributes+values where it allows duplicate attributes into a dictionary. But I don't know how to store them into a dictionary of dictionaries. This method seems to work best and this is what I need.
However, dictionary does not allow duplicate keys, the second identical one will overwrite the first one. Unless, I append the value to the key but the problem is I can't delete a specified value from the key that contains many duplicate values. Once I do del key[attrib], it will delete the whole key with all the values in it.


Do do have any template/example of how to create a dictionary that contains dictionaries??

Anyone that can help will be greatly appreciated.
Thanks.

 

[EBGreen]

Here is a quick and dirty example of a dictionary of dictionaries. I can make a more comprehensive example if this is not sufficient.

myDict = dict()
innerDict = dict()
innerDict["Key1"] = "Value1"
innerDict["Key2"] = "Value2"
myDict["Section1"] = innerDict
print myDict["Section1"]["Key1"]

[red]"... isn't sanity really just a one trick pony anyway?! I mean, all you get is one trick, rational thinking, but when you are good and crazy, oooh, oooh, oooh, the sky is the limit!" - The Tick[/red]

 

[EBGreen]

Here is a little more in depth example. You would of course need to come up with the routine to build the dictionaries dynamically from your config file.

myDict = dict()
innerDict = dict()
innerDict["Key1"] = "Value1"
innerDict["Key2"] = "Value2"
myDict["Section1"] = innerDict
innerDict = dict()
innerDict["Key1"] = "Value3"
innerDict["Key2"] = "Value4"
myDict["Section2"] = innerDict

for outerKey in myDict:
    print "[" + outerKey + "]"
    for innerKey in myDict[outerKey]:
        print "\t" + innerKey + "=" + myDict[outerKey][innerKey]

[red]"... isn't sanity really just a one trick pony anyway?! I mean, all you get is one trick, rational thinking, but when you are good and crazy, oooh, oooh, oooh, the sky is the limit!" - The Tick[/red]

 

[ericbrunson]

Yeah, if you weren't using duplicate keys you could use the built in config parser:

import ConfigParser
conf = ConfigParser.ConfigParser()
conf.read( 'config.ini' )

But in my tests subsequent sections and attributes of the same name overwrite previous entries.

If you're deviating from an RFC standard and creating your own config format, you're going to have to write your own parser. I don't see anything other than brute force. Here's a simple example without much error checking or exception handling:

# open the file
file = open( 'config.ini' )

# create an empty dict
sections = {}

for line in  file.readlines():
    # get rid of the newline
    line = line[:-1]

    # this will break if you have whitespace on the "blank" lines
    if line:
        # this assumes everything starts on the first column
        if line[0] == '[':
            # strip the brackets    
            section = line[1:-1]
            # create a new section if it doesn't already exist
            if not sections.has_key( section ):
                sections[ section ] = {}
        else:
            # split on first the equal sign
            ( key, val ) = line.split( '=', 1 )
            # create the attribute as a list if it doesn't
            # exist under the current section, this will
            # break if there's no section set yet
            if not sections[ section ].has_key( key ):
                sections[ section ][ key ] = []
            # append the new value to the list
            sections[ section ][ key ].append( val )

For an input file like this:

[My Section]
test=test
test=test2
dir=frob

[My Section]
test=test
test=test2
dir=frob

It'll parse it into one section called "My Section" and two attributes: "test" will have 4 elements and "dir" will have two.