Config File Help Too Many Values Needed

Question

I have a python script which uses settings.cfg which looks like:

[site1]
url=http://boots4tall.com,http://myboots.com,http://boots.com,http://xbox.com,http://xboxOne.com
category=Boots,Boots,Boots,Xbox,Xbox

And the python script function:

def main():
    config=ConfigParser.ConfigParser()
    config.readfp(open("settings.cfg"),"r")
    for site in config.sections():
       # ipdb.set_trace()
        settings=dict(config.items(site))
        for (url,category) in zip(settings['url'].split(","),settings['category'].split(",")):

My main goal is to use specific url for specific category. Most of the times it's 1-4 different url's for 1 category. So I pick 1 category 1 url and let the script do anything related to this url + category.

My problem: I have 1200 categories and around 3000 urls. It will be very hard for me to manually add those categories + urls without mistake in 1 line which is really hard. So I ask for your help can you help get a better system? I thought about reading lines from text files for example 1 file urls.txt 1 file categories.txt? How the code should even look for this kind of thing?

Answer 1

Given how many categories and urls you use / require, I would use an SQLite database to store the categories and URLs and retrieve them when needed. Yes, it adds a bit of overhead and more code but it's easier to maintain and update.