Corresponding float in two lists

Question

I have two CSV files.
The first one, when seen as a list, looks like this:

('Rubus idaeus', '10.0', '56.0')
('Neckera crispa', '9.8785', '56.803')
('Dicranum polysetum', '9.1919', '56.0456')
('Sphagnum subnitens', '9.1826', '56.6367')
('Taxus baccata', '9.61778', '55.68833')
('Sphagnum papillosum', '9.1879', '56.0442')

The columns are 'Species','Longitude' and 'Latitude'. They are observations made in the field.
The other file is also a CSV file. A test made to resemble the real thing. It looks like this:

{'y': '58.1', 'x': '22.1', 'temp': '14'}
{'y': '58.2', 'x': '22.2', 'temp': '10'}
{'y': '58.3', 'x': '22.3', 'temp': '1'}
{'y': '58.4', 'x': '22.4', 'temp': '12'}
{'y': '58.5', 'x': '22.5', 'temp': '1'}
{'y': '58.6', 'x': '22.6', 'temp': '6'}
{'y': '58.7', 'x': '22.7', 'temp': '0'}
{'y': '58.8', 'x': '22.8', 'temp': '13'}
{'y': '58.9', 'x': '22.9', 'temp': '7'}

Both of the files are very long indeed.

I have the observation, and now I want to find the closest lower number in the file containing climatic data, and then append that line to the other one, so the output becomes:

('Dicranum polysetum', '9.1919', '56.0456', 'y': '9.1', 'x': '56.0', 'temp': '7')

I tried creating nested loops by iterating through the CSV files using DictReader, but it gets very nested very fast. And it will take a tremendous amount of loops to get through the whole thing.
Does anybody know of a method?

The code I have at the moment is poor, but I tried looping in several ways and I expect there is something fundamentally wrong with my entire approach.

import csv
fil = csv.DictReader(open("TestData.csv"), delimiter=';')
navn = "nyDK_OVER_50M.csv"
occu = csv.DictReader(open(navn), delimiter='\t')

for row in fil:
    print 'x=',row['x']
    for line in occu:
        print round(float(line['decimalLongitude']),1)
        if round(float(line['decimalLongitude']),1) == row['x']:
            print 'You did it, found one dam match'

Here's the links for my two files, so you don't have to make up any data in case you know of something that can push me forward.

https://www.dropbox.com/s/lmstnkq8jl71vcc/nyDK_OVER_50M.csv?dl=0 https://www.dropbox.com/s/v22j61vi9b43j78/TestData.csv?dl=0

Best regards, Mathias

Answer 1

Because you say there are no missing temperature data points, then it is much easier to solve the problem:

import csv

# temperatures
fil = csv.DictReader(open("TestData.csv"), delimiter=';')
# species
navn = "nyDK_OVER_50M.csv"
occu = csv.DictReader(open(navn), delimiter='\t')

d = {}
for row in fil:
    x = '{:.1f}'.format(float(row['x']))
    y = '{:.1f}'.format(float(row['y']))
    try:
        d[x][y] = row['temp']
    except KeyError:
        d[x] = {y:row['temp']}

for line in occu:
    x = '{:.1f}'.format(round(float(line['decimalLongitude']),1))
    y = '{:.1f}'.format(round(float(line['decimalLatitude']),1))
    temp = d[x][y]
    line['temp'] = temp
    line['x'] = x
    line['y'] = y
    print(line)