[Tutor] dictionary of lists

[Chris Stinemetz]

I am trying to create a dictionary of lists as I read a file. I
envision it looking like: {key: [float_type],[string_type]}

For the first item in the list I am trying to add the value to the
existing value where the key matches but I am getting the following
error:

Resetting execution engine
Running C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py
The Python REPL process has exited
Traceback (most recent call last):
File "C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py",
line 22, in <module>
d[IMEI] += Elapsed_Mins
TypeError: 'float' object is not iterable

Here is the code. My question is how can I keep it list type and just
sumup the first element in the list as I read it?

d = defaultdict(list)
for fname in os.listdir('.'):
with open (fname) as csvfile:
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
for row in spamreader:

if row[8]:
device = row[36]
Elapsed_Mins = float(row[7])
IMEI = row[8].replace("'", "")

d[IMEI] += Elapsed_Mins ## this is where the error occurs.

Thanks in advance,

Chris
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alan Gauld]

Post by Chris Stinemetz
I am trying to create a dictionary of lists as I read a file. I
envision it looking like: {key: [float_type],[string_type]}

Thats not a dictionary of lists. You maybe mean:

{key: [[float_type],[string_type]]}

Which is a dictionary of lists of lists?

Post by Chris Stinemetz
For the first item in the list I am trying to add the value to the
existing value where the key matches

Sorry, I'm sure that made sense to you but not to me.
Which value are you adding to which existing value?
Can you give a before/after example?

Post by Chris Stinemetz
Resetting execution engine
Running C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py
The Python REPL process has exited

That's slightly unusual. How are you running this?

Post by Chris Stinemetz
File "C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py",
line 22, in <module>
d[IMEI] += Elapsed_Mins
TypeError: 'float' object is not iterable
d = defaultdict(list)
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
device = row[36]
Elapsed_Mins = float(row[7])
IMEI = row[8].replace("'", "")

So IMEA is a string and Elapsed_Mins is a float and d is a default dict
that sets its defaults to lists.

Post by Chris Stinemetz
d[IMEI] += Elapsed_Mins ## this is where the error occurs.

So this is trying to add a float to a list.

Post by Chris Stinemetz

L = []
L += f

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'float' object is not iterable

look familiar?

Now, what I don't know, is what you are trying to do.
Are you trying to append the float to the list?
Or to replace the list with the float?
Or to add the float to the value of the first(or last?)
element in the list - if it exists
(and if it doesn't? Then what?)
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Chris Stinemetz]

Post by Alan Gauld

Post by Chris Stinemetz
Resetting execution engine
Running
C:\Users\cs062x\Desktop\python\projects\PanHandle\PanHandle\PanHandle.py
The Python REPL process has exited

That's slightly unusual. How are you running this?

I am running it with Microsoft Visual Studio Community 2013 using
Python Tools for Visual Studio

Post by Alan Gauld
Now, what I don't know, is what you are trying to do.
Are you trying to append the float to the list?
Or to replace the list with the float?
Or to add the float to the value of the first(or last?)
element in the list - if it exists
(and if it doesn't? Then what?)

Although I am certain it is not very efficient I was able to
accomplish what I wanted with the following code I wrote:

import os
import pprint
import csv
from collections import defaultdict

print_map = {'MOU':0, 'Call_Att':1, 'Device':2}
header = ['IMEI','MOUs','Call_Att','Device']

path = 'C:/Users/cs062x/Desktop/Panhandle'

os.chdir(path)
running_MOU = {}
call_attempts = {}
d = defaultdict(list)
for fname in os.listdir('.'):
with open (fname) as csvfile:
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
for row in spamreader:

if row[8]:
device = row[36]
Elapsed_Mins = float(row[7])
IMEI = row[8].replace("'", "")

if IMEI in running_MOU.keys():
running_MOU[IMEI] += Elapsed_Mins
else:
running_MOU[IMEI] = Elapsed_Mins

if IMEI in call_attempts.keys():
call_attempts[IMEI] += 1
else:
call_attempts[IMEI] = 1

# if key matches append mou else append 0.
d[IMEI] = [running_MOU[IMEI]]
d[IMEI].append([call_attempts[IMEI]])
d[IMEI].append([device])


print ",".join(header)
for k,v in sorted(d.items()):
print k, ",", d[k][print_map['MOU']],",",
d[k][print_map['Call_Att']][0],",", d[k][print_map['Device']][0]

print "complete"
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Peter Otten]

Post by Chris Stinemetz
Although I am certain it is not very efficient I was able to
import os
import pprint
import csv
from collections import defaultdict
print_map = {'MOU':0, 'Call_Att':1, 'Device':2}
header = ['IMEI','MOUs','Call_Att','Device']
path = 'C:/Users/cs062x/Desktop/Panhandle'
os.chdir(path)
running_MOU = {}
call_attempts = {}
d = defaultdict(list)
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
device = row[36]
Elapsed_Mins = float(row[7])
IMEI = row[8].replace("'", "")

For big dicts in Python 2 the test

key in some_dict.keys()

is indeed very inefficient as it builds a list of keys first and then
performs a linear scan for the key. Much better:

key in some_dict

This test avoids building the list and can also use an efficient lookup
algorithm that is independent of the size of the dict.

Post by Chris Stinemetz
running_MOU[IMEI] += Elapsed_Mins
running_MOU[IMEI] = Elapsed_Mins
call_attempts[IMEI] += 1
call_attempts[IMEI] = 1
# if key matches append mou else append 0.
d[IMEI] = [running_MOU[IMEI]]
d[IMEI].append([call_attempts[IMEI]])
d[IMEI].append([device])
print ",".join(header)
print k, ",", d[k][print_map['MOU']],",",
d[k][print_map['Call_Att']][0],",", d[k][print_map['Device']][0]
print "complete"

Here's an alternative that uses only one dict:

import csv
import os
import sys

header = ['IMEI', 'MOUs', 'Call_Att', 'Device']

path = 'C:/Users/cs062x/Desktop/Panhandle'

d = {}
for fname in os.listdir(path):
with open(os.path.join(path, fname)) as csvfile:
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
for row in spamreader:
if row[8]:
device = row[36]
elapsed_mins = float(row[7])
IMEI = row[8].replace("'", "")

if IMEI in d:
record = d[IMEI]
record[1] += elapsed_mins
record[2] += 1
else:
d[IMEI] = [IMEI, elapsed_mins, 1, device]

writer = csv.writer(sys.stdout)
writer.writerow(header)
writer.writerows(sorted(d.itervalues()))

print "complete"


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Chris Stinemetz]

Post by Peter Otten

Post by Chris Stinemetz
Although I am certain it is not very efficient I was able to
import os
import pprint
import csv
from collections import defaultdict
print_map = {'MOU':0, 'Call_Att':1, 'Device':2}
header = ['IMEI','MOUs','Call_Att','Device']
path = 'C:/Users/cs062x/Desktop/Panhandle'
os.chdir(path)
running_MOU = {}
call_attempts = {}
d = defaultdict(list)
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
device = row[36]
Elapsed_Mins = float(row[7])
IMEI = row[8].replace("'", "")

For big dicts in Python 2 the test
key in some_dict.keys()
is indeed very inefficient as it builds a list of keys first and then
key in some_dict
This test avoids building the list and can also use an efficient lookup
algorithm that is independent of the size of the dict.

Post by Chris Stinemetz
running_MOU[IMEI] += Elapsed_Mins
running_MOU[IMEI] = Elapsed_Mins
call_attempts[IMEI] += 1
call_attempts[IMEI] = 1
# if key matches append mou else append 0.
d[IMEI] = [running_MOU[IMEI]]
d[IMEI].append([call_attempts[IMEI]])
d[IMEI].append([device])
print ",".join(header)
print k, ",", d[k][print_map['MOU']],",",
d[k][print_map['Call_Att']][0],",", d[k][print_map['Device']][0]
print "complete"

import csv
import os
import sys
header = ['IMEI', 'MOUs', 'Call_Att', 'Device']
path = 'C:/Users/cs062x/Desktop/Panhandle'
d = {}
spamreader = csv.reader(csvfile, delimiter=',', quotechar='|')
next(spamreader)
device = row[36]
elapsed_mins = float(row[7])
IMEI = row[8].replace("'", "")
record = d[IMEI]
record[1] += elapsed_mins
record[2] += 1
d[IMEI] = [IMEI, elapsed_mins, 1, device]
writer = csv.writer(sys.stdout)
writer.writerow(header)
writer.writerows(sorted(d.itervalues()))
print "complete"

Peter - Thank you for showing me how to do this with one dictionary
and a more efficient method to lookup dictionary keys. I originally
attempted to accomplish this by using one dictionary but could not
find a good example that is why I used the defaultdict module. Your
approach sped the parsing time up from about 3 minutes to about 15
seconds! Very cool.

Thanks,

Chris
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor