[Tutor] iterating through a directory

[richard kappler]

Yes, many questions today. I'm working on a data feed script that feeds
'events' into our test environment. In production, we monitor a camera that
captures an image as product passes by, gathers information such as
barcodes and package ID from the image, and then sends out the data as a
line of xml to one place for further processing and sends the image to
another place for storage. Here is where our software takes over, receiving
the xml data and images from vendor equipment. Our software then processes
the xml data and allows retrieval of specific images associated with each
line of xml data. Our test environment must simulate the feed from the
vendor equipment, so I'm writing a script that feeds the xml from an actual
log to one place for processing, and pulls images from a pool of 20, which
we recycle, to associate with each event and sends them to another dir for
saving and searching.

As my script iterates through each line of xml data (discussed yesterday)
to simulate the feed from the vendor camera equipment, it parses ID
information about the event then sends the line of data on. As it does so,
it needs to pull the next image in line from the image pool directory,
rename it, send it to a different directory for saving. I'm pretty solid on
all of this except iterating through the image pool. My idea is to just
keep looping through the image pool, as each line of xmldata is parsed, the
next image in line gets pulled out, renamed with the identifying
information from the xml data, and both are sent on to different places.

I only have one idea for doing this iterating through the image pool, and
frankly I think the idea is pretty weak. The only thing I can come up with
is to change the name of each image in the pool from something
like 0219PS01CT1_20111129_000004_00044979.jpg to 1.jpg, 2.jpg, 3.jpg etc.,
then doing something like:

i = 0

here begins my for line in file loop:
if i == 20:
i = 1
else:
i += 1
do stuff with the xml file including get ID info
rename i.jpg to IDinfo.jpg
send it all on

That seems pretty barbaric, any thoughts on where to look for better ideas?
I'm presuming there are modules out there that I am unaware of or
capabilities I am unaware of in modules I do know a little about that I am
missing.

[Albert-Jan Roskam]

Date: Wed, 9 Sep 2015 09:32:34 -0400
Subject: [Tutor] iterating through a directory
Yes, many questions today. I'm working on a data feed script that feeds
'events' into our test environment. In production, we monitor a camera that
captures an image as product passes by, gathers information such as
barcodes and package ID from the image, and then sends out the data as a
line of xml to one place for further processing and sends the image to
another place for storage. Here is where our software takes over, receiving
the xml data and images from vendor equipment. Our software then processes
the xml data and allows retrieval of specific images associated with each
line of xml data. Our test environment must simulate the feed from the
vendor equipment, so I'm writing a script that feeds the xml from an actual
log to one place for processing, and pulls images from a pool of 20, which
we recycle, to associate with each event and sends them to another dir for
saving and searching.
As my script iterates through each line of xml data (discussed yesterday)
to simulate the feed from the vendor camera equipment, it parses ID
information about the event then sends the line of data on. As it does so,
it needs to pull the next image in line from the image pool directory,
rename it, send it to a different directory for saving. I'm pretty solid on
all of this except iterating through the image pool. My idea is to just
keep looping through the image pool, as each line of xmldata is parsed, the
next image in line gets pulled out, renamed with the identifying
information from the xml data, and both are sent on to different places.
I only have one idea for doing this iterating through the image pool, and
frankly I think the idea is pretty weak. The only thing I can come up with
is to change the name of each image in the pool from something
like 0219PS01CT1_20111129_000004_00044979.jpg to 1.jpg, 2.jpg, 3.jpg etc.,
i = 0
i = 1
i += 1
do stuff with the xml file including get ID info
rename i.jpg to IDinfo.jpg
send it all on
That seems pretty barbaric, any thoughts on where to look for better ideas?
I'm presuming there are modules out there that I am unaware of or
capabilities I am unaware of in modules I do know a little about that I am
missing.

I do not really understand what you intend to do, but the following modules might come in handy
-os (os.rename, os.listdir)
-glob (glob.iglob or glob.glob)
-shutil (shutil.copy)




_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[richard kappler]

Albert-Jan, thanks for the response. shutil.copyfile does seem to be one of
the tools I need to make the copying, renaming the copy and saving it
elsewhere in one line instead of three or more.

Still not sure how to efficiently get the script to keep moving to the next
file in the directory though, in other words, for each iteration in the
loop, I want it to fetch, rename and send/save the next image in line. Hope
that brings better understanding.

Thanks for the tip!

regards, Richard

Post by richard kappler

Date: Wed, 9 Sep 2015 09:32:34 -0400
Subject: [Tutor] iterating through a directory
Yes, many questions today. I'm working on a data feed script that feeds
'events' into our test environment. In production, we monitor a camera

that

captures an image as product passes by, gathers information such as
barcodes and package ID from the image, and then sends out the data as a
line of xml to one place for further processing and sends the image to
another place for storage. Here is where our software takes over,

receiving

the xml data and images from vendor equipment. Our software then

processes

the xml data and allows retrieval of specific images associated with each
line of xml data. Our test environment must simulate the feed from the
vendor equipment, so I'm writing a script that feeds the xml from an

actual

log to one place for processing, and pulls images from a pool of 20,

which

we recycle, to associate with each event and sends them to another dir

for

saving and searching.
As my script iterates through each line of xml data (discussed yesterday)
to simulate the feed from the vendor camera equipment, it parses ID
information about the event then sends the line of data on. As it does

so,

it needs to pull the next image in line from the image pool directory,
rename it, send it to a different directory for saving. I'm pretty solid

on

all of this except iterating through the image pool. My idea is to just
keep looping through the image pool, as each line of xmldata is parsed,

the

next image in line gets pulled out, renamed with the identifying
information from the xml data, and both are sent on to different places.
I only have one idea for doing this iterating through the image pool, and
frankly I think the idea is pretty weak. The only thing I can come up

with

is to change the name of each image in the pool from something
like 0219PS01CT1_20111129_000004_00044979.jpg to 1.jpg, 2.jpg, 3.jpg

etc.,

i = 0
i = 1
i += 1
do stuff with the xml file including get ID info
rename i.jpg to IDinfo.jpg
send it all on
That seems pretty barbaric, any thoughts on where to look for better

ideas?

I'm presuming there are modules out there that I am unaware of or
capabilities I am unaware of in modules I do know a little about that I

am

missing.

I do not really understand what you intend to do, but the following
modules might come in handy
-os (os.rename, os.listdir)
-glob (glob.iglob or glob.glob)
-shutil (shutil.copy)

[Alan Gauld]

Post by richard kappler
Still not sure how to efficiently get the script to keep moving to the next
file in the directory though, in other words, for each iteration in the
loop, I want it to fetch, rename and send/save the next image in line. Hope
that brings better understanding.

Sounds like you want a circular list.

The traditional way to generate a circular
index into a list is to use the modulo (%) operator

But the itertools module gives you a better option with
the cycle function:

import itertools as it

for img in it.cycle(os.listdir(my_img_path)):
process(img)

HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Mark Lawrence]

Post by richard kappler
Yes, many questions today. I'm working on a data feed script that feeds
'events' into our test environment. In production, we monitor a camera that
captures an image as product passes by, gathers information such as
barcodes and package ID from the image, and then sends out the data as a
line of xml to one place for further processing and sends the image to
another place for storage. Here is where our software takes over, receiving
the xml data and images from vendor equipment. Our software then processes
the xml data and allows retrieval of specific images associated with each
line of xml data. Our test environment must simulate the feed from the
vendor equipment, so I'm writing a script that feeds the xml from an actual
log to one place for processing, and pulls images from a pool of 20, which
we recycle, to associate with each event and sends them to another dir for
saving and searching.
As my script iterates through each line of xml data (discussed yesterday)
to simulate the feed from the vendor camera equipment, it parses ID
information about the event then sends the line of data on. As it does so,
it needs to pull the next image in line from the image pool directory,
rename it, send it to a different directory for saving. I'm pretty solid on
all of this except iterating through the image pool. My idea is to just
keep looping through the image pool, as each line of xmldata is parsed, the
next image in line gets pulled out, renamed with the identifying
information from the xml data, and both are sent on to different places.
I only have one idea for doing this iterating through the image pool, and
frankly I think the idea is pretty weak. The only thing I can come up with
is to change the name of each image in the pool from something
like 0219PS01CT1_20111129_000004_00044979.jpg to 1.jpg, 2.jpg, 3.jpg etc.,
i = 0
i = 1
i += 1
do stuff with the xml file including get ID info
rename i.jpg to IDinfo.jpg
send it all on
That seems pretty barbaric, any thoughts on where to look for better ideas?
I'm presuming there are modules out there that I am unaware of or
capabilities I am unaware of in modules I do know a little about that I am
missing.

I'm not sure what you're trying to do so maybe
https://docs.python.org/3/library/collections.html#collections.deque or
https://docs.python.org/3/library/itertools.html#itertools.cycle
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor