[Tutor] Setting PYTHONPATH and other env vars dynamically

[Jramak]

Hello
We have developed three custom applications in Python. Each one of
these applications needs a different PYTHONPATH, in addition to
different environment variables to work. Instead of manually setting
the environment variables for each application, what would be the best
way to set PYTHONPATH and other environment variables for a specific
application? We only run one application at a time, not all of them.
We are running Python 2.5.2 and Python 2.4.1 on Win2K. Is a bash
script that sets the environment variables on the application start-up
way to go?

Any ideas? I was not aware of site.py until a co-worker bandied it
about - he says site.py is better than PYTHONPATH.

Ideas and recommendations welcome..
Regards
Jramak

[Kent Johnson]

Post by Jramak
Hello
We have developed three custom applications in Python. Each one of
these applications needs a different PYTHONPATH, in addition to
different environment variables to work. Instead of manually setting
the environment variables for each application, what would be the best
way to set PYTHONPATH and other environment variables for a specific
application? We only run one application at a time, not all of them.
We are running Python 2.5.2 and Python 2.4.1 on Win2K. Is a bash
script that sets the environment variables on the application start-up
 way to go?

You can modify the search path in each program my adding to sys.path.

Kent

[Douglas Philips]

Post by Jramak
Hello
We have developed three custom applications in Python. Each one of
these applications needs a different PYTHONPATH, in addition to
different environment variables to work. Instead of manually setting
the environment variables for each application, what would be the best
way to set PYTHONPATH and other environment variables for a specific
application? We only run one application at a time, not all of them.
We are running Python 2.5.2 and Python 2.4.1 on Win2K. Is a bash
script that sets the environment variables on the application start-up
way to go?
Any ideas? I was not aware of site.py until a co-worker bandied it
about - he says site.py is better than PYTHONPATH.

Ok, well, if you are on Windows and your talking about bash scripts,
it sounds like you might be using cygwin? Typically "scripts" on
Windows are .BAT files, so I am not sure what you're referring to here.
I think you have five-ish general options:

a) set variables by hand. run program(s) by hand. (not very much fun,
and possibly error prone.)

b) create scripts (bash or .BAT or whatever, depending on your
situation) that set the variables and runs your program(s).

c) install whatever your programs need under the site-packages
directory of your python install.
This assumes that all the packages your programs need have unique
names. I can't tell from your description if they are different
pacakges or if they might just be different versions of the same
package. If they are just different versions of the same package,
probably this option wouldn't work, it all depends on the specific
files. It also depends on where you have Python 2.5.2. and Python
2.4.1 installed and if you use them at the same time. This option
still requires another option for setting the other environment
variables.

d) Set up your environments in each of your main programs. Here is the
boilerplate you might use:
# This needs to be at the top of the program after the module doc
string (if any) and before any other statements or imports:

import sys
sys.path.append("dir1/dir2") # Add dir1/dir2 to your Python search
path. Can be absolute path or relative.
sys.path.append("dir3/dir4")
# ...

import os
os.environ['MYVARIABLE'] = 'MyVALUE'
os.environ['ANOTHERONE'] = 'someothervalue'
# ...

This has the nice property that everything is self contained. But that
also can be a downside if the values need to be changed by someone
else, they'd have to go and edit .py files.

e) option d and a (seems unlikely), option d and b, option d and c.
Depending on what you variables are, and how often they change, it
might make sense to put some of them (Python path or environment) into
the programs, and the rest in scripts.

Hope this helps,
--Doug

[Kent Johnson]

One more - put required packages and modules into the same directory
as the executable, usually the current directory is on the Python
search path.

Kent

[Jramak]

Post by Kent Johnson
One more - put required packages and modules into the same directory
as the executable, usually the current directory is on the Python
search path.
Kent

Hi Kent, there is no executable. It is just a collection of python scripts.
I assume you refer to the Python search path as PYTHONPATH or sys.path. Do
correct me if I am wrong.

Jramak

[Kent Johnson]

Post by Jramak

Post by Kent Johnson
One more - put required packages and modules into the same directory
as the executable, usually the current directory is on the Python
search path.
Kent

Hi Kent, there is no executable. It is just a collection of python scripts.
I assume you refer to the Python search path as PYTHONPATH or sys.path. Do
correct me if I am wrong.

Sorry, I mean put the packages in the same directory as the python
script that needs them. The search path is sys.path which is affected
by PYTHONPATH among other things...

Kent

[Jramak]

Post by Kent Johnson
Sorry, I mean put the packages in the same directory as the python
script that needs them. The search path is sys.path which is affected
by PYTHONPATH among other things...

sys.path is affected by PYTHONPATH ? how could it be affected by the
PYTHONPATH ?

Could you pls enlighten me ?

Jramak

[Kent Johnson]

Post by Jramak

Post by Kent Johnson
Sorry, I mean put the packages in the same directory as the python
script that needs them. The search path is sys.path which is affected
by PYTHONPATH among other things...

sys.path is affected by PYTHONPATH ? how could it be affected by the
PYTHONPATH ?

sys.path is the actual, runtime module search path. When Python is
started, it initialized sys.path from a variety of sources including
predefined locations, the contents of PYTHONPATH, and any .pth files
found.

http://docs.python.org/tutorial/modules.html#the-module-search-path

Kent

[Alan Gauld]

Post by Jramak
We have developed three custom applications in Python. Each one of
these applications needs a different PYTHONPATH, in addition to
different environment variables to work.

Environment variables should control the users (or oprocess) environment
and as such should really be fairly static, you could cause real problems
if you start changing the users environment dynamically because that might
affect how other applications run that also rely on PYTHONPATH or whatever.

Now, its true that each process gets a copy of the shell environment
so changing it within the process shouldn't affect the external environment
but to me it is still the wrong way to go about it. At the very least it
means
your environment variables no longer accurately reflect the users
environment
so you might get surprises if you try and launch a shell process or batch
job
which picks up the original environment!

In this case I'd much rather change sys.path. That is a Python list and
much
easier to add items at startup rather than mess with the environment.

Post by Jramak
the environment variables for each application, what would be the best
way to set PYTHONPATH and other environment variables for a specific
application?

Use config values in an ini file and set global variables in your program
at startup, and for paths use sys.path. (So far as I can tell thats what
Python does with PYTHONPATYH - it loads the values into sys.path
at startup...)

Post by Jramak
We only run one application at a time, not all of them.
We are running Python 2.5.2 and Python 2.4.1 on Win2K. Is a bash
script that sets the environment variables on the application start-up
way to go?

That is only slightly better since the environment no longer reflects the
users normal environment. It does mean the bash script could launch
multiple programs etc andd they would at lest share a common environment
but in your case I don't think its the best solution.

Post by Jramak
Any ideas? I was not aware of site.py until a co-worker bandied it
about - he says site.py is better than PYTHONPATH.

They do different things! site.py is about customising Python on that
site - which may be a shared installation - and environment variables
are about describing a users environment, and can be different for
each user. Now, there is an overlap, but Pythonpath is not one of them
since multiple users can share a site! (site.py is probably better
than setting a System wide PYTHONPATH though....) But again
site.py applies to every application you should not start modifying
it with application specific startup code - yuck! It could soon turn
into spaghetti and seriously slow down startup of everything.

HTH,

[Jramak]

Thanks for all the insight, everyone.

I was originally thinking of writing a bash or bat script that would set the
PYTHONPATH and other environment variables before running the specific
application code. It seemed to be the most simplest solution. There are no
other applications that rely on the PYTHONPATH. As Alan said this might
result in surprises

So a combo solution where in the env vars would be in an .ini file and the
path is set in sys.path would be, IMHO a good way to go.. it's simple. I
also looked at virtualenv http://pypi.python.org/pypi/virtualenv . Each
virtual env takes up 2.3 MB. To me this seems a bit of overkill, but this
will enable you to run Python24, Python25, Python26 etc specific code on one
computer.
Jramak

[Kent Johnson]

I
also looked at virtualenv http://pypi.python.org/pypi/virtualenv . Each
virtual env takes up 2.3 MB. To me this seems a bit of overkill, but this
will enable you to run Python24, Python25, Python26 etc specific code on one
computer.

You don't need virtualenv to run different versions of Python on one
computer, just specify which interpreter you want to use to run a
script.

By default virtualenv copies your site-packages directory to each
virtual environment, that is probably the bulk of the 2.3 MB. Disable
the copy with the --no-site-packages option. Of course you will then
have to install any needed modules yourself.
http://pypi.python.org/pypi/virtualenv#the-no-site-packages-option

Kent

[Kent Johnson]

Post by Jramak
Hello
We have developed three custom applications in Python. Each one of
these applications needs a different PYTHONPATH, in addition to
different environment variables to work. Instead of manually setting
the environment variables for each application, what would be the best
way to set PYTHONPATH and other environment variables for a specific
application? We only run one application at a time, not all of them.
We are running Python 2.5.2 and Python 2.4.1 on Win2K. Is a bash
script that sets the environment variables on the application start-up
 way to go?
Any ideas? I was not aware of site.py until a co-worker bandied it
about - he says site.py is better than PYTHONPATH.

Don't edit site.py, it is part of the std lib. If you do want to
customize it, add a site-customize.py in your site-packages folder and
put your customizations there.

Kent

[Jramak]

Post by Kent Johnson
Don't edit site.py, it is part of the std lib. If you do want to
customize it, add a site-customize.py in your site-packages folder and
put your customizations there.
Kent

There is no site-customize.py in Python25, how does Python know that
site-customize.py needs to be loaded on start-up? Or did you mean
sitecustomize.py ?

Jramak

[Jramak]

Post by Jramak

Post by Kent Johnson
Don't edit site.py, it is part of the std lib. If you do want to
customize it, add a site-customize.py in your site-packages folder and
put your customizations there.
There is no site-customize.py in Python25, how does Python know that

site-customize.py needs to be loaded on start-up? Or did you mean
sitecustomize.py ?
Jramak

[Kent Johnson]

Post by Jramak

Post by Kent Johnson
Don't edit site.py, it is part of the std lib. If you do want to
customize it, add a site-customize.py in your site-packages folder and
put your customizations there.

There is no site-customize.py in Python25, how does Python know that
site-customize.py needs to be loaded on start-up? Or did you mean
sitecustomize.py ?

Yes, sitecustomize.py.

Kent