[Tutor] pip install in a virtualenv *without* internet?

[Albert-Jan Roskam]

Hi,


I use Python(x y) (Python 2.7) on Win7. I need a higher version of openpyxl, because pandas.Dataframe.to_excel yields an error. So pandas and its own dependencies (e.g. numpy) could remain in the python(x y) site-packages, I just need a higher version of openpyxl without disturbing the x,y installation (I do not even have rights to install stuff there!)

So I would like to pip install a openpyxl AND its specific dependencies in a virtualenv.
The problem is that I can't use pip to download the packages from Pypi because I do not have a regular internet connection. Is there a generic way to install a package and its (pre-downloaded) dependencies, a way that requires little or no modifications to the original package?
Using pip 'editable' might help: http://stackoverflow.com/questions/15031694/installing-python-packages-from-local-file-system-folder-with-pip. I am hoping requirements.txt might somehow be used to install the dependencies from a local location --but how?


Thanks!

Albert-Jan



_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Laura Creighton]

In a message of Tue, 18 Aug 2015 20:10:15 -0000, Albert-Jan Roskam writes:
>So I would like to pip install a openpyxl AND its specific dependencies in a virtualenv.
>The problem is that I can't use pip to download the packages from Pypi because I do not have a regular internet connection. Is there a generic way to install a package and its (pre-downloaded) dependencies, a way that requires little or no modifications to the original package?

The usual fix for this problem is to head to your local library or
coffee shop that has free wifi and do the job there.

Laura

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Mike C. Fletcher]

On 15-08-18 04:10 PM, Albert-Jan Roskam wrote:
> Hi,
>
>
> I use Python(x y) (Python 2.7) on Win7. I need a higher version of openpyxl, because pandas.Dataframe.to_excel yields an error. So pandas and its own dependencies (e.g. numpy) could remain in the python(x y) site-packages, I just need a higher version of openpyxl without disturbing the x,y installation (I do not even have rights to install stuff there!)
>
> So I would like to pip install a openpyxl AND its specific dependencies in a virtualenv.
> The problem is that I can't use pip to download the packages from Pypi because I do not have a regular internet connection. Is there a generic way to install a package and its (pre-downloaded) dependencies, a way that requires little or no modifications to the original package?
> Using pip 'editable' might help: http://stackoverflow.com/questions/15031694/installing-python-packages-from-local-file-system-folder-with-pip. I am hoping requirements.txt might somehow be used to install the dependencies from a local location --but how?

To install without going out to the internet, you can use these arguments:

pip install --no-index --find-links=/path/to/download/directory
<packages>

that *won't* work for git/svn/bzr linked (editable) packages, but should
work for pre-downloaded "released" packages. If you need the editable
packages, you'll need to pull the git/whatever repositories and modify
your requirements file to point to the local git repo. But you likely
could just do a "python setup.py develop" for them if you've got the
source downloaded anyway.

I often use this with a separate "download dependencies" stage that
populates the packages directory so that our build server doesn't hit
PyPi every time we do a rebuild of our virtualenvs (which we do for
every testing build).

HTH,
Mike

--
________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alex Kleider]

On 2015-08-18 19:32, Mike C. Fletcher wrote:

> To install without going out to the internet, you can use these
> arguments:
>
> pip install --no-index --find-links=/path/to/download/directory
> <packages>


For this to work, /path/to/download/directory would, I assume, first
have to be populated.
I further assume that running wget from within that directory might do
the trick.
Can you suggest the correct parameter(s) that need to be provided?
If anyone happens to know approximately how much file space would be
required, that would be helpful.
Thanks,
Alex
(using python 3.4, Linux- Ubuntu LTS (14.4))
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Albert-Jan Roskam]

> Date: Wed, 19 Aug 2015 02:27:41 -0700
> From: ***@sonic.net
> To: ***@python.org
> Subject: Re: [Tutor] pip install in a virtualenv *without* internet?
>
> On 2015-08-18 19:32, Mike C. Fletcher wrote:
>
> > To install without going out to the internet, you can use these
> > arguments:
> >
> > pip install --no-index --find-links=/path/to/download/directory
> > <packages>
>
>
> For this to work, /path/to/download/directory would, I assume, first
> have to be populated.
> I further assume that running wget from within that directory might do
> the trick.


..but, but wget requires an internet connection, which I do not have (at least not a normal one).
But if you do have internet I think you could simply copy the URL-with-sha1, then for each package do
wget <url>

regards,
Albert-Jan


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alex Kleider]

On 2015-08-19 04:28, Albert-Jan Roskam wrote:
>> Date: Wed, 19 Aug 2015 02:27:41 -0700
>> From: ***@sonic.net
>> To: ***@python.org
>> Subject: Re: [Tutor] pip install in a virtualenv *without* internet?
>>
>> On 2015-08-18 19:32, Mike C. Fletcher wrote:
>>
>> > To install without going out to the internet, you can use these
>> > arguments:
>> >
>> > pip install --no-index --find-links=/path/to/download/directory
>> > <packages>
>>
>>
>> For this to work, /path/to/download/directory would, I assume, first
>> have to be populated.
>> I further assume that running wget from within that directory might do
>> the trick.
>
>
> ..but, but wget requires an internet connection, which I do not have
> (at least not a normal one).
> But if you do have internet I think you could simply copy the
> URL-with-sha1, then for each package do
> wget <url>
>
> regards,
> Albert-Jan


I guess if you 'never' have an internet connection what I'm trying to do
won't work,
but I'm addressing a different use case: I have connectivity in some
environments
but would like to be able to do a pip install at times when there is no
connectivity.
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Marc Tompkins]

On Wed, Aug 19, 2015 at 9:18 AM, Alex Kleider <***@sonic.net> wrote:

> I guess if you 'never' have an internet connection what I'm trying to do
> won't work,
> but I'm addressing a different use case: I have connectivity in some
> environments
> but would like to be able to do a pip install at times when there is no
> connectivity.


I'm wondering: does the solution absolutely have to involve pip? I ask
because I first started with Python right about the time pip was being
created, and I didn't actually start using it until about a year ago Prior
to that, I downloaded my dependencies on my development machine, saved them
to a flash drive, and wrote a script (well, technically a batch file - most
of my clients use Windows) to automate offline installations.

pip is certainly more convenient, and I'm quite grateful to its developers
- but it's a relatively recent solution to the problem, and it's far from
the only way to do things.
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Albert-Jan Roskam]

> Date: Wed, 19 Aug 2015 09:49:43 -0700
> From: ***@gmail.com
> To: ***@python.org
> Subject: Re: [Tutor] pip install in a virtualenv *without* internet?
>
> On Wed, Aug 19, 2015 at 9:18 AM, Alex Kleider <***@sonic.net> wrote:
>
> > I guess if you 'never' have an internet connection what I'm trying to do
> > won't work,
> > but I'm addressing a different use case: I have connectivity in some
> > environments
> > but would like to be able to do a pip install at times when there is no
> > connectivity.
>
>
> I'm wondering: does the solution absolutely have to involve pip? I ask
> because I first started with Python right about the time pip was being
> created, and I didn't actually start using it until about a year ago Prior
> to that, I downloaded my dependencies on my development machine, saved them
> to a flash drive, and wrote a script (well, technically a batch file - most
> of my clients use Windows) to automate offline installations.

The goal is most important: the installation should "just work", so "python setup.py install --user" for everything that is needed (in a .bat) might also work.

Btw, today I found out about the pip option "--target" that allows you to install to an alternative path. Handy in case you (like me) don't necessarily have write access to site-packages. You do need to prepend it to PYTHONPATH. That's nicer than prepending to sys.path, IMHO.


> pip is certainly more convenient, and I'm quite grateful to its developers
> - but it's a relatively recent solution to the problem, and it's far from
> the only way to do things.

I agree, but there could also be too many options (do we still need easy_install?). As if the ideal situation is yet to come. I played a bit with conda install and it seems *very* convenient. Like a combination of setuptools, pip, pythonbrew and virtualenv/wrapper.




_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Mike C. Fletcher]

On 15-08-19 05:27 AM, Alex Kleider wrote:
> On 2015-08-18 19:32, Mike C. Fletcher wrote:
>
>> To install without going out to the internet, you can use these
>> arguments:
>>
>> pip install --no-index --find-links=/path/to/download/directory
>> <packages>
>
>
> For this to work, /path/to/download/directory would, I assume, first
> have to be populated.
> I further assume that running wget from within that directory might do
> the trick.
> Can you suggest the correct parameter(s) that need to be provided?
> If anyone happens to know approximately how much file space would be
> required, that would be helpful.

I'm not sure what packages you are trying to install, so can't answer
the space question, but the easiest command to populate the directory is
pip on the internet-connected machine:

pip install --download=~/packages <packages>

now copy that directory onto your USB key (or whatever) and take it to
the offline machine. If you're planning to do a *lot* of installations,
you can also use (once you install the wheel package):

pip wheel --no-index --find-links=/path/to/download <packages>

to create fast-installing wheels from each of the dependencies (do that
on the target machine so that all libraries match).

HTH,
Mike

--
________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Albert-Jan Roskam]

Sorry, now with Reply All

From: ***@hotmail.com
To: ***@vrplumber.com
Subject: RE: [Tutor] pip install in a virtualenv *without* internet?
Date: Wed, 19 Aug 2015 11:25:49 +0000




> Date: Tue, 18 Aug 2015 22:32:28 -0400
> From: ***@vrplumber.com
> To: ***@python.org
> Subject: Re: [Tutor] pip install in a virtualenv *without* internet?
>
> On 15-08-18 04:10 PM, Albert-Jan Roskam wrote:
> > Hi,
> >
> >
> > I use Python(x y) (Python 2.7) on Win7. I need a higher version of openpyxl, because pandas.Dataframe.to_excel yields an error. So pandas and its own dependencies (e.g. numpy) could remain in the python(x y) site-packages, I just need a higher version of openpyxl without disturbing the x,y installation (I do not even have rights to install stuff there!)
> >
> > So I would like to pip install a openpyxl AND its specific dependencies in a virtualenv.
> > The problem is that I can't use pip to download the packages from Pypi because I do not have a regular internet connection. Is there a generic way to install a package and its (pre-downloaded) dependencies, a way that requires little or no modifications to the original package?
> > Using pip 'editable' might help: http://stackoverflow.com/questions/15031694/installing-python-packages-from-local-file-system-folder-with-pip. I am hoping requirements.txt might somehow be used to install the dependencies from a local location --but how?
>
> To install without going out to the internet, you can use these arguments:
>
> pip install --no-index --find-links=/path/to/download/directory
> <packages>
>
> that *won't* work for git/svn/bzr linked (editable) packages, but should
> work for pre-downloaded "released" packages. If you need the editable
> packages, you'll need to pull the git/whatever repositories and modify
> your requirements file to point to the local git repo. But you likely
> could just do a "python setup.py develop" for them if you've got the
> source downloaded anyway.
>
> I often use this with a separate "download dependencies" stage that
> populates the packages directory so that our build server doesn't hit
> PyPi every time we do a rebuild of our virtualenvs (which we do for
> every testing build).

Hi Mike,

Thank you so much! This looks very useful indeed. In fact, it is strange that pip does not cache packages by default (or does it?),
similar to apt-get. I have often been amazed by the number of downloads of some packages. Even with very popular packages, many thousands of downloads a day is probably mostly the result of build servers that re-download from Pypi with each and every push/commit. I always pin the exact version in requirements.txt, ie. I use pkg=1.0.1, not pkg>=1.0.1, so I really only use one version.

Best wishes,
Albert-Jan



_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor