Tool support for private modules

> That's where the distutils and setuptools place them by default,
> yes. I don't know what magic is required to put them elsewhere; that
> may be part of the answer, if someone can instruct me on how to do it.

You shoul dupload your work somewhere, "teaching" you is almost
impossible if one can't see what's wrong.

--
Bernd Zeimetz
<bernd.TakeThisOut@bzed.de> <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST.TakeThisOut@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.TakeThisOut@lists.debian.org

Bernd Zeimetz <bernd.TakeThisOut@bzed.de> writes:

> You shoul dupload your work somewhere, "teaching" you is almost
> impossible if one can't see what's wrong.

I'm not presenting something as "wrong". I'm asking for guidance on
how to do things right.

If the policy recommends that packages be set up a particular way
("put package-specific modules in '/usr/share/package/'"), but the
standard tools behave differently ("put all modules by default in
'/usr/lib/pythonX.Y/site-packages/'"), then there's a step that needs
to be taken to get from the default behaviour to the behaviour
recommended by policy.

I'm asking how to take that step, in a way that isn't brute-force
"manually hack each package to conform to policy".

--
\ "Dvorak users of the world flgkd!" —Kirsten Chevalier, |
`\ rec.humor.oracle.d |
_o__) |
Ben Finney


--
To UNSUBSCRIBE, email to debian-python-REQUEST.TakeThisOut@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.TakeThisOut@lists.debian.org

> If the policy recommends that packages be set up a particular way
> ("put package-specific modules in '/usr/share/package/'"), but the
> standard tools behave differently ("put all modules by default in
> '/usr/lib/pythonX.Y/site-packages/'"), then there's a step that needs
> to be taken to get from the default behaviour to the behaviour
> recommended by policy.

The standard tools Do it right. So there must be a bug somewhere in your
packaging or upstream's source. Unfortunately this bug is not shown in
my fishbowl.

You can read trough http://wiki.debian.org/DebianPython/NewPolicy and
check if your packaging contains all the pieces mentioned there. If you
think it does, show your code.

--
Bernd Zeimetz
<bernd RemoveThis @bzed.de> <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

Josselin Mouette <joss DeleteThis @debian.org> writes:

> Le lundi 01 octobre 2007 à 18:37 +1000, Ben Finney a écrit :
> > How can I use the tools available — distutils, setuptools,
> > debhelper — to install these package-specific modules to a
> > package-specific location, such that all the programs in the
> > package will be able to find them?
>
> The easiest way, if the modules are relocatable (99% of them are) is
> to simply move them after installation.
>
> Otherwise, you can pass specific arguments to setup.py. That would
> be, python setup.py install --home=/usr/share/$package
> --install-purelib=.
>
> More information:
> http://www.python.org/doc/2.4/inst/search-path.html

Thanks, this is a useful pointer. I wasn't aware such fine-grained
control was available over locations with distutils. (I've certainly
never seen any Python author using that control.)

> > Hmm. I am hoping that "modify the programs [to add an absolute
> > path to the search path]" is not a necessary part of this.
>
> If upstream hasn't thought of it, it is. You only need to add one
> line in the program, before the module is imported.

How can upstream anticipate the arbitrary path I pass to './setup.py
install --home="/foo/bar"' if the path "/foo/bar" is up to me as a
Debian packager? I don't see how any Python program can be written to
allow for that without modification every time the "/foo/bar" changes.

--
\ "Everything you read in newspapers is absolutely true, except |
`\ for that rare story of which you happen to have first-hand |
_o__) knowledge." -- Erwin Knoll |
Ben Finney


--
To UNSUBSCRIBE, email to debian-python-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Ben Finney <ben RemoveThis @benfinney.id.au> writes:

> How can I best conform to the [Debian policy for Python modules
> specific to a single package]?

As an example, here's a Python package I'm trying to get packaged for
Debian. (I am the upstream author of this one, but I'm interested in a
solution that *doesn't* involve significant changes to the upstream
code.)

<URL:http://cheeseshop.python.org/pypi/gracie/>

How should I modify 'setup.py' to allow binaries and modules for this
package to be installed properly, and have the programs continue to
find the modules?

How should I construct the command line for 'setup.py' when I create
the 'install-python%' target in 'debian/rules' for this package?

--
\ Eccles: "I just saw the Earth through the clouds!" Lew: "Did |
`\ it look round?" Eccles: "Yes, but I don't think it saw me." |
_o__) -- The Goon Show, _Wings Over Dagenham_ |
Ben Finney


--
To UNSUBSCRIBE, email to debian-python-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

> As an example, here's a Python package I'm trying to get packaged for
> Debian. (I am the upstream author of this one, but I'm interested in a
> solution that *doesn't* involve significant changes to the upstream
> code.)
>
> <URL:http://cheeseshop.python.org/pypi/gracie/>

The first thing I'd do here is to patch the ez_ stuff away, together
with setuptools. ez_... is known to break things badly (like trying to
download things on buildds and other really broken things). Setuptools
is broken by design imho (see a thread some time ago about this), but it
may work. But as I run into trouble with it too often (like missing
__init__.py files in random directories), I replace it by distutils.
Since we're not building eggs there's not need for setuptools at all
(afaik distutils is able to build eggs now, too - at least the egg info
files, which is enough for us).
Better to patch those things before getting FTBFS reports form the buildds.

Although I didn't test it, there should be no special thign to do with
your setup.py while building a package, python setup.py build/install
--root=.... should do the job (with and without ez_... and setuptools).

http://svn.debian.org/wsvn/python-modules/packages/python-contract/tru...debian/
is a pretty simple debian/rules file, which also tries to build the
package for all python versions and runs the tests to ensure nothing is
buggy.
If you like it really simple, cdbs is something you can use:
http://svn.debian.org/wsvn/python-modules/packages/ll-core/trunk/debia...ules?op

Aa you can see on the cdbs example, sometimes upstream does not install
all files in the way one would like it to have in debian, often out of
personal taste - that's easy to fix though. Although at least I
appreciate if upstras isntalls everything at the proper place, at least
distutils supports installing completely non-py related files into the
right directories.

Depending on what you like you have to call dh_py{central,support} to
move your installed files in their directories, of course you need to
add the neccessary informations for them at the apropriate places (like
which python versions you want to support).

> How should I construct the command line for 'setup.py' when I create
> the 'install-python%' target in 'debian/rules' for this package?

Have a look at the first example.
Both will work for arch: all and arch: any packages. For arch: all
packages you don;t need to build/install with all python versions, just
using the default one is enough, everythign else will be handled by
python-support/central while installing the package. It makes sense to
build them all, though, especially when you want to run tests.


Hope that helps,

Bernd
--
Bernd Zeimetz
<bernd.TakeThisOut@bzed.de> <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST.TakeThisOut@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.TakeThisOut@lists.debian.org

(Please preserve attribution lines on quoted material. I don't know
who wrote what in the following.)

Josselin Mouette <joss.TakeThisOut@debian.org> writes:

> Le lundi 01 octobre 2007 à 23:56 +1000, Ben Finney a écrit :
> > > > Hmm. I am hoping that "modify the programs [to add an absolute
> > > > path to the search path]" is not a necessary part of this.
> > >
> > > If upstream hasn't thought of it, it is. You only need to add
> > > one line in the program, before the module is imported.
> >
> > [...] I don't see how any Python program can be written to allow
> > for that without modification every time the "/foo/bar" changes.
>
> Distutils may be extended to do such a thing, but most of the
> software I've seen doing it is using automake or changing the files
> with custom hackery.

I'm confused, then. How does this fit with the implication (in the
above quoted teset) that upstream should have "thought of [changing
the location of the modules]"? If downstream hackery is required, I
don't see what upstream is expected to have done.

--
\ "The most merciful thing in the world... is the inability of |
`\ the human mind to correlate all its contents." -- Howard |
_o__) Philips Lovecraft |
Ben Finney


--
To UNSUBSCRIBE, email to debian-python-REQUEST.TakeThisOut@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.TakeThisOut@lists.debian.org

>
> I'm confused, then. How does this fit with the implication (in the
> above quoted teset) that upstream should have "thought of [changing
> the location of the modules]"? If downstream hackery is required, I
> don't see what upstream is expected to have done.
>
>

That is imho nothing upstream needs to take care of. PYTHONPATH needs to
include the directory where the modules are installed, and that's
something downstream needs to take care of.

If you 'import yourmodule' in your program, Python will know how to find.

http://docs.python.org/inst/inst.html is probably interesting to read
for you.

--
Bernd Zeimetz
<bernd RemoveThis @bzed.de> <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

Bernd, please follow the Debian mailing list code of conduct
<URL:http://www.debian.org/MailingLists#codeofconduct>, in particular
by *not* sending personal copies of messages that are also sent to the
list.

Also, please preserve attribution lines so we can keep track of who
wrote what quoted material.

Bernd Zeimetz <bernd.DeleteThis@bzed.de> writes:

> > As an example, here's a Python package I'm trying to get packaged
> > for Debian. [...]
> > <URL:http://cheeseshop.python.org/pypi/gracie/>
>
> The first thing I'd do here is to patch the ez_ stuff away, together
> with setuptools.

I presume you mean the 'ez_setup.py' module (there's no other 'ez_*'
files in that package).

> ez_... is known to break things badly (like trying to download
> things on buildds and other really broken things). Setuptools is
> broken by design imho (see a thread some time ago about this), but
> it may work. But as I run into trouble with it too often (like
> missing __init__.py files in random directories), I replace it by
> distutils.

The main reason I use distutils is to assist those people using
operating systems that *don't* have good package dependency
management, which seems to be the primary target market for
setuptools.

I also want my package listed properly at the Python Cheeseshop; this
is much easier using setuptools than distutils.

> Since we're not building eggs there's not need for setuptools at all
> (afaik distutils is able to build eggs now, too - at least the egg
> info files, which is enough for us). Better to patch those things
> before getting FTBFS reports form the buildds.

Okay. So you're suggesting that I should continue to maintain the
setuptools functionality upstream, but "patch it out" in the Debian
package of the same software?

I'm also unclear on what you mean by "replace it by distutils". What
does this mean for a package that already uses setuptools, and will
continue to do so upstream?

> Although I didn't test it, there should be no special thign to do with
> your setup.py while building a package, python setup.py build/install
> --root=.... should do the job (with and without ez_... and setuptools).

I'll address this in a separate thread, as it seems I'm not explaining
the problem very well.

--
\ "I bought some powdered water, but I don't know what to add." |
`\ -- Steven Wright |
_o__) |
Ben Finney


--
To UNSUBSCRIBE, email to debian-python-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Ben Finney <bignose+hates-spam@benfinney.id.au> writes:

> The main reason I use distutils is to assist those people using
> operating systems that *don't* have good package dependency
> management, which seems to be the primary target market for
> setuptools.

This should, of course, read "The main reason I use setuptools …"

--
\ "Pinky, are you pondering what I'm pondering?" "I think so, |
`\ Brain, but this time *you* put the trousers on the chimp." -- |
_o__) _Pinky and The Brain_ |
Ben Finney


--
To UNSUBSCRIBE, email to debian-python-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Josselin Mouette <joss RemoveThis @debian.org> writes:

> Le jeudi 11 octobre 2007 à 10:50 +1000, Ben Finney a écrit :
> > The main reason I use distutils is to assist those people using
> > operating systems that *don't* have good package dependency
> > management, which seems to be the primary target market for
> > setuptools.
>
> This is a laudable goal, but not when done at the expense of proper
> support of operating systems which have one.

Indeed. The rest of the message, which you chose not to address this
time, asks for help avoiding exactly that trap.

Care to answer some of the specific questions in that message and help
Python packagers improve their practices?

--
\ "Remember men, we're fighting for this woman's honour; which is |
`\ probably more than she ever did." -- Groucho Marx |
_o__) |
Ben Finney


--
To UNSUBSCRIBE, email to debian-python-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org