new package format

Eugene Gorodinsky <e.gorodinsky.RemoveThis@gmail.com> wrote:
> I've read the debian news announcement today
> (http://www.debian.org/News/2009/20090730). What got me very
> interested was the part about a new package format (in my oppinion
> this area can be vastly improved, and I'm interested in contributing).
> Searching the list archives I was unable to find any discussion
> relating to that, except for the multi-arch spec and a discussion that
> took place way back in 1999. Is there anything I might have missed?

http://wiki.debian.org/Projects/DebSrc3.0


--
To UNSUBSCRIBE, email to debian-devel-REQUEST.RemoveThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.RemoveThis@lists.debian.org

Eugene Gorodinsky <e.gorodinsky.TakeThisOut@gmail.com> wrote:
> On windows a program may contain some optional components, which you
> can choose at install time. This approach (I mean having some main
> package and some required and some optional subpackages inside it) is
> quite user-friendly. Neither dpkg nor apt have this functionality in
> them, and I do not think it's possible to implement it without
> changing the package format.

Recommends and Suggests do exactly that.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST.TakeThisOut@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.TakeThisOut@lists.debian.org

2009/7/31 Giacomo A. Catenazzi <cate.RemoveThis@debian.org>:
> Eugene Gorodinsky wrote:
>>
>> Hi all
>> I've read the debian news announcement today
>> (http://www.debian.org/News/2009/20090730). What got me very
>> interested was the part about a new package format
>
> There are two changes: one about the source package format
> (a true format change) and about binary package format
> (this is only a "formal" change about supported compressions
> algorithms, not real changes on functionality).
>
> Anyway these two format are already defined by long time
> (we need to have support of new format some releases
> before actual use, in order to provide smooth upgrades)
>
> So IMHO this is not the right period to discuss about a
> new format: we still have to use a "old new format".
>
>
>> (in my oppinion
>> this area can be vastly improved, and I'm interested in contributing).
>
> What are the problems of actual format?
>
For one the dependencies are specified as actual packages, rather than
the actual files themselves. Thus, if a user has installed some
library by `make install`, s/he cannot install a program packaged as a
deb, that relies on the library without some pain.

On windows a program may contain some optional components, which you
can choose at install time. This approach (I mean having some main
package and some required and some optional subpackages inside it) is
quite user-friendly. Neither dpkg nor apt have this functionality in
them, and I do not think it's possible to implement it without
changing the package format.

I also think some abstraction from the actual filesystem is a good
idea. For example currently the only way to install a lib in a
directory other than the one it was intended for is by using a hack
that would look at the directory of a file and move it somewhere. It
seems that with the current situation where you want to use
/lib/i386-linux-gnu tuples instead of the approach used before, would
be less painful if the current package format had some abstraction
from the filesystem. Since the programs don't usually care where the
library is, as long as it is in the LDD_LIBRARY_PATH.

The translations packages could be specifically marked as such, so
that a user could easily check if her package has translations for her
native language. The same applies for documentation, probably, which
could be made into an optional subpackage.

Currently debian policy is to have a .desktop file for each GUI
program. What would be better, IMHO, is having some sort of
abstraction, so that the package manager itself would create a
..desktop file entry, given an icon and some information about the
package.

Since programs usually store their settings in the user's home
directory, that aren't deleted when the program is uninstalled the
user's home directory becomes a mess. I'm not sure if it's possible to
change some functionality within dpkg without changing the format
itself.

>> Searching the list archives I was unable to find any discussion
>> relating to that, except for the multi-arch spec and a discussion that
>> took place way back in 1999. Is there anything I might have missed?
>
> Check the changelogs of dpkg to find when people discussed it.
>
> Anyway there were not real change since a lot of time. In last
> 10 years IIRC there was some support to "tar" extensions and
> compressing algorithm.
>
> ciao
>        cate
>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST.RemoveThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.RemoveThis@lists.debian.org

Eugene Gorodinsky wrote:
> 2009/7/31 Giacomo A. Catenazzi <cate DeleteThis @debian.org>:
>> Eugene Gorodinsky wrote:
>>> (in my oppinion
>>> this area can be vastly improved, and I'm interested in contributing).
>> What are the problems of actual format?
>>
> For one the dependencies are specified as actual packages, rather than
> the actual files themselves. Thus, if a user has installed some
> library by `make install`, s/he cannot install a program packaged as a
> deb, that relies on the library without some pain.

It is difficult to track the version of a file, and it is very difficult to
know if a version is compatible with our packages. There are a lot of
other important things to check (ABI), not only the version.
In general it is impossible to do right dependencies to external/local
installed packages.


> On windows a program may contain some optional components, which you
> can choose at install time. This approach (I mean having some main
> package and some required and some optional subpackages inside it) is
> quite user-friendly. Neither dpkg nor apt have this functionality in
> them, and I do not think it's possible to implement it without
> changing the package format.

In debian is done by different packages (within the same source).
Usually these related packages are listed in "Suggests" or/and
have a common prefix in package name.
User interface problems are outside package format.


> I also think some abstraction from the actual filesystem is a good
> idea. For example currently the only way to install a lib in a
> directory other than the one it was intended for is by using a hack
> that would look at the directory of a file and move it somewhere. It
> seems that with the current situation where you want to use
> /lib/i386-linux-gnu tuples instead of the approach used before, would
> be less painful if the current package format had some abstraction
> from the filesystem. Since the programs don't usually care where the
> library is, as long as it is in the LDD_LIBRARY_PATH.

Not sure I understand, and security implication should be handled
with care.

Anyway RH has support to install packages in own homes. This kind of
abstraction could be nice to have.


> The translations packages could be specifically marked as such, so
> that a user could easily check if her package has translations for her
> native language. The same applies for documentation, probably, which
> could be made into an optional subpackage.

I think this is not a problem of package format, but of user
interface.


> Currently debian policy is to have a .desktop file for each GUI
> program. What would be better, IMHO, is having some sort of
> abstraction, so that the package manager itself would create a
> .desktop file entry, given an icon and some information about the
> package.

This is outside package format. Anyway it is being standardized.


> Since programs usually store their settings in the user's home
> directory, that aren't deleted when the program is uninstalled the
> user's home directory becomes a mess. I'm not sure if it's possible to
> change some functionality within dpkg without changing the format
> itself.

This was already discussed, and it is impossible to solve.
Package managers must not touch homes. Check archives about
a lot discussions about this.


IMHO you are trying to solve problems on the wrong level.
Package format is a low level format. A lot of stuff can
be done with actual package format, but with improvement
of user interface or/and with triggers and postinst
stuffs.

Actual system is not perfect. You can help, but it is not
so easy to improve such things in a sane way without
causing troubles with existing users.

ciao
cate


--
To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

2009/7/31 Giacomo A. Catenazzi <cate DeleteThis @debian.org>:
> Eugene Gorodinsky wrote:
>>
>> 2009/7/31 Giacomo A. Catenazzi <cate DeleteThis @debian.org>:
>>>
>>> Eugene Gorodinsky wrote:
>>>>
>>>> (in my oppinion
>>>> this area can be vastly improved, and I'm interested in contributing).
>>>
>>> What are the problems of actual format?
>>>
>> For one the dependencies are specified as actual packages, rather than
>> the actual files themselves. Thus, if a user has installed some
>> library by `make install`, s/he cannot install a program packaged as a
>> deb, that relies on the library without some pain.
>
> It is difficult to track the version of a file, and it is very difficult to
> know if a version is compatible with our packages. There are a lot of
> other important things to check (ABI), not only the version.
> In general it is impossible to do right dependencies to external/local
> installed packages.
>
>
>> On windows a program may contain some optional components, which you
>> can choose at install time. This approach (I mean having some main
>> package and some required and some optional subpackages inside it) is
>> quite user-friendly. Neither dpkg nor apt have this functionality in
>> them, and I do not think it's possible to implement it without
>> changing the package format.
>
> In debian is done by different packages (within the same source).
> Usually these related packages are listed in "Suggests" or/and
> have a common prefix in package name.
> User interface problems are outside package format.
>
>
>> I also think some abstraction from the actual filesystem is a good
>> idea. For example currently the only way to install a lib in a
>> directory other than the one it was intended for is by using a hack
>> that would look at the directory of a file and move it somewhere. It
>> seems that with the current situation where you want to use
>> /lib/i386-linux-gnu tuples instead of the approach used before, would
>> be less painful if the current package format had some abstraction
>> from the filesystem. Since the programs don't usually care where the
>> library is, as long as it is in the LDD_LIBRARY_PATH.
>
> Not sure I understand, and security implication should be handled
> with care.
>
> Anyway RH has support to install packages in own homes. This kind of
> abstraction could be nice to have.
>
That's sort of what I had in mind, except on a larger scale where a
package is simply marked as being a shared library and perhaps given
some tag, e.g. "base". The package manager would then install it into
/lib or wherever it deems necessary.

>
>> The translations packages could be specifically marked as such, so
>> that a user could easily check if her package has translations for her
>> native language. The same applies for documentation, probably, which
>> could be made into an optional subpackage.
>
> I think this is not a problem of package format, but of user
> interface.
>
What I meant was to have some certain predefined types of packages, so
that it was easy for the package manager to tell which packages are
translations, which are libraries, which are documentation etc., and
also easy to tell which translation/documentation packages belong to a
particular piece of software.


>
>> Since programs usually store their settings in the user's home
>> directory, that aren't deleted when the program is uninstalled the
>> user's home directory becomes a mess. I'm not sure if it's possible to
>> change some functionality within dpkg without changing the format
>> itself.
>
> This was already discussed, and it is impossible to solve.
> Package managers must not touch homes. Check archives about
> a lot discussions about this.
>
Any pointers on what to search for? My search proved unsuccessful (again)
Anyway, wouldn't specifying which files need to be deleted from the
user's home directory solve the issue?

>
> IMHO you are trying to solve problems on the wrong level.
> Package format is a low level format. A lot of stuff can
> be done with actual package format, but with improvement
> of user interface or/and with triggers and postinst
> stuffs.
>
I hope I am. Would make things much easier

> Actual system is not perfect. You can help, but it is not
> so easy to improve such things in a sane way without
> causing troubles with existing users.
>
I'll look into the current format more, since I guess I might have
missed a few things.

> ciao
>        cate
>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

Klaus Ethgen <Klaus RemoveThis @Ethgen.de> writes:

> Hi,
>
> Am Fr den 31. Jul 2009 um 13:32 schrieb Eugene Gorodinsky:
>> Since programs usually store their settings in the user's home
>> directory, that aren't deleted when the program is uninstalled the
>> user's home directory becomes a mess. I'm not sure if it's possible to
>> change some functionality within dpkg without changing the format
>> itself.
>
> I don't want to go into the other arguments (yet). But this argument is
> very interesting.
>
> On one hand it ends in privacy problems if a user wants to keep his
> configuration stuff of one application (especially when he has a shared
> $HOME).
>
> On the other hand there could be a tool to cleanup such stuff. But this
> might be not connected with the real packages then better a own
> application with a data base of know applications.
>
> Regards
> Klaus

End becomes a real mess when home is shared between systems or users
have their own version of things in ~/bin/.

The right thing to do is to make software not create ~/.whatever
unless there is actualy something changed in there. There really is no
need to duplicate the system wide defaults or programs default unless
the user changes them.

So two things: Leave conffiles on removals and don't create them in
the first place.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

Adrian Perez <adrianperez.deb RemoveThis @gmail.com> writes:

> This may not be relevant in here, but I felt the need to ask.
> There's any plan of supporting another format - without breaking
> compatibility, I mean supporting - besides the RFC one?

There isn't any plan that I'm aware of.

> I think YAML would be a good one.

What would be the gain? I can think of one point at least, namely more
standard libraries for parsing the format, but parsing the current format
isn't particularly difficult and that doesn't seem to be worth the work to
support more formats. Are there other advantages?

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

Le Fri, Jul 31, 2009 at 03:32:43PM +0300, Eugene Gorodinsky a écrit :
>
> Currently debian policy is to have a .desktop file for each GUI
> program. What would be better, IMHO, is having some sort of
> abstraction, so that the package manager itself would create a
> .desktop file entry, given an icon and some information about the
> package.

Hi Eugene,

Debian policy is to have a Debian menu entry for each GUI program. This is not
to be confused with the FreeDesktop menu that is used by GNOME, KDE, Xfce, …

The Debian menu is sometimes made available inside the FreeDesktop menu as a
'Debian' category, and therefore each GUI program that has a Debian menu entry
also has a .desktop file on systems that can make use of it. But this one is
autogenerated and is not in the source package.

Many packages do however distribute a .desktop file that contain a fully
FreeDesktop-compliant menu entry, but this is up to the maintainer to write
such a file or install the one provided upstream.

There is an ongoing slow-pace discussion about factorising the information
between the two menu systems, and possibly use the same file format
(FreeDesktop), while preserving the essence of each menus. The Debian menu is
in particular needed in desktop environments – often lightweight – that are not
compliant to the FreeDesktop standards.

http://wiki.debian.org/Proposals/DebianMenuUsingDesktopEntries

Menu entries usually do not indicate the path to the program or the icon, and
finding them is delegated to the menu system itself, be it in Debian or
FreeDesktop standard. I think that what you propose is more their task than the
one of the packaging system.

Have a nice day,

--
Charles Plessy
Tsurumi, Kanagawa, Japan


--
To UNSUBSCRIBE, email to debian-devel-REQUEST.DeleteThis@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.DeleteThis@lists.debian.org

Le Fri, Jul 31, 2009 at 03:20:46PM +0200, Giacomo A. Catenazzi a écrit :
>
> Anyway RH has support to install packages in own homes. This kind of
> abstraction could be nice to have.

Hi all,

I would really love to have such a functionality in apt. At work, we use shared
workstations that run old Centos systems (that is, the current release at the
time the workstation was delivered), and we more and more often have to
recompile software in our home directories to benefit the latest updates. Given
the possibilies of breakage, it is of course not thinkable to just update one
program centrally without the unanimous consent of all the users, that nobody
has the time to seek for except it is really vital.

I think that if Debian could provide an easy way to install programs in user
directories (which means: not 'dpkg --root', that does not take care of
downloading the dependancies), this could be a real plus to become the OS of
choice on that type of machines.

This said, it only represents a small fraction of our user base…

Have a nice day,

--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan


--
To UNSUBSCRIBE, email to debian-devel-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org

]] Eugene Gorodinsky

| I also think some abstraction from the actual filesystem is a good
| idea. For example currently the only way to install a lib in a
| directory other than the one it was intended for is by using a hack
| that would look at the directory of a file and move it somewhere. It
| seems that with the current situation where you want to use
| /lib/i386-linux-gnu tuples instead of the approach used before, would
| be less painful if the current package format had some abstraction
| from the filesystem. Since the programs don't usually care where the
| library is, as long as it is in the LDD_LIBRARY_PATH.

Except that this doesn't work particularly well. Libraries embed
paths, and detecting when they do is painful.

[...]

| Currently debian policy is to have a .desktop file for each GUI
| program. What would be better, IMHO, is having some sort of
| abstraction, so that the package manager itself would create a
| .desktop file entry, given an icon and some information about the
| package.

like, the path, the description, translated into multiple languages and
so on? This is just .desktop files reinvented.

--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


--
To UNSUBSCRIBE, email to debian-devel-REQUEST.TakeThisOut@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster.TakeThisOut@lists.debian.org

2009/8/1 Tollef Fog Heen <tfheen DeleteThis @err.no>:
> ]] Eugene Gorodinsky
>
> | I also think some abstraction from the actual filesystem is a good
> | idea. For example currently the only way to install a lib in a
> | directory other than the one it was intended for is by using a hack
> | that would look at the directory of a file and move it somewhere. It
> | seems that with the current situation where you want to use
> | /lib/i386-linux-gnu tuples instead of the approach used before, would
> | be less painful if the current package format had some abstraction
> | from the filesystem. Since the programs don't usually care where the
> | library is, as long as it is in the LDD_LIBRARY_PATH.
>
> Except that this doesn't work particularly well.  Libraries embed
> paths, and detecting when they do is painful.
>
Haven't thought of that, thanks for the input.

> [...]
>
> | Currently debian policy is to have a .desktop file for each GUI
> | program. What would be better, IMHO, is having some sort of
> | abstraction, so that the package manager itself would create a
> | .desktop file entry, given an icon and some information about the
> | package.
>
> like, the path, the description, translated into multiple languages and
> so on?  This is just .desktop files reinvented.
>
Not quite. This separates the actual files the program uses and needs
from the files that are needed by the user. It's up to the package
manager to decide what to do with the information provided to it (e.g.
it could create a desktop icon, a menu icon, add an icon to a dockbar
or run the program once it is installed etc.). Granted, you can scan
the package and check if it has a .desktop file and read it if the
file exists, however this is a bit hackish.

> --
> Tollef Fog Heen
> UNIX is user friendly, it's just picky about who its friends are
>
>
> --
> To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian..org
>
>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST DeleteThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster DeleteThis @lists.debian.org

On Sat, Aug 1, 2009 at 3:31 AM, Charles Plessy<plessy RemoveThis @debian.org> wrote:
> Le Fri, Jul 31, 2009 at 03:20:46PM +0200, Giacomo A. Catenazzi a écrit :
>>
>> Anyway RH has support to install packages in own homes. This kind of
>> abstraction could be nice to have.
>
> Hi all,
>
> I would really love to have such a functionality in apt. At work, we use shared

There are various implementations of similar functionality out there,
like 0install, klik etc. I imagine dpkg would need some changes, here
is the bug for who want to work on implementing this:

http://bugs.debian.org/170850

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST RemoveThis @lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster RemoveThis @lists.debian.org