(copied from the gobolinux-users mailing list:)

Hi all,

Whilst searching for info on Listener I accidentally stumbled across a post from "kenneth marken" in 2005:

here is a blog or something that was posted on osnews the other day. while the first page is stock "whats wrong with linux" filler, the next couple of pages are interesting, mostly when he gets to talking about mounting /home as a database file system.

http://akaimbatman.blogspot.com/2005/06/linux-desktop-distribution-of-future_15.html

I was shocked to discover that there were no replies to this. This is one of the most spot-on articles I've ever read on Linux usability in my opinion. What's more, I think that a lot of what this guy is suggesting could be implemented reasonably easily in Gobolinux, except for the new database filesystem.

Just to be clear --- I agree that Gobolinux shouldn't contain special Gobo-specific versions of software like in Ubuntu. I don't see a big problem with small patches that can be included in recipes, like the Gobohide patch, but for feature changes we should really be pushing the application developers. Great, that'll be fun :p I am hoping to get my hands dirty eventually, when things start settling down a bit for me. I'm pretty much fresh out of university and am officially a software engineer now, which is a boost to my confidence as a programmer. I'm getting married in 5 days (!) but hopefully next time I have a bit of time I'll start looking in to what would be required to make some of these things a reality.

Before Gobolinux, I just couldn't contemplate switching to Linux. The filesystem was the number one reason I found Linux unusable. Now with the arrival of Listener, I think there's a lot that can be done for usability in Gobolinux that could actually start to make it a great user-friendly desktop. Software installation is still a pain, even though it's so much easier in Gobolinux than other systems. I have to use MacOS X somewhat for work, and I have to say that Apple completely nailed software installation. They obviously have the AppFolder idea that is present in Gobolinux, but the difference is that an application can be run from anywhere without any kind of manual SymlinkProgram command. If you want to try something out, you can download the DMG file, double-click to mount it, double-click the app inside to run. If you like it, you can copy it anywhere you want, but /Applications makes the most sense. That couldn't be too difficult to do with Listener. I know we're half-way there already. I'm thinking about possibilities involving unionfs. Instead of symlinking from a /Programs folder, why not "mount" a Program? Once it's mounted it could be run as normal from the command line until it's unmounted. I also like his idea of bundling some libraries with Programs. Dependencies are still a nightmare, even in Gobolinux. It would also be nice to have KDE treat app folders differently, but that's a huge topic best left for later.

Anyway, sorry to waffle. Guess I just need to let off steam :D

Thanks go to the GoboDevelopers for all their hard work so far. You're an inspiration!

I agree.

Personally I think gobolinux should

a) convince other smaller distributions that the Gobolinux philosophy is GOOD and that they
should assist their user base to switch to a similar layout (package tools *should* support
the user's wish IMHO)

b) continue radically with the idea, that a logical structure, a nice grouping etc.. is good for a
user.

/usr/lib is totally messy, Linus should really make a strong stance towards ideas that improve
Linux as a desktop OS ( recently some kernel developer left because he felt that his work
got not the same amount of look than server-code. he was doing quite a lot work for linux on
the desktop... the -ck patchset)

I think Linux is going a long way to having things install as easily as they do in Windows. Especially the big distros, lol. But I think what's hurting Linux is that people who use Windows have certain apps which can't easily be installed on Linux. e.g. games, you need to have Cedaga or Wine, it's not built into the distro usually.

[edit: capitals, spelling]

Posted By: exodusWell, i think one of the solutions to make Linux user-friendly is having an easy way to 'install' (or run) applications, for that i think Klik is perfect...

while I totally agree that this distro would really benefit from a kind of application installation mechanism, I don't think that "klik" is the answer. sure, it has it's own merits, but it is still in development and I have some doubts about the possible performance impact of klik-wrapped applications.
instead, i'd rather consider ".pbi" method of installing things. take a look here

Posted By: exodusAside from that: What defects you see in Klik? Because i like it, and perhaps I am not in the know of its deficiencies.

Well, for starters - it is too much complicated to my liking. There is the whole idea of application virtualization that gives me the creeps. No, I mean, I adore the simplicity that klik provides for the average Joe, but technically, the way it is implemented looks like an overkill to me. And I prefer the KISS principle :)

What advantages does klik have in comparison with standard package management? The most obvious is - distro-independency (as much as possible) and "one file - one program" paradigm. But aside from that - there is nothing special that other solutions couldn't provide.

Let's take PBI for example. It is a much more simple solution that closely resembles a kind of installer most of us have used in Windows environment. And it is also "one file - one application" paradigm. You download a program and then one-click-install-it. There are also no dependancies to resolve and nothing else from the internet to download. The most notable difference from "klik"-package is that this PBI-installer doesn't provide a virtual, sandboxed environment for your program, but instead it integrates (installs) the app with the system. The possible downside to this solution, as many classic-way-zealots mention, is that it bloats the system with many duplicate libraries and thus - eats the precious space on your HDD. However, that is not always true. First, because the "bloat" is almost negligible since the volume of nowadays HDD's is such big that you won't even notice 1GB more or less. Did you have that kind of problem with your Windows installation? Hardly, I presume. And no one says about bundling such monstrosity as GCC or something in every PBI-package. So, from this point as I see it - that is not an issue. Second, here is where Gobo FS-Hierarchy jumps in. It seems to me that with a kind of filesystem layout we have here it might be even easier to adapt PBI-method for our needs. I see a great potential here, but it would require somebody with greater expertise in scripting and programming (developers here? anyone?:) to give it a closer look and state their opinion.

btw, PC-BSD has the same FSH as in FreeBSD with one exception - they install all their applications to a specific "Programs" folder. When I saw this, funny, I immediately thought of the way GoboLinux does things :))

it seems that PCBSD site goes down from time to time, so I decided to put the PBI-features info here, if you don't mind

*****
Part of making a Desktop Operating System that people feel immediately comfortable with is ensuring that software installation is as easy and familiar as possible. PC-BSD has taken this approach when developing the PBI (Pc-Bsd Installer) file format. Programs under PC-BSD are completely self-extracting and self-installing, in a graphical format. These PBI's also ship with all the files and libraries necessary for the installed program to function, eliminating much of the hardship of dealing with broken dependencies and file incompatibilities. PBI files also provide developers and packagers with advanced scripting and user interaction in an entirely graphical format, making the entire install procedure similar to what a user would expect from other popular graphical operating systems.

PBI Features:
* Completely graphical extraction & installation process.
* Advanced scripting support - Use shell-scripts to control the installation process.
* Corruption detection - Ensures that a user's downloaded PBI is intact.
* Library auto population - Grabs all the library files a binary may need for operation during the creation process.
* Icon Management - Allows developers to set icons for both the desktop & K-Menu.
* Program Error Detection - If a PBI installed binary fails and silently outputs a stderr/stdout message, this is captured and displayed in a GUI for troubleshooting.
* Easy Removal - PBI's can be removed through the "Remove Programs" system utility.
*****
P.S. Does Gobolinux File System Hierarchy has the potential to become the standard in Linux? The short answer is - yes (and not only in Linux, but throughout the whole nix-family).
But will it happen in the nearest future? I highly doubt it :((
The only way to make it happen is to popularize GoboLinux as much as possible. But that is a whole another painful issue here... I have some thoughts however, but they are not for this particular post.

Posted By: MohjiveBut if you, like me, have ~520 entries in /Programs, where many has two or more versions, you'd notice the difference, I think. And I would mind. At this time I'm running GoboLinux on a laptop, that is some years by now, with 20GB of HDD. Every GB counts in my view.

Well, as I said - that is a known possible downside to this method. However, I don't know if there are many people with the same specs as you have (I mean - 520 entries and only 20 GB HDD ;)) so I'd rather have this little sacrifice in order to get system's "ease of use" to the next level.
And hey, who said this distro should adopt PBI-method as it is, unaltered to your needs? Since it is based on scripting (looks very similar to autopackage, btw), I presume it can be rather easily tailored to your liking, providing, for examle, all the functionality that Gobo's scripts have now, only in even more elegant and user-friendly manner? What do you think?

Posted By: MohjiveMy personal view on these package schemes is that I favor autopackage. It's similar to our packages and they are dynamically relocatable. One can install it in any prefix and they (should) work. Imagine using the *same* package on both GoboLinux and your rootless GoboLinux installation.

Yes, I thought of it too, however the PBI infrastructure is more mature than Autopackage nowadays and is easier to use while pertaining some similarities.

Posted By: MohjiveI don't like the payload of pbi. I think it's better to have a good dependencies resolution, that install what's needed. Why would one want one file, one application anyway?

I'd rather put it "one file - one installer" (like Win "setup.exe" - click and run it). That is contrary to "klik" - which doesn't install application at all, only provides sandboxed environment to it.
Anyway, I am not a strong proponent of PBI-method in particular. It is just as I tried it and found it to be very convenient and logical. Heck, the whole PCBSD system is much more pleasant and fun to use than even "The Holy KUBUNTU!" :)) Just try it!
What I (maybe even "we", in this thread) try to emphasize is that the system would greatly benefit from a kind of easy installation mechanism - be it klik, pbi, autopackage or anything else. I want to be able to find something on the net (not in repositories), download and install it without a hassle. Even without internet-connection if I have those packages locally. And to be able to remove them the same way. How does it work under the hood is a bit of a lesser concern to me as a user. But it would most certainly be a challenge to the devs :)

Posted By: MohjiveIf doable, great! But the way I read it, the scripts are packaged with the pbi package, which means that if the scripts are distro specific, the packages will be as well.

Is there a problem? I thought Gobo's current installation framework (scripts, recipes and packages) is also totally Gobo-specific, so I see no difference here. But yes, if you prefer one "package - any distro" way of things, than Autopackage would be your choice.
Also, I've just stumbled upon a very interesting discussion on PCBSD forums where they compare klik vs pbi. There is a very insightful post by someone nicknamed "pcbsdusr" with "Post subject: Plan B Concept". As I see it he suggests a very sensible solution on how to improve the PBI-method to avoid dependencies multiplication. Have a look at it here

@Vazub: I thought you were talking about implementing another package scheme into GoboLinux and that I think is unnecessary. What we need is some work to make our tools more usable. Like getting a graphical window when one clicks on a package/recipe (if one tries now, one should get a terminal running InstallPackage/Compile).

Looking at pbi again, I think that maybe we can make use of some of that functionality. I only skimmed the plan B concept, but it gave me an idea. He talks about having libraries linked into the program folder, I want it the other way around. Say that application Foo depends on library Bar. Then, in the Foo Resources directory, theres a directory called DependencyFiles and inside that directory is a /bin, /lib etc tree where one can place libraries and executables required to run Foo, i.e.
/Programs/Foo/x.y
/Programs/Foo/x.y/Resources
/Programs/Foo/x.y/Resources/DependencyFiles
/Programs/Foo/x.y/Resources/DependencyFiles/lib
/Programs/Foo/x.y/Resources/DependencyFiles/lib/libbar.so.x

and when symlinking Foo that library would be symlinked into /System/Links/Libraries, if no such link already exists. At the same time, every library provided by Foo overwrites any such "weak" link provided by other applications DependencyFiles. That way the "correct" application provides the libraries if it's installed, otherwice applications fall back on the libraries in DependencyFiles. With this solution we will have some duplication of libraries on the HDD, but they are shared in memory.
We also need to make up a list of libraries that should never be bundled in DependencyFiles, like Glibc, GCC and Xorg libraries. We can assume those (and some more) to always be present on a sane system.

Posted By: MohjiveI thought you were talking about implementing another package scheme into GoboLinux and that I think is unnecessary. What we need is some work to make our tools more usable. Like getting a graphical window when one clicks on a package/recipe (if one tries now, one should get a terminal running InstallPackage/Compile)

No, no :). Not copying the other scheme unaltered as it is. There are those drawbacks that everyone admits and that really need some redesign. What I meant was precisely what you mentioned later - to harvest PBI for an inspiration and implement only those of its tricks that are reasonable in our environment. As I mentioned previously, I don't really much care how does it work under the hood or how is it called (PBI, klik etc.) All I want is to be able to download a specific self-contained installer (maybe with a specified package extension, like ".gobo" or ".gob"), click-install it (without thinking about any additional dependencies - say I am offline) and remove it without a hassle when I don't need it anymore. And it crossed my mind, that with such a versatile FSH we have now (that allows for several versions of the same app to coexist peacefully) it might be a lot easier to implement than in the case of other distro or system.

Posted By: MohjiveHe talks about having libraries linked into the program folder, I want it the other way around.

Sounds like a good idea to start with.

Posted By: Mohjiveand when symlinking Foo that library would be symlinked into /System/Links/Libraries, if no such link already exists. At the same time, every library provided by Foo overwrites any such "weak" link provided by other applications DependencyFiles. That way the "correct" application provides the libraries if it's installed, otherwice applications fall back on the libraries in DependencyFiles. With this solution we will have some duplication of libraries on the HDD, but they are shared in memory.

However, I don't think that I get it quite right. Doesn't it resemble the way Gobo deals with library dependencies now? Or did I misunderstand something?

Posted By: MichaelThere is one case where unexplained breakage will come up with that scheme: you install program "Fred", which requires and ships a version of libfoo.so.2. You don't have that already, so it's linked in. You later install a program "Barney", which also requires libfoo.so.2, but since that file is already there there's no need to do anything. If you remove Fred, Barney will break, despite their having no relation at all.

Good point!

Posted By: MichaelRe-symlinking Barney would fix the problem, but that isn't an intuitive action to take.

And what if we have a re-symlinking script that automatically starts every time some programs get uninstalled? I mean, it could be the part of system's Installation/Uninsatllation framework.

Posted By: MichaelI don't like the idea of duplicating the whole dependency tree in each package

Neither do I. If you look at my last post again, I talk about a list of library never to bundle, KDE shold probably be one of them.

For the problem Michael brought up: yes, I've thought of that and my proposal is some kind of automatic symlinking, as Vazub has already mentioned. I'm not sure how it should work yet.

Hard links wont problably be used for the same reason we don't use it now: someone may want to place the link hierarchy and the application directory on different filesystems.

I've thought of package bundles as well, but not like that; that a tarball package would consist of a whole bundle, with multiple applications. That's an idea, but one drawback is that it would be harder to figure out what individual applications is installed by just looking at /Programs.

I don't think you can really assume that KDE is installed. It's present by default, but somebody could well remove it reasonably. Anyway, that was more an example of a generic large package that may not be installed but that is a dependency of many others. GTK+ is another, Ghostscript is a pretty common dependency, LibXML2 is used all the time, I have a couple of things depending on TeTeX. Bundling up Ruby or TCL with scripts each time is likely to multiply the package size many fold, and that's not even considering including the JRE.

The longer the list of exclusions the more inconsistent the user experience will be. "It just works, right out of the box!" until it doesn't, and you're thrown an error because you didn't have one of the dependencies it assumed you did. But the shorter it is the bigger the packages will be and the more duplication of data there is.

Nobody wants to download a package that's 90% programs they already have installed, especially on a slow connection. And then if it doesn't work anyway, because you're missing another dependency that wasn't included, you're not going to be very happy.

I've been thinking quite a lot about this on and off, and the more I think about it the more I think that we need to build a completely FUSE-based solution to this problem. The problems GoboLinux faces, in my opinion, are:

1) programs misbehave by assuming files are in a certain place when they're not
2) we want to avoid using millions of symlinks
3) we want to avoid duplicate files
4) we would like applications to be relocatable & renameable
5) I (anyone else?) would like to sandbox applications (non-system programs) so that they can only see the programs, libraries, and devices that they need. I don't like how cluttered the PATH is.

If we use a FUSE fs, I think it should be possible to achieve all of those things, and we may not need GoboHide any more either. Consider something along these lines:

There's a database on the system containing a map of what packages provide what libraries & programs. Two packages may provide the same library. When a package is moved, the database is updated. A package is merged into the path using a script (MergePackage). On boot, a base set of packages is merged (all in /System/Programs and /System/Links). A user downloads a package and tries to merge it. MergePackage looks up the executables and libraries that the package needs using the database. If it can't find a package that provides the required dependencies, it prompts the user to point it to a package that does (it may have missed it), or to download one. If it finds several packages that provide the library, it asks the user which package to use, and optionally remembers his choice for the future. The script calls MergePackage for all of the dependencies that are not already merged as well. The user can now run any binaries his package provides.

Now by "merging" I mean something a little different from the symlinking that's done at the moment. When you merge a program, the FUSE fs will provide a number of paths to its contents, and per-application sandboxing should be doable as well (you can have a different set of packages merged for each application). You can move a package while it's merged because its dependants are using an abstracted path which FUSE fs is providing.

I haven't yet decided if it would be better to have a FUSE fs mounted as root (if that's even possible), or just a special directory.

Any thoughts on any of this?

Wow, excellent! I wonder why Hisham hasn't mentioned that during my various posts on the subject in the past?

Posted By: Michael

Posted By: MohjiveThat way you'll only have duplication when downloading.

You say that like it's a good thing.

I agree — I think at least in the main store packages should not have dependencies bundled. However, I think it should be possible to package several programs together so that they can de distributed that way if need be. More than that, I think that packaging a program should literally be a simple as zipping the program directory — no script involved.

Posted By: giddie4) we would like applications to be relocatable & renameable

I do have to take issue with this one, though. I've never quite understood why that would be seen as useful, and I don't think it should be a goal for the system (it'll make things exceptionally complicated if it has to watch for moving programs...). I can see a case for being able to split across filesystems, in a put-it-and-leave-it kind of way, but not for moving them around.

Wikipedia lists a few definitions of usability in its main article on the subject, but they are mostly not specific to software design, so here is one of the more popular set, made slightly more relevant to this topic [source]:

* Learnability: How easy is it for users to accomplish basic tasks the first time they encounter the design?
* Efficiency: Once users have learned the design, how quickly can they perform tasks?
* Memorability: When users return to the design after a period of not using it, how easily can they reestablish proficiency?
* Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?
* Satisfaction: How pleasant is it to use the design?

The problem is that users are used to moving files around. They do it every day when they create, edit, and delete documents, send them by e-mail, etc... Then it comes to the point where they've downloaded a new program and they want to try it out. Suddenly the rules change... Why is this file treated differently? This is bad Learnability and Memorability, because the user has to use a completely different system for programs. This introduces Errors. Having to use a special program to install or move a program rather than just copying or moving the program on the filesystem slows down the user's workflow, which is bad Efficiency, and I know from experience that when my workflow is interrupted, I get frustrated, which certainly breaks the Satisfaction rule. As I see it, the way software is traditionally managed in Linux breaks pretty much all usability guidelines. It's understandable, because it's tricky to make easy. GoboLinux has gone so far toward fixing this problem, but we absolutely mustn't stop short of the mark!