0install

Aus Open Source Ecology - Germany
Version vom 18. Juni 2017, 11:37 Uhr von Jan R.B.-Wein (Diskussion | Beiträge) (Add section about What 0install (feed) is used for. Clarify dev vs. release versions. Improve Why.)
Zur Navigation springen Zur Suche springen

What

Release distribution

Developers provide a project feed somewhere on the net (no central repository to be maintained). Users feed the feed URI to 0install.

Platform and content independent dependency resolution

  • For releases: 0install reads the feed to resolve dependencies, downloads, installs them and creates read-only links.
  • For development: Manually or 0install triggered download of the repository (clone, pull which is fetch + merge). Git post merge hook triggers: The contained 0install feed is parsed, dependencies downloaded, installed, read-only links created (to prevent accidental override, which encourages single source principle, i.e. change it upstream where the link points to. This makes it easy to generate a patch and submit it at the origin of the dependency).

New dependencies are set up manually by developers first (linked, used in the project, tested) and then added to the feed.

One development version, but many release versions

Note in development projects effectively exist only in one version: the development version (branches are not considered).

In contrast, releases can exist in multiple versions due to containerization|virtualization.


Why

  • No need to package the projects' releases for multiple package managers.
  • Allows to depend on any open project released to the web. (While developers are trusted to manually set up requirements like unzip dependency releases or to overlay Git repositories, for the release this should be automated.)
  • Easy transition from user to developer. (Currently there is no 0install command to trigger download of the source repository of a project but the repo can be located easily by inspecting the feed URI as the feed is contained in the repository. Manual download is as easy as truncating the URI to the repo part and using `git clone` on that.)


The combination `development` in a `git repository` + `release` via `0install` generated directly from the repository allows:

  • amending or reverting incorrect releases because the release files are under version control.


  • modular, flexible development (git) __and__ distribution (0install)
  • single source principle, low level of complexity and few commits that logically belong together due to mostly small repository sizes.


  • easier finding of projects for forking / referencing by potential developers
  • potentially reducing redundancy or at least structuring it better because related projects are referenced automatically.


  • using the same toolchain for developing all kinds of projects (not only code)
  • on all operating systems.


  • depending on modules in any revision range (unlike Git submodules which was used for providing modularity prior to 0install and only allows to depend on one and only one commit, i.e. a single version instead of a range)
  • which may be distributed using any version control or package management system (not necessarily 0install which is just a convention that e.g. worlddevelopment uses)
  • what allows depending on almost all open source projects already developed.


  • following API dependent release versioning conventions (incompatible_api_change_increment.minor_version.fix-release_candidate_or_suffix, see Semantic versioning) for being able to specify a wider compatible version range.


  • automatic backup of projects (because of distributed development).


  • easy testing of modules locally without breaking the system due to multiple versions of a package.


  • developer interaction (world wide team work) without being forced to release just to allow other developers to fetch the new changes (because a distributed version control is used underneath).


  • users turning into developers with ease (because 0install is used for releases and can fetch the sources, i.e. the version controlled repository contents).


  • easily fetching, inspecting the sources of a project in a certain version. (e.g. to debug or further develop / fork a local version, this can be rebased later on)


  • preventing dependency mess.


  • linear project history (not mandatory but possible) and all other version control system powers.


  • having the 0install XML file under version control where it belongs: in the same repository where the project sources reside.


  • aliasing filenames a project depends upon to a functional general name to be able to exchange a file (e.g. a used image or schematic) later without updating all used references e.g. in FreeCAD or blender files (which may be binary where search replace does not work easily).
  • referencing files of dependencies using relative links (as if the dependencies were located as a folder in the current working directory).
  • to increase development consistency.


=> Setup flexibility (use/develop hybrid) All this together finally results in the possibility to use or develop projects in the newest revision available that is compatible on a per project basis instead of being scoped system wide!

This means that every project can be the latest stable working version instead of the lowest dependency definition limiting an update and thus an update of other projects which depend on the same project also.

Or it can be a full bleeding edge development setup (or a mix) which then also fulfills the package management principle of only one project installed like e.g. Arch, Gentoo and especially Void xbps package manager because development makes most sense in the most recent revision with the disadvantage of high in time demand due to bug and incompatibility fixing requirements to resolve broken packages. This may also be a benefit if viewed in a more global scope e.g. it is easier to find bugs when few changes were made compared to having to spend hours to bisect several related projects respectively to find the regression. In Void linux such broken shlibs actually resolve within a very short time span usually.

It thus is a rolling release setup.

How to package

It is recommended to use 0repo. The exact procedure is currently undocumented. An easy example to start with is freecad_convert. Key is to understand the 0install tools.

A module package version is released directly in the Git repository. See 0install, git interaction outlined under #Why.

It is packaged using the repository contents. => No magic extra ingredients to be figured out, then merged.

The resulting package can be hosted in the version control repository but it might be desirable to have it separate for big release data amounts. The package URI is provided in the 0install XML file.

Reference the Zero Install docs.

How to install, depend on a module

Install a module / package:

   0install add <repository.uri>/<module>.xml

Depend on a module by using the 0install XML file URI of the corresponding module repository.


Module naming convention

Generally this is a subjective matter and it will never be used consistently in the world. Thus the exact naming is not too important as long as it is used consistently within a project which 0install supports using symbolic links / aliases.

However within projects of engineering clubs, this can be kept as consistent as possible by following basic rules:


Use nouns

  • A module has a purpose which a conscious being is giving to the thing. (We are intellectual beings, we do develop this construct for a reason!)
  • It can have more than one functions, uses defined by its capabilities.
  • And can be made happen using one or multiple technologies.

When one considers that a module

  • can be based on other modules (inheritance)
  • and override/modify existing runtime files by providing the same file as a dependency. This will not take effect for compilation or preprocessing.
  • can use any number of other modules (dependencies).
  • may be used by 1..n other modules (reverse dependencies)

where n is a natural number.

=> Then it becomes apparent that a module name should use nouns, especially because we are interested in constructs (and constructs are things) that we desire to put together virtually or physically because we have a purpose for every one of it.


Why not to use a grammar like Lojban, English, Chinese?

Lojban

  • is not widely known.
  • Must be learned by developers first.
  • Is like programming and useful for conveying uniquely defined messages (the purpose of language?!).

Concerning Chinese and English grammar

  • most projects/modules would use transitive verbs that take objects while the subject is not important or generic (e.g. <subject>_control_motor), thus omitted (control_motor) and
  • subject_verb_object creates discrepancies when stacking modules recursively, because only things can be stacked and one can not e.g. stack control_spider_leg onto move_spider_leg_base without confusion.
  • verbs imply one function / use while a construct may have more functions, consider a motor controller that may limit current and control speed at the same time. Restricting a module to one function may be unwise especially for higher level modules, i.e. composites.

Please comment/ discuss this page if you think there is something important to be added or if there is a consistent naming scheme using language grammar that is generally applicable to project / module naming.

General to specific

This implies more reasons for using nouns instead of verbs, because

  • subject_verb_object contradicts from general to specific naming convention, e.g. electronics_control_motor_speed and mechanics_control_motor_speed versus motor_controller, motor_brake_mechanic, motor_brake_electromagnetical which eases depending on both modules for redundancy, too.

Goal: Provide at least one module for every purpose.

If we want to construct a mobile machine then we want it to transport something or someone. This mobile machine may be driven by an electromagnetic motor and because there is a desire to control vehicle speed, and many vehicle kinds may be useful for fulfilling some purpose, we hence define a motor controller module sooner or later. Its purpose is to control a motor. There may be variants for alternating and direct current.

Naming after purpose (transporter example)

When we develop a universal machine to help with the harvest, water transport and tree trunk transport why not call it as such?

e.g.

  • harvest_transporter
  • harvest_transport_vehicle
  • tree_trunk_transporter
  • transporter_tree_trunks
  • transporter
  • transporter_open
  • transporter_closed
  • transporter_container (module for transporting liquids)
  • transporter_mesh
  • transporter_mesh_container (for hay?)

Many possibilities.

At least one might agree that calling it:

  • transporter_for_liquid

might not be helpful because one thinks that this is a new transporter for liquid and not a submodule that adds liquid transport capabilities to the transporter.

A convention can be to use the base module according to its purpose:

  • transporter

and define modules that extend it depending on the object that the verb *transport* takes in a generic form (aggregrate state where applicable)

  • transporter_liquid
  • transporter_gas

Now that every object made from common matter on our known planets within reach has different shapes, different transport methods may be smart:

  • transporter_long_thin
  • transporter_small_pieces

Or directly use it as specific as initially:

  • transporter_tree_trunks
  • ...

A solution can be similar to *motor_controller__current_sensing*:

  • transporter
  • liquid_container
  • transporter__liquid_container (defining both previous modules as dependencies, i.e. tailoring the liquid_container to plug into the transporter platform)

Prefer tagging over suffixes for classification

Call a world module 'bending_machine' or 'sheet_bender' is a subjective decision. A machine can be automated turning the _machine suffix invalid. Yet the base module is a machine and not automated and if an automated self contained version were to be developed then a new repository 'bending_automaton' may be useful because it allows completely different design by turning the human machine interface into a radio control that interfaces the computers that actually control the machine automatically.

In the bender example one may extend/inherit the modules

  • bending (for the general process/documentation of such manufacturing process)
  • sheet_bender (may be a simple tool)
  • tube_bender (bender__tube_tool or bender_tube_bracket may inherit from this one for documentation or to reuse the manually operated tool, e.g. the brackets/clamps to create nicely shaped bends)

As it is questionable if a module bender rather refers to an active operator that bends something, i.e. a mobile automaton that changes location to bend something while bending_machine implies some static design. Consider a car, it is not static and yet a machine. => Thus bender is valid too and hence system -> machine -> automaton -> robot -> android classification left to tagging (categories).


Tags (like machine, automaton, manufacturing, energy, ...) should be included in every repository. Either as

  • tag / category list in the 0install feed or
  • as files in a directory called tags (.tags may not be found when not directly searching for hidden files too). This allows finding projects in the local file system without having to open XML files for reading, parsing.

The tags/<tag> empty files may be generated from the 0install XML feed file automatically which effectively turns this into an cache which may become invalid (something to be aware of).

Prepending versus Appending

bender__tube indicates an addition, i.e. the extension of a bending machine [sheet_]bender to also bend tube.

tube_bender indicates a stand alone / base module that may have its own extensions but is itself fully operational to fulfill its purpose.

To indicate a specialization/extension module via __ might be good practice [experience needed] because it is not meant to denote one thing anymore while bender_tube strictly means one thing and that is a kind of tube. (see also _ or __ as separator convention)

Compatible interfaces

This is a less useful solution if all modules are designed such that they have compatible interfaces. In the *liquid_container* example this means that the container has to fit any machine that exists as a module. While possible to adhere to coherent interfaces within a club's modules, it becomes virtually impossible for all existing open source modules. This lack of world wide consistent interfaces (standards) and the fact that it may not make sense to add the extra interface complexity because it increases costs (money, rare materials used, ...).

=> As compatible interfaces are only locally consistent and due to the increased complexity and costs it may be more valuable for the eco system of this world to prefer the tailoring solution as convention outlined in the previous chapter.

Room for discussion

It is seen that there is a lot of room for discussion because it is difficult to find an as smart as possible solution that is generic, yet specific enough to cover most needs while still fulfilling modularization. Please use the Talk/Dicuss function of this page.

Examples

The level of modularity is subjective because it depends on something atomar really existing. It may be sensible to start general and on demand subdivide further to allow reuse by other modules to follow the single source principle and Unix package principle (1 task, 1 program) without having to pull in a huge set of files that are not needed. Consider depending on spider_leg but it being contained within hexapod or octopod.

  • spider_leg
  • pod
  • hexapod
  • tank_track (Why
  • tank (Why build it? Because it protects us while fighting!)

Optionally (at e.g. a later point of time) this can be further modularized for arbitrary stacking/exchange of modules:

  • spider_leg_segment
  • spider_leg_segment_joint
  • spider_leg_segment_joint_transmission or just spider_leg_transmission


More examples

  • motor_controller
  • motor_controller_dc
  • motor_controller_ac
  • motor_controller_ac_three_phase
  • generator_load_controller
  • generator_load_controller_electronic
  • generator_load_controller_mechanic

are other examples for functionally separate, self contained repositories where the generator_load_controller may contain common files like documentation or evaluation of different control techniques.

current_sensing

motor_step_controller motor_step_controller__current_sensing is a modification/addition to allow micro stepping without redefining common circuits like power or the H bridge. It would depend on motor_step_controller obviously as well as on current sensing for then fine tuning the design for micro stepping.


Underscores (_ or __) as separator convention

The double __ is just one possible convention.

This is analog to Python modules which uses a hyphen e.g. python-numpy is serving a purpose and depends/operates on python itself. Of course it is not wise to put all depedencies into the name as these may change and are likely many. It is just that numpy (numeric python) is a part of the python scope because it is written in python, thus fine tuned for python versus using a similar library written in e.g. C.

=> This special fine tuning for one module is what can be used to decide when to prepend a base module's name to the module name.

The difference is that worlddevelopment.0install convention uses _ instead of - because underscore is in a more basic character set than -+,;:0123456789 are and allows easier machine interfacing later on because a machine has to do extra work to know when a hyphen - is a logical separator and when it is part of a word like "add-on", so if you have a module bending_machine-add-on and a submodule bending_machine-add-on-submodule then it becomes messy which is why the convention uses bending_machine__add-on-module.

Using __ also increases compatiblity as tools that rely on just a single underscore as separator can easily replace __ with _ and proceed as usual. Whereas when using hyphens - as separator it is not possible to replace all instances of - with _ without detecting which words are composites (e.g. add-on).