Mercurial (hg) Installation Without Being Root

Solutions for installing mercurial from a package repository or via a self-extracting or any other installer can easily be found for all operating system distributions. Thus this is not the topic of this Wiki entry. Here we cover the installation of a mercurial versioning system without root access. It is a necessary step for the creation of our symbiotic open source hard and software development system that at a later point shall be mirrored by the Wiki to keep it up to date automatically and at all times. This allows us to bundle our efforts for development in our versioning system (I use mercurial, but git, svn, bzr, ... will be fine too).

In the following the process of how we got mercurial working on our webspace (we don't own a server due to financial reasons) is documented. A lot of special information is added in the second chapter that is nice to know, but as it turned out this did not work for us.

So the only really relevant part is the Chapter: [wiki.opensourceecology.de/Mercurial_(hg)_Installation_Without_Being_Root#What_finally_worked_for_us: What finally worked for us]. If you want to know something about why the first undertakings did fail read all chapters, but I warn you, it's messy. Hence it's probably better to read the excerpt of [#Problems_that_might_occur_.28and_occurred_for_me.29 problems that occurred] for us instead.

With a Python installation readily available or with rights to install a python environment:

It's also no problem, if a python installation is available or you have a server where you have adminsitrative rights (root access) for installing a python environment and still want to have a local installation of mercurial into your home directory. It is recomended to use the Mercurial script: make install-home (to include documentation) or make install-home-bin (for the executable only). If you used a custom python location for the script it is required to add the following to the command to communicate it to hg:

$ PYTHON=/path/to/python2.7-or-newer #this will be supported for a long time

$ make install-home

Source: http://mercurial.selenic.com/wiki/UnixInstall#Per-user_installation

Without Root access and no Python available:

To install mercurial without being root, needs us to bundle Python 2.x (as Python 3 is not yet supported as there is only one advantage in favour of upgrading, so it's not worth the effort currently).

Windows

For Windows there currently is PortablePython as a ready made option.

UNIX/POSIX

For UNIX systems (Mac OS, GNU/Linux, Android, ...) or FreeBSD there are some packages floating around in the net, but it's sometimes hard to find the correct one for a certain architecture and operating system (what is needed to match the server's arch and os).

So if you have a system similar to the web server available where you have root access, the following is an option of how to get a working mercurial and python.

Virtual / bundled installation of Python

To create a virtual/bundled installation of Python and Mercurial there are two or more quite similar ways.

We will proceed in adapting this splendid blog entry of Fabrizio: https://weblogs.java.net/blog/fabriziogiudici/archive/2009/07/installing_merc.html .

$ cd $HOME #cd ~ also works

$ sudo apt-get install python2.7 python2.7-dev #if unmet dependencies problem occurs see below

$ wget peak.telecommunity.com/dist/virtual-python.py #if it can't be resolved, search the web for virtual-python.py .

$ export PYTHONPATH=$HOME/python2.7/lib/python2.7/site-packages/

$ python2.7 </path/to/>virtual-python.py --prefix=$HOME/python2.7/ #Note: This in conjunction with the following command copies the python files to the specified path in the $HOME directory!

Because of cd $HOME in our case it becomes: $ python2.7 ./virtual-python.py --prefix=$HOME/python2.7/

$ python2.7 ./virtual-python.py --prefix=$HOME/python2.7/

Two possibilities: Setuptools complete for creation of virtual python or - how we did it - only fetch the relevant part of setuptools package: (if another python version is required look here for the .egg file):

$ wget https://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg

I figured that one command was missing to get a successful install:

$ chmod +x ./setuptools-0.6c11-py2.7.egg

Now make sure that the PYTHONPATH is pointing to the correct python directory: $HOME/python2.7/lib/python2.7/site-packages/ . Check it using: echo $PYTHONPATH (If it's not correct, once again: export PYTHONPATH=$HOME/python2.7/lib/python2.7/site-packages/.

Then install easy_install binaries et alia using:

$ ./setuptools-0.6c11-py2.7.egg --prefix=$HOME/python2.7/

(Note that this is a required step - it must not fail! Check the terminal output.)

$ python2.7/bin/easy_install-2.7 --prefix=$HOME/python2.7/ mercurial

Change the rights of all types (user[read:owner], group, other) at once:

$ find python2.7 -type d | xargs chmod a+rx

Ensure files loose write permissions:

$ find python2.7 -type f | xargs chmod a+r

Avoid this: Bundle Mercurial and Python

This is only required if no compiler on target host/server and if using make install-home-bin is impossible and all other make install fail.

$ tar -cvzhf mercurial.tar.gz python2.7/ #compress, verbose, z: use gzip2, h: dump files that symbolic links point to instead of using the symbolic link, files, -C or --directory <target_directory>; if you have compatibility issues omit the '-' after the tar or use another archiver like 'pax' which is standard-compliant. tar might well come from times where dinosaurs were around. ; )

Remove no longer needed temporary files if desired: $ rm -rf python2.7 setuptools-0.6c11-py2.7.egg virtual-python.py #f: force : ignore non-existant files, never prompt. (so non-interruptable)

Alternatively:

$ mkdir mercurial_python_creation/ && mv mercurial.* mercurial_python_creation/ && mv virtual_python.py mercurial_python_creation/ && mv python2* mercurial_python_creation/

Upload to server

$ scp ~/mercurial.tar.gz <user>@<server>:mercurial.tar.gz

Eventually accept and store the server key footprint. Enter password and extract the uploaded file remotely via a shell session.

$ ssh <user>@<server>

Setting up virtual python installation

$ cd $HOME && tar -xzf mercurial.tar.gz #attention the resulting folder is python<version> in place of mercurial

Check for hg in the binary folder:

$ ls -a mercurial/bin/ # -a for all files (including hidden ones)

$ cd $HOME/python2.7/

Create wrapper shell script: Insert lines:

#!/bin/sh

export PYTHONPATH=$(HOME)/python-dev/lib/python2.7/site-packages

$(HOME)/python-dev/bin/hg "$@"

Now point the server to the script (via .bashrc or .bash_profile I guess?). Fabrizio also edited vi ~/python-dev/bin/hg so the "dang bash" line is #!$(HOME)/python-dev/bin/python in place of #!/usr/bin/python. He says, it just worked.

Configure Mercurial for a Python installation or PortablePython:

$ wget http://mercurial.selenic.com/release/mercurial-2.8.tar.gz

$ scp ~/mercurial-2.8.tar.gz ftp118002-2629078@dragontale.de:mercurial-2.8.tar.gz

Log on to the server. (see above)

Extracting into a folder requires this folder to already exist! (so then don't omit this step). We chose 'mercurial' as the target directory:

$ mkdir mercurial #alternatively call it 'hg'

Unpack the downloaded mercurial package:

$ tar -xzf mercurial-2.8.tar.gz --directory mercurial # -C is a synonym=alias for --directory

$ cd $HOME/<mercurial_extracted_folder>/

now

EITHER

$ make PYTHON=$HOME/python2.7/bin/python PREFIX=/usr/local/ install-home-bin

$ hg # should show version

OR

$ make PYTHON=$HOME/python2.7/bin/python local # build for inplace usage (only within current folder) and only if a compiler is available.

$ ./hg --version # should show the latest version

Alternative: VirtualEnv[ironment]

The third entry from top: http://stackoverflow.com/questions/11249901/creating-a-portable-python-local-install-for-linux

Problems that might occur (and occurred for me)

If a 'dependency unmet' error occurs when trying to install python2.7-dev then follow the solution I proposed in this thread:

1. sudo apt-get install aptitude
2. sudo aptitude install python2.7-dev
3. skip first proposed solution [Not installed] and wait for the next solution to be proposed:
4. take the solution to downgrade the dependencies.

Also a bunch of i386 programms will propably be removed during the process. (just trust Linux!)

Another severe problem and showstopper was that there are different mercurial versions around ... some of them use 4byte string representation instead of the common type of 2byte (As also using .tgz ending mercurial files did not stop errors I settled on a trick, see below:)

$ sudo apt-get source mercurial #called from $HOME

This made me feel strange once again, but at least this resolved the python issues.

The next problem was a old the python header files were missing once again. So that is a strong indicator of missing installation of python<version>-dev package on the target system (server in our case).

Just another trick helped resolve this: A bundled python or PortablePython as long as PortablePython has the header files included! And that may be uncertain.

For the bundled python solution see above: marked as obsolete, but now it no longer is. (- :

It's embarassing. There are plenty of libraries that mercurial needs missing now in the python installation. For yet another trick to solve this, see next chapter.

What finally worked for us;

As more and more problems occurred, I decided to split up the problem. So now all efforts went into first installing Python into a home directory and that works quite fluently, but the question was if all libraries were available - And they weren't! :/

With root access this all had not been problematic as the root problem were missing python-dev header files. So we had to grab the problem at its root and hence compiled our own python into our target $HOME directory.

This resolved the header problems. But when trying to install mercurial it complained once more - this time about missing bz2 which mercurial needs to compress files for versioning.

Of course the obvious solution is to fetch the sources for libbz2 but that left us with several other error messages as it is easily forgotten to copy the library (.so) file to $HOME/lib/ directory. For buliding a library .so file we also need yet another trick: using the alternative Makefile. So here go all the commands that finally succeeded:

On the local machine - that has to be almost identical to the remote target - issue the the following commands:

$ cd $HOME && sudo apt-get install python2.7 python2.7-dev && sudo apt-get source python2.7 python2.7-dev

$ scp $HOME/python2*.t* <user>@<target_remote>: #it's essentially a wget <python-dev-sources>

Fetch sources for Mercurial.

Change to the remote machine via ssh <user>@<target_remote> and enter:

$ wget http://mercurial.selenic.com/release/mercurial-2.8.tar.gz

$ tar -xf mercurial-2.8.tar.gz && mv mercurial-2.8 mercurial #if no error message is shown, everything went fine, omit the last command if desired.

Fetch missing library sources

$ wget http://www.bzip.org/1.0.5/bzip2-1.0.5.tar.gz && tar -xf bzip2-1.0.5.tar.gz && mv bzip2-1.0.5 bzip2 && cd bzip2

Don't forget to overwrite the PREFIX otherwise it will point to /usr/local and there we don't have access to! So:

$ export PYTHONPATH=$HOME/lib/python2.7/dist-packages/ && export PREFIX=$HOME

$ make -f Makefile-libbz2_so # build the library

$ make # build the library

$ make PREFIX=$HOME install

$ cp libbz2.so.1.0.4 $HOME/lib/libbz2.so.1.0.4 # make the library available in the general home lib path (so that it can be found)!

$ ln -s $HOME/lib/libbz2.so.1.0.4 $HOME/lib/libbz2.so.1.0 # in case applications try to find version *.so.1.0 instead of *.so.1.0.x

Resume the Python installation:

If you did not go the scp route as shown at the beginning of this chapter, then make sure you get the python<version>-dev packages! Otherwise header files will be missing. $ cd ~ && tar -xf <python-dev>.tar.gz && mv <python-dev> python-dev && cd python-dev

Now we have 'mercurial' and 'python-dev' directories and have changed our pwd (parent working directory) to the latter.

$ ./configure --prefix=$HOME && make # "We should not see bz2 on the not-built-module list now!" But don't worry, we will see it anyway in the next step. If we get no error concerning libbz2.so then all is fine!

$ make PREFIX=$HOME install

$ which python && $HOME/bin/python # Show which version is on the path and confirm our $HOME version works.

Now if we try to install mercurial, it might complain about missing docutils. So let's adapt another trick and install 'docutils' using 'easy_install', hence using setuptools.

Setuptools - Install docutils.

Fetch the relevant part of setuptools package: (if another python version is required look here for the .egg file):

$ wget https://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg

I figured that this command was missing to make sure we get a successful install: $ chmod +x ./setuptools-0.6c11-py2.7.egg # make it executable Now make sure that the PYTHONPATH is pointing to the correct python directory: $HOME/python2.7/lib/python2.7/site-packages/ . Check it using: echo $PYTHONPATH (If it's not correct, once again: export PYTHONPATH=$HOME/python2.7/lib/python2.7/site-packages/.

Then install easy_install binaries et alia using: $ ./setuptools-0.6c11-py2.7.egg --prefix=$HOME/python-dev/

$ ~/bin/easy_install docutils

Finish python installation

$ cd ~/python-dev/ && make altinstall prefix=$HOME exec-prefix=$HOME

$ ln -s ~/bin/python2.7 ~/bin/python # if python not already exists we add it as a symbolic link

Resume Mercurial installation.

Install mercurial and test it: If basic command list is shown, that'd be brilliant.

$ cd <mercurial-sources-dir> && make PYTHON=$HOME/bin/python install-home-bin && ~/bin/hg

Make python and hg commands available:

As the target server probably does have another installation of python in /usr/bin we want to overwrite the python command to automatically use the correct one [the one we installed ourselves]. So we append an alias to the .bashrc and .bash_profile that should get executed on ssh login too. Also we make the hg command available in our $HOME directory and all directories below included.

If .bashrc and .bash_profile not exist, create them:

If the files .bashrc and .bash_profile don't exist yet in $HOME directory - check it via ls -a $HOME - then use vi ~/.bashrc and type 'i' key for insert mode and insert #!/bin/sh. Then leave the Insert mode via ESC and save the file by typing :wq.

As an alternative:

$ touch ~/.bashrc && echo "#!/bin/sh" >> ~/.bashrc

$ touch ~/.bash_profile && echo "#!/bin/sh" >> ~/.bash_profile

Create alias:

$ echo "alias python='~/bin/python'" >> ~/.bashrc && echo "alias python='~/bin/python'" >> ~/.bash_profile && echo "alias hg='~/bin/hg'" >> ~/.bashrc && echo "alias hg='~/bin/hg'" >> ~/.bash_profile

The .hgrc file it used for defining exceptions and other settings. See this splendid [url=http://cgcookie.com/blender/2013/08/20/using-mercurial-version-control-with-blender-projects/]blender instructional mercurial video[/url] of Sebastian König.

$ '#mercurial setup\r\nexport HGRCPATH=$(HOME)/.hgrc' >> ~/.bash_profile && source ~/.bash_profile

Done - Follow the mercurial introduction.

Now follow the blender example tutorial a bit. You can omit the blender file manipulation and you will still learn a lot by simply following along and creating your own repository out of any files. Or you just start by navigating to your project folder via $ cd $HOME/<project_folder_within_home_including_below> and typing $ hg init and then hg for getting help or just $ hg add * to add files and hg commit for your first commit to check in where you start. Once you have changed something substantially and think it's required to go back to this version - e.g. because it's working and you plan a bigger change that probably breaks things - then just $ hg commit from within the project directory. Now have fun. : -)

Sources

• https://mail.python.org/pipermail/tutor/2002-March/012903.html
• http://isezen.com/2011/09/02/how-to-install-locally-python-on-linux-home-directory/
• http://ucla.jamesyxu.com/?p=95 <-- brilliant, this guy rocks