How to work on a CubicWeb cube without cloning the whole tree

cubicweb

pip install -e / setup.py develop on a cube

2 novembre 2014

3mn de lecture

This post was part of the now dead Unlish blog.

The current layout of CubicWeb cubes is not setuptools friendly, and does not permit to use the “develop” mode. The CWEP 004, currently under discussion, aims to solve that problem, and once adopted make life easier for the developers.

Meanwhile, some people (like us) would like to use the develop mode anyway, so that we can work on a cube without cloning the whole tree of CubicWeb requirements.

After a few experiments, it turns out we can tweak the virtualenv after pip install -e is run on the cube, and have the benefit of it.

The solution I describe below allows to work on both the cube and the unittests, importing from “cubes.thecube” and “cubes.anothercube” in a virtualenv. It can be applied to several cubes in the same virtualenv. It does not apply to the other cubicweb packages.

From this point, ’thecube’ is the cube that we work on, ‘anothercube’ is any other cube involved in the application we develop.

Problem 1 - the cubes path

Once pip install -e has been run the cube on which it was launched is not usable because it is not present in the cube path.

We do not want to add the parent directory of our cube to CW_CUBES_PATH because:

When importing from cubes.acube, and because “cubes” is not a proper namespace, we have issues
The spirit of virtualenv is that where the project was checked out does not matter -> we do not want to put restriction on where it can be put in the filesystem.

Solution

From within the virtualenv :

1

ln -s $(pwd) $VIRTUAL_ENV/share/cubicweb/cubes/thecube

Problem 2 - the python path

pip install -e add our directory to the interpreter path. It is a problem because it pollutes the root namespace, it not masks some modules when the names match (for example in unlish we have a ‘markdown.py’ file).

For some reason I did not identified, I could not import from cubes.xxx because the virtualenv cubes path is not added to the syspath like I expected it to (I may have done something wrong on this matter).

Solution

Remove our directory from the virtualenv path

1
2
3


PTH=$VIRTUAL_ENV/lib/python2.7/site-packages/easy-install.pth
grep -v $(pwd) $PTH > .pth.tmp
mv .pth.tmp $PTH

Add the virtualenv cubes path to the python path

1

export PYTHONPATH=$VIRTUAL_ENV/share/cubicweb

This last command has to be done each time the virtualenv is activated, it is recommended to add it to the ‘activate’ script, OR postactivate script if you are using virtualenvwrapper (which I recommend too).

Put it all together

The following script does all that for you, and has to be run after each run of pip install -e.

Note that it will overwrite your postactivate script, set CW_MODE=user and define a ‘ctl’ alias, so you may need to modify it before using it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33


!/bin/bash

MYPATH=$(pwd)
MODNAME=$(python -c "import __pkginfo__; print __pkginfo__.modname")

if [ -z "$VIRTUAL_ENV" ]; then
    exit 1
fi

if [ -n "$VIRTUAL_ENV/bin/postactivate" ]; then
    cat >>$VIRTUAL_ENV/bin/activate <<EOF
. $VIRTUAL_ENV/bin/postactivate
EOF
fi

cat >$VIRTUAL_ENV/bin/postactivate <<EOF
export CW_MODE=user
export PYTHONPATH=$VIRTUAL_ENV/share/cubicweb

alias ctl=cubicweb-ctl
EOF

rm -rf $VIRTUAL_ENV/share/cubicweb/cubes/$MODNAME
ln -s $MYPATH $VIRTUAL_ENV/share/cubicweb/cubes/$MODNAME

PTH=$VIRTUAL_ENV/lib/python2.7/site-packages/easy-install.pth
grep -v $MYPATH $PTH > .pth.tmp
mv .pth.tmp $PTH

echo "Tweaked virtualenv $VIRTUAL_ENV"
echo "Don't forget to re-activate it:"
echo ""
echo "   workon $(basename $VIRTUAL_ENV)"

Conclusion

To conclude, this is only a hackish workaround to wait for the real solution: reforming the cubes in cubicweb.

(this post was initially a mail on the cubicweb mailing-list)

cubicweb python pip

Voir tous les Tags

2 novembre 2014

3mn de lecture