Tag Archives: Buildout

Compiling lxml on 64bit CentOS

A few days ago I encountered a problem while deploying Plone 4 with collective.xdv to a CentOS cloud instance. Since CentOS’ repos were a bit out of date I needed to statically compile lxml and it’s dependencies with z3c.recipe.staticlxml. Here’s what you need to add to your buildout.cfg to do so:

parts += lxml
eggs += lxml

# ===================================================================
# For collective.xdv to work properly, we need a static build of lxml
# and it's dependencies on OS X and x86_64 Linux                     
# ===================================================================
recipe = z3c.recipe.staticlxml
egg = lxml
force = false
static-build = true

And here’s the kicker: on some 64-bit Linux systems compiling lxml produces an error like this at compile-time:

lxml: Building lxml ...
Building lxml version 2.2.6.
NOTE: Trying to build without Cython, pre-generated 'src/lxml/lxml.etree.c' needs to be available.
Using build configuration of libxslt 1.1.24
Building against libxml2/libxslt in one of the following directories:
/usr/bin/ld: /home/production/1.1/parts/lxml/libxslt/lib/libxslt.a(xslt.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/home/production/1.1/parts/lxml/libxslt/lib/libxslt.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
error: Setup script exited with error: command 'gcc' failed with exit status 1
An error occured when trying to install lxml 2.2.6. Look above this message for any errors that were output by easy_install.
  Updating lxml.
Error: Couldn't install: lxml 2.2.6

Some problem with -fPIC flag not being set or something? After googling around without much success I decided to take a look directly in trunk of z3c.recipe.staticlxml and luckily Reinout had the same problem before me and already committed a patch. So all that was needed was to pull z3c.recipe.staticlxml from trunk and the compile error went away.

UPDATE: I emailed Stephan Eletzhofer to make a new release of z3c.recipe.staticxml that would include Reinout’s patch. He responded quickly and now all you need to do is make sure you have z3x.recipe.staticxml of version of 0.7.2 or higher:

# we need this so that -fPIC flag is set when compiling lxml on 64 bit Linux
z3c.recipe.staticlxml = 0.7.2

MladiPodjetnik.si performance tweaks

MladiPodjetnik.si is a portal we’ve been actively developing since 2006 and has since seen 3 revisions. The latest one, running on Plone 3.1, was running out of juice due to enormous increase in traffic in the last year. It was time to do some performance optimization.

1. We started out by upgrading Plone to the latest 3.x realease – 3.3.5. A fairly straightforward procedure.

2. Then we focused on optimizing Zope settings:

  • setting python-check-interval to 1000
  • lowering ZServer threads (we could do this because we were adding ZEO clustering with several Zopes)
  • tweaking ZODB cache size
  • disabling Z2.log
  • specifying only one language (Slovenian) for the Placeless Translation Service
# add these lines to [instance] section of your buildout.cfg
z2-log = off
zodb-cache-size = 13500
zserver-threads = 1
zope-conf-additional =
    python-check-interval 1000
environment-vars =

3. Binding to a newer version of zope.i18nmessageid fixes severe memory leaks in Zope.

zope.i18nmessageid = 3.5.1

4. Finally we built and configured HAProxy load-balancer so we can have several Zopes serving those requests.

Result: Tweaking proved to be successful as we no longer get time-outs on traffic peaks and the site feels much more snappier than before. Munin graphs confirm this.