Plone Conf Tokyo Buildout Powered Continuous Integration Testing System

Plone Conference 2018 Tokyo CI Testing System implemented based on Buildout
  • Last Update:2018-11-08
  • Version:001
  • Language:en

Buildout powered CI Testing system

PloneConfTokyo Logo

Yusei Tahara

yusei@nexedi.com

This presentation introduces our CI testing system powered by buildout.

Agenda

  • Who is Nexedi?
  • Demo from user point of view
  • We use Buildout
  • Buildout + Resource Management
  • Inside Testing System

 

Nexedi: largest OSS publisher in Europe

 

Many different clients, all over the world

 

Buildout is excellent!

 

Buildout Profile

sqllite buildout.cfg at https://lab.nexedi.com/nexedi/slapos/blob/master/component/sqlite3/buildout.cfg

Demo

 

Overview

 

ERP5 : Powerful Business App Framework

 

ERP5 Buildout profile

https://lab.nexedi.com/nexedi/slapos/tree/master/software/erp5

453 sections

Following recipes are used to build ERP5 software.

    201  recipe= slapos.recipe.cmmi
          30   recipe= zc.recipe.egg:custom
          13   recipe= zc.recipe.egg
          12   recipe= plone.recipe.command
          9    recipe= slapos.recipe.template:jinja2
          8    recipe= slapos.recipe.build:gitclone
          8    recipe= hexagonit.recipe.download
          7    recipe= slapos.recipe.build:download
          6    recipe= zc.recipe.egg:develop
          3    recipe= zc.recipe.egg:scripts
          3    recipe= slapos.recipe.build:download-unpacked
          3    recipe= slapos.recipe.build
          1    recipe= hexagonit.recipe.cmmi
          1    recipe= erp5.recipe.w3validator
        

 

 

Buildout + Resource management = SlapOS

 

SlapOS Structure

Deployed Services and Supervisord

slapconfiguration recipe

    [slap-configuration]
          recipe = slapos.cookbook:slapconfiguration.serialised
          computer = ${slap-connection:computer-id}
          partition = ${slap-connection:partition-id}
          url = ${slap-connection:server-url}
          key = ${slap-connection:key-file}
          cert = ${slap-connection:cert-file}
        
    [jinja2-template-base]
          recipe = slapos.recipe.template:jinja2
          context =
          key slapparameter_dict slap-configuration:configuration
        
    #instance-zope.cfg.in
          {% set instance_index_list = range(slapparameter_dict['instance-count']) -%}
          {% for i in instance_index_list -%}
          {{   zope(
          i,
          next_port(),
          ) }}
          {% endfor -%}
        

slapconfiguration recipe is used when an instance is built. It receives parameters entered by user from SlapOS Master. Thanks to this, user can change application environment dynamically via SlapOS Master.

 

Zope cluster configuration

Let's review

  • Buildout is executed remotely by SlapOS Master.
  • Buildout is used to built software and setup an application execution environment in SlapOS.(we call them Software and Instance)
  • Deployed application environment(Instance) can be changed dynamically by receiving parameters from SlapOS Master.

Test Node (1)

User specifies the test master's location when requesting a test node.
 

Test Node (2)

 

Test Suite

Custom buildout.cfg is generated according to the test suite definition, then builtout uses it to build software and deploy an application environment inside test node dummy slapos environment.
 

Test Result (1)

Testnode looks for runTestSuite command inside instance directory. This command is executed if any of VCS repositories specified by test suite have new commit.
 

Test Result (2)

 

Thank You

All source code is available at https://lab.nexedi.com/nexedi/