WENDELIN combines Scikit Learn machine learning and NEO distributed storage for out-of-core data analytics in python
Table of Contents
Note: A binary cache has been uploaded to shacache as well as all of
the required python modules for Debian 8.1, 64 bit. This should speed up install
/ setup time a lot as there will be no need to wait for long downloads or
compilations. If you are using a different platform/arch, this cache cannot
be used and all software will be downloaded from source and compiled locally
which would mean much longer setup time!
- 4 GB of RAM
- At least 20GB of disk space.
A virtual machine running Debian 8, 64bit as only it has
been thoroughly tested. Debian 8 images are available here (
Note #1: openblast may fail to compile if CPU type of the VM is unknown.
This is an openblast issue which we can not resolve and it's needed that you
properly configure your VM's CPU type. You can see some examples below,
unfortunately this is different for every Virtual machine provider. In our example
we use '-cpu host' to use host machine's CPU type.
/usr/bin/qemu-kvm -cpu host -m 4096 -hda /home/vm/debian8.1.img -net nic,vlan=0,model=virtio,macaddr=00:16:35:AF:94:4B -net tap,vlan=0,ifname=$tap,script=no,downscript=no -vnc :1 -no-reboot -enable-kvm -smp 2
Note #2: it's your responsibility to setup and install Debian 8 on a VM.
- Note 3: If something went wrong or didn't work correctly, do not be
afraid to re-run the ansible script using bash wendelin-standalone.
This script is completely safe to rerun as many as times as you like and
doing so can resolve issues.
Once you have your virtual machine up and running, run (root permissions are needed):
root@debian8:~# wget http://deploy.nexedi.cn/wendelin-standalone && bash wendelin-standalone
Monitoring progress of Wendelin installation
To check up on the status of your build automatically please run:
root@debian8:~# watch -n 30 erp5-show -s
This way if something goes wrong you will be notified immediately. The total
time for install and setup should be around less than an hour, however it doesn't
need any interaction from user.
Another approach to monitor is to tail Wendelin installation status like so:
root@debian8:~# tail -f /opt/slapos/log/*.log
When is my Wendelin instance ready?
You can check if your Wendelin instance is ready using:
root@debian8:~# erp5-show -s
If your instance is ready the script will printout:
root@debian8:~# erp5-show -s
Build successful, connect to: https://zope:insecure@[2001::bd4d]:16001
Please note that this IPv6 is an internal one for your machine and the only way
to access is to run browser inside the VM machine for this see next chapter.
Access Wendelin instance
In case one needs to access the Wendelin instance from outside the VM machine
there are two ways described below:
Access through socat open ports on public VM's IP address (recommended)
We have developed a script which use socat command to make all internal Wendelin
services ports available through a public VM's IP address. Just run:
root@debian8:~# wget https://lab.nexedi.com/nexedi/wendelin/raw/master/utils/wendelin-standalone-bind.sh && bash wendelin-standalone-bind.sh
This script will allow you to connect to following services:
- Jupiter node at https://<YOUR_VM_IP>:20000/
- Zope node 1 at http://<YOUR_VM_IP>:20001/
- Zope node 2 at http://<YOUR_VM_IP>:20002/
- Zope node 3 at http://<YOUR_VM_IP>:20003/
Access VM through ssh tunnelling to it (not recommended)
- Find the internal IPv4 on whcih Wendelin is listening to
root@debian8:~# ps xa | grep runzope
root@debian8:~# vi /srv/slapgrid/slappart0/etc/zope.conf
#(and fine block with
# address 10.0.210.201:12001
- Create a ssh tunnel like above from host linux machine
LocalForward 2200 10.0.210.201:12001
- Access locally http://localhost:2200/ and use for username: zope and for password: insecure
This is a temporary approach until we provide a front end solution.
Configure Wendelin instance
The process is described in this HowTo.
Currently you may run into the following issues during installation:
OpenBLAS fails during compile as it cannot detect the CPU type. A long
error log will be spit out when this happens... again, you can check the
status of your build with
erp5-show -s and if there is
an error, view
Here you will find the output of
slapos node software.
To make it easier to read and follow, use
TARGET=CORE2 slapos node software
Sometimes when accessing Wendelin site you might get:
To fix one must restart MariaDB (in command we restart all just in case):
Error Type: OperationalError
Error Value: (1129, "Host '10.0.221.219' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'")
root@debian8w:~# slapos node restart all
If after your VM gets restarted not all processes are started (slapos node
reports them as EXITED) then one can do (see below). Same command will fix
" ERROR No IPv6 found on interface No valid IPv6 found on
lo. to construct IPv6 with" message (visible in /opt/slapos/log/slapos-node-format.log)
root@debian8:~# slapos node format --now
root@debian8:~# slapos node restart all