Wendelin Exanalytics Libre

WENDELIN combines Scikit Learn machine learning and NEO distributed storage for out-of-core data analytics in python

Table of Contents

Introducing Wendelin

Wendelin Core
NEO
ERP5
SlapOS
res6st
ShaCache
Scikit.Learn et al
  • This is Wendelin - stack written in 100% Python
  • Integrates with popular libs (Jupyter, scikit.learn, Pandas, ...)
  • Wendelin.Core component to compute beyond available RAM

Stack: Wendelin Core

Wendelin Core Computing
  • "Out-Of-Core"
  • Allows computation beyond limitation of physical RAM
  • Embedded within Wendelin stack, but also usable standalone

Stack: NEO

NEO Distributed Storage
  • Distributed Transactional NoSQL Object Database
  • Used as default storage in Wendelin

Stack: ERP5

ERP5 - Open Source Enterprise Application Framework
  • Based on 5 Theoretical Principles (Unified Business Model)
  • Generic: handles Big Data the same way as millions of ERP-related objects
  • ERP5 used under the hood of Wendelin

Stack: SlapOS

SlapOS | Hyperconvergent Platform as a Service
  • Cloud Provisionment, Deployment, Orchestration for single machines or clusters
  • Installable on any architecture
  • Uses Ansible recipes, fully automated, everything is a service
  • SlapOS is a bit like Gentoo (Linux) in that everything is compiled from scratch every time as described by a recipe. Recipes allow to reuse configurations across a cluster besides enabling a homogenous way to manage installable components.
  • Commercial Service available: Vifib.com (40€/4€ per server/vm per month)
  • SlapOS Website | Forum
  • SlapOS: Gitlab
  • -----------
  • Photo: Fotolia.fr - TOMO

Stack: re6st

Res6st | Resilient Mesh Overlay Network
  • IPv6-based Resilient Mesh Overlay Network, "internet on top of the internet"
  • Monitors possible connections between nodes, channels traffic on demand
  • Mitigates risks of unavialable routes/packet loss
  • Commercial service available: GrandeNet - Application Delivery Network
  • Res6st: Gitlab
  • -----------
  • Photo: Fotolia.fr - ake1150

Stack: ShaCache

Shacache | MD5-Signed Cache
  • MD5-signed cache, use signatures to validate cached files
  • Allows to reuse trusted compilations by providing key value
  • NoSQL storage with REST API, archives files through HTTP PUT method

Stack: Scikit.learn (et al)

Wendelin | Extensions
  • Integrate popular libraries for Data Science and Machine Learning
  • Provide familiar interface which connects to ERP5/Wendelin.Core
  • Scikit.Learn (project partner), Jupyter, SciPy, NumPy, Pandas
  • Tutorial to extend Wendelin with 3rd Party libraries using SlapOS: Coming soon
  • Scikit.Learn lectures
  • -----------
  • Photo: Fotolia.fr - weerapat1003

Wendelin: Advantages

Wendelin | Advantages
  • Single stack to handle big data from creation to utilization
  • Computation independent of available RAM
  • Fully written in Python, being built to handle Big Data

Wendelin: Challenges

Wendelin | Challenges
  • Simplify processes to make stack more accessable
  • Add Wendelin UI on top of ERP5
  • Provide tutorials and base documentation to foster adoption

Related Articles