WENDELIN combines Scikit Learn machine learning and NEO distributed storage for out-of-core data analytics in python
Table of Contents
Wendelin Architecture
Wendelin architecture is based on 5 layers:
- Analytics layer: Wendelin leverages a wide variety of Numpy based analytics libraries such as scikit-learn, Pandas, NLTK, OpenCV-python, etc
- Storage layer: Wendelin stores native python objects on NEO distributed storage and thus eliminates format conversion steps found in other NoSQL technologies.
- Elasticity layer: Wendelin distributes data processing scripts on a cluster thanks to ERP5 active python object technology. Scripts are stored on NEO and can be modified in real time without any system restart.
- Deployment layer: Wendelin deployment is automated thanks to SlapOS mesh computing operating system. Analytics libraries are optimized automatically by SlapOS based on the targert CPU.
- Infrastructure Layer: Wendelin can be deployed on commodity hardware, private cloud or public cloud.
Key Features
- Python based
- Native code compiler for key algorithms
- GPU compiler for key algorithms
- Native storage of low level matrix data structure
- Best machine learning algorithms
- Wide scientific community thanks to Numpy
- Support 30+ years of FORTRAN optimizations
- Distributed multi-index
- Orthogonal index/storage topology for high throughput and fast access
Wendelin architecture provides key features not found in other platforms.
Related Articles