June 23, 2015, written by Klaus Wölfel and Sven Franck
Today Mark Shuttleworth announced a new workaround for the IPv4 address space
shortage by claiming that "IPv6 is nowehre[sic] to be seen on the clouds"
This blog post is about re6stnet (repo,
our open source network, which solves the address space shortage problem by
providing IPv6 on top of existing IPv4. Re6stnet is in production at Nexedi and
clients since 2012 and is also used in our open source cloud-stack
SlapOS. We did not write much about
re6stnet until now, but thought this might be a good opportunity to explain what
it is and how it works.
"IPv4 address exhaustion is the depletion of the pool of unallocated
Internet Protocol Version 4 (IPv4) addresses, which has been anticipated since
the late 1980s. This depletion is the reason for the development and deployment
of its successor protocol, IPv6." The problem is particularly pressing
in the context of cloud computing, where hundreds of services can be instantiated
in a short time and want to be accessed individually from the internet. While
different short- and midterm mitigation efforts exist, the only long term-solution
to IPv4 is the deployment of IPv6
This is the reason our open source cloud-stack SlapOS was designed for IPv6 from
the ground up. Since IPv6 was and still is not available everywhere, we had to
find a solution to provide reliable IPv6 to machines running SlapOS.
Re6stnet is an IPv6-based low latency overlay network. It creates a resilient,
scalable IPv6 network "on top" of an existing ipv4 network by creating
tunnels (connections) on demand and then routing targeted traffic through
these tunnels. We use re6stnet to give IPv6 adresses to machines where only IPv4
is available and more importantly, to guarantee connectedness between computers which
have existing route connections to mitigate possible failures of the direct route.
So one could say we basically use it to establish our own routing system on the top
of DNS routing.
re6stnet routing technology shares similarities with peer-to-peer (P2P) technologies
such as Skype, PPStream or bittorrent. We initialize a mesh of direct routes
between selected edges in a network and leverage the mesh to create indirect
routes between all edges. Thanks to the babel (RFC 6126) distancevector routing
protocol, the best route is always selected to interconnect edges.
This way re6stnet allows to create very large, stable and private networks as
base architecture for our applications and implementations (side note: to ensure
resiliency we put part of our test (and live) system servers in hard to reach
places and countries).
A re6stnet network consists of at least one server (re6st-registry) and many nodes
(re6stnet). The server is only used to deliver certificates for secure
authentification in establishing tunnels, and to bootstrap new nodes. Re6stnet can
detect and take into account nodes present on the local network and
guarantees - that if there is an existing route between two machines - traffic will
be correctly routed between these two machines. Even if the registry node is down,
the probability that the network isn't connected is very low for big enough
networks (more than a hundred nodes).
Well, most importantly we have stable IPv6 everywhere - including on IPv4 legacy
networks. Besides that we use it to optimize routing for low latency or high
throughput based on application need and to aggregate bandwidth from multiple
ISPs for both uplink and downlink among others.
To conclude and come back to where we started, there is IPv6 in the cloud. It's
been there for a while. It's open-source and it works quite well. If you are
interested, check out the repo
or get in touch.