Skip to content

[Sandbox] Xline #11

Closed
Closed
@rogercloud

Description

@rogercloud

Application contact emails

jicheng.shi@datenlord.com,pu.wang@datenlord.com, zheng.pan@datenlord.com, tracy.zhu@datenlord.com

Project Summary

A geo-distributed KV store for metadata management

Project Description

Xline is a distributed KV storage for metadata management. Xline makes it possible to manage metadata, such as indexes, permissions, and configurations across multiple clusters. Xline can achieve high-performance data access and strong consistency in cross data center scenarios. Currently, Xline is compatible with the ETCD interface, so that existing ETCD users can seamlessly switch to Xline and get high performance metadata management across multiple clusters.

The Aim of the Project

Data isolation and data fragmentation resulting from cloud barrier become impediments to business growth. The biggest challenge of multi-data center architecture is how to maintain strong data consistency and ensure high performance in the scenarios of multi-data center competition and modification. Traditional single data center solutions cannot meet the availability, performance and consistency requirements of multi-data center scenarios. DatenLord launched the open source distributed KV storage called Xline, which is targeted to multi-data center scenarios. It aims to realize a high-performance metadata management solution across multiple clusters, which is critical for businesses with geo-distributed, and multi-active deployment requirements.

Characteristics of the project

  1. Accelerate data access across clouds by caching hot data in memory, and provide unified data management to automate data migration and backup.
  2. It is the first geo distributed consistency management service based on the WAN consensus protocol. It addresses the challenges of convergence and consistency across clouds.
  3. Unified data management for multi-cluster scenarios, making mutual access, discovery, and modification simple and convenient.
  4. High stability and excellent storage performance; Provide KV interface, Multi-Version Concurrency Control and is compatible with ETCD.

Org repo URL

https://github.com/datenlord

Project repo URL

https://github.com/datenlord/xline

Additional repos

No response

Website URL

No response

Roadmap

https://github.com/datenlord/Xline#roadmap

Roadmap context

No response

Contributing Guide

https://github.com/datenlord/Xline/blob/master/CONTRIBUTING.md

Code of Conduct (CoC)

https://github.com/datenlord/Xline/blob/master/CODE_OF_CONDUCT.md

Adopters

No response

Contributing or Sponsoring Org

No response

Maintainers file

https://github.com/datenlord/Xline/blob/master/MAINTAINERS

IP Policy

  • If the project is accepted, I agree the project will follow the CNCF IP Policy

Trademark and accounts

  • If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

Why CNCF?

The geo-distributed deployment has become increasingly popular nowadays, in which situation Xline is of help in terms of unified management of metadata with high performance and security. We believe that Xline is a missing piece of the CNCF landscape. The project is motivated by challenges observed in cloud-native environments, and will benefit users and organizations that store data in multi-clusters and across clouds. Xline is built naturally for cloud native, compatible with ETCD interface. The success of this project depends on maintaining its vendor-neutrality, as well as keeping collaboration with project adjacent in the ecosystem. Given its mission and relationship with the CNCF projects, we truly believe CNCF, as an open source platform, is a perfect place to host Xline.

We hope to acquire support (e.g., legal assistance in aspects of potential IP and trademark issues) from CNCF, and collaborate with other projects in CNCF to make geo-distributed deployment convenient. We believe the inclusion of Xline in the CNCF ecosystem could help increase the awareness of the project. With the guidance of CNCF, we can follow its procedures and standards, and set attainable goals to refine our open source tools, promote community engagement, and integrate Xline with other cloud native projects.

Benefit to the Landscape

Xline is focused on the infrastructure of cloud native storage, which is vital to the development of cloud native technology. The joining of Xline will help CNCF accelerate the evolution of cloud native storage system in terms of data access in geo-distributed scenarios, in turn benefiting the entire cloud native ecosystem.

First, Xline is fully aligned with the cloud native computing ecosystem because it is compatible with ETCD interface. ETCD enables K8S management in a single cluster, while Xline expands cloud native scenarios to multi clusters and allows users to experience task orchestration across multiple clusters by K8S. Moreover, Xline achieves metadata management among multi-clusters with high performance and security.

Cloud Native 'Fit'

No response

Cloud Native 'Integration'

No response

Cloud Native Overlap

No response

Similar projects

ETCD and ZooKeeper

Product or Service to Project separation

N/A

Project presentations

No response

Project champions

No response

Additional information

Here's the product intro slides

Activity

dims

dims commented on Jan 6, 2023

@dims
Member

@rogercloud looks like some notes on CURP vs RAFT would be a good thing, can you please tell us how they compare? thanks!

rogercloud

rogercloud commented on Feb 27, 2023

@rogercloud
Author

@dims For the requests don't conflict with each other, CURP takes half time to get the requests committed compared to Raft, because CURP requries only one RTT while Raft requires two. In the multi-cluster or multi-datecenter casese, CURP out-performs Raft. Even in the worst case that every request conflicts with others, CURP's performance is the same as Raft's. You get the benchmark result in our project page.

cathyhongzhang

cathyhongzhang commented on May 8, 2023

@cathyhongzhang

It is mentioned that the geo distributed consistency management service is based on the WAN consensus protocol. Which protocol is it? How does this protocol compare with raft and paxos? Could you list the Pros and cons?

cathyhongzhang

cathyhongzhang commented on May 8, 2023

@cathyhongzhang

I do not see the maintainer's name. Did I miss something?

justincormack

justincormack commented on May 9, 2023

@justincormack

Have you done any correctness testing on the project? eg Jepsen or equivalent?

self-assigned this
on May 9, 2023
moved this from 🏗 Upcoming to ⏲ Waiting in Sandbox Application Board - next Review is July 22on May 9, 2023
added
Need-infoThe TOC has requested more information from the project.
and removed
NewNew Application
on May 9, 2023
rogercloud

rogercloud commented on May 9, 2023

@rogercloud
Author

@cathyhongzhang
The protocol is called CURP, which is published in NSDI'19.

Compared to Raft or Paxos, it takes only 1 RRT to commit a request in the normal case while Raft takes 2. Even for the worst case, every request is changing the same key's value, CUPR's performance is the same as Raft. For more protocol comparison, please refer to the blog. For more performance result, you can refer to the README of the project.

The Pros of CURP:

  • Easy to implement.
  • Geo-distributed friendly.

The Cons of CURP:

  • More information is required for the requests, such as whether two requests conflict with each other. For KV store, it's easy to tell.

38 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

Status

✅ Done

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @dims@amye@justincormack@RichiH@rogercloud

      Issue actions

        [Sandbox] Xline · Issue #11 · cncf/sandbox