Cloud native

Jun 4, 2018 - 浏览量： - 次 |

Posted by YSL Blog on June 4, 2018
  • 云原生大神Jimmy Song耗时九个月苦心整理。小白拿来学习啦 ：）
  • A curated list of awesome cloud native architectures, tools and softwares. Inspired by awesome-go.

Contributing

Please take a quick gander at the contribution guidelines first. View in GitHub. Thanks to all contributors, you rock🤟!

Contents

AI

API gateway

Big Data

  • fast-data-dev - Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, Landoop Tools, 20+ connectors
  • spark - Apache Spark enhanced with native Kubernetes scheduler back-end
  • wallaroo - Ultrafast and elastic data processing https://www.wallaroolabs.com

Container engine

  • Containerd - An open and reliable container runtime https://containerd.io
  • Kata Containers - Kata Containers is a new open source project building extremely lightweight virtual machines that seamlessly plug into the containers ecosystem.
  • Moby - Moby Project - a collaborative project for the container ecosystem to assemble container-based systemshttps://mobyproject.org/
  • clear-containers - OCI (Open Containers Initiative) compatible runtime using Virtual Machines
  • cri-containerd - Containerd-based implementation of Kubernetes Container Runtime Interface
  • cri-o - Lightweight Container Runtime for Kubernetes
  • frakti - The hypervisor-based container runtime for Kubernetes.
  • gvisor - Sandboxed Container Runtime
  • hyperd - HyperContainer Daemon http://www.hypercontainer.io
  • pouch - Pouch is an open-source project created to promote the container technology movement.
  • railcar - RailCar: Rust implementation of the Open Containers Initiative oci-runtime
  • rkt - rkt is a pod-native container engine for Linux. It is composable, secure, and built on standards.

CI-CD

  • CircleCI - Continuous Integration and Deployment
  • Cyclone - A cloud native CI/CD platform built for container workflow
  • Drone - Drone is a Continuous Delivery platform built on Docker, written in Go https://drone.io
  • GitLab-CI - GitLab has integrated CI/CD pipelines to build, test, deploy, and monitor your code
  • Hygieia - CapitalOne DevOps Dashboard http://www.capitalone.io/Hygieia
  • Jenkins - The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project.
  • Travis - Free continuous integration platform for GitHub projects. https://travis-ci.org
  • Wercker - The Wercker CLI can be used to execute pipelines locally for both local development and easy introspection. http://wercker.com
  • argo - Get stuff done with container-native workflows for Kubernetes.
  • buddy.works - Build, Test & Deploy Code in Seconds Continuous Delivery, simplified.
  • concourse - BOSH release and development workspace for Concourse https://concourse.ci
  • cross-cloud - Cross Cloud Continuous Integration https://cncf.io
  • gitkube - Gitkube: Build and deploy docker images to Kubernetes using git push. https://gitkube.sh
  • jx - A command line tool for installing and working with Jenkins X http://jenkins-x.io/
  • kenyata - Automated Canary Service
  • pipeline - REST API to provision or reuse managed Kubernetes clusters in the cloud and deploy cloud native apps
  • skaffold - Easy and Repeatable Kubernetes Development

Database

  • ArangoDB - ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.
  • beringei - Beringei is a high performance, in-memory storage engine for time series data.
  • CockroachDB - CockroachDB - the open source, cloud-native SQL database. https://www.cockroachlabs.com
  • CouchDB - Apache CouchDB is one of a new breed of database management systems.
  • etcd - Distributed reliable key-value store for the most critical data of a distributed systemhttps://coreos.com/etcd/docs/latest/
  • InfluxDB - Scalable datastore for metrics, events, and real-time analytics https://influxdata.com
  • LevelDB - LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
  • MongoDB - MongoDB is an open source database that uses a document-oriented data model.
  • OpenTSDB - A scalable, distributed Time Series Database. http://opentsdb.net
  • Redis - Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, HyperLogLogs, Bitmaps. http://redis.io
  • RethinkDB - The open-source database for the realtime web. https://rethinkdb.com
  • TiDB - TiDB is a distributed NewSQL database compatible with MySQL protocol https://pingcap.com

Data Science

DevOps

Fault tolerant

  • Chaosmonkey - Chaos Monkey is a resiliency tool that helps applications tolerate random instance failures.
  • Hystrix - Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.

IoT

Logging

Observability

  • kiali - kiali project to help istio service mesh observability

Orchestration and scheduler

Portability

Proxy and load balancer

  • Contour - Contour is a Kubernetes ingress controller for Lyft’s Envoy proxy.
  • Envoy - C++ front/service proxy https://lyft.github.io/envoy
  • Haproxy - HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications.
  • Kong-ingress - A Kubernetes Ingress for Kong
  • Nginx-kubernetes-ingress - NGINX and NGINX Plus Ingress Controllers for Kubernetes
  • Nginx - nginx [engine x] is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server, originally written by Igor Sysoev.
  • Ribbon - Ribbon is a Inter Process Communication (remote procedure calls) library with built in software load balancers. The primary usage model involves REST calls with various serialization scheme support.
  • Traefik - Træfik, a modern reverse proxy https://traefik.io
  • caddy - Fast, cross-platform HTTP/2 web server with automatic HTTPS https://caddyserver.com
  • gimbal - Heptio Gimbal is an ingress load balancing platform capable of routing traffic to multiple Kubernetes and OpenStack clusters. Built by Heptio in partnership with Actapio. https://www.heptio.com
  • katran - A high performance layer 4 load balancer
  • kong/kubernetes-ingress-controller - Deploy Kong in a native Kubernetes Ingress Controller https://konghq.com/
  • skipper - An HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress
  • voyager - ✈️️ Secure Ingress Controller for Kubernetes by https://appscode.com

RPC

  • brpc - Most common RPC framework used throughout Baidu, with 600,000+ instances and 500+ kinds of services, called “baidu-rpc” inside Baidu.
  • finagle - A fault tolerant, protocol-agnostic RPC system http://twitter.github.io/finagle
  • gRPC - A high performance, open source, general-purpose RPC framework
  • proxygen - A collection of C++ HTTP libraries including an easy to use HTTP server.
  • sofa-bolt - SOFABolt is a lightweight, easy to use and high performance remoting framework based on Netty.
  • sofa-rpc - SOFARPC is a high-performance, high-extensibility, production-level Java RPC framework.
  • thrift - Apache thrift

Message broker

  • RabbitMQ - RabbitMQ is the most widely deployed open source message broker
  • flume - Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.
  • gnatsd - High-Performance server for NATS, the cloud native messaging system. https://nats.io
  • jocko - Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)https://twitter.com/travisjeffery
  • kafka - A distributed streaming platform.
  • nsq - A realtime distributed messaging platform http://nsq.io/

Miscellaneous

  • ballerina-lang - Ballerina is a new programming language for integration built on a sequence diagram metaphor.

Monitoring

Networking

Security and audit

  • AppArmor - AppArmor is an effective and easy-to-use Linux application security system.
  • authenticator - A tool for using AWS IAM credentials to authenticate to a Kubernetes cluster
  • Cilium - Linux Native, HTTP Aware Networking and Security for Containers
  • Clair - Vulnerability Static Analysis for Containers
  • OpenSCAP - Discover a wide array of tools for managing system security and standards compliance.
  • cert-manager - Automatically provision and manage TLS certificates in Kubernetes
  • dex - OpenID Connect Identity (OIDC) and OAuth 2.0 Provider with Pluggable Connectors https://coreos.com/blog/announcing-de…
  • docker-bench-security - The Docker Bench for Security is a script that checks for dozens of common best-practices around deploying Docker containers in production.
  • dockscan - dockscan is security vulnerability and audit scanner for Docker installations
  • drydock - drydock provides a flexible way of assessing the security of your Docker daemon configuration and containers using editable audit templates
  • falco - Behavioral Activity Monitoring With Container Support
  • goldfish - A HashiCorp Vault UI panel written with VueJS and Vault native Go API https://vault-ui.io
  • grafeas - Cloud artifact metadata CRUD API and resource specifications
  • guard - Kubernetes Authentication WebHook Server
  • k8guard - An auditing system for Kubernetes
  • kube-bench - The Kubernetes Bench for Security is a Go application that checks whether Kubernetes is deployed according to security best practices
  • kube-lego - Automatically request certificates for Kubernetes Ingress resources from Let’s Encrypt
  • kube2iam - kube2iam provides different AWS IAM roles for pods running on Kubernetes
  • kubed - 🛡️ A Kubernetes Cluster Operator Daemon by https://appscode.com
  • metallb - A network load-balancer implementation for Kubernetes using BGP
  • notary - Notary is a Docker project that allows anyone to have trust over arbitrary collections of data https://docker.com
  • opa - An open source project to policy-enable your service. http://openpolicyagent.org
  • spiffe - The SPIFFE Project http://spiffe.io
  • vault - A tool for managing secrets. https://www.vaultproject.io

Service broker

  • open-service-broker-sdk - A starting point for creating service brokers implementing the Open Service Broker API
  • rotor - Rotor is a fast, lightweight bridge between your service discovery and Envoy’s configuration APIs. Rotor supports Kubernetes, Consul, AWS (EC2 and ECS), DC/OS, flat files, and even other EDS/CDS implementations.
  • service-catalog - Consume services in Kubernetes using the Open Service Broker API
  • service-broker - Open Service Broker API Specification https://openservicebrokerapi.org/

Service mesh

  • Amalgam8 - Content and Version-based Routing Fabric for Polyglot Microservices
  • Conduit - The Ultralight Service Mesh for Kubernetes https://conduit.io
  • Istio - An open platform to connect, manage, and secure microservices.
  • Linkerd - Resilient service mesh for cloud native apps https://linkerd.io
  • nginmesh - Service Mesh using Nginx
  • nginx-unit - NGINX Unit is a new, lightweight, open source application server built to meet the demands of today’s dynamic and distributed applications.
  • ServiceComb - ServiceComb is a microservice framework that provides an easy way to develop and deploy applications in the cloud.

Service registry and discovery

Serverless

Storage

  • Ceph - Ceph is a distributed object, block, and file storage platform http://ceph.com
  • Convoy - A Docker volume plugin, managing persistent container volumes.
  • FastDFS - FastDFS is an open source high performance distributed file system (DFS). It’s major functions include: file storing, file syncing and file accessing, and design for high capacity and load balance.
  • Flocker - Container data volume manager for your Dockerized application https://clusterhq.com
  • GlusterFS - Gluster is a software defined distributed storage that can scale to several petabytes. It provides interfaces for object, block and file storage.
  • Heketi - RESTful based volume management framework for GlusterFS
  • Infinit - The Infinit policy-based software-defined storage platform. http://infinit.sh
  • LeoFS - The LeoFS Storage System https://leo-project.net/leofs/
  • Longhorn - We put storage on cows and move them around from rancher.
  • Minio - Minio is an open source object storage server compatible with Amazon S3 APIs https://minio.io
  • OpenEBS - OpenEBS is containerized block storage written in Go for cloud native and other environments w/ per container (or pod) QoS SLAs, tiering and replica policies across AZs and environments, and predictable and scalable performance. https://www.openebs.io
  • Rook - File, Block, and Object Storage Services for your Cloud-Native Environment https://rook.io
  • StorageOS - Enterprise persistent storage for containers and the cloud
  • Torus - Torus Distributed Storage https://coreos.com/blog/torus-distributed-storage-by-cores.html
  • Vitess - Vitess is a database clustering system for horizontal scaling of MySQL. http://vitess.io
  • Zenko - Because everyone should be in control of their data. http://zenko.io
  • glusterd2 - GlusterD-2.0 is the distributed management framework to be used for GlusterFS-4.0

Tools

  • Aglio - An API Blueprint renderer with theme support that outputs static HTML
  • Ark - Heptio Ark is a utility for managing disaster recovery, specifically for your Kubernetes cluster resources and persistent volumes. Brought to you by Heptio. http://www.heptio.com
  • Draft - A tool for developers to create cloud-native applications on Kubernetes.
  • Dragonfly - Dragonfly is an intelligent P2P based file distribution system.
  • Habitus - A Build Flow Tool for Docker http://www.habitus.io
  • Helm - The Kubernetes Package Manager
  • Smith - Smith: A microcontainer builder
  • Swagger - Swagger UI is a collection of HTML, Javascript, and CSS assets that dynamically generate beautiful documentation from a Swagger-compliant API. http://swagger.io
  • Vagrant - Vagrant is a tool for building and distributing development environments. https://www.vagrantup.com
  • armada - A python orchestrator for a installing, upgrading, and managing a collection of helm charts, dependencies, and values overrides.
  • chaostoolkit - An Open API to Chaos Engineering http://chaostoolkit.org
  • charitify - 📈 Generate Helm Charts from Kubernetes objects by https://appscode.com
  • client-go - Go client for Kubernetes.
  • container-structure-test - validate the structure of your container images
  • container-transform - Transforms docker-compose, ECS, and Marathon configurations
  • crashcart - CrashCart: sideload binaries into a running container
  • cri-tools - CLI and validation tools for Kubelet Container Runtime Interface (CRI) .
  • docker-elk - The ELK stack powered by Docker and Compose.
  • dockersh - A shell which places users into individual docker containers
  • dotmesh - dotmesh (dm) is like git for your data volumes (databases, files etc) in Docker and Kubernetes https://dotmesh.com
  • drakov - Mock Server that implements the API Blueprint specification
  • escalator - Escalator is a batch or job optimized horizontal autoscaler for Kubernetes
  • flux - A tool for turning container images into running Kubernetes services
  • freshpod - Restart Pods on Minikube automatically on image rebuilds
  • gockerize - Package golang service into minimal docker containers.
  • jsonnet - Jsonnet - The data templating language http://jsonnet.org
  • kail - kubernetes log viewer
  • kaniko - Build Container Images In Kubernetes
  • kd - Minimalistic kubernetes resources deployment tool with templating
  • kedge - Kedge - Concise Application Definition for Kubernetes http://kedgeproject.org
  • kismatic - Kismatic Enterprise Toolkit: Fully-Automated, Production-Grade Kubernetes Operations
  • kompose - Go from Docker Compose to Kubernetes http://kompose.io
  • kops - Kubernetes Operations (kops) - Production Grade K8s Installation, Upgrades, and Management
  • ksonnet - A CLI-supported framework that streamlines writing and deployment of Kubernetes configurations to multiple clusters.https://ksonnet.io/
  • ksonnet-lib - (technical preview) Simplify working with Kubernetes http://ksonnet.heptio.com
  • ktmpl - Parameterized templates for Kubernetes manifests.
  • kube-shell - Kubernetes shell: An integrated shell for working with the Kubernetes CLI
  • kubeadm - Aggregator for issues filed against kubeadm
  • kubeadm-offline-installer - Setup a cluster with kubeadm, without internet connections.
  • kubeapps - Discover & launch great Kubernetes-ready apps
  • kubecfg - A tool for managing complex enterprise Kubernetes environments as code.
  • kubedb - KubeDB CLI https://kubedb.com to manage kubernetes ready production-grade Databases
  • kubegen - kubegen – simple way to describe Kubernetes resources
  • kube-ps1 - Kubernetes prompt info for bash and zsh
  • kubernetes-client - Java client for Kubernetes & OpenShift 3 http://fabric8.io
  • kubernetes-deploy - A command-line tool that helps you ship changes to a Kubernetes namespace and understand the result
  • kubespray - Setup a kubernetes cluster also mentioned as kargo
  • kubeup - Cluster operation the Kubernetes way
  • kube-version-converter - Convert API Object file into specified version.
  • kustomize - Customization of kubernetes YAML configurations
  • minikube - Run Kubernetes locally
  • opencompose - OpenCompose - A higher level abstraction for Kubernetes Resource
  • operator-sdk - SDK for building Kubernetes applications. Provides high level APIs, useful abstractions, and project scaffolding. https://coreos.com/operators
  • prometheus-operator - Prometheus Operator creates/configures/manages Prometheus clusters atop Kuberneteshttps://coreos.com/operators/prometheus
  • searchlight - 🔦 Alerts for Kubernetes
  • sonobuoy - Heptio Sonobuoy is a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of Kubernetes conformance tests in an accessible and non-destructive manner. Brought to you by Heptio.http://www.heptio.com
  • source-to-image - A tool for building/building artifacts from source and injecting into docker images
  • squash - The debugger for microservices
  • stash - 🛅 Backup your Kubernetes Volumes by <htts://appscode.com>
  • stern - Multi pod and container log tailing for Kubernetes
  • tectonic-installer - Install a Kubernetes cluster the CoreOS Tectonic Way: HA, self-hosted, RBAC, etcd Operator, and more
  • telepresence - Local development against a remote Kubernetes or OpenShift cluster http://www.telepresence.io
  • terminus - Graceful shutdown and Kubernetes readiness / liveness checks for any Node.js HTTP applications
  • test-infra - Test infrastructure for the Kubernetes project.
  • watchtower - Automatically update running Docker containers

Tracing

  • Apache SkyWalking - An APM system for tracing, monitoring, diagnosing distributed systems, especially based on microservices, cloud native and container
  • OpenCensus - A single distribution of libraries that automatically collect traces and metrics from your app, display them locally, and send them to any backend. - https://opencensus.io
  • OpenTracing - Consistent, expressive, vendor-neutral APIs for distributed tracing and context propagation
  • Sentry - Sentry is a cross-platform crash reporting and aggregation platform. https://sentry.io
  • Zipkin - Zipkin is a distributed tracing system http://zipkin.io
  • appdash - Application tracing system for Go, based on Google’s Dapper. https://sourcegraph.com
  • jaeger - Jaeger, a Distributed Tracing System http://uber.github.io/jaeger/
  • sofa-tracker - SOFATracer is a component for the distributed system call trace. And through a unified traceId logging the logs of various network calls in the invoking link . These logs can be used for quick discovery of faults, service governance, etc.

Tutorial

FEATURED TAGS
FRIENDS