Stateful Kubernetes Applications Made Easier: PSO and FlashBlade
转自:https://medium.com/@joshua_robinson/stateful-kubernetes-applications-made-easier-pso-and-flashblade-aa3e2ebb0248
For stateful applications on Kubernetes, Pure Storage Orchestrator automates and productionizes those services with FlashBlade as shared storage. This repository presents three illustrative example applications: a simplistic shared scratch space as well as file-sharing services NextCloud and OwnCloud.
I assume the reader has only a basic understanding of Kubernetes and is interested in how to use FlashBlade with Kubernetes.
For a more general introduction and guide to PSO for both FlashArray and FlashBlade, see this explanation of Kubernetes Storage, Kubernetes documentation, and Containers-as-a-service architecture.
Introduction
The Pure Storage Orchestrator (PSO) automates the process of creating filesystems on FlashBlade and attaching them to the running applications in Kubernetes.
But first, why run applications in Kubernetes? There is a generic set of problems that almost all applications need to solve: recovering from hardware failures, scaling up or down resources, orchestrating connectivity between inter-linked services, securing and isolating environments, and provisioning resources for running applications. Kubernetes helps solve these problems, simplifying the operation of production services.
Why Use PSO and FlashBlade
In Kubernetes, applications run inside a set of pods, which are ephemeral and not tied to physical compute or storage resources. If the pod needs persistent storage for any reason (and there are many!), then PersistentVolumes are needed. But manual storage administration is complicated: creating volumes, attaching them to running pods regardless of protocol, and finally deleting volumes. In a fully automated system, none of these steps should involve a human!
PSO automates the process of provisioning storage and hides the details of storage creation and attachment to each pod. The result is self-service storagethat matches cloud-native workflows and applications.
FlashBlade provides shared filesystems for Read-Write-Many (RWX) volumes. These are critical for scale-out applications that spawn multiple pods; each additional pod is able to automatically share access to a common data store. FlashBlade can also support Read-Write-Once (RWO) volumes.
While I focus here on FlashBlade filesystems as volumes, PSO also serves FlashArray and block devices. The details of storage administration are different for block (FlashArray) and file (FlashBlade) volumes, and PSO hides all of these differences from the end-user. For example, a developer creating an application that uses a ReadWriteOnce volume can choose between latency sensitive or bandwidth sensitive performance simply by changing the text between “pure-file” and “pure-block.” All of the annoying differences between iSCSI, Fibre Channel, and NFS mounts are hidden automatically by PSO.
How to Use PSO with FlashBlade
Compared to traditional storage workflows, PSO automates most of the requests and interactions between users and administrators for file system creation and mounting. The result is a self-service storage infrastructure; no direct interaction is necessary between administrators and users.
Administrators install and occasionally upgrade the PSO software and users (developers) interact only via standard Kubernetes PersistentVolume mechanisms.
For the Kubernetes Administrator
PSO is a software layer for Kubernetes that utilizes the public REST API provided by FlashBlade to automate storage-related tasks and is installed via helm.
The one-time PSO configuration requires a FlashBlade management IP, data VIP, and the corresponding API token. These API tokens prove that PSO has permission to create and delete volumes or filesystems so safeguard them appropriately.
To get the API token, use the following command from the FlashBlade CLI:
pureuser@flashblade-ch1-fm2> pureadmin list --api-token --expose
Name API Token Created Expires
pureuser T-c4925090-c9bf-4033-8537-d24ee5669135 2017-09-12 13:26:31 PDT -
Then, add the following to the PSO values.yaml (example) file to add a FlashBlade under management:
FlashBlades:
- MgmtEndPoint: "10.62.64.20"
NFSEndPoint: "10.62.64.200",
APIToken: "T-ab0ed3e0-5438-4485-9503-863e6a9c1434"
The PSO works by fulfilling any PVC claims with a StorageClass with corresponding name, “pure-file,” by creating a matching filesystem.
Update the default storageClass to use the “pure-file” class if you want FlashBlade to be the default for PVCs that do not specify StorageClass.
> kubectl patch storageclass pure-file -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
After this change, view the configured StorageClasses and their associated provisioner with the following command:
> kubectl get storageclass
NAME PROVISIONER AGE
pure-block pure-provisioner 24d
pure-file (default) pure-provisioner 24d
Finally, to view the PersistentVolumes in the system which were automatically created by PSO:
> kubectl get persistentvolume
...
The PersistentVolumes reported here should match the filesystems created on the FlashBlade, as seen with the following CLI command in the FlashBlade:
> purefs list --filter "name = 'k8s-*'"
Name Size Used Hard Limit Created Protocols
k8s-pvc-98bb10ca-c037-11e8-9ada-525402501103 1T 0.00 False 2018-09-24 13:22:46 PDT nfs
k8s-pvc-98c51436-c037-11e8-9ada-525402501103 800G 52.50K False 2018-09-24 13:22:46 PDT nfs
k8s-pvc-98d6147c-c037-11e8-9ada-525402501103 20T 450.00K False 2018-09-24 13:22:48 PDT nfs
k8s-pvc-9fcfdf49-beb3-11e8-9ada-525402501103 5T 0.00 False 2018-09-22 15:05:33 PDT nfs
Adding a StorageQuota (example) allows the administrator to limit the number of claims or the total amount of storage requested. These limits cause graceful failures in the case of buggy jobs that consume too much storage.
For the Kubernetes User
The following examples create a PersistentVolumeClaim that will be satisfied by the PSO by using a storageClassName of “pure-file”. The storageClass signals to Kubernetes how to satisfy the claim request. Switching to use a FlashArray block device instead can be done by using “pure-block” as StorageClass.
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: file-claim
spec:
storageClassName: pure-file # This line is all that is required to use PSO
accessModes:
- ReadWriteMany
resources:
requests:
storage: 15Ti
A PersistentVolumeClaim is a request for a PersistentVolume of certain type and size. If there is a plugin for the specified storageClass, then it will create a matching PersistentVolume. To attach this storage to a container at a specific path, refer back to the claim as follows:
volumes:
- name: myvol
persistentVolumeClaim:
claimName: file-claim
containers:
...container config...
volumeMounts:
- name: file-claim
mountPath: /mountpoint
What did you NOT need to do?
- Create a filesystem via a UI of any kind (GUI or CLI)
- Log in to the node and issue the commands to mount the filesystem
- Think about what happens if the physical node fails and you need to move the app
- Track who uses which filesystem so you know when to cleanup
And what you do get is that the volume automatically follows your container if Kubernetes restarts or moves the container to a different physical host.
Example 1: Shared Scratch Space
The first example to illustrate PSO is a simple Deployment and PersistentVolumeClaim to create a shared scratch workspace. In other words, the application is a standard Linux shell and command line tools.
A shared scratch workspace is useful when a small team needs to work together, for example for forensics investigation of log files. The goal is to automatically create a shared scratch space to download necessary data to work with and produce derivative datasets. PSO automates the creation, connection, and cleanup of the workspace on FlashBlade.
The yaml file creates two pods that both mount a shared “/scratch” directory for collaborative work. This illustrates the core steps necessary to use a persistentVolumeClaim to attach a volume to pods in a Deployment.
To use this shared scratch space, each user connects to a pod and spawns a shell by running ‘exec’ on one of the pod instances:
> kubectl exec -it scratchspace-7877bc68f9-s8nxw /bin/bash
Once this shell has been started, the user can collaborate using the ‘/scratch’ directory and all results will be visible to users in other pods.
What if a third user wants to also collaborate? Adding additional workspaces that share the same filesystem is as simple as:
> kubectl scale deployment scratchspace --replicas 3
Each additional replica pod is automatically connected to the filesystem, i.e., the mounting and unmounting of the filesystem is automatically handled by PSO.
There are many directions to make this example even more useful: 1) add a unique name to the resources based on a ticket number, 2) automate cleanup of resources when finished, and 3) use a prebuilt container image with already installed tools.
Example 2: NextCloud
NextCloud and OwnCloud are both open-source file sharing applications providing an alternative to cloud services like DropBox. These applications provide similar file sharing services across multiple client platforms while keeping the data in-house instead of an offsite 3rd party cloud; either performance, cost, or data governance may drive this desire for an in-house service instead of external. Both applications rely upon persistent storage for the user’s data and metadata.
The NextCloud configuration borrows heavily from existing documentation for NextCloud. This configuration is kept simple in order to highlight the usage of persistent storage.
The config file combines three major elements:
- Service ‘nextcloud’ that exposes the application externally
- Deployment ‘nextcloud’ that starts the application server and mounts a filesystem
- PersistentVolumeClaim that PSO will use to create a matching filesystem on the FlashBlade
Create the nextcloud service as follows:
> kubectl apply -f nextcloud-pure.yaml
To make this application accessible externally, I used a NodePort service, which is not recommended in production. To access the NextCloud application, find the port assigned by Kubernetes:
> kubectl get svc nextcloud | grep NodePort
Connect to the NextCloud server by using that port number and the ip address of any Kubernetes node. Production environments should configure load balancers instead.
Using InitContainers for Setup
The NextCloud example utilizes an initContainer to solve a common problem: configuring the volume to meet the conditions expected by the application. An initContainer is a container that is run to completion before the application container is started and must complete successfully for the application to begin. InitContainers encode the steps necessary to satisfy pre-conditions or dependencies that the user does not want to or cannot build into the main application container. As an example, an initContainer can download a dataset to the persistent volume for the primary application to use.
The NextCloud server expects the “data/” directory to be owned by the www-data user and to have permissions of 770, otherwise the application fails. To accomplish this, an initContainer performs two simple commands to achieve this precondition:
initContainers:
- name: install
image: busybox
command:
- sh
- '-c'
- 'chmod 770 /var/www/html/data && chown www-data /var/www/html/data'
volumeMounts:
- name: nc-files
mountPath: /var/www/html/data
Many applications, especially legacy applications not originally built for containers, expect preconditions on the volumes and initContainers are the easiest way to achieve this. Beyond the simple example here with busybox, different container images can be used to place data or install necessary software on the volume.
Example 3: OwnCloud
The OwnCloud deployment involves multiple applications working together: mariadb and redis alongside the owncloud server. These applications use a mix of block devices and filesystems. For example, MariaDB is an RDBMS service that requires a block device backed volume whereas OwnCloud stores user data on a filesystem-backed volume. Derived from docker documentation for OwnCloud.
With PSO, multiple applications and volume types can be mixed and modified easily. For example, switching the Redis application between “pure-file” and “pure-block” is as simple as changing the value in the yaml config before deploying.
Running multiple different applications on FlashBlade works well because the system is 1) built to support high concurrency and many simultaneous applications, 2) scales-out seamlessly and non-disruptively, and 3) is architected natively to leverage the mixed IO performance of NAND flash.
Deploy this set of applications and volumes together with the following command:
> kubectl apply -f owncloud-pure.yaml
Note that there is a helm chart for OwnCloud here but it is currently not working due to a failed connection between OwnCloud and mariadb. With the default storageClass set to “pure-file,” this helm chart would automatically use FlashBlade as the backing store.
Quick Tips for Usability
Below are some quick-tips that I found useful in running the OwnCloud application.
Run OwnCloud reporting commands internally:
kubectl exec oc-owncloud-0 occ user:report
Log in to the server and look around at the contents of the filesystem.
kubectl exec -it oc-owncloud-0 /bin/bash
Summary
A good software engineer always looks to automate tedious and error-prone tasks in order to increase the team’s productivity. Running Kubernetes and the Pure Storage Orchestrator automates almost all storage related tasks: filesystem creation, mounting, and deletion. The result is simple and agile infrastructure that just works.
Stateful Kubernetes Applications Made Easier: PSO and FlashBlade的更多相关文章
- OpenState: Programming Platform-independent Stateful OpenFlow Applications Inside the Switch
文章名称:OpenState: Programming Platform-independent Stateful OpenFlow Applications Inside the Switch Op ...
- Announcing HashiCorp Consul + Kubernetes
转自:https://www.hashicorp.com/blog/consul-plus-kubernetes We're excited to announce multiple features ...
- SAP云平台对Kubernetes的支持
截至本文发稿(2019-2-10, 农历大年初六)时为止,访问SAP云平台的官方网站:https://cloudplatform.sap.com/enterprise-paas/kubernetes. ...
- kubernetes 基础
官网 kubernetes.io 有中文 中文网站 http://docs.kubernetes.org.cn kubectl 详细情况 https://kubernetes.io/docs/ref ...
- Kubernetes Helm
Helm is a tool for managing Kubernetes charts. Charts are packages of pre-configured Kubernetes reso ...
- Helm - Kubernetes包管理专家
What is Helm? - The package manager for kubernetes, Helm is the best way to find, share, and use sof ...
- Use PSO to find minimum in OpenCASCADE
Use PSO to find minimum in OpenCASCADE eryar@163.com Abstract. Starting from OCCT6.8.0 will include ...
- Kubernetes 技能图谱skill-map
# Kubernetes 技能图谱 ## Container basics (容器技术基础)* Kernel* Cgroups* Userspace runtime* Image* Registry ...
- CNCF CloudNative Landscape
cncf landscape CNCF Cloud Native Interactive Landscape 1. App Definition and Development 1. Database ...
随机推荐
- (C/C++学习笔记) 二十三. 运行时类型识别
二十三. 运行时类型识别 ● 定义 运行时类型识别(Run-time Type Identification, RTTI) 通过RTTI, 程序能够使用基类的指针或引用来检查(check)这些指针或引 ...
- Docker常用命令学习
sudo service docker startsudo docker run hello-worlddocker stats --helpdocker run -d -P training/web ...
- rancher中使用ingress-lbs做负载均衡
rancher 相关资料 http://rancher.com/docs/rancher/v1.6/zh/kubernetes/ingress/ lvs, haproxy, nginx负载均衡器比较 ...
- leetcode第11题:盛水最多的容器
给定 n 个非负整数 a1,a2,...,an,每个数代表坐标中的一个点 (i, ai) .在坐标内画 n 条垂直线,垂直线 i 的两个端点分别为 (i, ai) 和 (i, 0).找出其中的两条线, ...
- Could not load driverClass com.mysql.jdbc.Driver错误
在整合spring和mybatis的时候,在spring配置文件中已经加载了db.properties并配置了c3p0数据源 但在写了一个测试类测试是否取到了数据库的连接时,报错:Could not ...
- react state为数组时插入值
react state为数组时,如何插入值.在react里,一切皆是状态state,如果想通过改变state修改渲染效果,只能yongsetState.但是setState又是key:value格式, ...
- Transport & Buffer
Transport 传输API的核心是Channel接口,用于所有的出站操作. 每个Channel都会分配一个ChannelPipeline和ChannelConfig.ChannelConfig负责 ...
- 学习Java JDBC,看这篇就够了
JDBC (Java DB Connection)---Java数据库连接 JDBC是一种可用于运行SQL语句的JAVA API(ApplicationProgramming Interface应用程 ...
- 一个跳转提示页面---JS
//一个跳转提示页面 <script type="text/javascript"> var s=5; function go(){ do ...
- 增加临时表空间组Oracle11g单实例
#需求,测试库与生产库,临时表空间同步一致 #经过查询生产环境,数据库默认临时表空间,为临时表空间组,有三个成员,三个临时表空间,每个临时表空间一个数据文件,自动扩展 #使用临时表空间组的优点,减少不 ...