Portworx Storage with internal KVDB in on-premises Kubernetes cluster.

7 min readOct 12, 2022

Portworx

In this blog, we are going to add the Portworx storage solution to our on-premises Kubernetes cluster with internal KVDB.

What is Portworx?

Portworx is a software-defined storage platform. It was created for containers and microservices. To deliver a unified, overlay storage layer to cloud-native applications, it abstracts multiple storage devices. Users of Portworx can spread highly available stateful applications across a number of physical hosts in a data centre, compute instances running across a number of zones, regions, and even several cloud providers.

Prerequisite: -

The Kubernetes cluster is up and running.
To check Portworx compatibility with your Kubernetes cluster, Follow this link.
Portworx recommends running at least three storage nodes in a cluster.
To check Hardware, Network and Software compatibility, Follow this link. Make sure your Kernel version is supported.
An account on the Portworx portal. If you don’t have an account Follow this link to create one for you. (For this demo we are using Portworx Enterprise Essentials )
Your nodes are labelled with “px/metadata-node=true” which will run the internal KVDB. You can use the following command to label the nodes: -

kubectl label nodes <list of node-names> px/metadata-node=true

Create Manifest spec on Portworx

Go to the portworx website and log in. Navigate to the “Product Catalog ” from the left side of the window.

It will ask you to select the product which you want to deploy on your cluster. In our scenario, we are going to use portworx enterprise. Select your project and Click on “Continue”

Screenshot of PortworxProduct Catalog Page

A new window will open, here you can select the product which you want to deploy. In our scenario, we are going to deploy “Portworx Essentials”. Select your product and click on continue.

In the next window select the “Portworx Version” and click on “Next”.

In the new window, you will get an option to select your environment. As we are using an on-premises Kubernetes cluster. So, we have selected the environment “On Premises” and the type of On-Prem storage “Automatically scan disks”.

Select the check box for: -

Use unmounted disks even if they have a partition or filesystem on it. PX will never use a drive or partition that is mounted.
Auto-create journal device.

Add the device name for KVDB. Here “/dev/nvme2n1” is our disk name.

Screenshot of PortworxProduct Spec Storage Page

Once all the details are filled click on “Next”.

A new window will appear, here you will get an option to manage the Network. If you want to segregate the network you can choose to add the name of your network interfaces and if you're going to let portworx choose the network interfaces then keep it auto.

Now click on ”Advanced Settings” if you want the Portworx service to choose another port then change the Port number. In our scenario, we keep it default.

Screenshot of PortworxProduct Spec Network Page

Click on “Next”.

In the next window, it will ask for the details about the environment where you want to deploy the portworx. As we are using the On-premises Kubernetes cluster. So we have selected the “None” option.

Screenshot of PortworxProduct Spec Customize Page

Click on “Environment Variables”.

If you want to pass some Portworx “Environment Variables” you can add them in this window. In our scenario, we are not passing any env variables.

Click on “Registry And Image Settings”.

You can add your container registry details in this option. If you want to use your custom container repository. In our scenario, we are skipping this step, as we are using the default registry.

Click on “Advanced Settings”.

Enable “Stork, CSI, Monitoring”. If you want telemetry data, you can enable “Telemetry” in this step. If your cluster is air gaped don’t enable the telemetry option.

After making all the required changes click on “Finish”.

In the new window, you will get the portworx manifest files. You can download these files by making a curl request to the HTTPS URL or you can copy the content of the manifests by opening the URL in the web browser and pasting the manifest content to a file on your local machine Or you can simply click on the copy icon in front of the URL.

Screenshot of PortworxProduct Spec Final Page

Portworx installation steps.

To install the portworx on your cluster, we need to deploy the operator on our Kubernetes cluster. Use the operator manifest generated in the previous step.

You can use the following command to deploy the operator on your cluster. Use the URL mentioned in the portal to deploy an operator.

kubectl apply -f ‘https://install.portworx.com/X.XX?comp=pxoperator'

Once you see the portworx operator pod up and running in the kube-system namespace deploy the second manifest from the previous step on your Kubernetes cluster.

Use the URL mentioned in the portal to deploy the portworx workload.

kubectl apply -f ‘https://install.portworx.com/2.11?operator=true&mc=false&oem=esse&user=xxxxxxxxxxxxxxxxxx&b=true&f=true&j=auto&kd=%2Fdev%2Fnvme2n1&c=px-cluster-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx&stork=true&csi=true&mon=true&tel=false&st=k8s&promop=true'

Once the manifest is deployed, verify that all the pods are up and running.

kubectl get pods -n kube-system

And it will give you output like this.

Portworx pods running in kube-system Namespace.

As we can see all our portworx pods are up and running. To check the status of the pxctl run the below-mentioned command.

PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath=’{.items[0].metadata.name}’)kubectl exec $PX_POD -n kube-system -- /opt/pwx/bin/pxctl status

And you will get an output like this.

As you can see in the above screenshot the dedicated device for KVDB is used which we have mentioned in the portal.

Now your drives are ready to use. You can store container data on the disk. In the next step, we will guide step by step that how can you consume your drive.

Verifying Portworx installation.

To verify your portworx installation. Follow these steps.

Create a storage class. You can use the below given example to create a file name storage-class.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: portworx-sc
provisioner: kubernetes.io/portworx-volume
parameters:
  repl: "1"

Apply this configuration by running the below-given command.

kubectl apply -f storage-class.yaml

To check the status of the storage class run the below-given command.

kubectl  get storageclass

Make sure you have set the default storage class to the newly created storage class.

2. As the storage class is created now we need to create the persistent volume claim. You can use the below-mentioned example to create a PVC. Save this file with the name pvc.yaml.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: px-pv-t
spec:
  storageClassName:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi

To create the PVC run the below-given command.

kubectl apply -f pvc.yaml

To verify whether the PVC is created or not run the below-mentioned command.

kubectl get pvc

You will get an output like this.

Screenshot of the PVC created by using portworx storage class.

In the above output, you can check that your claimed volume of 50GB has been created.

3. Now we need to create the pod which will consume the created PVC. You can use the following manifest to create a pod.

apiVersion: v1
kind: Pod
metadata:
  name: portworx
spec:
  containers:
  - name: sagar
    image: sagar27/testing:latest
    volumeMounts:
    - name: portworx
      mountPath: /test-portworx-volume
  volumes:
  - name: portworx
    persistentVolumeClaim:
      claimName: px-pv

Save this file with the name pod.yaml. Now deploy this pod on a cluster by using the below-given command.

kubectl apply -f pod.yaml

To check the status of deployed pod run this command.

kubectl get pods

You will get an output like this.

4. To verify whether the volume is mounted or not in the pod. Run the following command.

kubectl exec -it portworx -- df -h

And you will get an output like this

Screenshot of Portworx Volume mounted inside the pod.

5. You can also verify the volume from the Portworx container. Use the below-mentioned command to get the details.

PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath=’{.items[0].metadata.name}’)kubectl exec $PX_POD -n kube-system -- /opt/pwx/bin/pxctl volume list

You will get an output like the one shown in the below-mentioned screenshot.

Screenshot of the volume created in portworx

From the output, you can see that the name in the volume list command is the same as the name of the PVC which we had created by using pvc.yaml file.