Service Broker for Ceph Object Storage
This broker is compliant with V2 of the Open Service Broker API. The broker provides access to Ceph's object storage,
and can currently be deployed as CloudFoundry app, on Kubernetes or on OpenShift. Deployment as a Bosh release is planned for the future.
Table of Contents
General Operation
The service provided by the broker and its plans are in the brokerConfig/service-config.json
file. You can edit this to your liking before deploying.
When an instance is provisioned a user is created on Ceph. Then when an
application binds to the broker, it returns access credentials for both the S3 and Swift
APIs supported by Ceph.
The credentials made available to the application (usually through environment variables) after a bind are:-
- s3User
- s3AccessKey
- s3SecretKey
- s3Endpoint
- swiftUser
- swiftSecretKey
- swiftEndpoint
Unbinding and deprovisioning are simply reverse operations of the provision and bind stages.
Deployment
Deployment to all platforms is done through the deploy.sh
file, so once prerequisites for a platform are fulfilled the script can be used to deploy the broker.
Prerequisites
Before deploying to a platform, you need to provide the required details about your Ceph installation. Specifically you will need a
Ceph object gateway setup. The broker will use the admin user on the gateway to manage users there as required to operate the
service, and so it requires a number of variables including the gateway's endpoint and access keys for the user.
To provide the required information you will need a file called vars-file.yml
. A template for this file called vars-file-template.yml
is available, and so can simply
be copied, renamed and then the details filled in.
Lastly, you will need Go installed as its used in the deployment script and in case you want to build yourself or run the integration tests.
The broker has been developed with Go V1.10.1. It should theoretically work with older releases, but keep in mind that is not verified.
CloudFoundry
Deployment of the broker as an app running on CloudFoundry is controlled by the manifest.yml
file, which requires no edits. To deploy simply
run ./deploy.sh cf ceph-objectstorage-broker
, with the second argument being the name of the app on CF.
Once the broker is running on CF, it needs to be registered with CF and then the plans need to be made public. To register the broker
use cf create-service-broker SERVICE_BROKER BROKER_USERNAME BROKER_PASSWORD BROKER_URL
. Then to make the service public
run cf enable-service-access ceph-object-storage
, where 'ceph-object-storage' is the name of the service provided in brokerConfig/service-config.json
.
Kubernetes & OpenShift
Deployment to k8s and OS are both done by using the following files:
- Automatically created/updated using your
vars-file.yml
via the update-cosb-vars/update-vars.go
GO program, which is run on each deploy
- config-map.yml
- secret.yml
- template.yml
- route.yml (only for OS)
- broker.yml (Manually used to register after deployment)
Before you deploy, please make sure you have kubectl
or oc
installed and that you are logged in to your cluster, as they are used to deploy to k8s and OS, respectively.
To deploy use ./deploy.sh k8s
or ./deploy.sh os
. These commands will set the config-map, secret, deploy the broker application and then create a service for it. In
the case of OS, it also creates a route for the broker and displays the url of the created route.
The default service created uses a NodePort to expose the broker, however depending on your
platform you might want to use something like a loadbalancer, in which case you can just edit the relevant yaml files and then use the deployment script to deploy with your
own configuration.
To register the broker you need to get the url of your broker (it could be deployed on a different platform), any certificates if you want encryption and then update
the deployment-configs/k8s/broker.yml
. If you don't use encryption then you simply need to set the url field. Once you have updated the broker file
you can run oc apply -f "deployment-configs/k8s/broker.yml"
or kubectl apply -f "deployment-configs/k8s/broker.yml"
, depending if you are using OpenShift or
Kubernetes, respectively.
NOTE: To apply the broker file you need to have the Service Catalog installed on your Kubernetes
cluster and be a user with sufficient privileges (e.g. system:admin on OpenShift).
Bosh Release
The BOSH release for the broker and related documentation can be found here.
Integration Tests
To run the tests:
- Fulfill the required prerequisites
- Run
go run update-cosb-vars/update-vars.go
- Run
source tests/tests.env
- Run
go run main.go
- In the
tests
folder run go test
or go test -v
for more details