Today I want to talk about SmartOS multi node central management.
When building your own SmartOS Cloud ‘s it is imperative to have some kind of central management or “Head Node” that can talk to all your nodes and manage them. The commercial product from Joyent called Smart Data Centre or SDC is what joyent uses to provision, manage and pull analytic’s from the SmartOS nodes.
All this is done via an openly available Joyent CloudAPI, but unfortunately without purchasing “Smart Data Centre” the SmartOS nodes do not natively have any API interaction service or daemon running on them that we can use to interact with them with.
Thankfully this all changed when one of the very talented members of the SmartOS Community Heinz N. Gies created an open source project called “vmwebadm” to implement Joyent’s cloud API on top of Vanilla SmartOS.
Before we get into the specifics of how to install and use “vmwebadm” I want to list some of the capabilities and attributes of the tool.
- “vmwebadm” may confusingly sound like it is a gui web interface management tool, it is not! The good news is that Heinz does have plans for a central web interface management GUI for this on the road-map. This will essentially be your headnode that will centrally manage all your SmartOS nodes that have “vmwebadm” installed.
- The “vmwebadm” service is persistent and will survive node reboots
- Since “vmwebadm” service is installed and lives in the zones dataset it will continue working even if you boot off a new SmartOS image version.
- vmwebadm is secure and uses ssh key authentication to interact with and provision Virtual Machines.
- vmwebadm is multi user aware which means individual authenticated users will only be able to see and manage virtual machines that belong to them.
Heinz released a screencast demonstrating the install and usage. Please watch the screencast as a visual overview. I will outline the steps below.
SmartOS API Installation
First we download the package to the /opt folder that is persistent and extract it.
cd /opt/ wget --no-check-certificate https://github.com/downloads/Licenser/vmwebadm/vmwebadm-0.3.3p3.tar.bz2 tar -jxvf vmwebadm-0.3.3p3tar.bz2
Once extracted lets go into the vmwebadm directory and create 2 users, an admin (admin) user and a normal user (user1). Once the users are created we will promote admin to be an administrative user which will be able to see and manage all vm’s regardless of who owns the Virtual machines. We then list them to verify creation and to make sure admin user has admin rights.
cd /opt/vmwebadm ./client.sh passwd admin adminpass ./client.sh passwd user1 user1pass ./client.sh promote admin
[root@00-15-17-ae-1f-d9 /opt/vmwebadm]# ./client.sh list users Admin | Key | UUID | Login ------+-----+--------------------------------------+------------------------ [ ] | [ ] | 00b481b4-8aac-11e1-ac7d-0f9e9f75292b | user1 [*] | [ ] | b39c24f2-879c-11e1-8946-af62f60c7836 | admin
Now we install and enable the service and confirm it is enabled.
[root@00-15-17-ae-1f-d9 /opt/vmwebadm]# ./vmwebadm install [root@00-15-17-ae-1f-d9 /opt/vmwebadm]# ./vmwebadm enable [root@00-15-17-ae-1f-d9 /opt/vmwebadm]# svcs vmwebadm STATE STIME FMRI online Apr_16 svc:/network/vmwebadm:default
Management Client Installation
We are now going to install the management client, this can be on any computer. In this example we are going to install it on my Mac laptop.
The CloudAPI command line interface (CLI) requires Node.js and npm to run.
wget http://nodejs.org/dist/v0.6.15/node-v0.6.15.pkg
Install the node-v0.6.15.pkg (npm has been included in the Node.js distribution since version 0.6.3.)
Once installed we proceed with installing “smartdc” and “jsontool”
sudo npm install smartdc -g sudo npm install jsontool -g
We now will now let sdc-tools know where our node lives and configure keys for “admin” and “user1” users. In this example we are setting up the same key for both users so that I can play with admin access to the node as well as simple user access for individual vm’s.
➜ ~ sdc-setup http://10.1.1.48 Username (login): (marks) admin Password: SSH public key: (/Users/marks/.ssh/id_rsa.pub) If you set these environment variables, your life will be easier: export SDC_CLI_URL=http://10.1.1.48 export SDC_CLI_ACCOUNT=admin export SDC_CLI_KEY_ID=id_rsa export SDC_CLI_IDENTITY=/Users/marks/.ssh/id_rsa ➜ ~ sdc-setup http://10.1.1.48 Username (login): (marks) user1 Password: SSH public key: (/Users/marks/.ssh/id_rsa.pub) If you set these environment variables, your life will be easier: export SDC_CLI_URL=http://10.1.1.48 export SDC_CLI_ACCOUNT=user1 export SDC_CLI_KEY_ID=id_rsa export SDC_CLI_IDENTITY=/Users/marks/.ssh/id_rsa
For easier management credentials you can export the variables for one of the accounts so that we do not have to enter in credentials every time we want to connect to our SmartOS node. This is useful if you only have 1 node to manage. Once the head node web GUI project is underway this will all be passed transparently to individual nodes for you.
We are now ready to interact with our nodes directly with our client via the API, here is an example of me listing all Virtual machines on the Node. All data returned is in JSON format.
➜ ~ sdc-listmachines --url http://10.1.1.48 --account admin [ { "id": "95352b5c-b9e8-4564-a5e7-1c099b4c673d", "ips": [ "10.1.1.220" ], "disk": 20, "name": "95352b5c-b9e8-4564-a5e7-1c099b4c673d", "type": "kvm", "state": "running", "dataset": "zones/95352b5c-b9e8-4564-a5e7-1c099b4c673d", "create": "2012-03-27T04:21:38.992Z", "memory": 2048 }, { "id": "d152947b-f42a-40f5-9c1b-a1e1b99b0f0a", "ips": [ "10.1.1.207" ], "disk": 40, "name": "d152947b-f42a-40f5-9c1b-a1e1b99b0f0a", "type": "kvm", "state": "running", "dataset": "zones/d152947b-f42a-40f5-9c1b-a1e1b99b0f0a", "create": "2012-02-11T07:14:34.783Z", "memory": 1536 }, { "id": "b084cd54-4536-47cb-af56-6f5da959c427", "ips": [ "10.1.1.217" ], "disk": 40, "name": "b084cd54-4536-47cb-af56-6f5da959c427", "type": "kvm", "state": "running", "dataset": "zones/b084cd54-4536-47cb-af56-6f5da959c427", "create": "2012-03-18T23:31:03.944Z", "memory": 1536 } ]
Thats it! Cool huh?
You can now proceed to manage your SmartOS nodes via the api with the following command available:
➜ ~ sdc- sdc-addmachinetags sdc-getmachinetag sdc-createinstrumentation sdc-getpackage sdc-createkey sdc-listdatacenters sdc-createmachine sdc-listdatasets sdc-createmachinesnapshot sdc-listinstrumentations sdc-deleteinstrumentation sdc-listkeys sdc-deletekey sdc-listmachines sdc-deletemachine sdc-listmachinesnapshots sdc-deletemachinemetadata sdc-listmachinetags sdc-deletemachinesnapshot sdc-listpackages sdc-deletemachinetag sdc-rebootmachine sdc-describeanalytics sdc-resizemachine sdc-getdataset sdc-setup sdc-getinstrumentation sdc-startmachine sdc-getkey sdc-startmachinefromsnapshot sdc-getmachine sdc-stopmachine sdc-getmachinemetadata sdc-updatemachinemetadata sdc-getmachinesnapshot
This is just the beginning, we are looking for other contributors that can help with creating the web front end for the proposed GUI Web Headnode as well as suggestions and feature requests. Please get in touch with me via the blog or with Heinz directly if you are interested.
Aug 24, 2012 @ 07:25:00
one side of me:
so, like, this is like using xenstore or xm list –long, just for people who could never do that because they were stuck on libvirt-based toolsets?
other side:
nice, this is much more structured than imgadm – thanks for the nice & complete intro.
Nov 19, 2012 @ 12:38:00
This looks pretty great. I’m evaluating options like this right now for my org.
Nov 19, 2012 @ 13:19:00
Hi there Blake,
Also have a look at http://project-fifo.net and specifically the new API that is going to be released shortly with our 0.3 release. You may find in addition Project FIFO will make your SmartOS cloud orchestration work very well. The backend is completed and we are now busy with the new UI. Feedback and contributions are always welcome, and we all hang out in IRC Freenode #project-fifo
Here are links to 0.3 architecture page and the new API
http://project-fifo.net/display/PF/FIFO+Architecture+Diagram
http://project-fifo.net/display/PF/API
vmwebadm – SmartOS management via API | b...
Aug 03, 2013 @ 03:09:38
[…] Today I want to talk about SmartOS multi node central management.When building your own SmartOS Cloud 's it is imperative to have some kind of central management or "Head Node" that can talk to all your nodes and manage them. […]