Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New VPS/CLOUD Control Panel!
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

New VPS/CLOUD Control Panel!

GoodHostingGoodHosting Member
edited June 2014 in Providers

Hello LowEndTalk,

We're currently working on a concept VPS/Cloud control panel that we intend to release as partially Open Source once we have had the initial code base security audited ( wouldn't want to release insecure software, it wouldn't help anyone. )

To be precise, this is an API driven panel that is both compatible with EC2-based API [ which includes most of Amazon's services ] as well as the OpenNebula XMLRPC2 API as described in their documentation.


Features


We are currently focusing on the feature support for KVM Virtualization, as this is what we use for all of our cloud services at GoodHosting. We will be expanding these features where possible to include Xen-PV and Xen HVM as well.

Implemented Features:

  • All basic management / control actions that the API exposes, including but not limited to "Boot/Reboot/Resume", "Suspend", "Reboot", "Poweroff", "Deploy", "Stop", "Undeploy", "Hold", "Release", "Reboot Hard", "Poweroff Hard", "Undeploy Hard", "Delete", "Delete Recreate", "Migrate", "Migrate Live"
  • Simpler interface for resizing capacity [ simple slider capped at userquota_total - userquota_used per resource, much more intuitive than a free input with no bounds. ]

Recently Implemented Features:

  • Ability to "grow" instances live [ where the guest Operating System supports hot-adding CPUs and the memory ballooning driver found in KVM. ]
  • Disk snapshot [ regardless of disk format, auto-converts to qcow2 from raw if necessary. ] and stores the snapshot as a compressed backing file, enabling full COW performance gains.
  • Ability to upload/download compressed (lzma/qcow2 compressed) snapshots.
  • [LIVE] CPU usage as reported by the host machine's qemu:// protocol
  • [LIVE] Disk usage as reported by the host machine's filesystem [qcow2 qemu-img]

As per the interface, it's very basic [ we need designers to give it a once-over, we know. ] but it should express all the currently available functionality just fine. Screenshots to follow:


Screenshots


Basic VM list, shows states of all VMs as listed. [Needs some work.]

Dashboard / VM List:

image

VM information as reported by the host machine, such as the real current CPU usage [ last 5 seconds as sampled ], the current REAL RAM usage, transit used in each direction as reported by the bridge statistics available at /sys/class/net/xx/statistics/yy_bytes, the real disk usage of all disks assigned to the machine, and even handles disks where a usage would not be applicable [ such as CDROM and other readonly file systems that only have a "full" state. ]

Example of features:

image

The RAM usage of a system can sometimes report a value higher than the allocation, this has something to do with the overhead of KVM virtualization such as the amount of RAM that gets reserved as virtual video memory for the guest operating system when using a graphical interface. We're working on separating this from the value reported.

One of our systems during a maldet scan:

image


Technical Details


Now, let's get into the technical details of things; such as the system requirements for our solution as it stands at the moment, and the technologies being used in the background. I'll split this up into various sections.

Agent

First, [ and probably most important ] are the reporting agents that must be installed on all hypervisor nodes that you would like real-time statistics available on, such as the live CPU usage and disk usage. This agent is written in JavaScript and runs on the popular Node.js server software. This agent uses small SH and python scripts to retrieve information from the local hypervisor's interface [ such as using the qemu:// protocol for KVM, or querying the information directly from the /sys/ or /proc/ psuedo-filesystems. ] We are currently working on a solution that would allow the agent to run without privileged access.

The agent itself uses less than 2MB of RAM, even during extremely high requests [ as tested with Blitz.io ] I ran a rush test directly on an agent responder using 1-1000 users over the course of 5 minutes, and the agents held up fine; keeping response times below 200ms near the top end of the scale, without a single error or timeout. The agent smartly queues requests, and responds with a cached result whenever possible [within 5 seconds] to avoid spawning too many worker processes.

Scripts

The scripts are currently extremely simple and act as only a proof of concept.

image

Panel

The panel itself currently runs on PHP 5.3+ / Apache, lighttpd or nginx, and can be installed alongside your billing or other panel [ It's cPanel compatible and doesn't require any special modules, it doesn't even use PHP's bastard child of an XMLRPC implementation; just cURL, which is a pretty standard module to find on most installations. ] I am currently working to rewrite the panel entirely in JavaScript to allow it to also run in Node.JS, which would increase the performance dramatically.

The panel requires no more resources than any standard PHP site would, and does not use any frameworks. Some classes have been borrowed from here and there, and credits given where due [of course]; but no full framework such as Symphony, YiiFramework, FuelPHP, CakePHP, etc. is being used due to the sheer size it would add to the package. The panel itself [due to cPanel's poor performance with default settings on Apache and PHP] was not able to stand up to the Blitz.io test of 1-1000 users over 5 minutes, giving rather poor response times and errors past the 100 user mark.


In Closing


My question to the community is, "What features would you like to see in the industries next VPS/cloud Control Panel?" We're also hoping that the community might have some feedback in concerns to the panel or operation thereof, and as such; will be releasing a limited beta very soon; available only to LET members.

And I repeat, we already know it's ugly; that's a future issue unrelated to features :).


Disclaimer: The blitz.io link is a referral link in which we gain one additional free test for each users that registers. We use blitz.io to load/stress test our control panel and other projects that we contribute to. It's a wonderful tool that I suggest you try out, referred or not.

Comments

  • Nice one, looking forward for this.

  • @theweblover007 said:
    Nice one, looking forward for this.

    Thanks for the words of encouragement. Do you have any ideal or dream features that you would like to see possible within the panel, or through the API? [ Yes, the panel includes an API that matches every function that the panel can support, including access to the agent functions such as real time statistics. ]

  • Cool!

  • @XxNisseGamerxX said:
    Cool!

    Thanks! The same question goes to you as well: Do you have any ideal or dream features that you would like to see possible within the panel, or through the API?

  • @GoodHosting said:
    Thanks! The same question goes to you as well: Do you have any ideal or dream features that you would like to see possible within the panel, or through the API?

    Hmmm, Pay invoice within the panel maybe?

  • @XxNisseGamerxX said:
    Hmmm, Pay invoice within the panel maybe?

    Well, the panel itself (planned feature) will do the following:

    • Expose an API through which integration with a billing system would be possible, including the ability to handle hourly billing either done with credit, or done on an adjustable frequency using invoice lines [ specifically for WHMCS, which has no sane means of doing hourly billing. ]

    That being said, I might consider making it a complete solution; but then I would need to add a Support Tickets system and a bunch of other unrelated junk too, so it's a possibility; but far off at the moment.

  • enitan092enitan092 Member
    edited June 2014

    @GoodHosting said: And I repeat, we already know it's ugly; that's a future issue unrelated to features :).

    looks okey

  • enitan092enitan092 Member
    edited June 2014

    @GoodHosting said:
    That being said, I might consider making it a complete solution; but then I would need to add a Support Tickets system and a bunch of other unrelated junk too, so it's a possibility; but far off at the moment.

    cant we have soemthing that works as whmcs /vps maanagement...

    then we dont have to worry about whmcs at all

    both billing and management not relying on whmcs(standalone)

    Thanked by 1Maximum_VPS
  • @enitan092 said:
    both billing and management not relying on whmcs(standalone)

    Well, honestly; if it's a feature that people want, then I really don't mind coding it. The thing is, that's a whole lot of features that need to be created to make a billing system in its' entirity.

    It's worth noting that the panel itself currently does not require any database whatsoever, as it simply acts as a gateway and interface to the underlying API. It pulls credentials from the API, pulls features from the API, etcetera. If I were to implement any of these billing related features, it would mean I'd have to implement a database and user registration/login first.

  • @GoodHosting said:
    It's worth noting that the panel itself currently does not require any database whatsoever, as it simply acts as a gateway and interface to the underlying API. It pulls credentials from the API, pulls features from the API, etcetera. If I were to implement any of these billing related features, it would mean I'd have to implement a database and user registration/login first.

    its the feature i want.........

    this will give alot of these established panel a run for there money

    anything on ipv6

  • @enitan092 said:
    anything on ipv6

    That's something I should also have mentioned in the OP, not only does this panel support IPv6; but it also explicitly supports the RFC definition and advisements for the use of IPv6 in that each VPS gets a /64 assigned, as it should.

    Thanked by 10xdragon
  • @GoodHosting said:
    That's something I should also have mentioned in the OP, not only does this panel support IPv6; but it also explicitly supports the RFC definition and advisements for the use of IPv6 in that each VPS gets a /64 assigned, as it should.

    waiting for a release i guess,

  • @enitan092 said:

    At the moment, the features/steps required before release are:

    • Rewrite the panel component in pure JavaScript [Node.JS] to increase performance dramatically, and refactor the code at the same time [may as well, good time to do it.]
    • Make the IPv6 assignments idiot friendly. [right now it requires IPv6 knowledge.]
    • Replace the proof of concept "Scripts" with cleaner, more secure agent scripts.
    • Figure out how to make the agent not require privileged system access on hypervisors.
  • Looks nice. How much did you spend for it ?

  • @ftpit said:
    Looks nice. How much did you spend for it ?

    I didn't spend anything for it, I'm developing it myself. It's based on top of a proven API, so there's not a whole lot of back-end coding I need to do. The only back-end coding I need to do is for new features that I implemented from scratch [ such as the Scripts / Agent system, and the live statistics engine. ]

    Thanked by 1support123
  • blackblack Member

    GoodHosting said: new features that I implemented from scratch

    Sounds like fun.

  • @black said:

    Yeah, as you've made quite a few projects yourself I'm sure you feel the hurt too.

    The scripts themselves weren't really that bad though:

    image

  • blackblack Member

    @GoodHosting I prefer backend development over frontend. Bash is one of the more "fun" languages I've experienced so I tend to code in that when I can.

  • AnthonySmithAnthonySmith Member, Patron Provider

    Requests:

    Please actually do this as well as PVH

    said: We will be expanding these features where possible to include Xen-PV and Xen HVM as well.

    Add OpenVZ.

    License it, get an actual team of local developers and actually continue development.

  • Any way for us to be notified via mail or something when a release or at least closed beta is available for testing ?

  • GoodHostingGoodHosting Member
    edited June 2014

    We have started a mailing list so you can get updated when this comes to a beta stage here: http://mailman.goodhosting.co/mailman/listinfo/panel

  • GoodHostingGoodHosting Member
    edited June 2014

    Currently working on storage back-end support, are there any requests for storage engines [ such as ones based on block devices ] that they would like to see supported? I've also started work on the OpenVZ hypervisor support, working from vSwap and Ploop as the standard deployment (any reason not to?)

    Currently on the table for KVM/XEN storage support:

    • Local files [qcow2 format images for KVM]
    • LVM2 Volumes symlinked to files
    • iSCSI, whereby each VM gets a LUN [XEN doesn't like this]

    Currently on the table for OVZ storage support:

    • Classic SIMFS via chroot (might just leave it unsupported?)
    • Local filesystem using Ploop

    Currently looking at a bunch of the nbd type storages to see if any are still woarkable, or they're all just too outdated to consider; will report back when I find anything there, or if someone has any storage they would like supported.

  • This is really grey-area self-promotion to the max, so I've set this to sink. Another one of these threads soon and it'll count towards the offer quota.

  • GoodHostingGoodHosting Member
    edited June 2014

    @mpkossen said:
    This is really grey-area self-promotion to the max, so I've set this to sink. Another one of these threads soon and it'll count towards the offer quota.

    Fair enough mpkossen. This one's not something that will be used at GoodHosting either way, we can plan to release it Open Source once the core components are working and it's got enough functionality for people to play around with.

    This thread [ or this kind of thread in general ] has been popping up a lot lately, so I figured I'd share whatever we had that was nearing enough completion for people to appreciate. Mind you, most of the threads of this kind lately have been monitoring scripts or solutions, haha.


    The idea here is to give a proper alternative to SolusVM, that actually has some features to it / isn't "more or less the same". (Sorry Feathur.)

  • A whmcs module

  • This is not a comment about your control panel (Which looks fantastic BTW).

    I just happened to click through the link to http://goodhosting.co/ and the text is way too light to read easily.

  • @mojojuju said:
    This is not a comment about your control panel (Which looks fantastic BTW).

    I just happened to click through the link to http://goodhosting.co/ and the text is way too light to read easily.

    Thanks for the feedback. I've darkened the main body text about 20%. I also increased the strength by a 100.

Sign In or Register to comment.