Uniview

Public Cloud Guides

Uniview is designed to have considered its user as from a non-trusted 3rd party organization such as public cloud tenant. Hence the process, user segregation, billing and payment have been in place to serve pubic cloud use cases. Besides that Unview provides the utilities to support daily operation for a public cloud administrator. As far as Public Cloud is concerned, the major activities include bill aggregation and generation, collecting payment, invoice management, Payment Gateway management etc.

User Onboarding Process over Uniview

Unview provide full process of user recruiting for the public cloud use cases. The major utilities including a free online user registration user interface, password resetting, Email based notification and confirmation, payment method user editing etc. Entire process can be management free and all can be conducted through user self-service.

The Process of User onboarding (Default):

Uniview comes with default user onboarding handling with process of below

The way to configure Uniview to have second validation of pay method before enabling a user project is done through manipulating portal properties at database as below. Since this is one time activity, no console is provided.

# mysql
INSERT INTO portal_properties (property_key_name, property_key_value, property_key_type ) 
values ('user_register_project_initial_status', 'enabled', 'string');
# in case the property row exist:
UPDATE portal_properties set property_key_value='disabled'  where property_key_name='user_register_project_initial_status';

User Onboarding Process is highly protected by security setup, such as HTTPS and Fraud Prevention. Hence administrators are required to setup following Security guides for the setup before a user can be registered!

User Onboarding Process through Keycloak

When user federation through KeyCloak is selected, the full process of user registration by KeyCloak will be the primary registration process. Over that Uniview complements Keycloak process by its landing portal to provide necessary payment filtering, permission grant and other utilities.

The Process of a User onboarding with Uniview+Keycloak:

For SSO and Uniview user federation, there might be some Uniview configurations required. For details, see Security Guides -> SSO User Federation

Keycloak as open-source solutions, has many references of customization for branding and registratin process, and the details will not be repeated here! Instead the official document provides a good guides on many of those

Metering by CloudKitty, Gnocchi and Ceilometer

When user has Ceilomter+Gnocchi+CloudKitty installed at backend, Uniview has a seamless support of data produced by those components for bill aggregation, payment and invoice!

Uniview generally works with generic installs. For the details of installing and administrating, see Open Infra official documentations!

By default Uniview is configured to support Metering/Rating of CloudKitty

Metering by Uniview-Collector

Uniview-Collector is an independnt docker service that collects events of instance/volume/floating IP lifecycles and scans the inventory periodically to form a billing database.

For those OpenStack deploys that Ceilomter/Gnocchi/Cloudkitty are not installed, Uniview collector can generate consumption data for billing purpose! Uniview collector works in the way of scanning the inventory, and listening to the message bus of RabbitMq for full lifecycle events of billable objects, e.g. Instance, Volume and Floating IPs. Other possible billing objects metering beyond those three can be extended on demand.

When OpenStack is multiple regions deploy, multiple Uniview-Collector needs be deployed per region and close to OpenStack region's Controller. All collectors will post data to a centralized Uniview database, and there is no need for regional database. From the docker host, it requirs connectivity to the Rabbit host on the designated port.

Uniview collector is optimized to produce moderate data, hence the storage space of Uniview DB is expected to grow moderately only. To compare Gnocchi or Prometheus based time serized database, only a small portion of corresponding data will be produced by Uniview-Collector. Once bill is generated, the raw data can be safely archived or deleted to free storage.

It's highly recommanded to have multiple docker replicas deployed to tolerate failure. Uniview-Collector runs as a pub/sub mode, so that there is no worry of data duplication at many replica cases. Events record loss, in case no collector service up at a time, the billing info can be incomplete that leads to billing incuracy of some resources.

To install Uniview collector, user would request access to the collector image. Install is fully automated with docker-compose or Ansible. The script is available at: git clone https://github.com/ComputingStack/Uniview-OpenStack-Portal.git

For administor bill aggregation, for it to consume data that Uniview-Collector produces instead of CloudKitty, a directive by portal attribute configuration is needed

insert into portal_properties (property_key_name, property_key_value, property_key_type) values( 'metering_db', "uniview-collector", "string"  );

Payment Gateway

Uniview supports Stripe Payment Gateway and will support Paypal in near future. Other payment gateways can be supported on demand as part of customizations.

As a very admin accessible info, the direct database manipulation is needed to set the Stripe Payment Gateway info

Steps of setting Payment Gateway Info:
Step 1: Get the database access and start mysql Console
Step 2: insert into database table portal_properties

mysql>insert into portal_properties ( "property_key_name","property_key_value","property_key_type") 
		  values ("stripe_server_keys", "your keys", "string");
		  exit;
$ #check result:
mysql> select * from  portal_properties where property_key_name="stripe_server_keys";
+-----------------+--------------------+--------------------------------------------------------------------------+-------------------+--------+
| property_key_id | property_key_name  | property_key_value                                                       | property_key_type | remark |
+-----------------+--------------------+--------------------------------------------------------------------------+-------------------+--------+
| NULL            | stripe_server_keys | sk_test_51xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | string            | NULL   |
+-----------------+--------------------+--------------------------------------------------------------------------+-------------------+--------+
1 row in set (0.00 sec)

Step 3: Restart Uniview Services. All uniview services need be restarted to pick up the latest value!

Cloudkitty Conducts Pricing and Rating

Uniview supports OpenStack resource pricing and plan by user selectable options, i.e. using Uniview collector+Uniview Rating, or using OpenStack CloudKittyCloudkitty capability.

1. Know Uniview dependencies on CloudKitty

Uniview integrates the latest solution of OpenStack Cloudkitty as the rating Engine! Once cloudkitty is configured properly (by clients), Uniview will be able to consume the data to produce bills per billing cycle with no other dependency! Cloudkitty has been considered mature and widely adopted by global cloud service providers. As it's open source generic, hence user has the advantange of esay development, cost-saving flexibility etc. Cloudkitty generates monetary unit of customer choice, and Uniview Bill Generation utility will aggregate into a bill per billing cycle

As it's generic and fully open source projects by upstream, OpenStack docs provide a full details of ratting and pricing.

2. Where/How to configure plan and rating

Following OpenStack community guides would get Pricing and Plan defined. Modules of Hashmap and Pyscripts are both working with Uniview. For the pricing defined by Hashmap, Uniview currently can display when user browses the instance type (Flavors). When defined by PyScripts module, service providers may need other places such as Incorporate website to publish such instead. As Rating and pricing can be a one time activity for administrators, hence Uniview doesn't overwrite the functionality. Instead it's highly recommanded to have your backend scripts of CLI defining those rating or use Horizon console to do so.

3. Cloudkitty module choice

Both hashmap and Pyscripts modules work, even though Pyscripts provide a better flexibilities, Hashmap can work out of box quickly for generating rating such as instance time based on flavor, Swift Storage, Volume etc.

4. Billing dashboard and Pricing and Display on Uniview

Uniview provides users a billing dashboard showing up 'Month to date spend' 'last 24 hours spending' and prices display possibility for those defined at CloudKitty. The purpose is to increase transparency and keep customers well tuned. For both billing dashboard and billing, clients will need to access rated data info for exmaple from cloudkitty, and pricing info in general. In order to do so, cloudkitty (policy.yaml) needs be configured to open certain read only API permission to the world, so that Uniview can show up the desired info to compose user dashboard and pricing. Alternatively, cloud service provider can publish pricing info on other static website for user lookup!

Based the module selected, some customization may be needed for those other than hashmap!
5. Discount and Promotion

Discount and promotion over Uniview can be very flexibile and versatile though integration with OpenStack upstream solutions!. Cloudkitty provides various flexibilities of defining discount, and promotion per project and flavor etc to facilitating marketing compaign!

It's fully on Service Provider choice of using CloudKitty or Uniview Rating/Collector and Uniview can support both seamlessly.

Uniview-Rating Conducts Pricing and Rating

Uniview supports a concise and dependeable rating and pricing other than the approach by OpenStack Cloudkitty. The reason to have Uniview-Rating and Collector is more about the manageability and extensibility. Users may find Uniview Collector+Rating are more dependable that provide accurate and traceable billing data to avoid controvercies.

A single switch is implemented to enable Uniview Rating cron job, tune the user console and backend to use Uniview Collector and Unview Rating for billing subject. At default, they works with Cloudkitty. Uniview supports everyone two hours cron job to conduct rating to generate interim billing data and user can query at their billing dashboards.

To define a Price Plan, Uniview provides administrator a console for conviniently editing the rating plan.

1. Clone Into a new price plan draft
2. Insert item by item with console guides for common billing items such as Instance, Volume and Floating IP




For three major billing items the plan is defined to reflect the nature of the services. Each price item is defined its service, i.e. instance/volume/floatingip etc; rating type as flat/threshold, the region, value and cost. Unit name is informative and will not be used by rating module. The price item must be explicitly to the region. An item without region, Rating Module can't find its applicability.

Instance price is defined per flavor for an hour. The price item is tagged by its corresponding flavor id. When no price is explicitly defined, the price is taken as 0.

Volume has two types of price items: flat and threshold. Only one flat price is allowed. Flat is applied at all storage volume size. The threshold is a percentage on flat rate when volume size reachies the threshold. Multiple thresholds are allowed in Uniview, and the rating module will apply sequentially to find the applicability for rating a volume

Floating IP is a single flat price as cost for hour.

3. Activate the draft to be the active plan

Activating the plan implies the plan will be applied to rating module immediately. Hence it's suggested to well plan the time when activation starts!

Bill Generation

From 2023-october, Uniview starts providing console based trigger for OpenStack administrator to aggregate billing data into project/billing cycle granular bill. The default billing cycle is per month. Other billing cycle such as bi-weekly can be provided too on customer's demand. Command based bill generation is still available too and will be decommissioned in future release!

Entry: Application entries->Vendor zone->Billing->Bill Generation.

There are two inputs to get bill generation started. One is the billing cycle of which the bill will be generated. The second is the mode of run. Run has two modes. Dry run which will print the result at the end of execution and prod run caculates and saves the bill into database. The user can browse the data that are generated in the dry run, then with confirmation, prod run can follow to generate the bill ready for billing cycle selected!
Example of Bill Generate Entry:

Collect Payment

Collect Payment can be through two approaches over Uniview. One is user initiated payment on the Unview console, and for pre-authroized pay users, Uniview provide utilities for OpenStack administrator to initiate batch payment at the end of billing cycle. Through administrator console(Vendor Zone), an admin can select the projects and a cycle to automate. Administrator console is an interactive process, so that the critical payment process is manageable and traceable! Administrator initiated payment process in Uniview provides auditability and traceability for offline or real time investagtion when there is error.

Ticketing System

Uniview provides an integrated ticketing system for those public cloud providers so that a user can conviniently report various technical or billing issues. There is console for administrators to reply and resolve the reported issues too.

There are two entried provided. One is for general users where they can create, reply, close and re-open a ticket: Application entries->Support->cases. Another is for administrator to reply, answer, close and re-open a ticket. Administrator console shows up all the open tickets that requires attention, and user console shows up only tickets created associated with the user project!

Each reply allows one attachment to upload, and in case multiple are needed, user can choose reply to attach more than one!

An example of the ticket console as below:

The email notification is configurable on whether a outgoing email will be sent upon activities of "user reply", "administrator reply", "re-opened", "resolved". All invovled users and administrators will be included in the list of recipients!

Email Template Customization

Uniview adopts technology of EJS for all outgoing email templates, and clients can fully customize to your themes, messages and branding

An example of the ticket console as below:

The proces can be as easy as:

• Git clone the install scripts, and get the email template folder! Currently each outgoing correspondence has one ejs file that contains header, body and footer. Execpt the keys that are in bracket <%= %> which are not changeable and will be populated in real time by the system when emails are sent, all rest of text, theme, css definition can be edited to your needs.
• Get familiar with various outgoing email from the default install and based on which design yours!
• Edit the ejs file with organization specific messages, themes, logo etc by CSS
• Once the template is complete, clients need send to ComputingStack Support Team, who will do sanity check and replace those into your orgnization's dedicated image
• If the email subjects need be customized, please contact ComputingStack Support for the messages.yml editing and packacking. It's fully doable by clients with no code skill needed!

When there is need of logo, or other image to be part of an email, it's highly recommanded to use external link, such as img src="https://mycompany.com/images/logo.png", where the image is saved and publicly available!

Invoice Customize

Uniview provides a standard invoice format that is estimated to meet majority business needs

An example of an invoice generated by Uniview as below:

The minimal items to customize:

• 1. Business logo, which will be the master logo of your uniview cloud portal (top left logo)!
• 2. Business Name
• 3. Business Address
• 4. Business website
• 5. Busines note, you can put terms and consition or other notes that clients need be aware
• 6. The business bar code if there is any, that you want to provide to the public

mysql> select property_key_name,  property_key_value from  portal_properties where property_key_name like 'business_%';
+-------------------+---------------------------------------------------------------------------------------------+
| property_key_name | property_key_value                                                                          |
+-------------------+---------------------------------------------------------------------------------------------+
| business_address  | High Tech Center Kanata, Ottawa Ontario Canada K2V9P6                                       |
| business_email    | admin@computingstack.com                                                                    |
| business_name     | ComputingStack Tech Inc.                                                                    |
| business_phone    | +1-613-400-4001                                                                             |
| business_qcode    | https://raw.githubusercontent.com/edisonneza/jspdf-invoice-template/demo/images/qr_code.jpg |
| business_tax_rate | 0.13                                                                                        |
| business_website  | https://www.computingstack.com                                                              |
+-------------------+---------------------------------------------------------------------------------------------+
7 rows in set (0.00 sec)

Client info on the invoice will be based on the user inputs when payment method is created, hence is not customizable!

Phone SMS Integration

By default, Uniview supports Twilio for SMS integration. For a region that TWILIO may not support, Uniview is open to other solutions too.

As one time of lifecycle of a cloud, there isn't console provided for such configuring. Steps of manually setting Twilio based SMS Integration:
Step 1: Get the database access and start mysql Console
Step 2: insert into database table portal_properties

mysql>insert into portal_properties ( "property_key_name","property_key_value","property_key_type") 
		  values ("twilio_sid", "your sid", "string");
		  insert into portal_properties ( "property_key_name","property_key_value","property_key_type") 
		  values ("twilio_token", "your token", "string");
		  insert into portal_properties ( "property_key_name","property_key_value","property_key_type") 
		  values ("twilio_send_from", "your sendfrom", "string");
		  exit;
$ #check result:
mysql> select * from portal_properties where property_key_name like "twilio%";
+-----------------+-------------------+------------------------------------+-------------------+--------+
| property_key_id | property_key_name | property_key_value                 | property_key_type | remark |
+-----------------+-------------------+------------------------------------+-------------------+--------+
| NULL            | twilio_send_from  | +13xxxxx                       | string            | NULL   |
| NULL            | twilio_sid        | ACxxxxxxxxxxxxxxxxxxxxxxxxxxxx | string            | NULL   |
| NULL            | twilio_token      | 4axxxxxxxxxxxxxxxxxxxxxxxxxx   | string            | NULL   |
+-----------------+-------------------+------------------------------------+-------------------+--------+
3 rows in set (0.00 sec)

Step 3: Restart Uniview Services. All uniview services need be restarted to pick up the latest value!

OpenStack Monitor Guides

To support cloud monitor by Uniview, there are some configurations and agents that are needed depending on what you want to monitor.

Agents normally will be installed along with what to be monitored, such as physical hosts, or a light compute resource needed daemons that will query metrics and status from the targeted elements.

All Uniview monitoring agents are self-contained binary to run. That's being said it has no repository dependencies etc of debian APT or centos/Redhat Linux YUM, so that no conflict with underlying OS. That makes binary install safe to be installed on generic OS. However other way such as docker based install is available too

There is a private repository by ComputingStack team that a user can easily pull then run to install all the agents and daemons that are needed to monitor OpenStack resources. Either clients will be provided with this repo along the artifact package as it's very light-weight. Alternatively the repository (based on AWS codecommit) can be cloned by users themselves if user/password are provided.

git clone https://git-codecommit.ca-central-1.amazonaws.com/v1/repos/uniview-openstack-monitor
#edit env/monitor/inventory/hosts file for the necessary IP address and the access
#edit env/monitor/inventory/group_vars/all for the variables that define the monitor
#one command below is a wrapper that have all services agents installed!
cd uniview-openstack-monitor
ansible-playbook -i env/monitor/inventory/ uniview_monitor_install.yml  
		  

Baremetal Monitor Agent Install

Uniview supports baremetal monitoring out of box. The major metrics such as CPU, Ram, Network and Storage will be monitored and alerted on demand.

The baremetal node nature are one of those linux OS, such as Ubuntu, Centos or Redhat where the monitor agent will be installed.

OpenStack Service Monitor

Uniview has service detection on major OpenStack daemons such as Nova, Neutron, Cinder etc. Any any un-expected service stop will be easily reported at Uniview.

There isn't extra configuration needed for the OpenStack service monitoring, as long as a Keystone user with admin role and its password is provided to uniview booting configuration, service will be monitored by Uniview!

Ceph Cluster Monitor

Uniview out of box of Ceph Dashboard and Ceph monitoring.

RabbitMq Monitor Support

As one volunerable component of OpenStack, Uniview provide various monitoring of heathy state of Rabbit Mq.

RabbitMq monitoring support is straightforward! RabbitMq uses the prometheus plugin to collect metrics. For the OS install of RabbitMq, below command will get enabled. For other container based install with such as Kolla ansible, your container extra configuration to enable the plugin might be needed. More detailed instructions of RabbitMq monitoring are available in the install reposity!

sudo rabbitmq-plugins enable rabbitmq_prometheus
Enabling plugins on node rabbit@ubuntu07:
rabbitmq_prometheus
The following plugins have been configured:
  rabbitmq_management_agent
  rabbitmq_prometheus
  rabbitmq_web_dispatch
Applying plugin configuration to rabbit@ubuntu07...
The following plugins have been enabled:
  rabbitmq_management_agent
  rabbitmq_prometheus
  rabbitmq_web_dispatch

started 3 plugins.

Database Monitor Agent Install and Configuration

Database is another volunberable component in OpenStack controller. A proper monitoring will help to identify and tackle issues before it has impact on business. In Uniview OpenStack monitoring, it provides state monitoring, and various metrics views and alerts.

Database monitor will require a daemon that collect the metric from the database. A user needs be created beforehand to talk with MySql server with permissions:

CREATE USER 'exporter'@'localhost' IDENTIFIED BY 'Password' WITH MAX_USER_CONNECTIONS 3;
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'localhost';

When Mysql metric collector is installed on a different host from where Mysql is running:

CREATE USER 'exporter'@'%' IDENTIFIED BY 'Password' WITH MAX_USER_CONNECTIONS 3;
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'%';

Edit the IP of a linux machine where the daemon will be installed, then run:

ansible -i env/monitor/inventory/ install_mysql_exporter.yml

Capacity Monitor Guide

Capacity management is one of the most desireable features of OpenStack cludster. Uniview provides aggregated metrics across hypervisor, bare metal OS, OpenStack services and real time monitoring to produce a most comprehensive capacity view for cluster scaling and also cost-saving!

Alerting and Webhook

Uniview supports comprehensive alerting, alert browsing, and communications with operation team through webhook such as Email, Slack, Pagerduty etc.

If the slack is aimed as a webhook, when an alert is triggered or resolved, below variable need be present and defined as below example

For Pagerduty and email based webhook, similar variables need be defined!

# Edit env/monitor/inventory/group_vars/all, by adding blow variable of slack API URL 
slack_api_url: https://hooks.slack.com/services/xxxx

Security Guides

Uniview provides a variety of security features, such as MFA, Encryption, HTTPS， reCAPTCHA, Organization Level Identify Management, Application Credential etc, so that enterprise users will have out of box tool kit to select for achieving infrastructure security without hassles.

SSO and User Federation

Uniview supports Single Sign on through integrating with Keystone and Keycloak technologies. To have best user experience, there are some configurations required!

Before Uniview is configured to support SSO, Keystone and the intended identify provider such as Keycloak should be configured. The reference of configuring such can be seen in OpenStack Docs Specifically for Uniview, Keystone needs be configured to allow:

[federation]
trusted_dashboard = http://horizon.ifneeded/auth/websso/
trusted_dashboard = http://uniview/sso-login/

Uniview user sign in pages provides two entries: entry for native Keystone user and entry for federated users. By default the entry to federated user is not enabled. To enable it, database access to insert a row of data:

INSERT INTO portal_properties (property_key_name, property_key_value, property_key_type) values("openid_federation", "1", "string");

In order to link user to edit its profile (through external identity providers other than Keystone), the identity provider profile is needed as below

INSERT INTO portal_properties (property_key_name, property_key_value, property_key_type) values("sso_default_idp", "your_idp_defined_at_keystone", "string"); 
INSERT INTO portal_properties (property_key_name, property_key_value, property_key_type) values("sso_default_idp_url", "http://your_keycloak_url/realms/yourrealm", "string");

MFA

Uniview provides full process for the MFA to individual project user and also the administrator.

MFA enabling is multiple steps configurations, instead of a single attribute. Those are such as when/how a user should enable, and once MFA enabled, there are considerations that the users should be able to continue OpenStack CLI etc the service providers can facilitate. For Keystone, it's recommended to have below authentication methods allowed:

methods = external,password,token,oauth1,totp,application_credential,openid,saml

"identity:create_credential": "(rule:admin_required) or user_id:%(target.credential.user_id)s"
"identity:update_user": "(rule:admin_required) or user_id:%(user_id)s"

Encryption

Uniview provides utility for the administrator and user for the encyption across Compute and Storage.

Organization Management

Uniview provides project level user management, such as user inviting etc.

Application Credentials

Uniview provides full support for application credentials, so that password and application credential based authentication is seamless.

RECAPTCHA (Deprecated)

Uniview provides most popular Google based fraudulence prevention utilities through configurations by users themselves. They can be applied to protect user recruitment process, or other such as user login process on customer demand.
Only HTTPS encrypted communciations accept reCaptcha fraud prevention, so that you will require to setup your frontend load balancer to listen at HTTPS before this is fully functional!

To be working with reCAPTCHA, tls(https) certificate, public domain name and reCAPTCHA v2 keys are required!

Tls certificate setup on your LB such as HaProxy or NGINX or other load balancers that sit in front of Uniview service is on clients' diligence.

Clients will be required to apply your own reCAPTCHA keys through Google Process!

Uniview provides a console for administrator to configure the website public URL, Google keys, from entry of: Vendor-Zone-> Operation Settings, collapse: Cloud Portal public DNS Settings

Once the variable is set on the console, Uniview services (local docker or Kubernetes Pods) need a resart to pick up the right value!

Entry of Portal public domain name and reCAPTCHA protection:

OpenStack Mutli-Domains Support

Uniview user console works with both multiple domains or single "Default" domain. By default, Keystone user signing in to Uniview will be assumed under the Default domain.

In majority of OpenStack install, the users and projects are created under the same domain of "Default". There are also some deploys such as by Canonical Juju Charms, the users are possibley created under a domain other than "Default".

For a public cloud, to ask customer to remember a domain other than user/password can be annoying, hence Uniview is designed to be working "Default" domain as the default. Uniview also can be conveniently configured to be working with multi domains too when security policy requires. When the users are created under domain other than "Default", sign-in process by Keystone will require the domain to be passed explicitly. To enable such users to sign in OpenStack over Uniview, a row of portal properties will be required.

insert into portal_properties (property_key_name, property_key_value) values('multi_domains_console_access', 'enabled'  );

FAQ

A FAQ is a list of frequently asked questions (FAQs) and answers on a particular topic.