Blog

Cloud Foundry Monitoring with Admin UI: Technical Overview

Alexander Lomov

14_12_24_admin_ui_techncial_overview_2In this blog post, you’ll find a technical overview of Admin UI, a Web service that allows to gather metrics from the Cloud Foundry components. I’ve shed some light on its core layers, featured functionality, integration with CF APIs, etc. You’ll also discover what issues you may face while logging in, scaling, and debugging—as well as how to address them.

 

Technology basics (Ruby, the Cuba framework, Redis)

The metrics of Admin UI are based on data taken from the UAA, Cloud Controller databases, and NATS. Admin UI checks the /varz and /healthz endpoints of the Cloud Foundry components. Find more information about these endpoints in the following documentation:

You can read about the Admin UI dependencies in the Gemfile.

Admin UI is a Sinatra-based application that uses SQLite3 to cache data and store stats. To support multiple possible UAA and CC databases (currently, Cloud Foundry supports PostgreSQL and MySQL), Admin UI uses the Sequel gem. That’s why it requires some additional packets to be installed, including the PostgreSQL and MySQL client libraries.

With Admin UI BOSH release, you can expose Admin UI through the CF Router, using cf-registrar. Still, it is not the best way to do it.

 

Components and layers

Admin UI is a JavaScript-based Web application: user’s browser gets a static file (application.html) that makes multiple AJAX requests to the server side (one AJAX request for each tab). These AJAX requests are sent only once, so to update information, you need to refresh a browser page. Data is mostly transferred in the JSON format.

UAA is used as an authentication tool in the latest version of Admin UI. When a user opens the application for the first time [link], he is redirected to the UAA endpoint with a login form. Then, a user is redirected back to Admin UI with username and access scope data in parameters. Username and scope are stored to a session and are further used to identify if a user has access to specific tabs.

The authentication process requires a special registration of Admin UI within UAA. In order to add Admin UI as the UAA client, you can use a script from Admin UI that provides access to UAA via the cf-uaac gem. If you install Admin UI with latest BOSH v4 release, you can run the errand jobs that will do it for you.

The Admin UI server has a special thread that updates the Cloud Foundry components data in the background. It has a low priority and doesn’t hamper the server’s work. This background thread gathers data from the Cloud Foundry components and stores it into SQLite3. Time period required for the thread to gather data is set in configurations. After data is saved to SQLite3, it is accessible by the main thread and is used in responses of a client. The server allows for performing some management tasks for the Cloud Foundry deployment, too. Since the Admin UI client access scope is used to get data or perform any management task, you will not be able to perform actions allowed to your user, but restricted for Admin UI.

14_12_24_admin_ui_techncial_overview

 

Functionality

Admin UI provides access to the CF components data and helps to manage Cloud Foundry deployment. Deeply integrated with Cloud Foundry, Admin UI brings forth loads of relevant information: a list of DEA components, a list of each DEA app containers, the UAA organizations, users and groups, statistics of resource consumption, etc.

 

Integration with Cloud Foundry (APIs)

To enjoy a full scope of possibilities offered by Admin UI, it should be connected to the following components:

  • NATS: the NATS client gem is used

  • the Cloud Controller REST API: no special library, all work with an API is implemented within Admin UI

  • the Cloud Controller DB URI: the sequel gem is used to connect the PostgreSQL or MySQL database

  • the UAA REST API: no special library, all work with an API is implemented within Admin UI

  • the UAA DB URI: the sequel gem is used to connect the PostgreSQL or MySQL database.

 

Performance / scaling

Admin UI is designed to work within the Ruby process, which implies strict constraints on scaling. While you can scale vertically without any limitations, horizontal scaling calls for using a load balancer with a sticky session. As a rule, Admin UI is used by a limited number of operators, so there is no data about request workload it can handle.

 

Error handling / logging and debugging

If you get “This page requires data from services that are currently unavailable,” it means that Admin UI doesn’t have access to the UAA or CC databases or is not able to get data from NATS.

14_12_24_admin_ui_technical_overview

To check if databases are available, you should SSH to the instance, where Admin UI is installed, and try to get access to the databases with client tools. Make sure Admin UI has correct settings for the databases, too. Learn more about this bug in this Github issue.

To find out if NATS is available, you need to get the NATS client and connect to NATS from within the Admin UI virtual machine. To do it, check out the NATS version in the Admin UI Gemfile and install a correspondent version of a gem. Then, you need to connect to NATS to prove it is possible:

gem install nats -v <nats-version>
nats-sub ‘>’ nats://nats-user@nats-password@nats-host:nats-port

Another issue occurs, when you have multiple instances of Admin UI and the Load Balancer that works in the “round robin” mode (you can find details here). In this case, only the Admin UI header will be shown.

If you get the “Wrong Scope” message after authentication to Admin UI, it means you haven’t configured the Admin UI UAA client properly.

 

Localization (Chinese and Russian)

The Russian and Chinese languages support is implemented in the Altoros’s fork of Admin UI in the i18n branch. Pull request, with the changes to the official repository mentioned, is on approval to be merged.

You can deploy a version with Сhinese and Russian support, using our special BOSH release. Discover more about the Admin UI BOSH release with i18n in this blog post.

 

Issues / bugs

While using Admin UI, you may face the following issues:

  1. The Admin UI server uses the SQLite3 file system database for caching information about Cloud Foundry. It may cause problems during horizontal scaling. See part “Performance / Scaling” to learn how to solve this issue.

  2. The authentication process doesn’t recognize the changes made to the user access scope. It means if a user is logged in Admin UI, the result is the same, even after s/he is removed from DEA or his/her access scope is changed.

Admin UI proved to be a great tool with a strong CF bond, using not only the Cloud Foundry REST API, but databases of such core components as the Cloud Controller and UAA. Therefore, Admin UI is able to monitor the Cloud Foundry vital data in real-time without network overloading. Hope, this blog post casts light on what Admin UI is. Feel free to ask any questions in the comments.

About the author: Alexander Lomov is a Cloud Foundry Engineer at Altoros. With extensive Ruby experience, Alexander is a fan of the open source movement, having contributed to Fog, RefineryCMS, Netzke, simple_form, and other projects. His professional interests include AWS, MySQL, PostgreSQL, Cassandra, MongoDB, C++, jQuery, JavaScript, and much more.


Further reading: Creating a BOSH Release for Admin UI, a Monitoring Tool for CF

7 Comments
  • I’m confused about the assertion that advertising the admin-ui via cf-registrar/gorouter isn’t “best way”. What’s wrong with it? Gorouter will happily receive routing requests from any system via NATS. It means that you have a clear networking boundary between users and the backend systems – users -> load balancer -> router -> any HTTP endpoint (CF or not).

  • I’m confused about the assertion that advertising the admin-ui via cf-registrar/gorouter isn’t “best way”. What’s wrong with it? Gorouter will happily receive routing requests from any system via NATS. It means that you have a clear networking boundary between users and the backend systems – users -> load balancer -> router -> any HTTP endpoint (CF or not).

    • Alexandr Lomov

      Very-very-very sorry for so late response. I didn’t get this message from disqus and noticed this comment only going through your timeline on the disqus.

      This conclusion was based on Admin UI team answer, you can find the discussion here – https://github.com/cloudfoundry-incubator/admin-ui/issues/123#issuecomment-60918676. The point is that Admin UI should be accessible even when any CF component is down (including NATS or GoRouter).

      I agree with you that using Gorouter is more natural and comfortable way to expose Admin UI. I will remove this point from the list soon.

      • np with late response; thanks for reply 🙂

        • Alexandr Lomov

          On this CF Summit in Berlin Diego Zamboni formalized the idea, that I had on my brain cortex, but couldn’t express it before. The idea came from quantum mechanics theory as I understand: it claims that you can’t make a reliable system observation being a part of the system. Liked this idea.

  • Bonus, just released the other day is a nats CLI client written in golang https://github.com/soutenniza/nats

  • V Kumar

    Hi Alexander my admin ui url is admin.devtest22.io and I have assigned floating ip to it.while accessing it from internet It redirects to http://uaa.devtest22.io/oauth/authorize?response_type=code&client_id=admin_ui_client&redirect_uri=https://admin.devtest22.io/login
    and says dns host name not found. Are doing any other configuration to access it .

Benchmarks and Research

Subscribe to new posts

Get new posts right in your inbox!