Congrats to our Hadoop team! Altoros has been recognized as a proven market leader in Hadoop and big data consulting, according to a recent study by SourcingLine (a Washington, DC-based research company). Their Leaders Matrix leverages proprietary research and methodology to identify top services firms and map their capabilities. Companies are plotted on the matrix based on their proven ability to deliver and focus on a service type.
The analysis based on verified customer interviews gave Altoros a high rank in two lists: Top Big Data & BI Consulting Companies and Top Hadoop Consulting Companies. (Bubble size indicates relative size of the firms.)
“All of the selected companies focus primarily on Big Data analytics or Hadoop and have proven they can provide tangible value to data driven enterprises,” says Tim Clarke, Senior Business Analyst at SourcingLine. “However, it is often challenging for prospective buyers to devise a short-list of leading firms. Our research aims to expedite the procurement process and help connect buyers with qualified service providers.”
Take a look at Altoros’s customer ratings or view the results of these two reports:
Or, read our latest Hadoop studies:
When I was digging into the architectures of Cloud Foundry and OpenShift for a high-level overview, it became obvious that many of their major components have similar functionality:
- Routers manage user traffic.
- Working Nodes are used to stage and run Web applications.
- Managers are the components that manage and monitor Working Nodes and take care of them in case of failures.
- The Messaging Bus enables collaboration between different parts of a distributed PaaS.
Although similar in functionality, these components use different technologies:
|Component / Function
||Brokers, HAProxy Gears
||Warden containers within DEA
||Cloud Controller, Health Managers
|Providing resources and services to applications
||Buildpacks and services
OpenShift uses unified abstractions to work with applications in the cloud. Its Gears are designed to run apps with separated access to shared resources and are implemented as lightweight containers. Cartridges provide the actual functionality necessary to run a user application. They add support for programming languages and access to various databases. Cartridges are the add-ons that contain binaries, setup, and control scripts that make it possible to deploy and maintain the functionality of applications.
OpenShift’s Brokers are the point of contact for all application management activities and traffic. They are implemented as daemons responsible for managing user logins, DNS, and application status. Nodes and Broker Support Nodes (BSN) represent the lower layer of the OpenShift architecture: Nodes are the physical machines where the Gears are allocated, while BSN are the Nodes that run Brokers. BSN and Nodes are connected through a Messaging Bus—for this purpose, OpenShift uses ActiveMQ.
Cloud Foundry has more direct and straightforward abstractions. The dynamic routing layer (Router) handles all the traffic. It resolves application traffic and developers’ Cloud Foundry REST API requests. Cloud Controller is responsible for all management tasks. Being the main endpoint for the Cloud Foundry REST API, it uses the UAA module to authenticate and authorize users. Health Manager monitors the status of applications and takes appropriate actions, when it changes—e.g. if an instance is down, it automatically triggers the system to create a new one. The NATS messaging system processes notifications. Services provide everything that an application may need to work properly with other resources, such as databases and external services.
Both OpenShift and Cloud Foundry support container management. Red Hat provides application container isolation via Docker, while Cloud Foundry relies on Warden. Although Docker and Warden serve the same purpose—providing resource isolation to applications (CPU, memory, etc.)—they use different approaches to achieve this. However, with the Decker project, Cloud Foundry is now a step closer to Docker.
It is also worth mentioning that one of the greatest advantages of Cloud Foundry is support for Heroku buildpacks. These bundles of detection and configuration scripts can deploy applications, as well as install all the application dependencies. When you push an application, the system automatically applies the appropriate buildpack—or you can set it manually.
You can find more details on the topic in my basic overview of OpenShift and Cloud Foundry (features and architectures). The document is by no means exhaustive, but only part of a bigger project. Currently, I’m occupied working on Juju charms for Cloud Foundry together with Pivotal and Canonical, so this comparison will be finalized a bit later.
Seems like great times are coming. More than 80 meetups, workshops, and webinars featuring Cloud Foundry have been announced for the next 3 months. Browse through this list to learn about the upcoming events around Cloud Foundry itself, as well as IBM’s BlueMix, Pivotal One (Pivotal CF), ActiveState’s Stackato, and other CF distributions. Please note that this list doesn’t include pure marketing presentations but is focused on meetups, workshops, and educational webinars. Feel free to add more if I’ve missed anything.
Continue reading this post…
This February was, probably, one of the busiest months in the history of Cloud Foundry. Literally, each day was filled with events, such as the announcement of the Cloud Foundry Foundation, open beta of BlueMix, the news about IBM’s $1B investment plans, etc.
1/02 ActiveState announced their new webinar,”How to Maximize Your Cloud Investment with a Private PaaS
” (59:25 video).
2/02 Jim Northrop posted a guide on how to use the Gradle Cloud Foundry plug-in to push the Caelyf Sample Template to a Cloud Foundry platform other than Pivotal’s.
3/02 Pivotal held a meetup on BOSH at its office in San Francisco. The discussion was dedicated to cluster packaging for continuous integration and disaster recovery.
4/02 AppDynamic announced a partnership with Pivotal and posted a detailed tutorial on how to bring built-in app performance monitoring to Cloud Foundry.
5/02 Anynines presented their PaaS—based on Cloud Foundry—at CLOUDZONE Karlsruhe, Germany. At the same time, Andy Piper of Pivotal was promoting Cloud Foundry in Paris, learning French meanwhile.
6/02 Having launched their CloudPaaS offering (based on CF) last September, NephoScale confirmed they already “have customers that run Cloud Foundry.” The IaaS provider also mentioned it “partners with a few PaaS providers that leverage their API.”
7/02 The 1.0.2 version of the Cloud Foundry Java Client Library (cloudfoundry-client-lib) was released.
8/02–9/02 During this weekend, the CF community was watching the LEGO® Movie. =)
10/02 Pivotal overviewed top 17 big data things that had happened since O’Reilly Strata 2013, including the launch of their Pivotal One platform.
11/02 Altoros introduced a MS SQL Service Broker for Cloud Foundry v2. The new broker enables developing and deploying services using the Microsoft technology stack.
12/02 Dr. Nic Williams works on “bringing Docker to the BOSH-land.” More on playing with private Docker from him: here.
13/02 IBM Cloud organized a #cloudchat dedicated to the evolution of PaaS. The discussion gathered recognized industry analysts, as well as experts from ActiveState, Pivotal, IBM, Altoros, etc.
14/02 “451 Research” released a paper on ActiveState’s Stackato 3.0; while The Church of Jesus Christ ofLatter-day Saints (LDS Church) revealed it is also using Cloud Foundry.
15/02 Overtimes on Saturdays: Matt Stine of Pivotal works on streaming logs into Splunk Enterprise via the Cloud Foundry loggregator.
16/02 Pivotal’s Sundays: looking for an office director in Chicago. The rest of us had the post-St. Valentine’s hangover. =)
17/02 Andy Piper published an explanatory blog post on debugging the Cloud Foundry runtime (or an app inside a Warden container) with BOSH-lite.
18/02 ActiveState were preparing for their webinar, “The Power of Buildpacks in a Platform as a Service” (watch the recorded video, 57:32).
19/02 AppFog cancelled its free plans to prevent malicious activities. On the same day, Verizon signed an agreement with CloudBees. This caused mixed reactions, since they had signed the Cloud Foundry community contributor license agreement (CCLA) on Nov 12, 2013.
20/02 RabbitMQ service became available on Pivotal CF.
21/02 Tensions heat up between the players behind OpenShift and Cloud Foundry: Red Hat kicked Piston out of RH Summit, then performed a $13,000 U-turn.
22/02 The new Cloud Foundry Health Manager (HM9000) written in Go was released.
23/02 IBM Pulse Open Summit started in Las Vegas; however, the most important announcements were made the next day.
24/02 Pivotal announced the Cloud Foundry Foundation backed by a number of industry leaders, such as HP, IBM, Rackspace, SAP, CenturyLink, etc. The company’s CEO, Paul Maritz, even created a Twitter account for that purpose. Read our detailed analysis on this topic to learn what it means for the community and what to expect next. Following this news, IBM launched open beta of the BlueMix PaaS and revealed that $1B would be invested in cloud initiatives.
25/02 Cloud Foundry for Eclipse 1.6.0 was released. In addition, version 6.0.1 of the official CLI for Cloud Foundry (written in Go) became available.
26/02 The Cloud Foundry Advisory Board held its February’s meeting. At the call, Iron Foundry joined the Cloud Foundry incubator; while ActiveState decided to open source the file-system service of their Stackato platform. Read the great recap from ActiveState on the summary of the CAB call and learn about other developer updates missed in our review. Another news on that day was brought by Pivotal: IronMQ message queueing and IronWorker task queue became available on Pivotal’s public PaaS.
27/02 The London Cloud Foundry Dev Community meetup took place in Shoreditch (feat. James Governor, Ban Hale, Jared Wray, and James Masson). A few hours later, Cloud Foundry Summit 2014 opened call for papers—the event will be held in San Francisco, June 9–11.
28/02 Altoros closed this month by introducing a basic overview of features and architectures implemented in Cloud Foundry and OpenShift—a chapter from a comparison study that may be published later this year.
Did we miss something? Feel free to add anything worth mentioning in the comments…or tweet @altoros.
Prepared by: Alex Khizhnyak (Director of Communications at Altoros) and Olga Kurylionak (Communications Manager).
Among all open source projects in the category known as Platform-as-a-Service, OpenShift and Cloud Foundry have amassed the strongest development communities. With similar functionality and goals, both make it possible to write code in a variety of languages and deploy applications to public or private clouds. Both are evolving extremely fast.
Though the launch of the Cloud Foundry Foundation (Feb 24, 2014) changed the game for the entire PaaS industry, few—if any—in-depth comparisons of OpenShift and Cloud Foundry exist yet. We wanted to contribute to this by combining a high-level overview of Cloud Foundry and OpenShift.
Today, we are publishing “OpenShift and Cloud Foundry PaaS: a High-level Overview of Features and Architectures,” a chapter from a deeper research that is still in progress. This document includes 2 tables and 2 diagrams that compare the available functionality as of February 2014.
Please note that this is only the initial version of the document, a 10-page chapter. We understand that for a comprehensive comparison much more info is needed (e.g., adoption, deployment, ALM, etc.). Depending on what will happen to these two platforms in the months coming after the Cloud Foundry Foundation announcement, an extended paper may be published later. We’re already working on it.
As for now, any feedback from the members of the community on the accuracy of data in this white paper is welcome. Your opinions will help to make this and future comparisons as fair as possible. If you have any suggestions on the possible roadmap for this document, feel free to let me know, as well.
The open source project Cloud Foundry exploded overnight. Just as we are witnessing the tipping point of the Ukrainian revolution, something similar just happened in the cloudy world of IT. Pivotal announced that a group of major IT vendors will form the Cloud Foundry Foundation, a legal entity aimed at providing a formal governance body for the open source software project. The group united around Cloud Foundry pulls a combined market capitalization of almost half a trillion dollars. IT superpowers have spoken.
What happened today might be the beginning of a new era for IT folks in the vendor ecosystem and their customers.
- The founding “Platinum sponsors” of the Cloud Foundry Foundation are IBM, HP, Rackspace, SAP, Pivotal, and Pivotal’s parents—VMware and EMC. Each of these companies made a pledge to pay $500,000 per year to the foundation for at least three years. ActiveState and Savvis joined as “Gold sponsors”, pledging $250,000 per year.
- Cloud Foundry is licensed under the Apache License 2.0. This is important because, compared to the General Public License, the Apache License is much more vendor-friendly. It allows modifications without the requirement to open source these modifications, thus empowering a larger ecosystem of vendors to monetize the code base.
- According to my observations, over the last 12 months, there were around 100 monthly active contributors to the project.
- The size of the project’s code-base is no joke. At the moment, it is approaching 600,000 lines of code.
What does this announcement mean for enterprise IT customers?
With this announcement, backers of the foundation are sending their customers a simple message: the traditional way of delivering applications is outdated, and Cloud Foundry will fix it. Here is why: At Altoros, we see three types of customers—those that use Cloud Foundry to make money (as the backbone of an innovation engine while they search for or roll out new business models); those that use Cloud Foundry to save money (by reducing the cost of compliance or increasing the utilization of infrastructure); and those that use Cloud Foundry to deliver productivity gains.
Capturing these benefits is not an easy job, especially when your IT teams design and manage almost each and every development stack nuts-to-bolts, in most cases—one at a time. What we learned is that the companies who became really good at capturing these benefits are using PaaS to limit the number of permutations in their development stacks all the way from virtual machines to the applications. Instead of providing flexibility for each and every stakeholder on each and every layer of the application architecture, they aim to provide a set of pre-defined, pre-approved combinations of run-time environments and database services in the form of APIs, focusing their application development teams on agility and business results.
What are the goals of the Cloud Foundry Foundation?
The goal of the Cloud Foundry Foundation is to promote the development and adoption of Cloud Foundry. Those of you who are bullish on the PaaS technology should read the goal as “to establish the project as a new standard for delivering software applications.”
Was forming the foundation really necessary?
To understand why the Cloud Foundry Foundation was created, let me share with you how the Cloud Foundry project got to where it is today.
- As of fall 2012, the Cloud Foundry project was led by VMware almost single-handedly. At the time, VMware drove adoption of the technology through a handful of small vendors mainly pushing a hosted “Heroku-style” PaaS offering based on Cloud Foundry. Yet, a few large companies became early adopters of private deployments, including Warner Music Group, Sky, NTT, and Rakuten.
- For the vast majority of enterprise IT folks, any given open source project is not “solvent,” unless it commands a diverse community of active contributors. When it comes to attracting community and contributors, many companies who were interested in sponsoring the Cloud Foundry project faced a competitive conflict of interest with VMware, either directly or indirectly. Because of that conflict of interest, the growth of the Cloud Foundry ecosystem was limited.
- In March 2013, EMC and VMware put together its business units (Cloud Foundry, Pivotal Labs, Greenplum, Spring, and Cetas) and spun them off as a separate company named Pivotal. The company employed a total of 1,250 employees and generated about $300 million in annual revenue. EMC commanded a 69% interest in Pivotal, with the remaining 31% owned by VMware.
- In April 2013, General Electric announced a strategic investment of $105 million in Pivotal. In exchange, GE received 10% of interest in the company. The investment aimed to support GE’s Internet of Things strategy, providing access to technology and talent.
- In July 2013, IBM announced that it was joining the Cloud Foundry project.
- Between July 2013 and December 2013, governance of the project became a community-driven process led by IBM. Some project decisions started to happen in collaboration with representatives of 10+ companies, including ActiveState, Altoros, Anynines, Canonical, IBM, Intel, Piston Cloud, Savvis, and Warner Music Group.
- By the end of 2013, this “community-driven process” became a “working prototype” for the forthcoming Cloud Foundry Foundation.
How will the Cloud Foundry Foundation help vendors and customers?
Speaking the language of the revolution, the foundation offers a set of rules, a legal basis for enforcement of such rules, and—most importantly—guarantees.
- Guarantees for vendors. In an industry that changes as fast as ours, building new revenue models with open source software requires the roadmap of the “project core” to match the strategy of a given vendor. The decisions about what to include or exclude from that “core” are pretty much the currency of the game. Voting rights on every level of the project (from forming a definition of what the “core” is or is not to accepting or denying an individual contribution) is the denomination of that currency. The charter of the foundation defines, among other things, the currency and its denomination. It specifies in precise detail how the money (decision power and voting rights) is distributed between citizens (various stakeholders in the project). In a nutshell, the foundation provides guarantees to stakeholders in exchange for source code contributions or funding.
- Guarantees for enterprise customers. Enterprise customers, on the one hand, are also building new (or protecting existing) revenue models. They want to avoid vendor lock-in and de-risk long-term technology liabilities. The foundation addresses the requirements of a typical due-diligence process, provides the needed transparency, and naturally pulls the vote of confidence from the conservative side of the enterprise IT.
What does the announcement mean for IT vendors?
Legacy vendors (infrastructure/hardware, software, and hosting) are forced to reinvent their businesses every now and then. The remarkable rise and domination of AWS disrupted many in the vendor ecosystem. Amazon is known for changing the landscape of many (but not all) industries it has played in. Legacy vendors have to move very fast. Cloud Foundry provides a perfect ground for many vendors to unite around a common goal.
Here is what some of the foundation members could do with Cloud Foundry:
- SAP could package Cloud Foundry to an as-a-Service layer above its database, creating a Hana-as-a-Service offering. If it does well, SAP might even build the successor to its NetWeaver business on Cloud Foundry.
- Cloud Foundry may help RackSpace to better fulfill its new mission as a hybrid cloud service provider, delivering a hybrid IaaS/PaaS offering.
- IBM. Last week, I got a chance to play with IBM BlueMix, a PaaS offering based on Cloud Foundry. Interestingly, BlueMix already contains Liberty, the cloud-ready version of the IBM WebSphere Application Server, SSO and mobile backend as a service. It seems like IBM is creating a unified platform for the cloud infrastructure, big data, and their existing middleware business. Cloud Foundry could very well play one of the key roles in their product strategy.
- For HP, Cloud Foundry may help in differentiating its software-defined infrastructure all the way from servers to applications. Its strength could be in the granularity of control, which HP’s customers could have over the form of infrastructure ownership, covering every type of deployment—from legacy to in-house private cloud, to hybrid and public clouds. In a deployment built for one of Altoros’s customers, Cloud Foundry performed remarkably well together with HP’s Moonshot, a next-generation 45-node, $65k-$100k micro-server chassis.
One vendor from the Cloud Foundry ecosystem was not present among the members of the foundation. It was Canonical, one of the key beneficiaries of the announcement. Ubuntu, Ubuntu OpenStack, and Cloud Foundry, all enjoy perfect synergies with the exception of conflicting deployment tool chains (Cloud Foundry’s BOSH vs. Canonical’s Juju—both can deploy anything, anywhere). In one of my next posts, I will cover in depth why the union of Cloud Foundry and Ubuntu OpenStack changes the game for many.
The question is—did anyone lose? Well, the PaaS vendors competing with Cloud Foundry just got more work to do, and lots of it. Red Hat will now face even more challenges in building an ecosystem around OpenShift. I really hope that folks at Red Hat will work with the Cloud Foundry community to standardize around run-time and services, which are now incompatible. Wouldn’t it be nice to see the cloud moving to the next level of interoperability, at least at the PaaS level?
A High-level Overview of OpenShift and Cloud Foundry PaaS: Features and Architectures
We’ve just finalized the .NET Cloud Foundry Service, a Microsoft .NET-based service broker that provides a possibility to use MS SQL Server with Cloud Foundry v2. The broker allows for easy developing and deploying Cloud Foundry services using the Microsoft technology stack. For example, you can quickly add support for unsupported databases, since you only need to implement a single interface.
Right now, the broker is distributed with one default service implementation (MS SQL Server). Below is a step-by-step video and tutorial on how to install and use it for your project.
Continue reading this post…
The main goal of this article was to find the best framework for a very basic but highly loaded Ruby application. This is the updated version of the comparison that was first posted in Jun 2013. Now we ran all the tests again, using the latest versions of Sinatra, Podrino, Goliath, and RoR. Unfortunately, the Espresso framework that we had tested last time disappeared from all the repositories, so it is no longer included.
We used the following test infrastructure and settings: ApacheBench 2.4.3 ab -n 1000 -c 100 localhost, a 2.3 GHz Intel Core i5 Sandy Bridge CPU, 4 GB of 1333 MHz DDR3 RAM, a 120 GB Intel 330 SSD, OS X Mavericks 10.9.1, Ruby 2.0.0p247, and MySQL 5.6.14 in a development environment. All the tables below contain data on the time it takes to process 1,000 requests.
||No Views and DB
||Views (Slim) + MySQL (Sequel)
Let’s start with Sinatra. It turned out that Unicorn works faster with MySQL and Views than with Views only, but since the difference is very small (0.005 s), this might be due to an error.
To eliminate any inaccuracies, we re-checked all the controversial results many times and got pretty stable values. So, you should be extremely careful when building high-load applications, because any small changes may slow them down. We recommend that you always run benchmark tests to the Web server you want to use in production.
Since Thin demonstrated the best results while keeping the application very simple, we decided to use it for the rest of the tests.
||No Views and DB
||Views (Slim) + MySQL (Sequel)
Although the Padrino framework is built on Sinatra with a lot of helpful features, like in Rails, its performance is still close to that of a clean Sinatra application. This is very good news for us.
|No Views and DB
||Views (Slim) + MySQL (Sequel)
The results for Goliath look a bit strange, because we expected this framework to be the fastest, thanks to its own HTTP server. If you test it and get different results, please do not hesitate to mention that in the comments and/or propose your version of the benchmark test.
|No Views and DB
||Views (Slim) + MySQL (Sequel)
We also decided to test Rails 4.0.2. For these tests, we enabled caching classes in development.rb and disabled sessions and protection from forgery. We also used the Sequel database toolkit instead of ActiveRecord to make the results more trustworthy. As you can see, they turned out to be close to Goliath’s, but slightly better.
You can take a look at the applications I created for these tests to make sure they were really similar. Hope this article helps you to choose the right Ruby framework. Feel free to leave your comments below.
p.s. Special thanks to Alexander Kuntsevich, our Ruby developer, for his assistance.
This brief presentation will introduce you to Cloud Foundry, an open source Platform-as-a-Service that can run on private clouds. Watch the slides to learn:
1. The history of Cloud Foundry
2. Why Cloud Foundry matters?
3. How to work with Cloud Foundry: deployment, application scaling, and services
4. Developing with a PaaS: workflow for developers, DevOps, and system administrators
5. The architecture of Cloud Foundry
In this brief overview, we’ve gathered Top 10 Platform-as-a-Service news for Jan 2014.
- Gartner Released Magic Quadrant for Enterprise Application Platform-as-a-Service
- CenturyLink Cloud Supports the BOSH Tool Chain for Cloud Foundry Deployments
- January’s Cloud Foundry Advisory Board Meeting
- Docker Received $15 M in Series B Funding
- Microsoft’s Windows Azure Updates
- OpenShift Origin Now Supports CentOS
- CloudMunch Expanded Their DevOps Platform to Windows Azure
- Apprenda 5.0: Support for Java and Oracle
- Jelastic PaaS Integrates with the SendGrid E-mail Infrastructure
- Mendix Raised $25 Million in Funding
Continue reading this post…