As you move more workloads into the cloud, you’ll be looking to expand your overall governance, and operational practices around it to ensure application reliability, security, performance, and cost optimization. You may also be investing in DevOps practices such as continuous integration, continuous delivery, infrastructure as code, scaling, and other automation.

The question you may ask yourself is whether implementing your cloud strategy and DevOps practices is moving fast enough with the right level of talent and expertise. Many organizations will look for managed service providers (MSP) to either supplement their existing staff or to outsource most of the cloud management services. Some will seek a MSP early to aid in developing their cloud strategy, identify vendors, and to oversee a mass migration to the cloud. Others will begin the journey on their own and will look to add an MSP to address skill gaps or to raise service levels. In a recent survey, 56% of respondents claimed that they were still in the migration process to the cloud so many should still be considering the help of service providers to aid in migration or to establish best practices in architecture and operations.

Cloud MSP Versus Hosting MSP

When looking at service providers for cloud infrastructure and workloads, it’s reasonable to leverage some of the same questions, criteria, and due-diligence applied to data center providers, hosting services, and SaaS offerings. You’ll want to know:

• What uptime and performance service levels they guarantee
• Security expertise and services
• Approach to incident management
• Cloud optimized disaster recovery methodologies
• Tools and approach to monitoring applications
• Skill levels in targeted technologies
• Approach to hiring and retaining talent
• Overall financial status
• Pricing structure
• Customer mix
• AND other criteria applied

But that’s not enough!

Remember why you moved to cloud in the first place. Cloud is not just about hosting applications, it’s about the flexibility in scaling computing resources up and down as demand fluctuates. It’s about nimble practices to add and scale infrastructure so that you can meet new computing demands. It’s about attaining higher levels of reliability by having architectures that span multiple availability zones. Lastly, and most important, it’s about getting the automation and achieving a DevOps culture so that you can meet business needs around digital transformation, innovation, and other strategic programs.

A hosting provider that’s moved into the business to provide cloud services may not have all the skills and practices to achieve goals that leverage cloud services or DevOps practices. If a hosting service company sees their role primarily around cloud transition and hosting operational practices, your cloud migration may fall short of achieving the targeted agility and cost optimizations.

Questions to Draw Our Cloud Competencies

AWS currently lists 80+ Managed Service Providers and 45+ partners under DevOps so identifying the right mix of service providers, deciding how best to use internal talent, selecting an optimal set of technologies and tools, and maturing the optimal set of operational and DevOps practices may seem overwhelming. In reviewing cloud service providers as a CIO and now with several clients, there are several key questions that helped separate cloud providers that truly understand cloud operations and can partner with you on transformations.

1. What’s your role in enabling DevOps, security, and automation with your clients and what’s your expertise with different platforms and tools?

Service providers should be certified by the cloud vendor (AWS, Azure, etc) and should also have their best practices in implementing the architecture, security, and DevOps capabilities. You’ll want to review these practices, understand where they have successfully implemented them, discuss with references, and determine whether their approach is appropriate for your business and workloads.

Some vendors will hyper-specialize on specific tools and practices. If you’re already using Jenkins, Docker, and Ansible and the service provider has demonstrated expertise with these technologies then you’re probably in good shape. Even so, it’s good to understand the overall flexibility of the service provider. Are they leading you into a one-size fits all architecture or are they technically strong enough and culturally flexible to adapt their best practices to your business needs?

2. What constitutes “support” and what types of issues are you escalating back to application developers?

A classic hosting provider will support the underlying computing infrastructure and escalate application, database, and sometimes environment issues back to the development team to respond and resolve. In the cloud era where infrastructure management is being handled by the cloud vendor and where development teams are chartered with digital transformation programs, cloud service providers have to “step up” and handle a larger responsibility in diagnosing production issues and resolving them.

I have a simple test.

If the issue’s root cause is in code, then the development team must resolve it. If it’s outside of code, I expect a cloud service provider to mature it’s understanding of the underlying applications so that they can resolve the majority of non-code incidents. Even it if it’s a code issue, I expect them to be able to navigate log files or leverage an application monitoring tool to pinpoint the services that are at the root cause of the issue. Lastly, I expect them to be able to collaborate with developers to get a reasonable knowledge transfer and to feedback support needs back into the development backlog.

3. What types of requests and changes will incur additional costs and how do you help reduce cloud hosting costs?

If you operate the cloud like a data center with fixed assets and computing capacity then it’s really hard to optimize costs. Cloud service providers should be monitoring cloud accounts and utilization, recommending optimizations, and automating the scaling of resources using standard tools that help to automate these processes. They also have to be well versed in new vendor capabilities especially regarding containerized workloads and serverless computing to drive workload migrations to less expensive service offerings. Lastly, they should demonstrate how they’ve handled shifting or explosive demand.

Not all providers are equally skilled or incentivized to implement these optimizations especially providers that prices their services as a percentage of the underlying cloud costs. The best way to evaluate service providers is to ask these three questions and for their track records in helping clients optimize their environments and the cost improvements they instrumented.