Enterprise integration solutions these days tend to span multiple platforms across Cloud and On-Premise locations to deliver large complex programs. The management of these platforms needs to be fast, precise and continuous to ensure there are no bottlenecks to delivery as the solution development flows through its lifecycle
This means creating environments across multiple platforms, creating configuration, deploying applications, managing properties, endpoints and lots more. Environment / Platform teams have their hands-full!
So how do platform teams manage to provision, deploy, change, monitor etc. when everything is running somewhere else especially given how modern applications (from vendors) are hosted in some-else’s-cloud?
Also, managing platforms by hand is hard, error-prone and time consuming
In order to manage the chaos, accelerate delivery and deliver platform environments for complex programs safely while reducing impact from platform / environment issues – Platform Engineers use interfaces build to create, update, query the Platform and Platform resources
These interfaces are called Platform APIs (as opposed to Service APIs) and are the key to automating platform provisioning, platform resource management, configuration management and key to building complex solutions over distributed platforms
So what are platform APIs
Internally automating your platform environments requires a skilled team of platform engineers but what if you are integrating platforms-as-services which host applications which you need to pull together to build end-to-end solutions; how then do you orchestrate these multi-vendor platforms into your environment
Platform APIs are interfaces to a platform that allow consumers to authenticate, authorise, provision, query, monitor the platform. The consumers of these APIs are teams that need to manage and maintain such platforms for application services teams, in most cases this could be a single platform and set of platform APIs they manage but in more complex use cases these could be a range of platforms that are part of their solution
What functionality do Platform APIs provide?
Modern Platforms (Cloud or On-Prem) should offer a basic set of functionality via Platform APIs. The base set of Platform APIs provided can be
- Authentication for platform administrator
- Create/Query the organisation for the platform (who is this instance for)
- Create/Update, Query and Delete environments for the organisation
- Deploy applications to environments
- Query environment configuration and properties (network, gateway etc)
- Update/Query application configuration (environment scoped properties)
- Update/Query application platform resource utilisation and health
Some platform providers also extend these APIs to provide logging and monitoring information. The base set of APIs allow the consumers to automate the platform provisioning, environment configuration and application deployment tasks
How do we use platform APIs to accelerate delivery?
Platform APIs can automate provisioning, shakedown and monitoring of Cloud and On-prem platforms. Orchestrating platform APIs from different vendors, client platform engineers can rapidly deploy environments for small to large, simple to complex programs of delivery
In IT programs the platform teams can be requested to provision, validate, monitor and manage a large number of environments each containing multiple enterprise applications hosted in one or more platforms and platform environments and hosting one or more application with different configuration values. Doing this by hand can be time consuming and error-prone and automating environment provisioning, validation and monitoring becomes a key ask
We can refine these asks into the following
- Automating platform provisioning / de-provisioning specially for modern cloud or hybrid solutions given platform is externally hosted. The ability to remotely manage them is key
- Automating provisioning of multi-vendor Platform-as-Service environments especially for modern cloud or hybrid solutions can reduce lead times for programs of work
- Automating and establishing Validating environments and application configurations through automation vs manual shakedown can reduce the time to for environment delivery from weeks to days in multi-platform solutions
- Automating monitoring and de-provisioning of environments when paying per resource in multiple platform environment can reduce operational budget the death-by-thousand-papercuts
One example was when we built an Environment Configuration Consistency Checker using Platform APIs to continuously monitor cloud platforms for changes to agreed/consistent view of configurations. This approach reduced environment shakedown times from weeks (2-3) to days (1-2) and notified change owners in real-time if they broke a dependent system in an integrated environment. How sweet!
What are some of the key things to look out for or consider?
Platform APIs are amazing to use and orchestrate platforms for delivering environments etc however there are a few things to watch out for when using them and a few things to consider
- Rate limits: Like every other API out there, be careful about SLAs and limits for your platform APIs. Ensure you read the documentation and contract to know how many calls you can make. Check if Platform API provider distinguishes between reads and writes because if they do not then reading (like polling for platform status) can limit activities like provisioning (preventing UI or API based access)
- Security: This is a no-brainer but do not share your platform API “samples” with platform credentials. These are always probably “admin” credentials and yeah it hurts to lose them. I once took a screenshot of a Platform Orchestration script which showed the Base64 encoded basic auth credentials and was about to present it to a large audience – luckily it was noticed before I could leak out the admin credentials :face-palm:
- Manage Differences with Anti-corruption Layer: When orchestrating multiple platform APIs, do not expect all vendors to provide the same functionality or data-model. Expect them to be different and build consumer adapters which expose a common internal interface and convert the external platform APIs to the internal format by mapping. This will make your code maintainable and extensible
- Extend to Application Services: Newly deployed applications on the platform especially microservices can extend the platform API by including endpoints such as “/health” to enable continuous validation of the platform and its products. Encourage your application services team to extend platform services
- Continuous Platform Health Monitoring: Speaking of health checks, one thing to consider is using the platform APIs to ensure the foundations are working okay and doing so continuously. Some platform providers provide inbound streams of monitoring data while others provide polling endpoint
- Continuous Platform Endpoint Configuration Monitoring: When we build integrated solutions across platforms, these solution components need to know the endpoint configuration of one another to discover and exchange data. However, when services are unavailable or change configuration then the consumers may have the wrong address for their endpoints. Use the platform APIs to investigate the configured endpoints and ensure these map to the registry of services and do this continuously – this can speed up environment shakedown, provide proactive early notifications of service outages and pin-point root cause (reducing time to root cause analysis) when integration failures happen
Summary
As we build more solutions leveraging platforms and applications-as-a-service platform teams have have a lot more to provision, manage and configure. Complex and simple solutions in multi-platform or even single platform led programs require environments provisioning, configuration, validation, continuous monitoring tasks etc to deliver a modern software product
These tasks can be time consuming, complex and buggy if done manually. We want them to be fast, safe, repeatable by automating them and orchestrating them through Platform APIs
There are 4 basic functionality that you as platform engineers and platform team owners should ask of service providers when they offer to host their application and build on their platform. These include ability to create and monitor environments, configurations etc.
Also there are about 5 key things to consider from experience, hope to see you add more to this list as you unleash the power of automation and introspection on your platform!
Alok Mishra 