Saturday, December 9, 2023
HomeCloud ComputingAWS Cloud Management API, a Uniform API to Entry AWS & Third-Social...

AWS Cloud Management API, a Uniform API to Entry AWS & Third-Social gathering Companies

At this time, I’m comfortable to announce the supply of AWS Cloud Management API a set of frequent utility programming interfaces (APIs) which can be designed to make it straightforward for builders to handle their AWS and third-party providers.

AWS delivers the broadest and deepest portfolio of cloud providers. Builders leverage these to construct any sort of cloud infrastructure. It began with Amazon Easy Storage Service (Amazon S3) 15 years in the past and grew over 200+ providers. Every AWS service has a selected API with its personal vocabulary, enter parameters, and error reporting. For instance, you utilize the S3 CreateBucket API to create an Amazon Easy Storage Service (Amazon S3) bucket and the Amazon Elastic Compute Cloud (Amazon EC2) RunInstances API to create an EC2 situations.

A few of you utilize AWS APIs to construct infrastructure-as-code, some to examine and robotically enhance your safety posture, some others for configuration administration, or to provision and to configure excessive efficiency compute clusters. The use circumstances are numerous.

As purposes and infrastructures grow to be more and more refined and you’re employed throughout extra AWS providers, it turns into more and more tough to be taught and handle distinct APIs. This problem is exacerbated if you additionally use third-party providers in your infrastructure, since it’s important to construct and preserve customized code to handle each the AWS and third-party providers collectively.

Cloud Management API is a normal set of APIs to Create, Learn, Replace, Delete, and Checklist (CRUDL) sources throughout a whole bunch of AWS Companies (extra being added) and dozens of third-party providers (and rising).

It exposes 5 frequent verbs (CreateResource, GetResource, UpdateResource, DeleteResource, ListResource) to handle the lifecycle of providers. For instance, to create an Amazon Elastic Container Service (Amazon ECS) cluster or an AWS Lambda operate, you name the identical CreateResource API, passing as parameters the kind and attributes of the useful resource you wish to create: an Amazon ECS cluster or a Lambda operate. The enter parameters are outlined by a unified useful resource mannequin utilizing JSON. Equally, the return sorts and error messages are uniform throughout all verbs and all sources.

Cloud Management API supplies help for a whole bunch of AWS sources at present, and we are going to proceed so as to add help for current AWS sources throughout providers resembling Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Easy Storage Service (Amazon S3) within the coming months. It is going to help new AWS sources usually on the day of launch.

Till at present, after I wish to get the small print a couple of Lambda operate or a Amazon Kinesis stream, I exploit the get-function API to name Lambda and the describe-stream API to name Kinesis. Discover within the instance under how completely different these two API calls are: they’ve completely different names, completely different naming conventions, completely different JSON outputs, and so forth.

aws lambda get-function --function-name TictactoeDatabaseCdkStack
    "Configuration": {
        "FunctionName": "TictactoeDatabaseCdkStack",
        "FunctionArn": "arn:aws:lambda:us-west-2:0123456789:operate:TictactoeDatabaseCdkStack",
        "Runtime": "nodejs14.x",
        "Function": "arn:aws:iam::0123456789:function/TictactoeDatabaseCdkStack",
        "Handler": "framework.onEvent",
        "CodeSize": 21539,
        "Timeout": 900,
        "MemorySize": 128,
        "LastModified": "2021-06-07T11:26:39.767+0000",


aws kinesis describe-stream --stream-name AWSNewsBlog
    "StreamDescription": {
        "Shards": [
                "ShardId": "shardId-000000000000",
                "HashKeyRange": {
                    "StartingHashKey": "0",
                    "EndingHashKey": "340282366920938463463374607431768211455"
                "SequenceNumberRange": {
                    "StartingSequenceNumber": "49622132796672989268327879810972713309953024040638611458"
        "StreamARN": "arn:aws:kinesis:us-west-2:012345678901:stream/AWSNewsBlog",
        "StreamName": "AWSNewsBlog",
        "StreamStatus": "ACTIVE",
        "RetentionPeriodHours": 24,
        "EncryptionType": "NONE",
        "KeyId": null,
        "StreamCreationTimestamp": "2021-09-17T14:58:20+02:00"

In distinction, when utilizing the Cloud Management API, I exploit a single API identify get-resource, and I obtain a constant output.

aws cloudcontrol get-resource        
    --type-name AWS::Kinesis::Stream 
    --identifier NewsBlogDemo
   "TypeName": "AWS::Kinesis::Stream",
   "ResourceDescription": {
      "Identifier": "NewsBlogDemo",
      "Properties": "{"Arn":"arn:aws:kinesis:us-west-2:486652066693:stream/NewsBlogDemo","RetentionPeriodHours":168,"Title":"NewsBlogDemo","ShardCount":3}"

Similary, to create the useful resource above I used the create-resource API.

aws cloudcontrol create-resource    
   --type-name AWS::Kinesis::Stream 
   --desired-state "{"Title": "NewsBlogDemo","RetentionPeriodHours":168, "ShardCount":3}"

In my view, there are three sorts of builders which can be going to undertake Cloud Management API:

The primary group is builders utilizing AWS Companies APIs to handle their infrastructure or their buyer’s infrastructure. Those requiring utilization of low-level AWS Companies APIs moderately than larger degree instruments. For instance, I do know corporations that manages AWS infrastructures on behalf of their shoppers. Many developed options to checklist and describe all sources deployed of their consumer’s AWS Accounts, for administration and billing functions. Typically, they constructed particular instruments to deal with their necessities, however discover it arduous to maintain up with new AWS Companies and options. Cloud Management API simplifies any such instruments by offering a constant, resource-centric strategy. It makes simpler to maintain up with new AWS Companies and options.

One other instance: Stedi is a developer-focused platform for constructing automated Digital Information Interchange (EDI) options that combine with any enterprise system. “We’ve got a robust give attention to infrastructure as code (IaC) inside Stedi and have been in search of a programmatic method to uncover and delete legacy cloud sources which can be now not managed via CloudFormation – serving to us cut back complexity and handle price,” stated Olaf Conjin, Serverless Engineer at Stedi, Inc. “With AWS Cloud Management API, our groups can simply checklist every of those legacy sources, cross-reference them towards CloudFormation managed sources, apply extra logic and delete the legacy sources. By deleting these unused legacy sources utilizing Cloud Management API, we will handle our cloud spend in a less complicated and sooner method. Cloud Management API permits us to take away the necessity to writer and preserve customized code to find and delete every sort of useful resource, serving to us enhance our developer velocity”.

APN Companions
The second group that advantages from Cloud Management API is APN Companions, resembling HashiCorp (maker of Terraform) and Pulumi, and different APN Companions providing options that depends on AWS Companies APIs. When AWS releases a brand new service or function, our associate’s engineering groups must be taught, combine, and check a brand new set of AWS Service APIs to reveal it of their choices. This can be a time consuming course of and sometimes results in a lag between the AWS launch and the supply of the service or function of their resolution. With the brand new Cloud Management API, companions at the moment are in a position to construct a novel REST API code base, utilizing unified API verbs, frequent enter parameters, and customary error sorts. They simply should merge the standardized pre-defined uniform useful resource mannequin to work together with new AWS Companies uncovered as REST sources.

Launch Companions
HashiCorp and Pulumi are our launch companions, each options are built-in with Cloud Management API at present.

HashiCorp supplies cloud infrastructure automation software program that permits organizations to provision, safe, join, and run any infrastructure for any utility. “AWS Cloud Management API makes it simpler for our groups to construct options to combine with new and current AWS providers,” stated James Bayer – EVP Product, HashiCorp. “Integrating HashiCorp Terraform with AWS Cloud Management API means builders are ready to make use of the newly launched AWS options and providers, usually on the day of launch.”

Pulumi’s new AWS Native Supplier, powered by the AWS Cloud Management API, “offers Pulumi’s customers sooner entry to the newest AWS improvements, usually the day they launch, with none want for us to manually implement help,” stated Joe Duffy, CEO at Pulumi. “The complete floor space of AWS sources supplied by AWS Cloud Management API can now be automated from acquainted languages like Python, TypeScript, .NET, and Go, with customary IDEs, package deal managers, and check frameworks, with excessive constancy and nice high quality. Utilizing this new supplier, builders and infrastructure groups can develop and ship fashionable AWS purposes and infrastructure sooner and with extra confidence than ever earlier than.”

To be taught extra about HashiCorp and Pulumi’s integration with Cloud Management API, consult with their weblog put up and bulletins.

AWS Clients
The third sort of builders that may profit from Cloud Management API is AWS clients utilizing resolution resembling Terraform or Pulumi. You possibly can profit from Cloud Management API too. For instance, when utilizing the brand new Terraform AWS Cloud Management supplier or Pulumi’s AWS Native Supplier, you’ll be able to profit from availability of recent AWS Companies and options usually on the day of launch.

Now that you simply perceive the advantages, let’s see Cloud Management API in motion.

How It Works?
To begin utilizing Cloud Management API, I first be sure that I exploit the newest AWS Command Line Interface (CLI) model. Relying on how the CLI was put in, there are completely different strategies to replace the CLI. Cloud Management API is offered from our AWS SDKs as nicely.

To create an AWS Lambda operate, I first create an handler, I zip it, and add the zip file to considered one of my personal bucket. I concentrate that the S3 bucket is in the identical AWS Area the place I’ll create the Lambda operate:

cat << EOF >  
import json 
def lambda_handler(occasion, context):
    return {
        'statusCode': 200,
        'physique': json.dumps('Whats up from Lambda!')

aws s3 cp s3://private-bucket-seb/

Then, I name the create-resource API, passing the identical set of arguments as required by the corresponding CloudFormation useful resource. On this instance, the Code, Function, Runtime, and Handler arguments are obligatory, as per the CloudFormation AWS::Lambda::Perform documentation.

aws cloudcontrol create-resource          
       --type-name AWS::Lambda::Perform   
       --desired-state '{"Code":{"S3Bucket":"private-bucket-seb","S3Key":""},"Function":"arn:aws:iam::0123456789:function/lambda_basic_execution","Runtime":"python3.9","Handler":"index.lambda_handler"}' 
       --client-token 123

    "ProgressEvent": {
        "TypeName": "AWS::Lambda::Perform",
        "RequestToken": "56a0782b-2b26-491c-b082-18f63d571bbd",
        "Operation": "CREATE",
        "OperationStatus": "IN_PROGRESS",
        "EventTime": "2021-09-26T12:05:42.210000+02:00"

I’ll name the identical command once more to get the standing or to study an eventual error:

aws cloudcontrol create-resource          
       --type-name AWS::Lambda::Perform   
       --desired-state '{"Code":{"S3Bucket":"private-bucket-seb","S3Key":""},"Function":"arn:aws:iam::0123456789:function/lambda_basic_execution","Runtime":"python3.9","Handler":"index.lambda_handler"}' 
       --client-token 123

    "ProgressEvent": {
        "TypeName": "AWS::Lambda::Perform",
        "Identifier": "ukjfq7sqG15LvfC30hwbRAMfR-96K3UNUCxNd9",
        "RequestToken": "f634b21d-22ed-41bb-9612-8740297d20a3",
        "Operation": "CREATE",
        "OperationStatus": "SUCCESS",
        "EventTime": "2021-09-26T19:46:46.643000+02:00"

Right here, the OperationStatus is SUCCESS and the operate identify is ukjfq7sqG15LvfC30hwbRAMfR-96K3UNUCxNd9 (I can cross my very own identify if I would like one thing extra descriptive 🙂 )

I then invoke the Lambda operate to make sure it really works as anticipated:

aws lambda invoke 
    --function-name ukjfq7sqG15LvfC30hwbRAMfR-96K3UNUCxNd9 
    out.txt && cat out.txt && rm out.txt 

    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
{"statusCode": 200, "physique": ""Whats up from Lambda!""}

When completed, I delete the Lambda operate utilizing Cloud Management API:

aws cloudcontrol delete-resource 
     --type-name AWS::Lambda::Perform 
     --identifier ukjfq7sqG15LvfC30hwbRAMfR-96K3UNUCxNd9 
    "ProgressEvent": {
        "TypeName": "AWS::Lambda::Perform",
        "Identifier": "ukjfq7sqG15LvfC30hwbRAMfR-96K3UNUCxNd9",
        "RequestToken": "8923991d-72b3-4981-8160-4d9a585965a3",
        "Operation": "DELETE",
        "OperationStatus": "IN_PROGRESS",
        "EventTime": "2021-09-26T20:06:22.013000+02:00"

You might need seen the client-token parameter I handed to the create-resource API name. Create, Replace, and Delete requests all settle for a ClientToken, which is used to make sure idempotency of the request.

  • We advocate all the time passing a consumer token. This may disambiguate requests in case a retry is required. In any other case, you might encounter surprising errors like ConcurrentOperationException or AlreadyExists.
  • We advocate that consumer tokens all the time be distinctive for each single request, resembling by passing a UUID.

One Extra Factor
On the coronary heart of AWS Cloud Management API supply of knowledge, there may be the CloudFormation Public Registry, which my colleague Steve introduced final June in this weblog put up. It permits anybody to reveal a set of AWS sources via CloudFormation and AWS CDK. That is the mechanism AWS Service groups at the moment are utilizing to launch their providers and options as CloudFormation and AWS CDK sources. A number of third-party distributors are additionally publishing their options within the CloudFormation Public Registry. All sources printed are modelled with a normal schema that defines the useful resource, its properties, and their attributes in a uniform means.

AWS Cloud Management API is a CRUDL API layer on high of sources printed within the CloudFormation Public Registry. Any useful resource printed within the registry exposes its attributes with customary JSON schemas. The useful resource can then be created, up to date, deleted, or listed utilizing Cloud Management API with no extra work.

For instance, think about I resolve to reveal a public CloudFormation stack to let any AWS buyer create VPN servers, based mostly on EC2 situations. I mannequin the VPNServer useful resource sort and publish it within the CloudFormation Public Registry. With no extra work on my aspect, my customized useful resource “VPNServer” is now obtainable to all AWS clients via the Cloud Management API REST API. Not solely, it is usually robotically obtainable via options like Hashicorp’s Terraform and Pulumi, and probably others who undertake Cloud Management API sooner or later.

It’s value mentioning Cloud Management API isn’t aimed toward changing the standard AWS service-level APIs. They’re nonetheless there and can all the time be there, however we expect that Cloud Management API is less complicated and extra constant to make use of and it’s best to use it for brand new apps.

Availability and Pricing
Cloud Management API is offered in all AWS Areas, besides China.

You’ll solely pay for the utilization of underlying AWS sources, resembling a CloudWatch logs or Lambda features invocations, or pay for the variety of handler operations and handler operation period related to utilizing third-party sources (resembling Datadog screens or MongoDB Atlas clusters). There aren’t any minimal charges and no required upfront commitments.

I can’t wait to find what you will construct on high of this new Cloud Management API. Go construct!

— seb



Please enter your comment!
Please enter your name here

Most Popular

Recent Comments