Api Testing

API Description

According to wikipedia an API is:

An application programming interface (API) is a set of subroutine definitions, protocols, and tools for building application software. In general terms, it is a set of clearly defined methods of communication between various software components.

Modern Web APIs are Restful

There are 6 constraints for an API to be Restful :

  1. Client-server architecture

  2. Statelessness

  3. Cacheability

  4. Layered system

  5. Code on demand (optional)

  6. Uniform interface

Notice that Code on demand is an optional constraint so APIs that don’t have this constraint can still be restful

Restful data formats

Most modern Web APIs use JavaScript Object Notation (JSON) instead of XML

Although it is not required that you use JSON for a restful API most modern Web APIs do use JSON

JSON Format

JSON is an open-standard file format that uses human-readable text to transmit data objects consisting of attribute–value pairs and array data types.

JSON’s basic data types are:

  1. Number a signed decimal number that may contain a fractional part and may use exponential E notation, but cannot include non-numbers such as NaN.

    1. The format makes no distinction between integer and floating-point.
    2. JavaScript uses a double-precision floating-point format for all its numeric values, but other languages implementing JSON may encode numbers differently.
  2. String: a sequence of zero or more Unicode characters. Strings are delimited with double-quotation marks and support a backslash escaping syntax.

  3. Boolean: either of the values true or false

  4. Array: an ordered list of zero or more values, each of which may be of any type.

    1. Arrays use square bracket notation and elements are comma-separated.
  5. Object: an unordered collection of name–value pairs where the names (also called keys) are strings. Since objects are intended to represent associative arrays, it is recommended, though not required, that each key is unique within an object.

    1. Objects are delimited with curly brackets and use commas to separate each pair, while within each pair the colon ‘:’ character separates the key or name from its value.
  6. null: An empty value, using the word null

Here is a sample JSON payload from the paypal API:

{
    "id": "PAY-23S34684677003128LI5LXMQ",
    "intent": "sale",
    "state": "created",
    "payer": {
        "payment_method": "paypal"
    },
    "transactions": [
        {
            "amount": {
                "total": "30.11",
                "currency": "USD",
                "details": {
                    "subtotal": "30.00",
                    "tax": "0.07",
                    "shipping": "0.03",
                    "insurance": "0.01",
                    "handling_fee": "1.00",
                    "shipping_discount": "-1.00"
                }
            },
            "description": "The payment transaction description.",
            "custom": "EBAY_EMS_90048630024435",
            "invoice_number": "48787589673",
            "soft_descriptor": "ECHI5786786",
            "payment_options": {
                "allowed_payment_method": "INSTANT_FUNDING_SOURCE",
                "recurring_flag": false,
                "skip_fmf": false
            },
            "item_list": {
                "items": [
                    {
                        "name": "hat",
                        "sku": "1",
                        "description": "Brown hat.",
                        "price": "3.00",
                        "currency": "USD",
                        "tax": "0.01",
                        "quantity": 5
                    },
                    {
                        "name": "handbag",
                        "sku": "product34",
                        "description": "Black handbag.",
                        "price": "15.00",
                        "currency": "USD",
                        "tax": "0.02",
                        "quantity": 1
                    }
                ],
                "shipping_address": {
                    "recipient_name": "Brian Robinson",
                    "line1": "4th Floor",
                    "line2": "Unit #34",
                    "city": "San Jose",
                    "state": "CA",
                    "postal_code": "95131",
                    "country_code": "US",
                    "phone": "011862212345678"
                }
            },
            "related_resources": []
        }
    ],
    "note_to_payer": "Contact us for any questions on your order.",
    "create_time": "2017-12-20T19:36:18Z",
    "links": [
        {
            "href": "https://api.sandbox.paypal.com/v1/payments/payment/PAY-23S34684677003128LI5LXMQ",
            "rel": "self",
            "method": "GET"
        },
        {
            "href": "https://www.sandbox.paypal.com/cgi-bin/webscr?cmd=_express-checkout&token=EC-08C03207VR002793P",
            "rel": "approval_url",
            "method": "REDIRECT"
        },
        {
            "href": "https://api.sandbox.paypal.com/v1/payments/payment/PAY-23S34684677003128LI5LXMQ/execute",
            "rel": "execute",
            "method": "POST"
        }
    ]
}

XML Format

Extensible Markup Language (XML) is a markup language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable.

XML Terminology:

Character

An XML document is a string of characters. Almost every legal Unicode character may appear in an XML document.

Processor and application

The processor analyzes the markup and passes structured information to an application.

The specification places requirements on what an XML processor must do and not do, but the application is outside its scope.

Markup and content

The characters making up an XML document are divided into markup and content, which may be distinguished by the application of simple syntactic rules.

Generally, strings that constitute markup either begin with the character < and end with a >, or they begin with the character & and end with a ;.

Strings of characters that are not markup are content.

Tag

A tag is a markup construct that begins with < and ends with >.

Tags come in three forms in xml:

  1. start-tag: such as <section>

  2. end-tag: such as </section>

  3. empty-element tag: such as <line-break />

Element

An element is a logical document component that either begins with a start-tag and ends with a matching end-tag or consists only of an empty-element tag.

The characters between the start-tag and end-tag, if any, are the element’s content, and may contain markup, including other elements, which are called child elements.

An example is <message>Hello, world!</message>. Another is <line-break />

Attribute

An attribute is a markup construct consisting of a name–value pair that exists within a start-tag or empty-element tag.

An example is <img src="picture.jpg" alt="Mona Lisa" />

The names of the attributes are “src” and “alt”, and their values are “picture.jpg” and “Mona Lisa” respectively.

XML declaration

XML documents may begin with an XML declaration that describes some information about themselves.

An example is <?xml version="1.0" encoding="UTF-8"?>

####### Comments in XML

One distinct difference between xml and json is that xml can have comments

Douglas Crockford Statement on why comments don’t exist in JSON

Comments may appear anywhere in a document outside other markup.

Comments cannot appear before the XML declaration.

Comments begin with <!-- and end with -->.

Here is a sample XML Payload from the Walmart Labs Search API:

<searchresponse>
    <query>chromebook</query>
    <sort>relevance</sort>
    <responseGroup>base</responseGroup>
    <totalResults>303</totalResults>
    <start>1</start>
    <numItems>10</numItems>
    <facets/>
    <items>
        <item>
            <itemId>48172060</itemId>
            <parentItemId>48172060</parentItemId>
            <name>HP Chromebook 14 G4 - 14" - Celeron N2840 - 4 GB RAM - 16 GB SSD</name>
            <msrp>234.0</msrp>
            <salePrice>242.88</salePrice>
            <upc>889894582294</upc>
            <categoryPath>Electronics/Computers/Laptops/Shop Laptops by Type/Google Chromebook</categoryPath>
            <shortDescription>Make the move to a cloud-based infrastructure with the HP Chromebook 14. Get down to business with Intel Celeron processors Citrix-certified VDI compatibility and access to Google Apps for Work.</shortDescription>
            <longDescription>HP Chromebook 14 G4 - 14&amp;quot; - Celeron N2840 - 4 GB RAM - 16 GB SSD</longDescription>
            <thumbnailImage>https://i5.walmartimages.com/asr/448176d1-516c-4362-a215-71cba7a07d44_1.a01c748d62cbd8fa07148eea7063ae7d.jpeg?odnHeight=100&amp;odnWidth=100&amp;odnBg=FFFFFF</thumbnailImage>
            <mediumImage>https://i5.walmartimages.com/asr/448176d1-516c-4362-a215-71cba7a07d44_1.a01c748d62cbd8fa07148eea7063ae7d.jpeg?odnHeight=180&amp;odnWidth=180&amp;odnBg=FFFFFF</mediumImage>
            <largeImage>https://i5.walmartimages.com/asr/448176d1-516c-4362-a215-71cba7a07d44_1.a01c748d62cbd8fa07148eea7063ae7d.jpeg?odnHeight=450&amp;odnWidth=450&amp;odnBg=FFFFFF</largeImage>
            <productTrackingUrl>http://linksynergy.walmart.com/fs-bin/click?id=|LSNID|&amp;offerid=223073.7200&amp;type=14&amp;catid=8&amp;subid=0&amp;hid=7200&amp;tmpid=1082&amp;RD_PARM1=https%253A%252F%252Fwww.walmart.com%252Fip%252FHP-Chromebook-14-G4-14-Celeron-N2840-4-GB-RAM-16-GB-SSD%252F48172060%253Faffp1%253DjNVX3nQxlVchQvsmT4mYZBb50I_klgLjVTSUfdTaI18%2526affilsrc%253Dapi</productTrackingUrl>
            <standardShipRate>0.0</standardShipRate>
            <marketplace>false</marketplace>
            <modelNumber>T4M32UT#ABA</modelNumber>
            <productUrl>http://c.affil.walmart.com/t/api02?l=https%3A%2F%2Fwww.walmart.com%2Fip%2FHP-Chromebook-14-G4-14-Celeron-N2840-4-GB-RAM-16-GB-SSD%2F48172060%3Faffp1%3DjNVX3nQxlVchQvsmT4mYZBb50I_klgLjVTSUfdTaI18%26affilsrc%3Dapi%26veh%3Daff%26wmlspartner%3Dreadonlyapi</productUrl>
            <customerRating>4.444</customerRating>
            <numReviews>9</numReviews>
            <customerRatingImage>http://i2.walmartimages.com/i/CustRating/4_4.gif</customerRatingImage>
            <categoryNode>3944_3951_1089430</categoryNode>
            <bundle>false</bundle>
            <stock>Available</stock>
            <addToCartUrl>http://c.affil.walmart.com/t/api02?l=http%3A%2F%2Faffil.walmart.com%2Fcart%2FaddToCart%3Fitems%3D48172060%7C1%26affp1%3DjNVX3nQxlVchQvsmT4mYZBb50I_klgLjVTSUfdTaI18%26affilsrc%3Dapi%26veh%3Daff%26wmlspartner%3Dreadonlyapi</addToCartUrl>
            <affiliateAddToCartUrl>http://linksynergy.walmart.com/fs-bin/click?id=|LSNID|&amp;offerid=223073.7200&amp;type=14&amp;catid=8&amp;subid=0&amp;hid=7200&amp;tmpid=1082&amp;RD_PARM1=http%253A%252F%252Faffil.walmart.com%252Fcart%252FaddToCart%253Fitems%253D48172060%257C1%2526affp1%253DjNVX3nQxlVchQvsmT4mYZBb50I_klgLjVTSUfdTaI18%2526affilsrc%253Dapi</affiliateAddToCartUrl>
            <giftOptions>
                <allowGiftWrap>false</allowGiftWrap>
                <allowGiftMessage>false</allowGiftMessage>
                <allowGiftReceipt>false</allowGiftReceipt>
            </giftOptions>
            <imageEntities>
                <imageEntities>
                    <thumbnailImage>https://i5.walmartimages.com/asr/5a378a80-82e1-4f06-9e71-6219e03ef568_1.be37a5f6ee49fc2777a13ef41214151c.jpeg?odnHeight=100&amp;odnWidth=100&amp;odnBg=FFFFFF</thumbnailImage>
                    <mediumImage>https://i5.walmartimages.com/asr/5a378a80-82e1-4f06-9e71-6219e03ef568_1.be37a5f6ee49fc2777a13ef41214151c.jpeg?odnHeight=180&amp;odnWidth=180&amp;odnBg=FFFFFF</mediumImage>
                    <largeImage>https://i5.walmartimages.com/asr/5a378a80-82e1-4f06-9e71-6219e03ef568_1.be37a5f6ee49fc2777a13ef41214151c.jpeg?odnHeight=450&amp;odnWidth=450&amp;odnBg=FFFFFF</largeImage>
                    <entityType>SECONDARY</entityType>
                </imageEntities>
                <imageEntities>
                    <thumbnailImage>https://i5.walmartimages.com/asr/cf9351fd-5857-4f58-86ca-77b4c644798e_1.d56cc64ab726ad0f9d16e96237d0fbf8.jpeg?odnHeight=100&amp;odnWidth=100&amp;odnBg=FFFFFF</thumbnailImage>
                    <mediumImage>https://i5.walmartimages.com/asr/cf9351fd-5857-4f58-86ca-77b4c644798e_1.d56cc64ab726ad0f9d16e96237d0fbf8.jpeg?odnHeight=180&amp;odnWidth=180&amp;odnBg=FFFFFF</mediumImage>
                    <largeImage>https://i5.walmartimages.com/asr/cf9351fd-5857-4f58-86ca-77b4c644798e_1.d56cc64ab726ad0f9d16e96237d0fbf8.jpeg?odnHeight=450&amp;odnWidth=450&amp;odnBg=FFFFFF</largeImage>
                    <entityType>SECONDARY</entityType>
                </imageEntities>
                <imageEntities>
                    <thumbnailImage>https://i5.walmartimages.com/asr/d923862d-49c9-4ec1-b55d-863ec74a82e7_1.a8bed47384d4f9a5d5cd30a731fdea95.jpeg?odnHeight=100&amp;odnWidth=100&amp;odnBg=FFFFFF</thumbnailImage>
                    <mediumImage>https://i5.walmartimages.com/asr/d923862d-49c9-4ec1-b55d-863ec74a82e7_1.a8bed47384d4f9a5d5cd30a731fdea95.jpeg?odnHeight=180&amp;odnWidth=180&amp;odnBg=FFFFFF</mediumImage>
                    <largeImage>https://i5.walmartimages.com/asr/d923862d-49c9-4ec1-b55d-863ec74a82e7_1.a8bed47384d4f9a5d5cd30a731fdea95.jpeg?odnHeight=450&amp;odnWidth=450&amp;odnBg=FFFFFF</largeImage>
                    <entityType>SECONDARY</entityType>
                </imageEntities>
                <imageEntities>
                    <thumbnailImage>https://i5.walmartimages.com/asr/448176d1-516c-4362-a215-71cba7a07d44_1.a01c748d62cbd8fa07148eea7063ae7d.jpeg?odnHeight=100&amp;odnWidth=100&amp;odnBg=FFFFFF</thumbnailImage>
                    <mediumImage>https://i5.walmartimages.com/asr/448176d1-516c-4362-a215-71cba7a07d44_1.a01c748d62cbd8fa07148eea7063ae7d.jpeg?odnHeight=180&amp;odnWidth=180&amp;odnBg=FFFFFF</mediumImage>
                    <largeImage>https://i5.walmartimages.com/asr/448176d1-516c-4362-a215-71cba7a07d44_1.a01c748d62cbd8fa07148eea7063ae7d.jpeg?odnHeight=450&amp;odnWidth=450&amp;odnBg=FFFFFF</largeImage>
                    <entityType>PRIMARY</entityType>
                </imageEntities>
            </imageEntities>
            <offerType>ONLINE_ONLY</offerType>
            <isTwoDayShippingEligible>true</isTwoDayShippingEligible>
            <availableOnline>true</availableOnline>
        </item>
        ...
    </items>
</searchresponse>

HTTP Verbs

The HTTP 1.1 Specification has 9 Request Methods:

  1. GET

    1. The GET method requests a representation of the specified resource.
    2. Requests using GET should only retrieve data and should have no other effect.
    3. The W3C has published guidance principles on this distinction, saying, “Web application design should be informed by the above principles, but also by the relevant limitations.
  2. HEAD

    1. The HEAD method asks for a response identical to that of a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.
  3. POST

    1. The POST method requests that the server accept the entity enclosed in the request as a new subordinate of the web resource identified by the URI.
    2. The data POSTed might be, for example, an annotation for existing resources; a message for a bulletin board, newsgroup, mailing list, or comment thread; a block of data that is the result of submitting a web form to a data-handling process; or an item to add to a database.
  4. PUT

    1. The PUT method requests that the enclosed entity be stored under the supplied URI. If the URI refers to an already existing resource, it is modified; if the URI does not point to an existing resource, then the server can create the resource with that URI.
  5. DELETE

    1. The DELETE method deletes the specified resource.
  6. TRACE

    1. The TRACE method echoes the received request so that a client can see what (if any) changes or additions have been made by intermediate servers.
  7. OPTIONS

    1. The OPTIONS method returns the HTTP methods that the server supports for the specified URL. This can be used to check the functionality of a web server by requesting ‘*’ instead of a specific resource.
  8. CONNECT

    1. The CONNECT method converts the request connection to a transparent TCP/IP tunnel, usually to facilitate SSL-encrypted communication (HTTPS) through an unencrypted HTTP proxy.
  9. PATCH

    1. The PATCH method applies partial modifications to a resource.

Most modern Web APIs only use 4 out the 9 HTTP verbs when implementing Restful APIs

  1. GET
  2. POST
  3. PUT
  4. DELETE

URL Design

Keep your base URL simple and intuitive

Nouns are good; verbs are bad

The base URL is the most important design affordance of your API.

A simple and intuitive base URL design makes using your API easy.

A URL identifies a resource.

For consistency, only use plural nouns (e.g. “books” instead of “book”).

Restful URLs

Here is a list of Good Urls for a restful API:

  1. List of Books:

    1. GET https://api.sample.com/v1/books
  2. List of Books using query:

    1. GET https://api.sample.com/v1/books?name=Batman
  3. Create an access token

    1. POST https://api.sandbox.paypal.com/v1/oauth2/token

Notice that in all of these apis there is a major version number depicted by v1.

Also notice that nouns are used instead of verbs.

Testing an API

For the purposes of this blog post we will use curl and Postman to make Rest calls

Let use use the Walmart Search API:

Curl Usage
curl -X GET \
  'http://api.walmartlabs.com/v1/search?query=chromebook&format=xml&apiKey=someApiKey1234av' \
  -H 'Content-Type: application/xml'

First create an account with Walmart Labs Open API

You can go to Mykeys at Walmart Labs Developer Portal to get your api key from the developer portal

The above curl GET request returns an xml response payload as we saw above

With curl you specify the HTTP verb by using the option -X, --request, {GET,PUT,POST,DELETE

To pass headers into a curl command you pass in the -H, --header, {Content-Type: application/xml, etc.}

In order to post data with curl you use the -d, --data options

Here is a sample POST request with curl

curl -X POST \
  https://localhost:3000/api/v1/users \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "user": {
    "email": "[email protected]",
    "first_name": "Big",
    "gender": "Male",
    "id": 57,
    "last_name": "Kahuna"  
  }
}'

Postman Usage

Here is a screenshot of the same rest call using Postman

Notice that in the screenshot both URL and API_key are wrapped in {{}} double curly braces

In Postman there are several different scopes for variables:

Variables in Postman

The following scopes are available to you:

  1. Global

  2. Collection

  3. Environment

  4. Local

  5. Data

Some advantages of using Postman over curl is that you can use Pre request scripts and post request scripts to automate API development testing and for scripting

If we wanted to make a payment to the paypal API we first need to get an access token from the POST /v1/oauth2/token endpoint

after that we can make the rest call to the POST /v1/payments/payment endpoint.

Here we need to make 2 calls in order to copy first the access token from the token endpoint and use that as our bearer token for the payment endpoint.

I will show a series of snapshots where I utilize Postman’s Pre request script and Tests section to automate this flow.

In order to make the rest call for POST /v1/oauth2/token endpoint you need your client id and secret from the paypal developer portal which you can find by going Here and clicking the Log into Dashboard button

If you were to make this call with curl you would have to Base 64 Encode your clientId and secret.

With JavaScript btoa method you can do this like this

btoa("clientId1234" + ":" + "asecret8888")

You can also use Node.js to base 64 encode a string like this:

const encodedVal = new Buffer("clientId1234" + ":" + "asecret8888").toString("base64");

Both of these methods return the following base 64 encoded client id and secret: Y2xpZW50SWQxMjM0OmFzZWNyZXQ4ODg4

Now using curl we can make the following POST request

curl -X POST \
  https://api.sandbox.paypal.com/v1/oauth2/token \
  -H 'Authorization: Basic Y2xpZW50SWQxMjM0OmFzZWNyZXQ4ODg4' \
  -H 'Content-Type: application/x-www-form-urlencoded'
  -d grant_type=client_credentials

and then using the access token we get from the JSON payload from the above call:


{
    "scope": "https://uri.paypal.com/services/subscriptions https://api.paypal.com/v1/payments/.* https://api.paypal.com/v1/vault/credit-card https://uri.paypal.com/services/applications/webhooks openid https://uri.paypal.com/payments/payouts https://api.paypal.com/v1/vault/credit-card/.*",
    "nonce": "2017-12-20T19:26:8888onelongguid",
    "access_token": "zdd12344alongGuid",
    "token_type": "Bearer",
    "app_id": "APP-553322344T",
    "expires_in": 31275
}

Now we need to copy the access_token and we can make the following rest call

curl -X POST \
  https://api.sandbox.paypal.com/v1/payments/payment \
  -H 'Authorization: Bearer zdd12344alongGuid' \
  -H 'Content-Type: application/json' \
  -d '{
  "intent": "sale",
  "payer": {
  "payment_method": "paypal"
  },
  "transactions": [
  {
    "amount": {
    "total": "30.11",
    "currency": "USD",
    "details": {
      "subtotal": "30.00",
      "tax": "0.07",
      "shipping": "0.03",
      "handling_fee": "1.00",
      "shipping_discount": "-1.00",
      "insurance": "0.01"
    }
    },
    "description": "The payment transaction description.",
    "custom": "EBAY_EMS_90048630024435",
    "invoice_number": "48787589673",
    "payment_options": {
    "allowed_payment_method": "INSTANT_FUNDING_SOURCE"
    },
    "soft_descriptor": "ECHI5786786",
    "item_list": {
    "items": [
      {
      "name": "hat",
      "description": "Brown hat.",
      "quantity": "5",
      "price": "3",
      "tax": "0.01",
      "sku": "1",
      "currency": "USD"
      },
      {
      "name": "handbag",
      "description": "Black handbag.",
      "quantity": "1",
      "price": "15",
      "tax": "0.02",
      "sku": "product34",
      "currency": "USD"
      }
    ],
    "shipping_address": {
      "recipient_name": "Brian Robinson",
      "line1": "4th Floor",
      "line2": "Unit #34",
      "city": "San Jose",
      "country_code": "US",
      "postal_code": "95131",
      "phone": "011862212345678",
      "state": "CA"
    }
    }
  }
  ],
  "note_to_payer": "Contact us for any questions on your order.",
  "redirect_urls": {
  "return_url": "https://www.example.com/return",
  "cancel_url": "https://www.example.com/cancel"
  }
}'

to post a payment to paypal.

Automation with Postman

Using Postman we can automate the copy and pasting of the Basic Auth using Pre-request script part

Here is the Pre-request script block:

require('btoa');

var client_id = postman.getEnvironmentVariable("client_id");
var client_secret = postman.getEnvironmentVariable("client_secret");

// Base 64 Encode client id and secret
var encodeCredentials = btoa(client_id + ":" + client_secret);

postman.setEnvironmentVariable("encodeCredentials", encodeCredentials);

Here is a screenshot of the script:

Notice here that we saved client_id and client_secret into environment variables and require('btoa') into the script

You can read more about the Pre-request script in Postman

To get a full listing of the Postman Sandbox API look Here

One thing to note is that to get the response payload we must use the Test section in Postman, the Pre-request script will not have access to the response body

Here is the Test script to set access_token:

// Parse JSON response
var responseBody = JSON.parse(responseBody);

var access_token = responseBody.access_token;
var token_type = responseBody.token_type;

postman.setEnvironmentVariable("access_token", token_type + " " + access_token);

pm.test("Token Type is Bearer", function () {
    var jsonData = pm.response.json();
    pm.expect(jsonData.token_type).to.eql("Bearer");
});

Here is a screenshot of the Test script section:

You can read more about the Test Script Section in Postman

We can completely automate the POST https://api.sandbox.paypal.com/v1/payments/payment endpoint with environment variables set in test script

Now the environment variable access_token has been set in Postman

We can write the following Test Script now:

var responseBody = JSON.parse(responseBody);

pm.test("Status code is 201", function () {
    pm.response.to.have.status(201);
});

pm.test("State is created", function () {
    var jsonData = pm.response.json();
    pm.expect(jsonData.state).to.eql("created");
});

Notice that Postman has its own flavor of testing using pm.test method.

When you click the Send button you will see the following in Test results:

If you like this Blog post please follow me at jbelmont at Github

comments powered by Disqus