Dedicated to Providing Insight Into the Enablement of Cloud Hosting Environments.

Cloud Hosting Journal

Subscribe to Cloud Hosting Journal: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get Cloud Hosting Journal: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Cloud Hosting Authors: Vaibhaw Pandey, Matt Brickey, Harry Trott, Yeshim Deniz, Liz McMillan

Blog Feed Post

Monitor and Analyze Performance on AWS

Our goal at Catchpoint has always been to simplify things, make it more simple and scalable for our end users. In our constant endeavor to achieve scalability at lower costs, we introduced Webhooks. Webhooks are HTTP callbacks that are triggered by events specified in the code by the application developer; the event data is then recorded and pushed to an end-point URL. Webhooks are a powerful way of exchanging data.

 Let’s take an example:
Imagine, you go to your neighborhood grocery store to pick up mangoes, but unfortunately today the shop is out of the mangoes that you want. The delivery boy who is supposed to bring the mangoes to the store, is stuck in traffic. So, the shop owner asks you to come back in an hour. You go back home and return to the shop after an hour, but the delivery boy has not yet arrived with the mangoes. The story repeats and after multiple such attempts you finally get your beloved mangoes. This is exactly how API’s work. You keep making requests, you keep wasting resources, but you won’t get data until it is available.

http://blog.catchpoint.com/wp-content/uploads/2017/07/web1-300x48.png 300w" sizes="(max-width: 702px) 100vw, 702px" />

Now imagine, instead of you going to the shop, you leave your address with the shopkeeper and ask him to deliver mangoes as soon as they arrive, this will help you save your time, effort and resources. This is exactly how Webhooks work.

http://blog.catchpoint.com/wp-content/uploads/2017/07/web2-300x48.png 300w" sizes="(max-width: 698px) 100vw, 698px" />

One of the advantages of Webhooks is, since there’s no request required for a Webhook, it sends the data to the endpoint as soon as it’s available and hence the data is made available in near real time

In this blog, we explore the idea of using Webhooks to analyze web performance data from Catchpoint in your Amazon Elastic Search service.

Step 1: Setting up the AWS instance:

Following are the steps to setup your own AWS instance.

1: Log in to your AWS Account.

2: Go to Services and select Elastic Search Service under the Analytics section.

3: Create a Domain and select the elastic search version and click next.

http://blog.catchpoint.com/wp-content/uploads/2017/07/web3-300x133.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web3-768x340.png 768w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web3-1024x454.png 1024w" sizes="(max-width: 1167px) 100vw, 1167px" />

4: Configure the cluster as per your requirements and click next

http://blog.catchpoint.com/wp-content/uploads/2017/07/web4-300x143.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web4-768x367.png 768w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web4-1024x489.png 1024w" sizes="(max-width: 1206px) 100vw, 1206px" />

http://blog.catchpoint.com/wp-content/uploads/2017/07/web5-300x150.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web5-768x384.png 768w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web5-1024x512.png 1024w" sizes="(max-width: 1247px) 100vw, 1247px" />

5: Setup access policy, make sure you allow catchpoint Webhook domain/ip so that it can post data to elastic and then click next.

http://blog.catchpoint.com/wp-content/uploads/2017/07/web6-300x152.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web6-768x389.png 768w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web6-1024x519.png 1024w" sizes="(max-width: 1234px) 100vw, 1234px" />

http://blog.catchpoint.com/wp-content/uploads/2017/07/web7-300x152.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web7-768x388.png 768w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web7-1024x518.png 1024w" sizes="(max-width: 1236px) 100vw, 1236px" />

6: Confirm your changes and create the AWS ES instance. Wait for the Domain Status to change to Active, that’s when the service is ready to ingest data. Here endpoint URL and Kibana URL are important.

http://blog.catchpoint.com/wp-content/uploads/2017/07/web8-300x146.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web8-768x374.png 768w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web8-1024x498.png 1024w" sizes="(max-width: 1256px) 100vw, 1256px" />

7: Create an index to push data to AWS ES.

For example: The following curl request will create an index called cpindex

curl -X PUT http://search-catchpoint-test-imwp2rphrbhue6k2gizw6c5afm.ap-south.es.amazonaws.com/cpindex  -H 'cache-control: no-cache' -H 'content-length: 225' -H 'content-type: application/json'

8: While sending data don’t forget to add a type to this index.

So, the new post URL will look like, here data is the AWS ES type.:

http://search-catchpoint-test-imwp2rphrbhue6k2gizw6c5afm.ap-south.es.amazonaws.com/cpindex /data

The following lines are taken from Elastic search official documentation:

For example, let’s assume you run a blogging platform and store all your data in a single index. In this index, you may define a type for user data, another type for blog data, and yet another type for comments data

Step 2:  Setting up the Webhook to post data to AWS ES instance endpoint.

1:  Select a test for which you want to enable Webhook. Go to advanced options in the test setup page and enable test data Webhook.

http://blog.catchpoint.com/wp-content/uploads/2017/07/web9-300x192.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web9-768x491.png 768w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web9-750x480.png 750w" sizes="(max-width: 984px) 100vw, 984px" />

2: In the Webhook Settings, copy the AWS ES endpoint URL and save

http://blog.catchpoint.com/wp-content/uploads/2017/07/web10-300x142.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web10-768x363.png 768w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web10-1024x484.png 1024w" sizes="(max-width: 1114px) 100vw, 1114px" />

Step 3: Verifying the setup

Once the test settings have been saved and a test run executed, you can view the index in AWS ES indices tab.

You should be able to view the results in Kibana.

http://blog.catchpoint.com/wp-content/uploads/2017/07/web11-300x144.png 300w, http://blog.catchpoint.com/wp-content/uploads/2017/07/web11-768x368.png 768w" sizes="(max-width: 909px) 100vw, 909px" />

That’s how you can leverage the power of Webhooks for your Performance analysis.

See how Google uses Catchpoint’s Webhooks to access their performance data in real time.

The post Monitor and Analyze Performance on AWS appeared first on Catchpoint's Blog - Web Performance Monitoring.

Read the original blog entry...

More Stories By Mehdi Daoudi

Catchpoint radically transforms the way businesses manage, monitor, and test the performance of online applications. Truly understand and improve user experience with clear visibility into complex, distributed online systems.

Founded in 2008 by four DoubleClick / Google executives with a passion for speed, reliability and overall better online experiences, Catchpoint has now become the most innovative provider of web performance testing and monitoring solutions. We are a team with expertise in designing, building, operating, scaling and monitoring highly transactional Internet services used by thousands of companies and impacting the experience of millions of users. Catchpoint is funded by top-tier venture capital firm, Battery Ventures, which has invested in category leaders such as Akamai, Omniture (Adobe Systems), Optimizely, Tealium, BazaarVoice, Marketo and many more.