Posted on by JesterXL

AWS Adventures: Part 1 – Lambda

Introduction

At the beginning of October, I started a new project to help my team move a 12 year old application to the cloud. I knew next to nothing about Amazon, or their infrastructure offerings called AWS. In the past 3 months I’ve learned a ton. Today I wanted to share what I’ve learned about Lambdas: Functions you run in AWS.

Also known as serverless architecture.

What is AWS?

AWS is short for Amazon Web Services. Amazon provides you a bunch of different services that help you build software, almost all of which they host.

What Are Lambdas?

Lambdas are one of their offerings that have been out over a year. The selling points of Lambdas are Amazon sets up and maintains the server for you, and you’re only charged for how long your code actually runs. You upload code and it just works.

Why Not Use an EC2 Instead?

An EC2 is Amazon’s virtual machine servers. You have to build and maintain them yourself. You pay to keep them running even when they aren’t handling any traffic. You have to rebuild them when new AMI’s (virtual machine templates) come out from Amazon. Things get really challenging when you scale dynamically for spikes in traffic.

Why Not Docker?

While Docker makes some of the above less painful, especially when testing in Jenkins or testing on other developers machines, you still have all the same issues. You build the docker image and manage them through a container service like DockerSwarm or Consul. The images themselves have to be managed by someone in DockYard.

Why Lambdas

I’m A Front-End Developer

If I want to run my JavaScript in the cloud, I don’t care about the OS, what’s installed on it, and all the details about keeping it running and refreshed with security updates. Amazon does and handles all that for you.

Lamba’s are just code, you don’t have to maintain the server or infrastructure, you only pay for what you use, and they automatically scale based on traffic.

If you’re a front-end developer like me who just wants to write code, and you’re keen on building a back-end API for the front end, they’re perfect.

Lambdas Are Reactive and Event Driven

One additional feature of Lambdas is their built in events. They are “reactive” by default. It’s assumed Lambdas are triggered by other AWS things. Things in this case being API Gateway for REST API’s such as GET and POST, S3 when you drop files on it, SNS for a basic message bus, or even JSON written into logs through CloudWatch. There also are the building blocks for distributed architecture in Step Functions.

Simpler Continuous Deployment

Trying to get monoliths, big applications where all the code is bundled together, deployed to production is hard. Getting this done often is even harder. One of the benefits microservices give you is easier deployment. You can deploy “one thing” without having to worry about the “other things”.

Anyone who’s had back-end developers deploy new API’s, or you deploying new front end code knows that’s a loaded statement. Unless you’ve done some integration and functional testing or you’ve deployed the code to a QA or staging environment which mimics production, you don’t “know” the code works.

If you define clear API’s upfront, keep the microservices truly simple in their functionality, this process becomes much simpler, and eases CI down the line.

Are Lambdas Microservices?

They can be microservices, but no, not by default. They are just a function. Microservices typically are accessed by a REST API. If you trigger them by an API Gateway (a restful URL provided by Amazon), then yes, they become microservices.

Build A Microservice

If you want to build a RESTful microservice with Lambda’s, you first have to have an AWS account. Assuming you have that, login and access the console (the AWS web interface).

Step 1: Build A HelloWorld Lambda

Click the “Services” button/menu, and select/search for “Lambda”, then click “Create a Lambda Function”.

In Runtime select “Node”; version doesn’t matter, just ensure not Edge for now. For template, click on “Blank Function”.

Step 2: Choose A Trigger

Click the empty box and choose “API Gateway”. Make up an API Name, choose “api” for deployment stage, and choose “Open” for security. Click “Next”.

Step 3: Configure

Pick a camel case (ex: likeThisMan) name. Scroll down to Role, and choose “Create a new role from template(s)”. Create a basic camel case role name such as “myCustomTemplateRole”. Scroll to the bottom and click “Next”, then on the next screen scroll to the bottom and click “Create function”.

You’re done!

Note it usually dumps you on the Trigger tab (what causes your Lambda function to run). In our case, it should show the URL for our API Gateway. Copy this to your clipboard or a notepad/textedit for later use.

It generates a basic function for you. You can see it by clicking the Code tab:

exports.handler = (event, context, callback) => {
    callback(null, 'Hello from Lambda');
};

 

Test & Iterate On Your Microservice

Step 4: Test

Click the blue “Test” button. In the new popup, the sample event template should be “Hello World”. Click “Save and Test”, and you should see “Hello from Lambda” below the execution result. Great, your Lambda function runs.

Step 5: Test Your API Gateway

If you copy and paste your API Gateway URL in the browser, it should say “internal server error”. This is because that Lambda creation wizard you ran created a new REST API for you, but returns a JavaScript string. By default, you need to use JSON. Let’s modify your Lambda code to do that.

exports.handler = (event, context, callback) => {
    
    const response = {
        statusCode: '200',
        body: JSON.stringify('Hello from Lambda'),
        headers: {
            'Content-Type': 'application/json',
        },
    }
    callback(null, response);
};

Now re-open the API Gateway URL in your browser, and you should see your “Hello from Lambda” message.

Debug Your Microservice

Step 6: Custom Logging

The event and context parameters allow you to pass dynamic information to your Lambda, and let you know how your Lambda was invoked. Add 2 log statements to show the event and context variables:

exports.handler = (event, context, callback) => {
    console.log("event:", event);
    console.log("context:", context);

Now click “Save and Test”. The log output at the bottom right should show your 2 log messages. The event is probably the custom Hello World default test data of key1, value1, etc.

Think of “event” as parameters your Lambda can choose to parse and use to dynamically act on. The context gives your Lambda information about itself while running (log group name, how long your lambda function has to run, etc). Let’s see how the event can differ.

Step 7: CloudWatch Logs

Click the “Monitoring” tab, and to the far right you should see “View logs in CloudWatch”. Click it.

Hopefully you should see at least 1 log, but probably more.

They are sorted by most recent, so click on the top one. These should match what’s in your log output when you tested it.

Click the back button, select all the log streams by clicking the box to their left, and click “Delete Log Stream” then “Yes Delete”.

Reopen the API Gateway URL, then go back to your CloudWatch logs and click the little refresh button (the spinning circle arrows) on the top right. It should have just 1 log which you just caused by re-running your API Gateway URL. Click it, and you’ll notice the data is much different.

CloudWatch is pretty good at parsing arbitrary strings and JSON. Click the event area in the list, and it should expand it to show you the full JSON. Notice API Gateway sends us a lot of parameters.

What Languages Can I Use?

You can write your Lambdas in JavaScript, Python, Java, and C#. You can also use your normal libraries. The tutorial above edits the code inline in the AWS console, but a future tutorial shows you how you can automate your code.

What Else Can Lambdas Do?

Amazon’s documentation gives you a list of events that can trigger your lambda. However, some real world use cases I’ve used are:

REST API’s

API Gateway for basic REST Services instead of using Node + Restify + Docker + DockerSwarm (You can still use Node + Restify, heh)

Backend For Front End

Database guy had 1 Lambda for his DynamoDB access. My Lambda was an API for my front-end, and took his JSON and formatted it the way I needed it (my microservice invoked his microservice).

Unexpected Spikes In Traffic == No Worries

Trying to sell a client on how Lambda’s scaled better, and were cheaper than a dozen WebSphere instances in a row. We wired up Gatling to blast it with hundreds of thousands of users over 2 minutes. Amazon was like, “That’ll be $10 please.” lol! #awesome (This as opposed to the Terminator 2 nuke scene that typically resulted in a spike of traffic with the servers melting).

Content Mangement Updating Servers

Dropping updated content from a Content Management System on S3 (a hard drive in the cloud). Then, updating content on some EC2 instances behind an ELB. Using the SDK to describe instances (what servers do I have, and of those, are they running or not), you can then snag their IP addresses. Our EC2’s were in a different security group, so we put a lambda in the same group, gave it a URL from API Gateway, then had Lambda A call Lambda B with the IP addresses to call & updated content to give them.

Conclusions

As you can see, Lambdas provide a wonderful way to build microservices for both your application, and your infrastructure. You can start with a monolith first and make your Lambda’s 1 function turn into a ton, or build a bunch of little ones that may or may not know about each other, but all have a well defined API contract.

Not worrying about servers has freed up me and others to no longer worry about infrastructure, and instead write code.