API Gateway Backed By SQS Featuring CORS

I show how you can return dynamic headers based on the request origin to satisfy CORS when API gateway is setup with an AWS service instead of a proxy lambda

API Gateway Backed By SQS Featuring CORS

I'm over a project that has the potential of handling high volume traffic. Right now the company is just getting started so not a ton of traffic is coming through but we currently handle around 1.7M requests per week. This could be potentially much much higher and API gateway may not be the platform we want to stay on. But for now, this works for our clients starting out.

Originally our infrastructure was pretty standard. You have REST API Gateway endpoints pointing to a proxied lambda. The lambda would process some data, save it somewhere and spit out a response. Saving data isn't a huge deal until you start needing to query that data. There are several ways you can balance out the load for this kind of scenario (like an Aurora instance), however, we wanted to solve one problem first. When you start to have millions of rows in a table and you search an indexed column for something. If your box isn't very big it can lock up the requests until it finds just a few matching records. During that lockup requests coming into API gateway start failing because the lambda can't insert records.

What I wanted to do is make all requests coming into our API be durable. I wanted the requests to be accepted and saved for eventual processing. It is acceptable for our data to be delayed by a few seconds before being readily available. So the way I accomplished this was by backing the API gateway with SQS instead of lambda.

This means while the database is churning out results from a query, failing insert requests get backed up on SQS until the box comes back up. If the box never comes back up, we have a dead letter queue waiting for failed records that have been pushed back onto the main queue too many times. This way, we never lose - or are less likely to lose - the data even if something terrible happens to the DB. Our API also never sees failing requests because (hopefully) SQS is always ready to take on new events.

This really only works with an API that is only consuming information and does not need to return any data back to the requesting client. One other problem I ran into was CORS. CORS being a rather simple constraint can be one of the most frustrating settings to try to get right. Especially from proxies. Something I wasn't sure was possible is having the request origin be forwarded back to the client in API Gateway. All the guides I read were less than helpful as they showed how to get the origin from the backing service - typically lambda - which isn't what I needed.

I was using a serverless plugin serverless-api-gateway-service-proxy to help setup the CloudFormation template requirements, but it seems to not setup the CORS headers correctly when you have multiple allowed origins. I was seeing it set the options headers correctly but it wasn't setting up the response headers for the PUT/POST requests. Since I didn't have lambda anymore to handle those and there not being a way for SQS to be configured to return headers when something is inserted into the queue, I needed API gateway to return those instead.

So, if you need CORS headers set from a service-based API Gateway configuration, you will need to configure it with this:

custom:
  apiGatewayServiceProxies:
      - sqs:
        path: /my/path
        method: put
        queueName: { 'Fn::GetAtt': ['myQueue', 'QueueName'] }
        cors:
          origins:
            - https://*.test.com
            - https://*.other.test
            - https://test.com
          allowCredentials: true
        response:
          - statusCode: 200
            responseTemplates:
              application/json: |-
                #set($origin = $input.params("Origin"))
                #if($origin == "")
                  #set($origin = $input.params("origin"))
                #end
                #set($context.responseOverride.header.Access-Control-Allow-Origin = $origin)
                #set($context.responseOverride.header.Access-Control-Allow-Credentials = 'true')

resources:
  Resources:
    myQueue:
      Type: 'AWS::SQS::Queue'
      Properties:
        QueueName: "my-queue"
        RedrivePolicy:
          deadLetterTargetArn:
            'Fn::GetAtt':
              - updateSessionDLQ
              - Arn
          maxReceiveCount: 5
    myQueueDLQ:
      Type: 'AWS::SQS::Queue'
      Properties:
        QueueName: "my-queue-dlq"

You may want to add additional status codes in the case that something goes down, but for me, it doesn't really matter. We've lost the data at that point if API gateway is returning 400 or 500's.

Hopefully, this helps someone. I figured this stuff out without much help from AWS support and hours of searching. So if I saved you a few hours of searching, please send this out to others.