Sign in Subscribe
AWS

Delivering dynamic non-HTTPS content over HTTPS with the help of AWS

Ivan Sieder

• 3 min read
Delivering dynamic non-HTTPS content over HTTPS with the help of AWS

Having a HTTPS-encrypted website is becoming more and more important. In fact, by now it should already be the standard and we should not even have to talk about this topic anymore.

As we started migrating our customers' websites to HTTPS, we faced the problem, that webcams most of the time weren't hosted on HTTPS-enabled domains and/or their providers had no plans to migrate them to HTTPS anytime soon.

But our customers of course wanted to have them on their websites. So we had to work out a solution, which could be realized fast and doesn't cause any maintenance costs. There are of course a few different ways to do this. But I'll explain the one we found the most useful in our case.

TL;DR

We have an AWS Lambda function, which fetches images from URLs stored inside an array and saves them to a S3 Bucket. This Lambda function is triggered every 10 minutes by a CloudWatch Event. To allow us to serve the images from a custom domain (with HTTPS-support), we set up CloudFront with TTL set to 0 (more on this below).

AWS Lambda

Our AWS Lambda function has the sole purpose of fetching all images from a URL and store them to an S3 bucket with a provided file name (to avoid duplicate names). For this purpose, we use Node.js' internal http.get() function.

To save the image from Node.js to an S3 bucket, the best and most stable way was to use the AWS SDK (https://aws.amazon.com/documentation/sdk-for-javascript/)

Additionally to just fetching the image, we also do some pre-checks, if the URL is being redirected and follow those redirects to make sure we actually fetch the real image instead of the redirection page.

AWS CloudWatch

AWS CloudWatch Event rules allow us to trigger a Lambda function every 10 minutes, so we regularly get fresh images from the origin servers.

S3 Bucket

AWS S3 (Simple Storage Service) provides us with persistent and replicated storage for all our webcam images and simplifies the process of adding HTTPS to our webcams. It also keeps us distant from managing storage permissions etc.

CloudFront

You might wonder, why we had to set up CloudFront additionally to the S3 bucket. At the time of writing it was not possible to have a S3 bucket with a custom domain and a valid TLS-certificate, that's why we needed to set up CloudFront additionally to the S3 bucket.

Now there's usually a problem with CDNs when delivering dynamic content, as it's primary purpose is to deliver static content which doesn't change too frequently. But luckily, CloudFront has an option for this purpose.

AWS CloudFront can be set up with a TTL of 0 seconds, which means, that CloudFront will still cache the content, but it will make a GET request with an If-Modified-Since header to allow the origin to tell CloudFront, if the image can still be served from the cache or if it has to be pulled from the origin.

Conclusion

Using cloud services for such simple services can really save up time in terms of maintenance and management, as you don't have to care about OS updates, public-facing storage, distribution

Useful resources

Here's a list with a few resources you might check out to learn more about Amazon Web Services (AWS):

If you have any further questions, aspects you are unsure with or found anything, that is incorrect in this article, don't hesitate to leave a comment below or reach out to me on Twitter: @ivansieder.