I built a thing - price monitor using AWS CDK, Lambda, DynamoDB and SES

Published: 2021.10.21 · 4 minutes read

I wanted to buy a PlayStation 5 but it was out of stock all over the place, so I spent a short amount of time building a price monitor using AWS CDK, Lambda, DynamoDB and SES. The implementation turned out to be simple enough, so I decided to share the core parts with you. This article assumes that you have at least some basic understanding of the AWS platform and CDK framework.

Price monitor architecture

Everything starts from a scheduled EnventBridge rule (ScheduledEvent) that triggers the Lambda function (PriceCheckLambda) every so often (in my case, 15 minutes). This function reads current product prices stored in DynamoDB (PriceTable), compares with a current price online and updates DB accordingly. Updates in DynamoDB trigger a second Lambda (NotificationLambda) that sends an email to my mailbox. Done!

I deliberately stripped down the noise and kept only the core parts. You can find my full implementation of the price monitor on my GitHub.

CDK stack

To provision architecture, I used AWS CDK. On one of the previous projects, I used pure CloudFormation templates, and I don’t miss these times since the day I embraced CDK.

// stack.ts

import * as path from "path";
import * as cdk from "@aws-cdk/core";
import * as events from "@aws-cdk/aws-events";
import * as targets from "@aws-cdk/aws-events-targets";
import * as lambdaNodejs from "@aws-cdk/aws-lambda-nodejs";
import * as lambda from "@aws-cdk/aws-lambda";
import * as dynamodb from "@aws-cdk/aws-dynamodb";
import * as iam from "@aws-cdk/aws-iam";
import * as awsLambdaEventSources from "@aws-cdk/aws-lambda-event-sources";

export class Stack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string) {
    super(scope, id);

    const dynamoDbTable = new dynamodb.Table(this, "PriceTable", {
      partitionKey: {
        name: "id",
        type: dynamodb.AttributeType.STRING,
      },
      stream: dynamodb.StreamViewType.NEW_AND_OLD_IMAGES,
    });

    const lambdaPriceCheck = new lambdaNodejs.NodejsFunction(
      this,
      "PriceCheckLambda",
      {
        entry: "./price-check.ts",
        environment: {
          TABLE_NAME: dynamoDbTable.tableName,
        },
      }
    );

    const lambdaNotification = new lambdaNodejs.NodejsFunction(
      this,
      "NotificationLambda",
      {
        entry: "./price-notification.ts",
      }
    );

    dynamoDbTable.grantReadWriteData(lambdaPriceCheck);

    lambdaNotification.addEventSource(
      new awsLambdaEventSources.DynamoEventSource(dynamoDbTable, {
        startingPosition: lambda.StartingPosition.TRIM_HORIZON,
        batchSize: 1,
      })
    );

    lambdaNotification.addToRolePolicy(
      new iam.PolicyStatement({
        actions: ["ses:SendEmail"],
        resources: ["*"],
      })
    );

    new events.Rule(this, "ScheduledEvent", {
      schedule: events.Schedule.rate(cdk.Duration.minutes(15)),
      targets: [new targets.LambdaFunction(lambdaPriceCheck)],
    });
  }
}

Price check Lambda

This function is responsible for comparing prices from the database to current prices online. I used cheerio for the web scraping part, but most likely, I will replace it with Puppeteer.

// price-check.ts

import fetch from "node-fetch";
import cheerio from "cheerio";
import {
  DynamoDBClient,
  ScanCommand,
  PutItemCommand,
} from "@aws-sdk/client-dynamodb";
import { ScheduledHandler } from "aws-lambda";
import { marshall, unmarshall } from "@aws-sdk/util-dynamodb";

interface PriceRow {
  selector: string;
  id: string;
  url: string;
  email: string;
  item: string;
  price: string;
}

const { AWS_REGION: region, TABLE_NAME: TableName } = process.env;

const dbClient = new DynamoDBClient({ region });

const handler: ScheduledHandler = async (event) => {
  try {
    const { Items, Count } = await dbClient.send(
      new ScanCommand({
        TableName,
      })
    );

    if (!Count) {
      return;
    }

    const itmesUnmarshall = Items?.map((i) => unmarshall(i)) as PriceRow[];

    const newPrices = await Promise.all(
      itmesUnmarshall.map(({ url }) =>
        fetch(url).then((response) => response.text())
      )
    );

    const diff = itmesUnmarshall.reduce((acc, item, index) => {
      const $ = cheerio.load(newPrices[index]);
      const price = $(item.selector).text();

      if (price === item.price) {
        return acc;
      }

      return [...acc, { ...item, price }];
    }, [] as PriceRow[]);

    if (diff.length) {
      const updateCommands = diff.map((item) =>
        dbClient.send(
          new PutItemCommand({
            TableName,
            Item: marshall(item),
          })
        )
      );
      await Promise.all(updateCommands);
    }

    return;
  } catch (error) {
    console.error(error);
    throw new Error("Uuuups!");
  }
};

export { handler };

Notification Lambda

This function is responsible for parsing the DynamoDB stream that contains old and new prices and sending a notification email. I used SES to handle email communication, but you know — you do you.

// price-notification.ts

import { unmarshall } from "@aws-sdk/util-dynamodb";
import { SESClient, SendEmailCommand } from "@aws-sdk/client-ses";
import { DynamoDBStreamHandler } from "aws-lambda";

const { AWS_REGION: region } = process.env;

const sesClient = new SESClient({ region });

const handler: DynamoDBStreamHandler = async (event) => {
  const record = event.Records[0];
  const {
    // @ts-ignore
    dynamodb: { NewImage, OldImage },
    eventName,
  } = record;

  if (eventName !== "MODIFY") {
    return;
  }

  const unmarshalledNewImage = unmarshall(NewImage);
  const unmarshalledOldImage = unmarshall(OldImage);

  const { price, item, email } = unmarshalledNewImage;

  try {
    await sesClient.send(
      new SendEmailCommand({
        Source: email,
        Destination: {
          ToAddresses: [email],
        },
        Message: {
          Body: {
            Html: {
              Charset: "UTF-8",
              Data: `Old price: ${unmarshalledOldImage.price}, new price: ${price}`,
            },
          },
          Subject: {
            Charset: "UTF-8",
            Data: `💰 Price alert - ${item}`,
          },
        },
      })
    );

    return;
  } catch (error) {
    console.error(error);
    throw new Error("Uuuups!");
  }
};

export { handler };

To summarise

Creating this service was very enjoyable — good Philips Hue deals on Amazon and my new discounted pair of kicks I like even more. I pay for this service absolutely nothing, thanks to generous AWS free tier. By the way, I don’t want to buy PS5 anymore. Hopefully, you found it helpful 🤪