Native Service Integrations

Add automated error reporting with Simple Notification Service (SNS)

Now that we’ve implemented an end-to-end image processing pipeline, we will prepare our application for production usage by implementing e-mail alerts to be sent by our Step Functions state machine in the event of any failures.

The intent of the PhotoDoesNotMeetRequirement step is to send notification to the user that the verification of their profile photo failed so they might try uploading a different picture. It currently uses the AWS Lambda function NotificationPlaceholderFunction which simply returns the message instead of actually sending the notification. We will update this task to actually send e-mail alerts by creating a Simple Notification Service (SNS) topic and e-mail subscription to send any alerts to your inbox.

First, we will create the SNS topic.

Create SNS Topic

  1. From the AWS Management Console, choose Services then select Simple Notification Service under the Application Integration section.

  2. Select Topics on the left-hand navigation menu.

  3. Next, select Create Topic in the top right corner of the screen.

    SNS Topic Step 1
  4. Name your topic NotifyMe and leave the Display name blank.

    SNS Topic Step 2
  5. Scroll to the bottom of the page and select Create topic.

Subscribe to SNS Topic

Now that you’ve created the topic, you will subscribe your personal e-mail address so you can receive notifications and alerts.

  1. Select Create subscription in the Subscriptions section of the page.

    SNS Subscription Step 1

  2. Select Email as the Protocol for delivery.

  3. For Endpoint, type your personal e-mail address address.

    SNS Subscription Step 2

  4. Select Create subscription.

Verify your e-mail subscription

SNS e-mail subscriptions require confirmation of the end user’s desire to receive messages before messages start being sent to a new e-mail subscriber. Accordingly, we’ll now go ahead and validate our subscription.

  1. Check your inbox for a mail from AWS Notifications and click Confirm subscription within the body of the e-mail message.

    SNS Subscription Step 3
  2. You should now be directed to a webpage that shows your subscription as confirmed.

    SNS Subscription Step 4

Update State Machine to send messages to SNS topic

Now that we have our SNS topic and e-mail subscription setup, we will update our state machine to send any errors as an SNS topic publication so we receive e-mail alerts.

  1. Edit your rider-photo-state-machine.json file to update the existing step, PhotoDoesNotMeetRequirement, to be like the following.

    "PhotoDoesNotMeetRequirement": {
    	"Type": "Task",
    	"Resource": "arn:aws:states:::sns:publish",
    	"Parameters": {
    		"TopicArn": "REPLACE_WITH_YOUR_SNS_TOPIC_ARN",
    		"Subject": "Non-compliant photo encountered",
    		"Message.$": "$"
    	},
    	"End": true
    },	

    Note: By leaving the message as “$”, you will send the full initial state payload, as well as the error details via e-mail for review.

  2. Next, replace the REPLACE_WITH_YOUR_SNS_TOPIC_ARN with the ARN you noted down after creating your SNS topic. IT should look something like arn:aws:sns:us-west-2:012345678912:NotifyMe.

    Since the last step in our state machine remains the same, we do not need to update the “End” property.

  3. At this point, your rider-photo-state-machine.json file should look like this (the AWS Lambda ARNs are examples):

    (expand to see)

  4. Go back the AWS Step Functions Console, click the Edit state machine button to update the RiderPhotoProcessing state machine.

  5. Paste the updated JSON definition and click the refresh button in the preview panel to visualize the changes:

    Update state machine with persistence step
  6. Click the Save button to save the state machine.

  7. Click the Start execution button to test the new state machine with with test input (or any other input from the validation section that you’ve already used with the state machine at least once):

    {
    	"userId": "user_a",
    	"s3Bucket": "REPLACE_WITH_YOUR_BUCKET_NAME",
    	"s3Key": "1_happy_face.jpg"
    }	

    If you reference an image that’s already indexed when you were testing the previous state machine, the execution would fail the CheckFaceDuplicate step like this:

    Previously indexed face

    Even though the same visual flow has occurred, because we’ve now updated the PhotoDoesNotMeetRequirement step to publish to an SNS topic, you should have received an e-mail with the error message.

    Since we left the message text set to $ above, the full execution state payload including the error details object is included in the message.

    SNS E-mail Error

    You can use the aws rekognition list-faces and aws rekognition delete-faces commands to clean up the previous indexed faces during testing to try triggering different types of errors. Note that our e-mail alerts will only be sent when our state machine execution ends in an error state.

Other Native Service Integrations

This example demonstrates how to natively use SNS publications with Step Functions, but there are many other native service integrations avaialble, which allow you to integrate with many services in orchestration of batch processing, machine learning training and deployments, containerized tasks, data transformations with AWS Glue and EMR, and more.

Feel free to review our Step Functions documentation for a full list of native AWS service integrations.