Scheduled EC2 start/stop: Serverless, AWS SSM, IAM roles

This is a Serverless Framework project which will automatically start and stop EC2 instances.

It creates two scheduled events. One event will start a list of EC2 instances and the other will stop a list of EC2 instances. The list of instances is defined in the AWS Systems Manager Parameter Store. The project also sets up the necessary IAM policies. No manual AWS configuration is required!

How it works

The important files are:

• serverless.yml: creates the Lambda functions, defines the IAM roles and defines the schedule for running the start and stop scripts
• ec2_start.py and ec2_stop.py: python scripts to start and stop the instances
• parameter.py: defines the ParameterStore that provides an easy way to interact with AWS SSM Parameter Store.

serverless.yml

The provider section:

• Defines the IAM role to use: Ec2StartStopRole. The role is defined further down in serverless.yml
• Creates an environment variable API_ENV and sets the value to the stage.

provider:
  name: aws
  role: Ec2StartStopRole
  runtime: python3.7
  environment:
    API_ENV: $
  stage: ${opt:stage, 'dev'}
  region: us-east-1

The events section of each function define when the function will run, using cron syntax: in this case daily at 19:00.

    events:
     - schedule:
         rate: cron(0 19 * * ? *)

The Resources section sets the policy to be associated with the IAM role.

                - Action:
                    - logs:PutLogEvents
                    - logs:CreateLogStream
                    - logs:CreateLogGroup
                  Effect: Allow
                  Resource: arn:aws:logs:$-*:*

This allows the function to write Cloudwatch log files to the relevant log group. The variables will be substituted so for a dev deployment for example this will result in arn:aws:logs:us-east-1:*:log-group:/aws/lambda/ec2-start-stop-dev-*:*

                - Action:
                    - ssm:GetParameters
                    - ssm:GetParameter
                  Effect: Allow
                  Resource: arn:aws:ssm:$/*

This allows the function to get parameters from AWS Systems Manager Parameter Store. The parameter store is read at run-time to determine which instances to start and stop. Note that the stage is used to determine which parameters to access so it is possible to have separate lists for each environment e.g. dev, prod.

                - Action:
                    - ec2:StartInstances
                    - ec2:StopInstances
                  Effect: Allow
                  Resource: arn:aws:ec2:$:*:instance/*
                  Condition:
                    StringEquals:
                      ec2:ResourceTag/Type: $

This allow the function to start and stop instances provided the instance is tag with a Type equal to the stage e.g. dev

ec2_start.py and ec2_stop.py

    parameter_store = ParameterStore()
    region = parameter_store.get_parameter("/aws_region")
    instances = parameter_store.get_parameter("/ec2_instances")

This code loads parameters from SSM Parameter Store for the region and the instances to start and stop.

parameter.py

        if env is None and 'API_ENV' in os.environ:
            environment = f"/"
        else:
            environment = DEFAULT_ENV

This reads the API_ENV environment variable which will be set to the name of the stage e.g. prod

        response = self.client.get_parameter(Name=self.env + path,
                                             WithDecryption=decrypt)
        parameter = response['Parameter']['Value']

This takes the parameter key that has been passed in to the function e.g. /ec2_instances, prepends it with the stage e.g. prod and returns the value stored in that parameter.

Install

These instructions are for Linux (specifically Ubuntu). For other environments, a quick web search will help!

Install Serverless

# Install node and npm
sudo apt update
sudo apt install nodejs
sudo apt install npm

# Install the serverless cli
npm install -g serverless

# Create and deploy a new service/project
serverless

Install AWS CLI

# Install
sudo apt-get install awscli

# Configure
aws configure

Install EC2 Start/Stop

# Install git
sudo apt install git

# Clone the repo
git clone git@gitlab.com:raytio-public/tools/ec2_start_stop.git
cd ec2_start_stop

Set up

Decide which instances you want to start and stop

To get a list of instance ids:

aws ec2 describe-instances --output json

Note the values of the InstanceId fields in the returned response.

Create the relevant parameters in SSM

Create two parameters with the Type of String. The name of the parameters will be as follows:

//aws_region
//ec2_instances

The stage is a Serverless concept that allows deployments to be grouped. In the following examples we use a stage named dev. To create a schedule for a separate set of instances just use the name of that stage e.g. staging.

Set the value of each to be the relevant region and the list of instances to start and stop. For example to have the script start and stop 2 dev instances in the us-east-1 region:

aws ssm put-parameter --name /dev/aws_region --type String --value us-east-1 --output json
aws ssm put-parameter --name /dev/ec2_instances --type String --value ['i-0c7cdb180150fabc', 'i-0c7c3760bd2fdc4e9'] --output json

Tag the EC2 instance

Against each instance type, create a Type tag with the value of the stage. This tag is checked by the IAM policy to ensure that the event has permissions for the instances in the list. The tag should correspond to the stage name. For example, for your development EC2 instances, set Type to dev.

aws ec2 create-tags --resources i-0c7cdb180150fabc i-0c7c3760bd2fdc4e9 --tags Key=Type,Value=dev --output json

Set up the schedule

Update the schedule for the ec2_start and ec2_stop functions in the serverless.yml file. Use either the AWS Cloudwatch events cron or rate syntax. For example the following will run the function every day at 1900:

cron(0 19 * * ? *)

Deploy

Use serverless to deploy the code to AWS which:

1. Creates two Lambda functions
2. Creates an IAM role with the necessary permissions
3. Creates two scheduled Cloudwatch events

For example to deploy to the dev stage:

sls deploy --stage dev