How to Stop/Start an AWS EC2 instance using AWS CLI command line

by Laurence Posted on February 3, 2023

In my experience the smaller the AWS EC2 instance the more often it freezes / completely locks up. Something goes wrong inside the AWS infrastructure and poof, no ability to SSH or do anything with it. This is regardless of system load, memory status, the type of application, etc. The frequency of the crash is inversely proportional to the instance size.

  • t3.micro -> 1-8 times a month (free tier instances)
  • t3.medium -> 1-3 times a month (on a commercial Django product I maintain)
  • t2.xlarge -> maybe once a year if ever (another client of mine running a high traffic website)

One “solution” is to login to the AWS Console and stop / start the instance. That is a pain because 1) I have MFA setup, 2) the AWS console is a beast to navigate.  It can also take time for the instance to stop (15 minutes worst case) or require a force stop, before it can be restarted.

So I decided to figure out how to query the server status with the AWS CLI, and if needed stop/start it from the command line. This procedure is locked down to my IP, so even if the IAM user credentials leaked, it would have to be from my machine. I’m pretty happy with this balance of ease and security.  If your EBS volume is encrypted (which it should be) there are a few important yet obscure things to pay attention to as you proceed with the configuration.

In this guide I’ll show you how I use AWS cli to stop/start an EC2 instance from the command line using an IAM user with the right permissions, including support for encrypted volumes.

1) Install the AWS CLI tool if you haven’t already.

2) In the AWS Console, under IAM, create an IAM user, name it something like ec2-stop-start. Get the new user’s credentials (Access Key ID and Secret Access Key) and save them somewhere safe.

If your EBS Volume is encrypted, find the key it uses under the EC2 -> EBS Volume configuration. If this is a customer managed key, under AWS KMS Console, add the new IAM user to the key under Key Users.

3) In the AWS Console, under IAM create a policy that allows querying EC2 status and stop/starting instances.

This policy will be locked down to your local IP!  For me this works great because my ISP changes my IP once ever couple of years. The IP will also change if the MAC address on my wifi router changes (this is a trick to force a new IP to be allocated).

IAM policy outline:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "0",
            "Effect": "Allow",
            "Action": [
                "ec2:StartInstances",
                "ec2:StopInstances"
            ],
            "Resource": "arn:aws:ec2:*:*:instance/*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "xyz.0.0.1/32"
                }
            }
        },
        {
            "Sid": "1",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ec2:DescribeInstanceStatus"
            ],
            "Resource": "*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "xyz.0.0.1/32"
                }
            }
        },
        {
            "Sid": "2",
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:DescribeKey",
                "kms:GenerateDataKey*",
                "kms:GetPublicKey",
                "kms:GetKeyPolicy"
            ],
            "Resource": "arn:aws:kms:us-west-2:0000:/xyz",
            "Condition": {
                "StringEquals": {
                    "kms:ViaService": [
                        "ec2.us-west-2.amazonaws.com"
                    ]
                }
            }
        }
    ]
}

Summary of each block:

Sid 0 -> EC2 allow stop/start from YOUR IP
Sid 1 -> EC2 allow describe instances from YOUR IP
Sid 2 -> Grant user access to encryption keys via EC2

Make sure to change the following:

  • Change the aws:SourceIP to be YOUR PUBLIC IP, which is in two places.
  • If your EBS volume is encrypted, change the region under the kms:ViaService block to the region your EC2 instance is actually in. Also change the `arn:aws:kms:us-west-2:0000:/xyz` value to be the ARN of the actual key that goes with the EBS volume.
  • Note – you don’t need the section with "SiD": "2" if your EBS volume is not encrypted. This section gives the IAM user the ability to work with the encryption key when requesting on behalf of another service (that is what the kms:ViaService tag does).
  • Note – this policy gives access to ALL EC2 instances under the account, but maybe that is way too much? Customize the Resource blocks as needed for your situation.

4) Associate the policy you just created with your ec2-stop-start user in the IAM control panel.

5) Add the IAM user credentials in your local aws credentials file (~/.aws/credentials), using the profile named
aws_ec2_stop_start:

[aws_ec2_stop_start]
aws_access_key_id = ******
aws_secret_access_key = ******

6)  Find your server’s instance ID in the EC2 control panel (something like i-00000).

7) Try querying the instance status:

$ aws ec2 describe-instances --region us-west-2 --profile aws_ec2_stop_start --output text

$ aws ec2 describe-instance-status --instance-ids i-00000 --region us-west-2 --profile aws_ec2_stop_start --output text

(change the region, and instance-ids parameters for your case, output can be text, table or json)

8) Try running stop start with the dry-run argument:

$ aws ec2 stop-instances --instance-ids i-00000 --region us-west-2 --profile aws_ec2_stop_start --dry-run

$ aws ec2 start-instances --instance-ids i-00000 --region us-west-2 --profile aws_ec2_stop_start --dry-run

(again, change the region, and instance-ids parameters for your case)

 

9) And for real, with out the –dry-run argument:

# STOP INSTANCE FOR REAL - MAKE SURE YOU KNOW WHAT YOU ARE DOING!!
$ aws ec2 stop-instances --instance-ids i-00000 --region us-west-2 --profile aws_ec2_stop_start

# if you need to force it
$ aws ec2 stop-instances --instance-ids i-00000 --region us-west-2 --profile aws_ec2_stop_start --force

# at this point you can query the status
$ aws ec2 describe-instances \
    --region us-west-2 --output text --profile aws_ec2_stop_start \
    --query 'Reservations[*].Instances[*].{Instance:InstanceId,Instance:State}'

# START FOR REAL
$ aws ec2 start-instances --instance-ids i-00000 --region us-west-2 --profile aws_ec2_stop_start --output text

 

Troubleshooting:

If the instance is not starting (stuck in pending), describe-instance-status shows:

"StateReason": {
  "Message": "Client.InternalError: Client error on launch",
  "Code": "Client.InternalError"
},

This might be because the EBS volume is encrypted. In that case double check the Sid 2 block in the IAM profile above, and see this AWS post for details on fixing that, and more about IAM policies for keys and the kms:ViaService configuration setting. The notes above work for me, but you may need to fiddle with the settings to get it to work. Again, if your volume is encrypted with a customer managed key, under AWS KMS Console, add the IAM user you created in step 2 to the key under Key Users.

Links to AWS CLI EC2 command docs:

AWS CLI has a lot of extra command line options you might want to get familiar with.  See the docs here:

This entry was posted in Sys Admin, Work and tagged , . Bookmark the permalink.

Comments are closed.