
AWS Cloudformation – Metadata
- Blog
In a serie of blog articles I’ll take a closer look at AWS CloudFormation. Read more about what AWS CloudFormation is, how to design templates and bootstrapping an EC2 with userdata in my previous blog articles:
- AWS CloudFormation – Templates, stacks and change sets
- AWS CloudFormation – Designing Templates
- AWS CloudFormation – Bootstrapping an Ec2 instance with Userdata
In this blog article I’ll take a closer look at using metadata in the resource section to prevent a mess.
CloudFormation Metadata
Additional to bootstrap scripts you can include meta data on an EC2 instance. In comparison to the UserData, where basic shell scripting is used, metad ata will follow a declarative approach to setting up the EC2 instances. Use the AWS::CloudFormation::Init type to include meta data on an Amazon EC2 instance for the cfn-init helper script. When your template calls the cfn-init script, the script looks for resource meta data rooted in the AWS::CloudFormation::Init metadata key. The cfn-init
command can be found in UserData property. Basically the previous UserData example will be extended with this extra line:
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} \
--resource MyInstance --region ${AWS::Region}
More about this later.
The Metadata configuration is separated into sections. The following example shows how to attach Metadata for cfn-init to an Amazon EC2 instance resource within the template.
Resources:
MyInstance:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
config:
packages:
:
groups:
:
users:
:
sources:
:
files:
:
commands:
:
services:
:
Properties:
Metadata is organized into config keys, which you can group into config sets. You can specify a config set when you call cfn-init in your template. If you don’t specify a config set, cfn-init looks for a single config key named config. The cfn-init helper script processes these configuration sections in the following order: packages, groups, users, sources, files, commands, and then services. If you require a different order, separate your sections into different config keys, and then use a config set that specifies the order in which the config keys should be processed.
AWS::CloudFormation::Init:
configSets:
ascending:
- "config1"
- "config2"
descending:
- "config2"
- "config1"
config1:
commands:
test:
command: "echo \"$CFNTEST\" > test.txt"
env:
CFNTEST: "I come from config1."
cwd: "~"
config2:
commands:
test:
command: "echo \"$CFNTEST\" > test.txt"
env:
CFNTEST: "I come from config2"
cwd: "~"
With cfn-init -c <configSet>
you can call a specific set and force an ordering. Also, using configSets allows you to combine multiple configSets within your template.
AWS::CloudFormation::Init:
1:
commands:
test:
command: "echo \"$MAGIC\" > test.txt"
env:
MAGIC: "I come from the environment!"
cwd: "~"
2:
commands:
test:
command: "echo \"$MAGIC\" >> test.txt"
env:
MAGIC: "I am test 2!"
cwd: "~"
configSets:
test1:
- "1"
test2:
- ConfigSet: "test1"
- "2"
default:
- ConfigSet: "test2"
Let’s dive into more detail and explore each section keys. Every key contains a set of (sub) keys. Some of them are required.
Commands key
The only required key in this section is command
. This can either be a string or an array. Commands are processed in alphabetical order by name. Optionally you can specify environment variables, working directory and test-runs before being executed by cfn-init (dry run). Additionally you can ignore errors by setting ignoreErrors to true.
Files key
You can use the files
key to create files on the EC2 instance. The content can be specified inline in the template or can be pulled from a URL.
files:
/tmp/setup.mysql:
content: !Sub |
CREATE DATABASE ${DBName};
CREATE USER '${DBUsername}'@'localhost' IDENTIFIED BY '${DBPassword}';
GRANT ALL ON ${DBName}.* TO '${DBUsername}'@'localhost';
FLUSH PRIVILEGES;
mode: "000644"
owner: "root"
group: "root"
Creating a symlink can easily done by specify the symlink target in the content key. The mode key uses the first three digits for symlinks and the last three digits for setting permissions. To create a symlink, specify 120000. To specify permissions for a file, use the last three digits, such as 000644
files:
/tmp/file1:
content: "/tmp/file2"
mode: "120644"
Sources key
Instead of declaring the content inside the template, you can use the source
key to specify a specific URL, like GitHub etc. This may also be a S3 bucket. Additionally, you can configure your access keys in the authentication
key to successfully pull the content from S3. You need to configure the type key correctly, and tell CloudFormation about the kind of authentication. Use s3
to authenticate to S3 buckets and basic
to authenticate to GitLab or another site. The type of authentication determines which properties to use. s3
requires secretKey, accessorized and bucket. basic
on the oterh hand requires an url, user name and password.
Packages key
This is very similar to configuration management. It allows you to install packages and specify the version. Version is not required, leaving this blank, CloudFormation assumes the latest version.
yum:
httpd: []
in this example CloudFormation leverages the yum repository to install httpd with the latest version.
Services key
This key manages the services, but has some extra keys, which you may not expect at first sight. The following code snippet shows two examples of services managed by CloudFormation init.
services:
sysvinit:
nginx:
enabled: "true"
ensureRunning: "true"
files:
- "/etc/nginx/nginx.conf"
php-fastcgi:
enabled: "true"
ensureRunning: "true"
packages:
yum:
- "php"
- "spawn-fcgi"
Besides declaring the run- and startup-state, you can also manage the files, sources and packages that are required to run the services. Any changes to this will trigger a restart of the service.
User and Group keys
These keys will manage your user and groups during creation.
groups:
groupOne:
gid: "45"
users:
myUser:
groups:
- "groupOne"
uid: "50"
homeDir: "/tmp"
As already mentioned the Metadata is accessed by cfn-init via UserData. With Fn::Sub
you can substitute the stackname and region pseudo parameters by the actual stackname and region at run-time. Always update aws-cfn-bootstrap to have to latest version installed.
Ec2Instance:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
config:
..
Properties:
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -xe
yum update -y aws-cfn-bootstrap
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} \
--resource MyInstance --region ${AWS::Region}
In this example cfn-init
will look for the meta data of the resource MyInstance within the CloudFormation template and executes the configuration (search for config by default).
As described above, cfn-init
only applies to the bootstrap process of your EC2 instance. Updating the configuration of your instances can be done with the cfn-hup
daemon. cfn-hup
runs user defined actions when a change is detected in the resource Metadata. Next time cfn-hup
will be covered in more detail. You can read it here.
Vincent Lamers, Linux-consultant @ AT Computing