Tag / vRA

by Mayank Goyal

An Introduction to Cloud-Config Scripting for Linux based VMs in vRA Cloud Templates

Disclaimer This article is referenced from digitalocean.com

  1. Introduction
  2. General Information about Cloud-Config
  3. YAML Formatting
  4. User and Group Management
  5. Change Passwords for Existing Users
  6. Write Files to the Disk
  7. Update or Install Packages on the Server
  8. Configure SSH Keys for User Accounts and the SSH Daemon
  9. Set Up Trusted CA Certificates
  10. Configure resolv.conf to Use Specific DNS Servers
  11. Run Arbitrary Commands for More Control
  12. Shutdown or Reboot the Server
  13. Conclusion
  14. References

Introduction

The cloud-init program that is available on recent distributions (only Ubuntu and CentOS at the time of this writing) is able to consume and execute data from the user-data field. This process behaves differently depending on the format of the information it finds. One of the most popular formats for scripts within user-data is the cloud-config file format.

Cloud-config files are special scripts designed to be run by the cloud-init process. These are generally used for initial configuration on the very first boot of a server. In this guide, we will be discussing the format and usage of cloud-config files.

General Information about Cloud-Config

The cloud-config format implements a declarative syntax for many common configuration items, making it easy to accomplish many tasks. It also allows you to specify arbitrary commands for anything that falls outside of the predefined declarative capabilities.

This “best of both worlds” approach lets the file acts like a configuration file for common tasks, while maintaining the flexibility of a script for more complex functionality.

YAML Formatting

The file is written using the YAML data serialization format. The YAML format was created to be easy to understand for humans and easy to parse for programs.

YAML files are generally fairly intuitive to understand when reading them, but it is good to know the actual rules that govern them.

Some important rules for YAML files are:

  • Indentation with whitespace indicates the structure and relationship of the items to one another. Items that are more indented are sub-items of the first item with a lower level of indentation above them.
  • List members can be identified by a leading dash.
  • Associative array entries are created by using a colon (:) followed by a space and the value.
  • Blocks of text are indented. To indicate that the block should be read as-is, with the formatting maintained, use the pipe character (|) before the block.

Let’s take these rules and analyze an example cloud-config file, paying attention only to the formatting:

#cloud-config
users:
  - name: demo
    groups: sudo
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    ssh-authorized-keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf0q4PyG0doiBQYV7OlOxbRjle026hJPBWD+eKHWuVXIpAiQlSElEBqQn0pOqNJZ3IBCvSLnrdZTUph4czNC4885AArS9NkyM7lK27Oo8RV888jWc8hsx4CD2uNfkuHL+NI5xPB/QT3Um2Zi7GRkIwIgNPN5uqUtXvjgA+i1CS0Ku4ld8vndXvr504jV9BMQoZrXEST3YlriOb8Wf7hYqphVMpF3b+8df96Pxsj0+iZqayS9wFcL8ITPApHi0yVwS8TjxEtI3FDpCbf7Y/DmTGOv49+AWBkFhS2ZwwGTX65L61PDlTSAzL+rPFmHaQBHnsli8U9N6E4XHDEOjbSMRX user@example.com
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcthLR0qW6y1eWtlmgUE/DveL4XCaqK6PQlWzi445v6vgh7emU4R5DmAsz+plWooJL40dDLCwBt9kEcO/vYzKY9DdHnX8dveMTJNU/OJAaoB1fV6ePvTOdQ6F3SlF2uq77xYTOqBiWjqF+KMDeB+dQ+eGyhuI/z/aROFP6pdkRyEikO9YkVMPyomHKFob+ZKPI4t7TwUi7x1rZB1GsKgRoFkkYu7gvGak3jEWazsZEeRxCgHgAV7TDm05VAWCrnX/+RzsQ/1DecwSzsP06DGFWZYjxzthhGTvH/W5+KFyMvyA+tZV4i1XM+CIv/Ma/xahwqzQkIaKUwsldPPu00jRN user@desktop
runcmd:
  - touch /test.txt

By looking at this file, we can learn a number of important things.

First, each cloud-config file must begin with #cloud-config alone on the very first line. This signals to the cloud-init program that this should be interpreted as a cloud-config file. If this were a regular script file, the first line would indicate the interpreter that should be used to execute the file.

The file above has two top-level directives, users and runcmd. These both serve as keys. The values of these keys consist of all of the indented lines after the keys.

In the case of the users key, the value is a single list item. We know this because the next level of indentation is a dash (-) which specifies a list item, and because there is only one dash at this indentation level. In the case of the users directive, this incidentally indicates that we are only defining a single user.

The list item itself contains an associative array with more key-value pairs. These are sibling elements because they all exist at the same level of indentation. Each of the user attributes are contained within the single list item we described above.

Some things to note are that the strings you see do not require quoting and that there are no unnecessary brackets to define associations. The interpreter can determine the data type fairly easily and the indentation indicates the relationship of items, both for humans and programs.

By now, you should have a working knowledge of the YAML format and feel comfortable working with information using the rules we discussed above.

We can now begin exploring some of the most common directives for cloud-config.

User and Group Management

To define new users on the system, you can use the users directive that we saw in the example file above.

The general format of user definitions is:

#cloud-config
users:
  - first_user_parameter
    first_user_parameter
    
  - second_user_parameter
    second_user_parameter
    second_user_parameter
    second_user_parameter

Each new user should begin with a dash. Each user defines parameters in key-value pairs. The following keys are available for definition:

  • name: The account username.
  • primary-group: The primary group of the user. By default, this will be a group created that matches the username. Any group specified here must already exist or must be created explicitly (we discuss this later in this section).
  • groups: Any supplementary groups can be listed here, separated by commas.
  • gecos: A field for supplementary info about the user.
  • shell: The shell that should be set for the user. If you do not set this, the very basic sh shell will be used.
  • expiredate: The date that the account should expire, in YYYY-MM-DD format.
  • sudo: The sudo string to use if you would like to define sudo privileges, without the username field.
  • lock-passwd: This is set to “True” by default. Set this to “False” to allow users to log in with a password.
  • passwd: A hashed password for the account.
  • ssh-authorized-keys: A list of complete SSH public keys that should be added to this user’s authorized_keys file in their .ssh directory.
  • inactive: A boolean value that will set the account to inactive.
  • system: If “True”, this account will be a system account with no home directory.
  • homedir: Used to override the default /home/<username>, which is otherwise created and set.
  • ssh-import-id: The SSH ID to import from LaunchPad.
  • selinux-user: This can be used to set the SELinux user that should be used for this account’s login.
  • no-create-home: Set to “True” to avoid creating a /home/<username> directory for the user.
  • no-user-group: Set to “True” to avoid creating a group with the same name as the user.
  • no-log-init: Set to “True” to not initiate the user login databases.

Other than some basic information, like the name key, you only need to define the areas where you are deviating from the default or supplying needed data.

One thing that is important for users to realize is that the passwd field should not be used in production systems unless you have a mechanism of immediately modifying the given value. As with all information submitted as user-data, the hash will remain accessible to any user on the system for the entire life of the server. On modern hardware, these hashes can easily be cracked in a trivial amount of time. Exposing even the hash is a huge security risk that should not be taken on any machines that are not disposable.

For an example user definition, we can use part of the example cloud-config we saw above:

#cloud-config
users:
  - name: demo
    groups: sudo
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    ssh-authorized-keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf0q4PyG0doiBQYV7OlOxbRjle026hJPBWD+eKHWuVXIpAiQlSElEBqQn0pOqNJZ3IBCvSLnrdZTUph4czNC4885AArS9NkyM7lK27Oo8RV888jWc8hsx4CD2uNfkuHL+NI5xPB/QT3Um2Zi7GRkIwIgNPN5uqUtXvjgA+i1CS0Ku4ld8vndXvr504jV9BMQoZrXEST3YlriOb8Wf7hYqphVMpF3b+8df96Pxsj0+iZqayS9wFcL8ITPApHi0yVwS8TjxEtI3FDpCbf7Y/DmTGOv49+AWBkFhS2ZwwGTX65L61PDlTSAzL+rPFmHaQBHnsli8U9N6E4XHDEOjbSMRX user@example.com
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcthLR0qW6y1eWtlmgUE/DveL4XCaqK6PQlWzi445v6vgh7emU4R5DmAsz+plWooJL40dDLCwBt9kEcO/vYzKY9DdHnX8dveMTJNU/OJAaoB1fV6ePvTOdQ6F3SlF2uq77xYTOqBiWjqF+KMDeB+dQ+eGyhuI/z/aROFP6pdkRyEikO9YkVMPyomHKFob+ZKPI4t7TwUi7x1rZB1GsKgRoFkkYu7gvGak3jEWazsZEeRxCgHgAV7TDm05VAWCrnX/+RzsQ/1DecwSzsP06DGFWZYjxzthhGTvH/W5+KFyMvyA+tZV4i1XM+CIv/Ma/xahwqzQkIaKUwsldPPu00jRN user@desktop

To define groups, you should use the groups directive. This directive is relatively simple in that it just takes a list of groups you would like to create.

An optional extension to this is to create a sub-list for any of the groups you are making. This new list will define the users that should be placed in this group:

#cloud-config
groups:
  - group1
  - group2: [user1, user2]

Change Passwords for Existing Users

For user accounts that already exist (the root account is the most pertinent), a password can be suppled by using the chpasswd directive.

Note: This directive should only be used in debugging situations, because, once again, the value will be available to every user on the system for the duration of the server’s life. This is even more relevant in this section because passwords submitted with this directive must be given in plain text.

The basic syntax looks like this:

#cloud-config
chpasswd:
  list: |
    user1:password1
    user2:password2
    user3:password3
  expire: False

The directive contains two associative array keys. The list key will contain a block that lists the account names and the associated passwords that you would like to assign. The expire key is a boolean that determines whether the password must be changed at first boot or not. This defaults to “True”.

One thing to note is that you can set a password to “RANDOM” or “R”, which will generate a random password and write it to /var/log/cloud-init-output.log. Keep in mind that this file is accessible to any user on the system, so it is not any more secure.

Write Files to the Disk

In order to write files to the disk, you should use the write_files directive.

Each file that should be written is represented by a list item under the directive. These list items will be associative arrays that define the properties of each file.

The only required keys in this array are path, which defines where to write the file, and content, which contains the data you would like the file to contain.

The available keys for configuring a write_files item are:

  • path: The absolute path to the location on the filesystem where the file should be written.
  • content: The content that should be placed in the file. For multi-line input, you should start a block by using a pipe character (|) on the “content” line, followed by an indented block containing the content. Binary files should include “!!binary” and a space prior to the pipe character.
  • owner: The user account and group that should be given ownership of the file. These should be given in the “username:group” format.
  • permissions: The octal permissions set that should be given for this file.
  • encoding: An optional encoding specification for the file. This can be “b64” for Base64 files, “gzip” for Gzip compressed files, or “gz+b64” for a combination. Leaving this out will use the default, conventional file type.

For example, we could write a file to /test.txt with the contents:

Here is a line.
Another line is here.

The portion of the cloud-config that would accomplish this would look like this:

#cloud-config
write_files:
  - path: /test.txt
    content: |
      Here is a line.
      Another line is here.

Update or Install Packages on the Server

To manage packages, there are a few related settings and directives to keep in mind.

To update the apt database on Debian-based distributions, you should set the package_update directive to “true”. This is synonymous with calling apt-get update from the command line.

The default value is actually “true”, so you only need to worry about this directive if you wish to disable it:

#cloud-config
package_update: false

If you wish to upgrade all of the packages on your server after it boots up for the first time, you can set the package_upgrade directive. This is akin to a apt-get upgrade executed manually.

This is set to “false” by default, so make sure you set this to “true” if you want the functionality:

#cloud-config
package_upgrade: true

To install additional packages, you can simply list the package names using the “packages” directive. Each list item should represent a package. Unlike the two commands above, this directive will function with either yum or apt managed distros.

These items can take one of two forms. The first is simply a string with the name of the package. The second form is a list with two items. The first item of this new list is the package name, and the second item is the version number:

#cloud-config
packages:
  - package_1
  - package_2
  - [package_3, version_num]

The “packages” directive will set apt_update to true, overriding any previous setting.

Configure SSH Keys for User Accounts and the SSH Daemon

You can manage SSH keys in the users directive, but you can also specify them in a dedicated ssh_authorized_keys section. These will be added to the first defined user’s authorized_keys file.

This takes the same general format of the key specification within the users directive:

#cloud-config
ssh_authorized_keys:
  - ssh_key_1
  - ssh_key_2

You can also generate the SSH server’s private keys ahead of time and place them on the filesystem. This can be useful if you want to give your clients the information about this server beforehand, allowing it to trust the server as soon as it comes online.

To do this, we can use the ssh_keys directive. This can take the key pairs for RSA, DSA, or ECDSA keys using the rsa_private, rsa_public, dsa_private, dsa_public, ecdsa_private, and ecdsa_public sub-items.

Since formatting and line breaks are important for private keys, make sure to use a block with a pipe key when specifying these. Also, you must include the begin key and end key lines for your keys to be valid.

#cloud-config
ssh_keys:
  rsa_private: |
    -----BEGIN RSA PRIVATE KEY-----
    your_rsa_private_key
    -----END RSA PRIVATE KEY-----

  rsa_public: your_rsa_public_key

Set Up Trusted CA Certificates

If your infrastructure relies on keys signed by an internal certificate authority, you can set up your new machines to trust your CA cert by injecting the certificate information. For this, we use the ca-certs directive.

This directive has two sub-items. The first is remove-defaults, which, when set to true, will remove all of the normal certificate trust information included by default. This is usually not needed and can lead to some issues if you don’t know what you are doing, so use with caution.

The second item is trusted, which is a list, each containing a trusted CA certificate:

#cloud-config
ca-certs:
  remove-defaults: true
  trusted:
    - |
      -----BEGIN CERTIFICATE-----
      your_CA_cert
      -----END CERTIFICATE-----

Configure resolv.conf to Use Specific DNS Servers

If you have configured your own DNS servers that you wish to use, you can manage your server’s resolv.conf file by using the resolv_conf directive. This currently only works for RHEL-based distributions.

Under the resolv_conf directive, you can manage your settings with the nameservers, searchdomains, domain, and options items.

The nameservers directive should take a list of the IP addresses of your name servers. The searchdomains directive takes a list of domains and subdomains to search in when a user specifies a host but not a domain.

The domain sets the domain that should be used for any unresolvable requests, and options contains a set of options that can be defined in the resolv.conf file.

If you are using the resolv_conf directive, you must ensure that the manage-resolv-conf directive is also set to true. Not doing so will cause your settings to be ignored:

#cloud-config
manage-resolv-conf: true
resolv_conf:
  nameservers:
    - 'first_nameserver'
    - 'second_nameserver'
  searchdomains:
    - first.domain.com
    - second.domain.com
  domain: domain.com
  options:
    option1: value1
    option2: value2
    option3: value3

Run Arbitrary Commands for More Control

If none of the managed actions that cloud-config provides works for what you want to do, you can also run arbitrary commands. You can do this with the runcmd directive.

This directive takes a list of items to execute. These items can be specified in two different ways, which will affect how they are handled.

If the list item is a simple string, the entire item will be passed to the sh shell process to run.

The other option is to pass a list, each item of which will be executed in a similar way to how execve processes commands. The first item will be interpreted as the command or script to run, and the following items will be passed as arguments for that command.

Most users can use either of these formats, but the flexibility enables you to choose the best option if you have special requirements. Any output will be written to standard out and to the /var/log/cloud-init-output.log file:

#cloud-config
runcmd:
  - [ sed, -i, -e, 's/here/there/g', some_file]
  - echo "modified some_file"
  - [cat, some_file]

Shutdown or Reboot the Server

In some cases, you’ll want to shutdown or reboot your server after executing the other items. You can do this by setting up the power_state directive.

This directive has four sub-items that can be set. These are delay, timeout, message, and mode.

The delay specifies how long into the future the restart or shutdown should occur. By default, this will be “now”, meaning the procedure will begin immediately. To add a delay, users should specify, in minutes, the amount of time that should pass using the +<num_of_mins> format.

The timeout parameter takes a unit-less value that represents the number of seconds to wait for cloud-init to complete before initiating the delay countdown.

The message field allows you to specify a message that will be sent to all users of the system. The mode specifies the type of power event to initiate. This can be “poweroff” to shut down the server, “reboot” to restart the server, or “halt” to let the system decide which is the best action (usually shutdown):

#cloud-config
power_state:
  timeout: 120
  delay: "+5"
  message: Rebooting in five minutes. Please save your work.
  mode: reboot

Conclusion

The above examples represent some of the more common configuration items available when running a cloud-config file. There are additional capabilities that we did not cover in this guide. These include configuration management setup, configuring additional repositories, and even registering with an outside URL when the server is initialized.

You can find out more about some of these options by checking the /usr/share/doc/cloud-init/examples directory. For a practical guide to help you get familiar with cloud-config files, you can follow our tutorial on how to use cloud-config to complete basic server configuration here.

References

by Mayank Goyal

Getting Started: Cloud Assembly Code Examples [CB10094]

  1. Introduction
  2. Samples
    1. A blank template
    2. Literals
    3. Environment variables
    4. Resource variables
    5. Resource self variables
    6. Cluster count index
    7. Conditions
    8. Arithmetic operators
    9. String concatenation
    10. Operators [ ] and .
    11. Construction of map
    12. Construction of array
    13. Functions
    14. Referencing input parameters
    15. Optional Inputs
    16. Explicit dependencies
    17. Property bindings
    18. Encrypt access credentials
    19. Passing inputs to ABXs
    20. Fetching value from Property Groups
    21. Adding a secret property
    22. Enum & oneOf (Dropdowns in Input form)
    23. Dynamic Enums (binding with vRO action)
    24. vRO Custom DataTypes
    25. Resource Flags
    26. cloudConfig (cloud-init & Cloudbase-Init)
  3. Examples of vSphere resources
  4. Market Place
  5. Other ways to create Cloud Assembly templates
    1. Cloud template cloning
    2. Uploading and downloading
    3. Integrating Cloud Assembly with a repository
  6. References

Introduction

Starting your journey on vRA 8.x can be a little challenging. Out of numerous demanding areas, one of them is creating blueprints or do i say, cloud templates in Cloud Assembly. Converting your existing vRA 7.x blueprints to vRA 8.x cloud templates requires some clarity on YAML development. Even though, a good deal of things are already provided out of the box, considering YAML as a Code which can single-handedly establish all your cloud deployments, getting the desired state out of it can be a trickster. Hence, I would like to share some quick bits on what we can do in vRA provided YAML editor to get the job done faster.

Samples

We will start with the basics and then will move to more advanced template options.

A blank template

name: templateName
formatVersion: 1
inputs: {}
resources: {}

Literals

The following literals are supported:

  • Boolean (true or false)
  • Integer
  • Floating point
  • StringBackslash escapes double quote, single quote, and backslash itself:" is escaped as \"' is escaped as \'\ is escaped as \\Quotes only need to be escaped inside a string enclosed with the same type of quote, as shown in the following example."I am a \"double quoted\" string inside \"double quotes\"."
  • Null

Environment variables

Environment names:

  • orgId
  • projectId
  • projectName
  • deploymentId
  • deploymentName
  • blueprintId
  • blueprintVersion
  • blueprintName
  • requestedBy (user)
  • requestedAt (time)
${env.blueprintId}

Resource variables

Resource variables let you bind to resource properties from other resources

format: resource.RESOURCE_NAME.PROPERTY_NAME

${resource.db.id}
${resource.db.networks[0].address}
${resource.app.id} (Return the string for non-clustered resources, where count isn't specified. Return the array for clustered resources.)
${resource.app[0].id} (Return the first entry for clustered resources.)

Resource self variables

Resource self variables are allowed only for resources supporting the allocation phase. Resource self variables are only available (or only have a value set) after the allocation phase is complete.

${self.address} (Return the address assigned during the allocation phase.)

Note that for a resource named resource_x, self.property_name and resource.resource_x.property_name are the same and are both considered self-references.

Cluster count index

${count.index == 0 ? "primary" : "secondary"} (Return the node type for clustered resources.)

Limitations:

Use of count.index for resource allocation is not supported. For example, the following capacity expression fails when it references the position within an array of disks created at input time.

inputs:
  disks:
    type: array
    minItems: 0
    maxItems: 12
    items:
      type: object
      properties:
        size:
          type: integer
          title: Size (GB)
          minSize: 1
          maxSize: 2048
resources:
  Cloud_vSphere_Disk_1:
    type: Cloud.vSphere.Disk
    properties:
      capacityGb: '${input.disks[count.index].size}'
      count: '${length(input.disks)}'

Conditions

  • Equality operators are == and !=.
  • Relational operators are < > <= and >=.
  • Logical operators are && || and !.
  • Conditionals use the pattern:condition-expression ? true-expression : false-expression
${input.count < 5 && input.size == 'small'}
${input.count < 2 ? "small" : "large"}

Arithmetic operators

Operators are + – / * and %.

${(input.count + 5) * 2}

String concatenation

${'ABC' + 'DEF'}

Operators [ ] and .

The expression follows ECMAScript in unifying the treatment of the [ ] and . operators.

So, expr.identifier is equivalent to expr["identifier"]. The identifier is used to construct a literal whose value is the identifier, and then the [ ] operator is used with that value.

${resource.app.networks[0].address}

In addition, when a property includes a space, delimit with square brackets and double quotes instead of using dot notation.

Incorrect:

input.operating system

Correct:

input["operating system"]

Construction of map

${{'key1':'value1', 'key2':input.key2}}

Construction of array

${['key1','key2']}

Functions

${function(arguments...)}

${to_lower(resource.app.name)}

List of supported functions

FunctionDescription
abs(number)Absolute number value
avg(array)Return average of all values from array of numbers
base64_decode(string)Return decoded base64 value
base64_encode(string)Return base64 encoded value
ceil(number)Returns the smallest (closest to negative infinity) value that is greater than or equal to the argument and is equal to a mathematical integer
contains(array, value)Check if array contains a value
contains(string, value)Check if string contains a value
digest(value, type)Return digest of value using supported type (md5, sha1, sha256, sha384, sha512)
ends_with(subject, suffix)Check if subject string ends with suffix string
filter_by(array, filter)Return only the array entries that pass the filter operationfilter_by([1,2,3,4], x => x >= 2 && x <= 3)returns [2, 3]filter_by({'key1':1, 'key2':2}, (k,v) => v != 1)returns [{"key2": 2}]
floor(number)Returns the largest (closest to positive infinity) value that is less than or equal to the argument and is equal to a mathematical integer
format(format, values…)Return a formatted string using Java Class Formatter format and values.
from_json(string)Parse json string
join(array, delim)Join array of strings with a delimiter and return a string
json_path(value, path)Evaluate path against value using XPath for JSON.
keys(map)Return keys of map
length(array)Return array length
length(string)Return string length
map_by(array, operation)Return each array entry with an operation applied to itmap_by([1,2], x => x * 10)returns [10, 20]map_by([1,2], x => to_string(x))returns ["1", "2"]map_by({'key1':1, 'key2':2}, (k,v) => {k:v*10})returns [{"key1":10},{"key2":20}]
map_to_object(array, keyname)Return an array of key:value pairs of the specified key name paired with values from another arraymap_to_object(resource.Disk[*].id, "source")returns an array of key:value pairs that has a key field called source paired with disk ID stringsNote thatmap_by(resource.Disk[*].id, id => {'source':id})returns the same result
matches(string, regex)Check if string matches a regex expression
max(array)Return maximum value from array of numbers
merge(map, map)Return a merged map
min(array)Return minimum value from array of numbers
not_null(array)Return the first entry which is not null
now()Return current time in ISO-8601 format
range(start, stop)Return a series of numbers in increments of 1 that begins with the start number and ends just before the stop number
replace(string, target, replacement)Replace string containing target string with target string
reverse(array)Reverse entries of array
slice(array, begin, end)Return slice of array from begin index to end index
split(string, delim)Split string with a delimiter and return array of strings
starts_with(subject, prefix)Check if subject string starts with prefix string
substring(string, begin, end)Return substring of string from begin index until end index
sum(array)Return sum of all values from array of numbers
to_json(value)Serialize value as json string
to_lower(str)Convert string to lower case
to_number(string)Parse string as number
to_string(value)Return string representation of the value
to_upper(str)Convert string to upper case
trim(string)Remove leading and trailing spaces
url_encode(string)Encode string using url encoding specification
uuid()Return randomly generated UUID
values(map)Return values of map

Referencing input parameters

In the resources section, you can reference an input parameter using ${input.property-name} syntax. If a property name includes a space, delimit with square brackets and double quotes instead of using dot notation: ${input["property name"]}.

inputs:
  sshKey:
    type: string
    maxLength: 500
resources:
  frontend:
    type: Cloud.Machine
    properties:
      remoteAccess:
        authentication: publicPrivateKey
        sshKey: '${input.sshKey}'

Important In cloud template code, you cannot use the word input except to indicate an input parameter.

Optional Inputs

Inputs are usually required and marked with an asterisk. To make an input optional, set an empty default value as shown.

owner:
  type: string
  minLength: 0
  maxLength: 30
  title: Owner Name
  description: Account Owner
  default: ''

List of available input properties

PropertyDescription
constUsed with oneOf. The real value associated with the friendly title.
defaultPrepopulated value for the input.The default must be of the correct type. Do not enter a word as the default for an integer.
descriptionUser help text for the input.
encryptedWhether to encrypt the input that the user enters, true or false.Passwords are usually encrypted.You can also create encrypted properties that are reusable across multiple cloud templates. See Secret Cloud Assembly properties.
enumA drop-down menu of allowed values.Use the following example as a format guide.enum: – value 1 – value 2
formatSets the expected format for the input. For example, (25/04/19) supports date-time.Allows the use of the date picker in Service Broker custom forms.
itemsDeclares items within an array. Supports number, integer, string, Boolean, or object.
maxItemsMaximum number of selectable items within an array.
maxLengthMaximum number of characters allowed for a string.For example, to limit a field to 25 characters, enter  maxLength: 25.
maximumLargest allowed value for a number or integer.
minItemsMinimum number of selectable items within an array.
minLengthMinimum number of characters allowed for a string.
minimumSmallest allowed value for a number or integer.
oneOfAllows the user input form to display a friendly name (title) for a less friendly value (const). If setting a default value, set the const, not the title.Valid for use with types string, integer, and number.
patternAllowable characters for string inputs, in regular expression syntax.For example, '[a-z]+' or '[a-z0-9A-Z@#$]+'
propertiesDeclares the key:value properties block for objects.
readOnlyUsed to provide a form label only.
titleUsed with oneOf. The friendly name for a const value. The title appears on the user input form at deployment time.
typeData type of number, integer, string, Boolean, or object.Important:A Boolean type adds a blank checkbox to the request form. Leaving the box untouched does not make the input False.To set the input to False, users must check and then clear the box.
writeOnlyHides keystrokes behind asterisks in the form. Cannot be used with enum. Appears as a password field in Service Broker custom forms.

Explicit dependencies

Sometimes, a resource needs another to be deployed first. For example, a database server might need to exist first, before an application server can be created and configured to access it.

An explicit dependency sets the build order at deployment time, or for scale in or scale out actions. You can add an explicit dependency using the graphical design canvas or the code editor.

  • Design canvas option—draw a connection starting at the dependent resource and ending at the resource to be deployed first.
  • Code editor option—add a dependsOn property to the dependent resource, and identify the resource to be deployed first.An explicit dependency creates a solid arrow in the canvas.
Explicit dependency

Property bindings

Sometimes, a resource property needs a value found in a property of another resource. For example, a backup server might need the operating system image of the database server that is being backed up, so the database server must exist first.

Also called an implicit dependency, a property binding controls build order by waiting until the needed property is available before deploying the dependent resource. You add a property binding using the code editor.

  • Edit the dependent resource, adding a property that identifies the resource and property that must exist first.A property binding creates a dashed arrow in the canvas.
Implicit dependency or property binding

Encrypt access credentials

resources:
  apitier:
    type: Cloud.Machine
    properties:
      cloudConfig: |
        #cloud-config
        runcmd:
          - export apikey=${base64_encode(input.username:input.password)}
          - curl -i -H 'Accept:application/json' -H 'Authorization:Basic :$apikey' http://example.com

Passing inputs to ABXs

resources:
  db-tier:
    type: Cloud.Machine
    properties:
      # Command to execute
      abxRunScript_script: mkdir bp-dir
      # Time delay in seconds before the script is run
      abxRunScript_delay: 120
      # Type of the script: shell (Linux) or powershell (Windows)
      abxRunScript_shellType: linux
      # Could be aws, azure, etc.
      abxRunScript_endpointType: '${self.endpointType}'

Fetching value from Property Groups

appSize: '${propgroup.propGroup_name.const_size}'

Adding a secret property

type: Cloud.Machine
properties:
  name: ourvm
  image: mint20
  flavor: small
  remoteAccess:
    authentication: publicPrivateKey
    sshKey: '${secret.ourPublicKey}'
    username: root

Enum & oneOf (Dropdowns in Input form)

enum and oneOf both are used to provide a set of default values in the input form. However, the only difference in them is that oneOf allows for a friendly title.

inputs:
  osversion:
    type: string
    title: Select OS Version
    default: SLES12
    enum:
      - SLES12
      - SLES15
      - RHEL7
inputs:
  platform:
    type: string
    title: Deploy to
    oneOf:
      # Title is what the user sees, const is the tag for the endpoints.
      - title: AWS
        const: aws
      - title: Azure
        const: azure
      - title: vSphere
        const: vsphere
    default: vsphere

Dynamic Enums (binding with vRO action)

input1:
    type: string
    title: Environment
    default: ''
    $dynamicEnum: '/data/vro-actions/com.org.utils/getEnvironment?environment={{vmenvironment}}'
.
.
.
.
input1:
    type: string
    title: podId
    $dynamicEnum: /data/vro-actions/com.org.helpers/getPod

This is calling a vRO action getEnvironments inside a action module com.org.utils with 1 input environment. Actions with no inputs can be called with using quotes.

vRO Custom DataTypes

inputs:
  accountName:
    type: string
    title: Account name
    encrypted: true   
  displayName:
    type: string
    title: Display name   
  password:
    type: string
    title: Password
    encrypted: true 
  confirmPassword:
    type: string
    title: Password
    encrypted: true   
  ouContainer: 
    type: object
    title: AD OU container
    $data: 'vro/data/inventory/AD:OrganizationalUnit'
    properties:
        id:
            type: string
        type:
            type: string

Resource Flags

Cloud Assembly includes several cloud template settings that adjust how a resource is handled at request time. Resource flag settings aren’t part of the resource object properties schema. For a given resource, you add the flag settings outside of the properties section as shown. 

resources:
  Cloud_Machine_1:
    type: Cloud.Machine
    preventDelete: true
    properties:
      image: coreos
      flavor: small
      attachedDisks:
        - source: '${resource.Cloud_Volume_1.id}'
  Cloud_Volume_1:
    type: Cloud.Volume
    properties:
      capacityGb: 1

Available resource flags

Resource FlagDescription
allocatePerInstanceWhen set to true, resource allocation can be customized for each machine in a cluster.The default is false, which allocates resources equally across the cluster, resulting in the same configuration for each machine. In addition, day 2 actions might not be separately possible for individual resources.Per instance allocation allows count.index to correctly apply the configuration for individual machines. For code examples, see Machine and disk clusters in Cloud Assembly.
createBeforeDeleteSome update actions require that the existing resource be removed and a new one be created. By default, removal is first, which can lead to conditions where the old resource is gone but the new one wasn’t created successfully for some reason.Set this flag to true if you need to make sure that the new resource is successfully created before deleting the previous one.
createTimeoutThe Cloud Assembly default timeout for resource allocate, create, and plan requests is 2 hours (2h). In addition, a project administrator can set a custom default timeout for these requests, applicable throughout the project.This flag lets you override any defaults and set the individual timeout for a specific resource operation. See also updateTimeout and deleteTimeout.
deleteTimeoutThe Cloud Assembly default timeout for delete requests is 2 hours (2h). In addition, a project administrator can set a different default timeout for delete requests, applicable throughout the project.This flag lets you override any defaults and set the individual timeout for a specific resource delete operation. See also updateTimeout and createTimeout.
dependsOnThis flag identifies an explicit dependency between resources, where one resource must exist before creating the next one. For more information, see Creating bindings and dependencies between resources in Cloud Assembly.
dependsOnPreviousInstancesWhen set to true, create cluster resources sequentially. The default is false, which simultaneously creates all resources in a cluster.For example, sequential creation is useful for database clusters where primary and secondary nodes must be created, but secondary node creation needs configuration settings that connect the node to an existing, primary node.
forceRecreateNot all update actions require that the existing resource be removed and a new one be created. If you want an update to remove the old resource and create a new one, independent of whether the update would have done so by default, set this flag to true.
ignoreChangesUsers of a resource might reconfigure it, changing the resource from its deployed state.If you want to perform a deployment update but not overwrite the changed resource with the configuration from the cloud template, set this flag to true.
ignorePropertiesOnUpdateUsers of a resource might customize certain properties, and those properties might be reset to their original cloud template state during an update action.To prevent any properties from being reset by an update action, set this flag to true.
preventDeleteIf you need to protect a created resource from accidental deletion during updates, set this flag to true. If a user deletes the deployment, however, the resource is deleted.
recreatePropertiesOnUpdateUsers of a resource might reconfigure properties, changing the resource from its deployed state. During an update, a resource might or might not be recreated. Resources that aren’t recreated might remain with properties in changed states.If you want a resource and its properties to be recreated, independent of whether the update would have done so by default, set this flag to true.
updateTimeoutThe Cloud Assembly default timeout for update requests is 2 hours (2h). In addition, a project administrator can set a different default timeout for update requests, applicable throughout the project.This flag lets you override any defaults and set the individual timeout for a specific resource update operation. See also deleteTimeout and createTimeout.

cloudConfig (cloud-init & Cloudbase-Init)

You can add a cloudConfig section to Cloud Assembly template code, in which you add machine initialization commands that run at deployment time. cloudConfig command formats are:

  • Linux—initialization commands follow the open cloud-init standard.
  • Windows—initialization commands use Cloudbase-init.

Linux cloud-init and Windows Cloudbase-init don’t share the same syntax. A cloudConfig section for one operating system won’t work in a machine image of the other operating system.

To ensure correct interpretation of commands, always include the pipe character cloudConfig: | as shown. Learn more about cloud-config here.

cloudConfig: |
        #cloud-config
        repo_update: true
        repo_upgrade: all
        packages:
         - apache2
         - php
         - php-mysql
         - libapache2-mod-php
         - php-mcrypt
         - mysql-client
        runcmd:
         - mkdir -p /var/www/html/mywordpresssite && cd /var/www/html && wget https://wordpress.org/latest.tar.gz && tar -xzf /var/www/html/latest.tar.gz -C /var/www/html/mywordpresssite --strip-components 1
         - i=0; while [ $i -le 5 ]; do mysql --connect-timeout=3 -h ${DBTier.networks[0].address} -u root -pmysqlpassword -e "SHOW STATUS;" && break || sleep 15; i=$((i+1)); done
         - mysql -u root -pmysqlpassword -h ${DBTier.networks[0].address} -e "create database wordpress_blog;"
         - mv /var/www/html/mywordpresssite/wp-config-sample.php /var/www/html/mywordpresssite/wp-config.php
         - sed -i -e s/"define( 'DB_NAME', 'database_name_here' );"/"define( 'DB_NAME', 'wordpress_blog' );"/ /var/www/html/mywordpresssite/wp-config.php && sed -i -e s/"define( 'DB_USER', 'username_here' );"/"define( 'DB_USER', 'root' );"/ /var/www/html/mywordpresssite/wp-config.php && sed -i -e s/"define( 'DB_PASSWORD', 'password_here' );"/"define( 'DB_PASSWORD', 'mysqlpassword' );"/ /var/www/html/mywordpresssite/wp-config.php && sed -i -e s/"define( 'DB_HOST', 'localhost' );"/"define( 'DB_HOST', '${DBTier.networks[0].address}' );"/ /var/www/html/mywordpresssite/wp-config.php
         - service apache2 reload

If a cloud-init script behaves unexpectedly, check the captured console output in /var/log/cloud-init-output.log when troubleshooting. For more about cloud-init, see the cloud-init documentation.

Advertisements

Examples of vSphere resources

vSphere virtual machine with CPU, memory, and operating system

resources:
  demo-machine:
    type: Cloud.vSphere.Machine
    properties:
      name: demo-machine
      cpuCount: 1
      totalMemoryMB: 1024
      image: ubuntu

vSphere machine with a datastore resource

resources:
  demo-vsphere-disk-001:
    type: Cloud.vSphere.Disk
    properties:
        name: DISK_001
        type: 'HDD'
        capacityGb: 10
        dataStore: 'datastore-01'
        provisioningType: thick

vSphere machine with an attached disk

resources:
  demo-vsphere-disk-001:
    type: Cloud.vSphere.Disk
    properties:
      name: DISK_001
      type: HDD
      capacityGb: 10
      dataStore: 'datastore-01'
      provisioningType: thin
  demo-machine:
    type: Cloud.vSphere.Machine
    properties:
      name: demo-machine
      cpuCount: 2
      totalMemoryMB: 2048
      imageRef: >-
        https://packages.vmware.com/photon/4.0/Rev1/ova/photon-ova-4.0-ca7c9e9330.ova
      attachedDisks:
        - source: '${demo-vsphere-disk-001.id}'

vSphere machine with a dynamic number of disks

inputs:
  disks:
    type: array
    title: disks
    items:
      title: disks
      type: integer
    maxItems: 15
resources:
  Cloud_Machine_1:
    type: Cloud.vSphere.Machine
    properties:
      image: Centos
      flavor: small
      attachedDisks: '${map_to_object(resource.Cloud_Volume_1[*].id, "source")}'
  Cloud_Volume_1:
    type: Cloud.Volume
    allocatePerInstance: true
    properties:
      capacityGb: '${input.disks[count.index]}'
      count: '${length(input.disks)}'

vSphere machine from a snapshot image. Append a forward slash and the snapshot name. The snapshot image can be a linked clone.

resources:
  demo-machine:
    type: Cloud.vSphere.Machine
    properties:
      imageRef: 'demo-machine/snapshot-01'
      cpuCount: 1
      totalMemoryMB: 1024

vSphere machine in a specific folder in vCenter

resources:
  demo-machine:
    type: Cloud.vSphere.Machine
    properties:
      name: demo-machine
      cpuCount: 2
      totalMemoryMB: 1024
      imageRef: ubuntu
      resourceGroupName: 'myFolder'

vSphere machine with multiple NICs

resources:
  demo-machine:
    type: Cloud.vSphere.Machine
    properties:
      image: ubuntu
      flavor: small
      networks:
        - network: '${network-01.name}'
          deviceIndex: 0
        - network: '${network-02.name}'
          deviceIndex: 1
  network-01:
    type: Cloud.vSphere.Network
    properties:
      name: network-01
  network-02:
    type: Cloud.vSphere.Network
    properties:
      name: network-02

vSphere machine with an attached tag in vCenter

resources:
  demo-machine:
    type: Cloud.vSphere.Machine
    properties:
      flavor: small
      image: ubuntu
      tags:
        - key: env
          value: demo

vSphere machine with a customization spec

resources:
  demo-machine:
      type: Cloud.vSphere.Machine
      properties:
        name: demo-machine
        image: ubuntu
        flavor: small
        customizationSpec: Linux

vSphere machine with remote access

inputs:
  username:
    type: string
    title: Username
    description: Username
    default: testUser
  password:
    type: string
    title: Password
    default: VMware@123
    encrypted: true
    description: Password for the given username
resources:
  demo-machine:
    type: Cloud.vSphere.Machine
    properties:
      flavor: small
      imageRef: >-
        https://cloud-images.ubuntu.com/releases/16.04/release-20170307/ubuntu-16.04-server-cloudimg-amd64.ova
      cloudConfig: |
        ssh_pwauth: yes
        chpasswd:
          list: |
            ${input.username}:${input.password}
          expire: false
        users:
          - default
          - name: ${input.username}
            lock_passwd: false
            sudo: ['ALL=(ALL) NOPASSWD:ALL']
            groups: [wheel, sudo, admin]
            shell: '/bin/bash'
        runcmd:
          - echo "Defaults:${input.username}  !requiretty" >> /etc/sudoers.d/${input.username}
Advertisements

Market Place

The Marketplace provides VMware Solution Exchange cloud templates and images that help you build your template library and access supporting OVA or OVFs.

Source: https://learncloudassembly.github.io/Marketplace/Cloud-Templates/

Other ways to create Cloud Assembly templates

Cloud template cloning

To clone a template, go to Design, select a source, and click Clone. You clone a cloud template to create a copy based on the source, then assign the clone to a new project or use it as starter code for a new application.

Uploading and downloading

You can upload, download, and share cloud template YAML code in any way that makes sense for your site. You can even modify template code using external editors and development environments.

Note A good way to validate shared template code is to inspect it in the Cloud Assembly code editor on the design page.

Integrating Cloud Assembly with a repository

An integrated git source control repository can make cloud templates available to qualified users as the basis for a new deployment. See How do I use Git integration in Cloud Assembly.

References