kiwi-doc-ec2.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
                         "http://www.docbook.org/xml/4.5/docbookx.dtd">

<chapter id="chap.ec2">
  <title>EC2 Image — Amazon Elastic Compute Cloud</title>
  <indexterm>
    <primary>KIWI</primary>
    <secondary>EC2 image</secondary>
  </indexterm>
  <indexterm>
    <primary>images</primary>
    <secondary>EC2</secondary>
  </indexterm>
  <indexterm>
    <primary>Amazon Elastic Compute Cloud</primary>
    <see>EC2 image</see>
  </indexterm>
  <indexterm>
    <primary>EC2 images</primary>
  </indexterm>
  <para>The <trademark>Amazon Elastic Compute Cloud</trademark> (Amazon
    EC2) provides an environment known as 
    <ulink url="http://en.wikipedia.org/wiki/IAAS">
      <citetitle>IaaS</citetitle></ulink> (Infrastructure as a Service).
    In this environment you have the ability to run Virtual
    Machines (VMs) on hardware managed by Amazon and the virtualization
    infrastructure provided by Amazon.</para>

  <para>The virtualization infrastructure for EC2 is setup to work with
    Amazon Machine Images (AMIs). There are two storage models for AMIs:
    <orderedlist>
      <listitem>
        <simpara>
          <ulink url="http://aws.amazon.com/s3/">
             <citetitle>S3</citetitle></ulink> (Simple Storage Service)
          backed AMI
        </simpara>
      </listitem>
      <listitem>
        <simpara>
          <ulink url="http://aws.amazon.com/ebs/">
            <citetitle>EBS</citetitle></ulink> (Elastic Block Store)
          backed AMI 
        </simpara>
      </listitem>
    </orderedlist>
    The image created with KIWI can be used to create an AMI for both
    storage models. For an S3 backed AMI a bundle with a manifest XML file
    is required. The bundle can be created using the ec2-ami-tools provided
    by Amazon in a post processing step using the image created by KIWI.
    </para>

  <para>For an EBS backed AMI the procedure to get to a working AMI
    requires more manual steps when compared to the S3 backed AMI approach.
    The KIWI created image needs to be uploaded to EC2 and then it needs to
    be dumped to an EBS volume. This implies that you need to have a running
    AMI in EC2</para>

  <para>The procedures to handle both storage options are outlined
    below.</para>

  <para>You can work with EC2 using the Amazon Web application found
    at <ulink url="http://aws.amazon.com"/> or you
    can use the Amazon provided command line tools. In this example we will
    exclusively interact with EC2 using the command line tools. The command
    line tools are divided into <emphasis>AMI</emphasis> and
    <emphasis>API</emphasis> tools. The AMI tools are designed
    to operate on images, while the API tools are designed to work with the
    Amazon <ulink url="http://en.wikipedia.org/wiki/Representational_state_transfer">
      <citetitle>REST API</citetitle></ulink>. In order
    for KIWI to create the bundle for S3 backed AMIs the Amazon AMI tools
    must be installed. It is recommended that you install both, the AMI
    and API tools on your build system.</para>

  <para>The Amazon tools are not distributed with KIWI and can be
    installed using packages from the openSUSE Build Service <ulink
      url="http://download.opensuse.org/repositories/Cloud:/EC2/"><citetitle>Cloud:EC2</citetitle></ulink> repository, or can be downloaded from Amazon at 
    <ulink url="http://aws.amazon.com/developertools/368"/> and
    <ulink url="http://aws.amazon.com/cli/"/>.</para>

  <para>Documentation for Amazon EC2 can be found at <ulink
        url="http://aws.amazon.com/documentation/ec2/"/>. The documentation for the command line tools may be
     accessed at <ulink url="http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference"/>
    and <ulink url="http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference"/>
    All commands also support the customary <option>--help</option>
    command line option to display the supported command line arguments
    for the given command.</para>

  <para>When working with the Amazon AMI tools it is useful to set the
    EC2_HOME, EC2_PRIVATE_KEY, and EC2_CERT environment variables. Setting
    EC2_PRIVATE_KEY, and EC2_CERT allows you to forego specification of the
    <option>--private-key</option> and
    <option>--cert</option> command line options with every
    command. The EC2_HOME environment variable is used by the tools to
    find required libraries. This also transfers to ec2-api-tools. Using
    the aws-cli command line tools one first needs to run
    <command>aws configure</command>. Using the aws-cli command
    line tools has the advantage that these tool provide a consistent
    interface across many Amazon services, while the *-api-tools have
    separate downloads and installs for each service. The aws-cli tools do
    not take the environment variables into consideration and provide a
    <option>--profile</option> argument to allow you to manage
    multiple accounts. Something that is more challenging with the *-api-tools.
    <itemizedlist>
      <listitem><para>EC2_HOME</para>
        <para>Location of the bin and lib directories installed by the
            Amazon tools. A good location for the tools on your
            system is <filename>/usr/local</filename>.
        </para>
      </listitem>
      <listitem><para>EC2_PRIVATE_KEY</para>
        <para>Path to your private key file (including the filename). For
            example <filename>/home/USERNAME/AWS/keys/pk-....pem</filename>
        </para>
      </listitem>
      <listitem><para>EC2_CERT</para>
        <para>Path to your certificate file (including the filename). For
            example <filename>/home/USERNAME/AWS/keys/cert-....pem</filename>
        </para>
      </listitem>
    </itemizedlist>
  </para>
  <para>Please note that your account will be billed by Amazon at the 
   published rate for any computing resources you consume in EC2. This
   includes but is not limited to, running instances, storing data
   (your image) on S3 or EBS, and network traffic.</para>

  <para>One final remark before we get started, the default region for any
    <command>ec2-</command> command that communicates with the
    REST API or sends files to EC2 is the US-East region, i.e. us-east-1.
    Therefore, if you want to upload any data to other EC2 regions you must
    specify the desired target region. Specifying a region is accomplished
    by setting the EC2_URL environment variable, by using the
    <option>--url</option> command line option, or by using
    the <option>--region</option> argument. The
    <option>--region</option> argument is used for the aws-cli
    tools. The EC2-URL environment variable and the
    <option>--url</option> argument expect a value in the form
    <replaceable>https://ec2.amazonaws.com</replaceable>
    (us-east-1). The <option>--region</option> argument expects
    the name of a region as returned by the
    <command>aws ec2 describe-regions</command> command.
  </para>
    
  <sect1 id="sec.ec2.building">
    <title>Building the suse-ec2-guest Example</title>
    <para>The example provided with KIWI uses openSUSE as the base
      distribution and includes the base pattern plus the vim editor. Also
      included is the suse-ami-tools package that provides tools needed
      in the EC2 environment.
    </para>

    <para>Lets assume you copied the example configuration directory to
      <filename class="directory">/tmp</filename>
      such that you could add packages to the
      <filename>config.xml</filename> file without
      modifying the original example provided by KIWI.</para>
 
      <screen><command>kiwi</command> --prepare <!--
        -->/tmp/suse-ec2-guest --root /tmp/myec2</screen>

      <screen><command>kiwi</command> --create <!--
        -->/tmp/myec2 -d /tmp/myec2-result -y</screen>
  </sect1>

  <sect1 id="sec.ec2.using">
    <title>Using EC2 and the created image</title>
    <para>The file that serves as the basis for the AMI is the 
      <emphasis>.raw</emphasis> file. This is the disk image
      and is used for both S3 and EBS backed AMIs. The file is found
      in your destination directory,
      <filename class="directory">/tmp/myec2-result</filename>, if you followed the commands given above. Prior to describing
      the specifics about using the KIWI produced images the following
      section will address some rudimentary general EC2 concepts and
      commands that can be used with existing AMIs and the AMIs you can
      register with the KIWI created images.</para>

    <sect2 id="sec.ec2.using.general">
      <title>Using a registered AMI</title>
      <para>This section is not a replacement for the EC2 documentation
        mentioned earlier. We will only cover the concepts and commands
        necessary to get you started such that you can launch the KIWI
        created image in this example.</para>

      <para>Prior to launching any instance in EC2 you need to have a
        key-pair. If you do not already have a key-pair in EC2 you can create
        one using <command>aws ec2 create-key-pair --key-name</command> command. This creates a public/private key-pair that is used to grant
        you access to your running instance via the ssh tools. Generate the
        key-pair as shown below, the <replaceable>gsgkey</replaceable> name is arbitrary and used in this example, you can choose any
        name you like. The use of the key is quite frequent. Therefore, you
        probably want to choose a name that is easy to remember and not too
        terribly long to type.</para>
  
      <screen><command>aws ec2 create-key-pair --key-name</command> <replaceable>gsgkey</replaceable> </screen>
  
      <para>Save the private key returned by the command in a local file.
        Using your favorite text editor, paste everything between
        (and including) the <literal>-----BEGIN RSA PRIVATE KEY-----</literal>
        and <literal>-----END RSA PRIVATE KEY-----</literal>
        lines into your editor and save the key to a file. The file can have
        any name. However, it makes sense to name the file after the key-pair
        name you have chosen earlier. If the file is named differently from
        the key-pair you will end up launching instances with
        <option>--key-name</option> <replaceable>mykey</replaceable> and then accessing the instance with
        <option>-i</option> <replaceable>yourkey</replaceable>, which may be a bit weird. As indicated by the heading,
        this is your private key, thus make sure you safe guard it
        appropriately. On Linux the ssh tools will complain if the key
        file does not have the proper permissions. Change the permissions
        of your private key file to be read-write by you, the owner, only.

        <screen><command>chown</command> 600 <!--
          --><replaceable>gsgkey</replaceable> </screen>

        The public key of your key-pair is stored in the EC2 infrastructure.
        EC2 allows you to have multiple key-pairs, to review your existing
        key-pairs use the <command>aws ec2 describe-key-pairs</command> command.
      </para>
  
      <para>When you launch an instance of an AMI you must specify a
        key-pair name. This selects the public key to be injected into the
        instance. The key injection occurs through the <filename>amazon</filename> init script provided by the
        <filename>suse-ami-tools</filename> package. This
        package, as mentioned previously, is already included
        in the example's <filename>config.xml</filename> file.
        Do not forget to include this package when you create your own image
        descriptions for EC2 or you will not be able to log into your running
        instances. Additionally you need to activate this service by adding
        <emphasis>suseInsertService amazon</emphasis> in your
        <filename>config.sh</filename> file.</para>
  
      <para>The key injection mechanism needs to access the network.
        Therefore, you must configure the network when you build your image.
        Configuration of the network can be accomplished through the overlay
        mechanism or via commands in config.sh. The network interface of a
        guest in EC2 is always eth0 and it needs to be configured to use DHCP.
        In the example the overlay mechanism is used to setup the network
        configuration.</para>

      <para>Note that the naming of network devices changed to a
        persistent naming scheme based on location of the device. While
        this naming scheme provides persistent names on a given system
        across reboots, with the underlying assumption that a network
        device would not be moved to a different slot on real hardware,
        this makes it more difficult to configure the network for
        machines with unknown topology. Therefore, kiwi examples for
        effected distributions inject a udev rule in config.sh.</para>

      <para>Another prerequisite to launching an instance in EC2 is knowing
        the AMI you want to instantiate. The
        <command>aws ec2 describe-images</command> command
        will provide information about all publicly available AMIs, a rather
        lengthy list. Use the <option>--filters</option>
        option or other qualifiers to reduce the list to a manageable size.
      </para>

      <para>The Amazon EC2 infrastructure uses <ulink
        url="http://www.linode.com/wiki/index.php/PV-GRUB">
        <citetitle>PVGrub (Para-Virtual Grub)</citetitle></ulink> to boot instances of an AMI. This allows instances to run the kernel
        that is part of the AMI, rather than some kernel provided by the
        Amazon infrastructure. However, an Amazon provided kernel is still
        required to kick things off and in the startup process PVGrub
        eventually picks up the <filename>/boot/grub/menu.lst</filename> file in your image and then boots the kernel specified. Note, that
        during the boot process you do not have access to a console and thus
        it makes no sense to have multiple kernel entries in your
        <filename>menu.lst</filename> file. Without console
        access you do not have an opportunity to choose a kernel. The kernel
        command line options are important, please refer to the examples to
        see the required options for EC2 images. Each EC2 region has it's own
        independent copy of this boot mechanism and the boot mechanism is
        differentiated between 32 bit and 64 bit. The boot kernels are named
        with an ID that starts with the TLA (Three Letter Acronym)
        <emphasis>aki</emphasis> followed by a dash ("-") and
        a hex number.
        The <xref linkend="tab.ec2.kernel-image-ids" xrefstyle="select:title"/>
        table below provides guidelines for the selection of the boot kernel
        ID based on Region and image architecture.</para>
  
        <table id="tab.ec2.kernel-image-ids">
          <title>Amazon Kernel Image IDs</title>
          <tgroup cols="4">
            <colspec colwidth="1*"/>
            <colspec colwidth="1*"/>
            <colspec colwidth="1*"/>
            <colspec colwidth="4*"/>
            <thead>
              <row>
                <entry>Region</entry>
                <entry>AKI</entry>
                <entry>Arch</entry>
                <entry>Name</entry>
              </row>
            </thead>
            <tbody>
              <row>
                <entry>AP-Northeast</entry>
                <entry>aki-196bf518</entry>
                <entry>x86</entry>
                <entry><filename>ec2-public-images-ap-northeast-1/pv-grub-hd00-V1.04-i386.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>AP-Northeast</entry>
                <entry>aki-1f6bf51e</entry>
                <entry>x86-64</entry>
                <entry><filename>ec2-public-images-ap-northeast-1/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>AP-Southeast</entry>
                <entry>aki-563e7404</entry>
                <entry>x86</entry>
                <entry><filename>ec2-public-images-ap-southeast-1/pv-grub-hd00-V1.04-i386.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>AP-Southeast</entry>
                <entry>aki-5e3e740c</entry>
                <entry>x86-64</entry>
                <entry><filename>ec2-public-images-ap-southeast-1/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>AP-Southeast2</entry>
                <entry>aki-c162fffb</entry>
                <entry>x86</entry>
                <entry><filename>ec2-public-images-ap-southeast-1/pv-grub-hd00-V1.04-i386.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>AP-Southeast2</entry>
                <entry>aki-3b1d8001</entry>
                <entry>x86-64</entry>
                <entry><filename>ec2-public-images-ap-southeast-1/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>EU-West</entry>
                <entry>aki-5ea34529</entry>
                <entry>x86</entry>
                <entry><filename>ec2-public-images-eu/pv-grub-hd00-V1.04-i386.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>EU-West</entry>
                <entry>aki-58a3452f</entry>
                <entry>x86-64</entry>
                <entry><filename>ec2-public-images-eu/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>SA-East</entry>
                <entry>aki-5753f44a</entry>
                <entry>x86</entry>
                <entry><filename>ec2-public-images-sa/pv-grub-hd00-V1.04-i386.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>SA-East</entry>
                <entry>aki-5153f44c</entry>
                <entry>x86-64</entry>
                <entry><filename>ec2-public-images-sa/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>US-East</entry>
                <entry>aki-659ccb0c</entry>
                <entry>x86</entry>
                <entry><filename>ec2-public-images/pv-grub-hd00-V1.04-i386.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>US-East</entry>
                <entry>aki-499ccb20</entry>
                <entry>x86-64</entry>
                <entry><filename>ec2-public-images/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>US-West</entry>
                <entry>aki-960531d3</entry>
                <entry>x86</entry>
                <entry><filename>ec2-public-images-us-west-1/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>US-West</entry>
                <entry>aki-920531d7</entry>
                <entry>x86-64</entry>
                <entry><filename>ec2-public-images-us-west-1/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>US-West2</entry>
                <entry>aki-e28f11d2</entry>
                <entry>x86</entry>
                <entry><filename>ec2-public-images-us-west-2/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
              <row>
                <entry>US-West2</entry>
                <entry>aki-e68f11d6</entry>
                <entry>x86-64</entry>
                <entry><filename>ec2-public-images-us-west-2/pv-grub-hd00-V1.04-x86_64.gz.manifest.xml</filename></entry>
              </row>
            </tbody>
          </tgroup>
        </table>
    
      <para>The information in the table above was extracted from the Amazon
        documentation found at: <ulink
        url="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/UserProvidedKernels.html"/>. As of version 1.04 of the akis the distinction between <emphasis>h0</emphasis> and <emphasis>h00</emphasis> is no longer relevant, they are in fact the same aki.
        </para>

      <para>AMIs in EC2 already have the aki ID embedded in their
        description. Therefore, you only need to know the aki ID when
        registering the image or when creating an S3 image bundle. The

      <screen><command>aws ec2 run-instances --image-id </command> <replaceable>IMAGE-ID</replaceable></screen>

        command is used to start an instance.</para>

      <para>Once the instance state for an instance is shown as
        <emphasis>running</emphasis> you can log into the
        instance using ssh as follows:</para>
  
      <screen><command>ssh</command> -i<!--
        --> <replaceable>PATH_TO_PRIVATE_KEY</replaceable> <!--
        -->root@<replaceable>PUBLIC_IP_OF_YOUR_INSTANCE</replaceable></screen>
  
      <para>The <replaceable>PUBLIC_IP_OF_YOUR_INSTANCE</replaceable> is
        displayed as part of the output of the
        <command>aws ec2 describe-instances</command> command.
        If you are unable to log in, it is most likely that the security
        setting for the instance is blocking the ssh access, or that you did
        not enable the ssh daemon process on startup. Your
        <filename>config.sh</filename> script should have
        a line activating the ssh daemon as follows,
        <emphasis>suseInsertService sshd</emphasis>. If you
        do not have this entry in <filename>config.sh</filename>
        you will have to rebuild your image. Addressing the problem with
        port blocking is accomplished using the
        <command>aws ec2 authorize-security-group-ingress</command> command.</para>
  
    </sect2>

    <sect2 id="sec.ec2.using.S3">
      <title>Using a bundle for an S3 backed AMI</title>
      <para>The destination directory,
        <filename class="directory">/tmp/myec2-result</filename>, if you followed the commands given above, contains the disk
        image, <emphasis>.raw</emphasis> file that we will use
        to create the image bundle. The ec2-ami-tools must be installed to
        create the image bundle. It is a good practice to keep the image
        bundle files and the kiwi results separated. Thus you may want to
        create a directory for the bundle. If you target different regions you
        want to have one directory per region. The following command will
        create an image bundle you can upload to an S3 bucket.

        <screen><command>ec2-bundle-image</command> <option>-k</option> <replaceable>PRIVATE_KEY</replaceable> <option>-c</option> <replaceable>CERT_FILE</replaceable> <option>-u</option> <replaceable>ACCOUNTNUMBER</replaceable> <option>-p</option> <replaceable>IMAGE_NAME</replaceable> <option>--block-device-mapping ami=sda,root=/dev/sda1</option> <option>--kernel</option> <replaceable>AKI_ID</replaceable> <option>-r</option> <replaceable>ARCHITECTURE</replaceable> <option>-i</option> <replaceable>PATH_TO_RAW_FILE</replaceable> <option>-d</option> <replaceable>PATH_TO_BUNDLE_DIR</replaceable></screen>
      </para>

      <para>The generated bundle needs to be transfered to Amazon using the
        <command>ec2-upload-bundle</command> command line tool.
        This tool is part of the Amazon AMI tools. Upload the AMI as follows,
        replacing <replaceable>AWS_Key_ID</replaceable> and
        <replaceable>AWS_secret_Key_ID</replaceable> with
        your Amazon key information. Also you may want to choose a different
        name for your bucket than <filename>myImages</filename>.
        If the bucket does not exist in S3 it will be created.
    
      <!-- formating is ugly here but looks reasonably good in the html-->
      <screen><command>ec2-upload-bundle</command> <!--
        --> -b <replaceable>myImages</replaceable> <!--
        --> -a <replaceable>AWS_Key_ID</replaceable> <!--
        --> -s <replaceable>AWS_secret_Key_ID</replaceable> \
   -m <replaceable>PATH_TO_MANIFEST_FROM_PREVIOUS_STEP</replaceable></screen>
      </para>

      <para>After the upload process is complete, register your image
        with the EC2 infrastructure using the
       <command>aws ec2 register-image</command>
        command as shown below. The result of the registration
        process is an AMI ID returned on the command line in the form
        <quote>ami-</quote> followed by a random key sequence.
        Use this AMI ID to launch your instance as described in the
        <xref  linkend="sec.ec2.using.general" xrefstyle="select:title"/>
        section.</para>

      <screen><command>aws ec2 register-image</command>
		--image-location <replaceable>myImages/MANIFEST_NAME</replaceable>
		<replaceable>ARCH</replaceable></screen>

      <para>This completes the S3 specific setup. The next section
        explains the use of the disk image file created by KIWI to create
        and EBS backed AMI.</para>
    </sect2>

    <sect2 id="sec.ec2.using.EBS">
      <title>Using the disk image for and EBS backed AMI</title>
      <para>For the EBS backed image we will also use the
        <emphasis>.raw</emphasis> file. Find this file in the
        destination directory, <filename class="directory">/tmp/myec2-result</filename>, if you followed the commands given above.
        We will use the raw disk image file to create a tarball to speed up
        the upload process. After unpacking the tarball in an instance in
        EC2 we will dump the image onto a volume with the
        <command>dd</command> command. Thus,
        <command>dd</command> must be available in the image
        you are running. You may also use the <command>dd_rescue</command> if itis available to dump the image. The SUSE Linux Enterprise
        AMIs available in EC2 have the
        <command>dd_rescue</command> command available. For the
        openSUSE AMIs available in EC2 it is easy to install the
        <command>dd_rescue</command> command by executing
        zypper as shown below.
      
        <screen><command>zypper</command> in dd_rescue</screen></para>

      <para>The first step in creating an EBS backed AMI is to create a
        tarball of the disk image file. This will significantly reduce your
        upload time and generate less network traffic. The following commands
        are executed on your build machine.</para>
      
        <screen><command>cd</command> /tmp/myec2-result</screen>
        <screen><command>tar</command> -cjf myImage.tar.bz *.raw</screen>
      
      <para>The tarball needs to be uploaded to EC2 and unpacked. This
        implies that the running instance of your chosen AMI needs to have
        sufficient space to store the tarball and the unpacked tarball.
        Therefore, it is recommended to create a storage volume as shown
        below. Some ami automatically create ephemaral storage that may be
        sufficiently big to hold the tarbal and image. For simplicity we
        will create our own storage volume. The command
        used will return information about the created volume including the
        a volume ID that you want to remember. This will
        be referred to as <replaceable>STORE_VOL_ID</replaceable> in this example. Execute this command on your local machine.
      
        <screen><command>aws ec2 create-volume</command> <option>--size</option> <replaceable>X</replaceable> <option>--availability-zone</option> <replaceable>AV_ZONE</replaceable></screen>
      
        The <replaceable>X</replaceable> is an integer
        value representing the size of the volume to be created in GB
        (Giga Bytes). The <replaceable>AV_ZONE</replaceable>
        value is one of the Amazon availability zones. For all commands
        presented here the <replaceable>AV_ZONE</replaceable>
        value must be the same. Possible values for <replaceable>AV_ZONE</replaceable> are obtained with the
        <command>aws ec2 describe-availability-zones</command></para>
      
      <para>The next step is to launch an instance of your liking in EC2.
        This instance will be used to create the EBS volume that will
        eventually function as the backing store for your AMI, therefore
        this instance must have the <command>dd</command> as
        described earlier. Launching the instance is accomplished with the
        <command>aws ec2 run-instances</command> command
        executed on your local machine.
      
        <screen><command>aws ec2 run-instances</command> <option>--image-id</option> <replaceable>AMI_ID</replaceable> <option>--key-name</option> <replaceable>SSH_KEY_NAME</replaceable> <option>--security-groups</option> <replaceable>SECURITY_GROUP_NAME</replaceable> <option>--placement</option> <replaceable>AvailabilityZone=AV_ZONE</replaceable> <option>--instance-type</option> <replaceable>INSTANCE_TYPE</replaceable></screen>
      
        The <replaceable>SECURITY_GROUP_NAME</replaceable>
        is optional, but it is best to have a group setup that allows ssh
        access by default to avoid having to open the port all the time as
        outlined earlier. The <replaceable>INSTANCE_TYPE</replaceable> specified with the <option>--instance-type</option>
        depends on the architecture of your image. The <command>aws ec2 run-instances</command> command returns information about the instance,
        including the instance ID, and you want to remember this ID. This ID
        will be referred to as <replaceable>INST_ID</replaceable> in this example.
      </para>
      
      <para>Wait until the instance is running, check the status with the
        <command>aws ec2 describe-instance-status</command>
        command. Once the instance is indicated as <emphasis>running</emphasis> attach the previously created volume to your instance,
        by executing the <command>aws ec2 attach-volume</command> 
        command on your local machine.
      
        <screen><command>aws ec2 attach-volume</command> <option>--volume-id</option> <replaceable>STORE_VOL_ID</replaceable> <option>--instance-id</option> <replaceable>INST_ID</replaceable> <option>--device</option> /dev/sdf</screen>
      
        The chosen device specified with the <option>--device</option> is arbitrary, however, you do obviously not want to pick a device node
        that is already in use.</para>

      <para>In a different shell login to the running instance as shown in
        the <xref  linkend="sec.ec2.using.general" xrefstyle="select:title"/> section.</para>

      <para>Check that the storage volume is attached using the
        <command>aws ec2 describe-volumes</command> command on
        your local machine. Once the volume is attached the "State" will be
        shown as <emphasis>in-use</emphasis>.</para>

      <para>With the EBS storage volume attached to your instance create a
        filesystem on the volume. There is no need to partition the volume.
        In the EC2 instance that you previously logged into execute the
        <command>mkfs</command> command.
      
        <screen><command>mkfs</command> -t ext3 /dev/sdf</screen>
      
        Once the filesystem creation is complete mount the volume.
      
        <screen><command>mount</command> dev/sdf /mnt</screen></para>

      <para>With the storage volume attched, formated and mounted you
        now have sufficient space to transfer the image tarball you created
        earlier. From the destination directory on your local machine that
        contains your tarball you can use sftp of scp to copy the tarball
        to your instance.
      </para>
      
        <screen><command>scp</command> <option>-i</option> <replaceable>PATH_TO_PRIVATE_KEY</replaceable> <replaceable>PATH_TO_THE_TARBALL</replaceable> root@<replaceable>PUBLIC_IP_OF_YOUR_INSTANCE</replaceable>:/mnt</screen>

      <para>While the image is transfering you can create a new EC2 volume.
        This new volume will become the backing store for your new AMI and
        be referred to as <replaceable>VOL_ID</replaceable>
        in this example. Use the <command>aws ec2 create-volume</command> command on your local machine as shown previously. This time you want
        to make sure that the specified size matches the size of your
        image file.
      
        <screen><command>aws ec2 create-volume</command> <option>--size</option> <replaceable>X</replaceable> <option>--availability-zone</option> <replaceable>AV_ZONE</replaceable></screen>
      
        It is good practice to use the <sgmltag class="element">size</sgmltag> element in your
        <filename>config.xml</filename> file when creating EC2
        images. Using the <sgmltag class="element">size</sgmltag> element ensures that you have additional space on your root volume
        and that you can match the volume size exactly to your image size.
     </para>

      <para>As previously, wait for the volume to be created and check the
        status with the <command>aws ec2 describe-volumes</command> command. When the volume creation process is complete, attach the new
        volume to your running instance by executing the
        <command>aws ec2 attach-volume</command> command on
        your local machine.

        <screen><command>aws ec2 attach-volume</command> <option>--volume-id</option> <replaceable>VOL_ID</replaceable> <option>--instance-id</option> <replaceable>INST_ID</replaceable> <option>--device</option>/dev/sdg</screen>

        Wait until the status for the volume changes to
        <emphasis>in-use</emphasis> before proceeding.
      </para>

      <para>By now the upload of your image tarball has probably
        completed and you can unpack the image on your storage volume in
        your EC2 instance. In the shell on your EC2 instance unpack the
        tarball as shown below.
      </para>

        <screen><command>cd</command> /mnt
<command>tar</command> <option>-xjf</option> <replaceable>myImage.tar.bz2</replaceable></screen>

      <para>After the unpack operation completes you can dump the
        image to your EBS volume that will be your backing store for your AMI,
        this volume is attached to /dev/sdg in this example. In the shell
        on your EC2 instance execute te <command>dd</command>
        command as shown below.

        <screen><command>dd</command> <option>if=</option> <replaceable>/mnt/IMAGE_FILE_NAME</replaceable> <option>of=</option> /dev/sdg <option>bs=32k</option></screen>

        You have to wait until the dump process is complete before proceeding
         to the next step.
      </para>

      <para>With the dump process complete you can now unmount the storage
        volume from your EC2 instance by executing the following in the
        shell of your running instance.

        <screen><command>umount</command> /mnt</screen>

        You can also exit the shell in your running EC2 instance as all
        remaining commands are executed on your local machine.</para>

      <para>You must detach the volume that is intended as your AMI backing
        store from your running instance. Detaching and deleting the storage
        volume as well as terminating the running instance are optional.

        <screen><command>aws ec2 detach-volume</command> <option>--volume-id</option> <replaceable>VOL_ID</replaceable> 

<command>aws ec2 detach-volume</command> <option>--volume-id</option> <replaceable>STORE_VOL_ID</replaceable></screen>

         Prior to shutting down the instance or deleting the storage volume,
         wait until the detach operation has completed. This is indicated by
         the <emphasis>available</emphasis> status in the
         output of the <command>aws ec2 describe-volumes</command> command.

         <screen><command>aws ec2 delete-volume</command> <option>--volume-id</option> <replaceable>STORE_VOL_ID</replaceable>

<command>aws ec2 terminate-instances</command> <option>--instance-ids</option> <replaceable>INST_ID</replaceable></screen></para>

      <para>Your next step is to create a snapshot of the EBS volume that
        contains your image. The <command>aws ec2 create-snapshot</command> command returns a snapshot ID that you want to remember and will be
        referred to as <replaceable>SNAP_ID</replaceable> in this example.

        <screen><command>aws ec2 create-snapshot</command> <option>--description</option><replaceable>A_SHORT_DESCRIPTION</replaceable> <option>--volume-id</option> <replaceable>VOL_ID</replaceable></screen>

        The process of creating the snapshot will take a while and depends on
        the size of your volume. Check the status of the snapshot creation
        process using the <command>aws ec2 describe-snapshots</command><option>--owner-ids  self</option> command. Ignoring the
        <option>--owner-ids</option> will create a long list of all
        available snapshots. When the process is complete the "Progress" will
        change to <emphasis>100%</emphasis>.</para>

      <para>Once the snapshot is complete you can register it as an AMI
        with the EC2 infrastructure using the <command>aws ec2 register-image</command> command.

        <screen><command>aws ec2 register-image</command> <option>--name</option> <replaceable>A_NAME</replaceable> <option>--description</option> <replaceable>A_DESCRIPTION</replaceable> <option>--architecture</option> <replaceable>ARCH</replaceable> <option>--kernel-id</option> <replaceable>BOOT_KERNEL_ID</replaceable> <option>--root-device-name</option> /dev/sda <option>--block-device-mappings</option> '[{"DeviceName":"/dev/sda","Ebs": {"SnapshotId": "<replaceable>SNAP_ID</replaceable>}}]'</screen>

        The <command>aws ec2 register-image</command> command
        will return the AMI ID that you can then use to launch your
        instance.</para>

      <para>The process of creating an EBS backed AMI is a bit tedious. If
        you create EBS backed AMIs more often it might be well worth your time
        to script this process using the Amazon REST API. The aws-cli
        command line interface is written in Python and you can import the
        various modules to incorporate functionality in your script. Another
        scripting option is to use the bot interface, another module
        implemented in Python. Other interfaces in other scripting languages
        are available.</para>

    </sect2>
  </sect1>
</chapter>