Release 05.08.2021

* DataSphere: added instruction how to connect to S3 via UI.
* DataSphere: added information about datasets.
* Fixes and improvements.
* Translations updated.

en/_includes/compute/image-create-requirements.md

@@ -0,0 +1,12 @@
+Boot disk images must meet the following requirements:
+
+* The `virtio-net` and `virtio-blk` drivers are installed. See the [instructions](../../compute/operations/image-create/custom-image.md#virtio).
+* The `ttyS0` terminal (COM1 port) is set up as a [serial console](../../compute/operations/serial-console/index.md). See the [instructions](../../compute/operations/image-create/custom-image.md#serial-console).
+* The network interface runs correctly when starting a VM and obtains the IP address via DHCP.
+* The `cloud-init` package is installed and configured to work with our [metadata service](../../compute/operations/vm-info/get-info.md#inside-instance). To install the package for CentOS, run the `sudo yum install cloud-init` command. To install it for Debian or Ubuntu, run `sudo apt update && sudo apt install -y cloud-init`.
+* In the system firewall settings, the minimum required set of ports for running your applications and a port for SSH access (by default, this is port 22 TCP) are open.
+* The SSH server starts automatically at VM startup.
+* Services running your application are resistant to VM reboots.
+* GPT disk partitioning is used.
+* The disk is mounted by its UUID rather than by name.
+

en/compute/operations/image-create/custom-image.md

@@ -0,0 +1,180 @@
+---
+
+__system: {"dislikeVariants":["No answer to my question","Recomendations didn't help","The content doesn't match title","Other"]}
+---
+# Preparing your disk image
+
+You can use your own file with a Linux-based VM disk image. After preparing your image, [upload it](upload.md) to {{ compute-name }}.
+
+If you made software that might be helpful to other others, [offer](../../../marketplace/operations/create-product.md) it in {{ marketplace-name }}.
+
+## Configure the OS to meet the requirements {#requirements}
+
+{% include [image-create-requirements](../../../_includes/compute/image-create-requirements.md) %}
+
+### Install the virtio drivers {#virtio}
+
+To upload your image successfully, make sure to install the `virtio-blk` and `virtio-net` drivers. When uploading it from `initramfs`, you need at least the `virtio-blk` driver. To upload the image completely, both the `virtio-blk` and `virtio-net` are required.
+
+Most modern distributions contain the `virtio` drivers by default. They can be compiled as separate `.ko` files or be part of the kernel itself.
+
+Follow the instructions below to check if the drivers are installed and, if not, add them.
+
+1. Find out if the drivers are included in the kernel configuration:
+
+   {% cut "To find out" %}
+
+   Run the command:
+
+   ```sh
+   grep -E -i "(VIRTIO_BLK|VIRTIO_NET)" /boot/config-$(uname -r)
+   ```
+
+   If no lines starting with `CONFIG_VIRTIO_BLK=` and `CONFIG_VIRTIO_NET=` are displayed, you should recompile the Linux kernel with the virtio drivers. Otherwise, proceed to the next steps.
+
+   {% endcut %}
+
+1. If the `CONFIG_VIRTIO_BLK=y` and `CONFIG_VIRTIO_NET=y` lines are returned in step 1, check that the drivers are included in the kernel:
+
+   {% cut "To check drivers in the kernel" %}
+
+   Run the command:
+
+   ```sh
+   grep -E "(virtio_blk|virtio_net)" /lib/modules/"$(uname -r)"/modules.builtin
+   ```
+   * If the lines with the `virtio_net.ko` and `virtio_blk.ko` filenames are returned, the drivers are part of the kernel and you don't need to install them.
+   * If not, recompile the Linux kernel with the virtio drivers.
+
+   {% endcut %}
+
+1. If the `CONFIG_VIRTIO_BLK=m` and `CONFIG_VIRTIO_NET=m` lines are returned in step 1, check that the drivers are installed as kernel modules:
+
+   {% cut "To check kernel modules" %}
+
+   {% list tabs %}
+
+   - CentOS, Fedora
+
+     Run the following command:
+
+     ```sh
+     sudo lsinitrd /boot/initramfs-$(uname -r).img | grep -E "(virtio_blk|virtio_net)"
+     ```
+
+     * If the lines with the `virtio_net.ko.xz` and `virtio_blk.ko.xz` filenames are returned, the drivers are installed as kernel modules.
+
+     * If not, create a backup of the `initramfs` file and install the drivers:
+
+       ```sh
+       sudo cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak
+       sudo mkinitrd -f --with=virtio_blk --with=virtio_net /boot/initramfs-$(uname -r).img $(uname -r)
+       ```
+
+       Then restart the OS and check that the drivers appear in the `initramfs` file and are loaded:
+
+       ```sh
+       sudo lsinitrd /boot/initramfs-$(uname -r).img | grep -E "(virtio_blk|virtio_net)"
+       find /lib/modules/"$(uname -r)"/ -name "virtio*" | grep -E "(blk|net)"
+       ```
+
+       After running each of the commands, the lines with the `virtio_net.ko.xz` and `virtio_blk.ko.xz` filenames should be returned.
+
+   - Debian, Ubuntu
+
+     Run the following command:
+
+     ```sh
+     lsinitramfs /boot/initrd.img-$(uname -r) | grep -E "(virtio_blk|virtio_net)"
+     ```
+
+     * If the lines with the `virtio_net.ko` and `virtio_blk.ko` filenames are returned, the drivers are installed as kernel modules.
+
+     * If not, install the drivers:
+
+       ```sh
+       echo -e "virtio_blk\nvirtio_net" | sudo tee -a /etc/initramfs-tools/modules
+       sudo update-initramfs -u
+       ```
+
+       Then restart the OS and check that the drivers appear in the `initrd` file and are loaded:
+
+       ```sh
+       lsinitramfs /boot/initrd.img-$(uname -r) | grep -E "(virtio_blk|virtio_net)"
+       find /lib/modules/"$(uname -r)"/ -name "virtio*" | grep -E "(blk|net)"
+       ```
+
+       After running each of the commands, the lines with the `virtio_net.ko` and `virtio_blk.ko` filenames should be returned.
+
+   {% endlist %}
+
+   {% endcut %}
+
+### Configure the serial console {#serial-console}
+
+The serial console allows you to access your VM regardless of the network or OS status. Use the serial console, for example, for troubleshooting VM issues or when there are problems with SSH access. For more information, see [{#T}](../serial-console/index.md).
+
+To connect to your VM using the serial console, set up the `ttyS0` terminal (COM1 port) as a system console for your image:
+
+1. In the `/etc/default/grub` file with GRUB settings, add `console=ttyS0` to the `GRUB_CMDLINE_LINUX` parameter value. The line with this parameter should look like this:
+
+   ```sh
+   GRUB_CMDLINE_LINUX="foo=bar baz console=ttyS0"
+   ```
+
+1. Check that the `/etc/securetty` file with a list of terminals that the `root` user can log in to the system through contains the `ttyS0` line. If the file doesn't exist, create it.
+
+   {% note info %}
+
+   In some operating systems, the `/etc/securetty` file is disabled and the `root` user is allowed to access all terminals by default. To configure access using this file, add to the `/etc/pam.d/login` file a pointer to the `pam_securetty.so` module. For more information, see `man pam_securetty` and `man pam.d`.
+
+   {% endnote %}
+
+1. Run the following commands:
+
+   {% list tabs %}
+
+   - CentOS, Fedora
+
+     ```sh
+     sudo stty -F /dev/ttyS0 9600                 # Sets the recommended baud rate for the ttyS0 terminal at 9600 
+     sudo grub2-mkconfig -o /boot/grub2/grub.cfg  # Generates a configuration file for GRUB
+     sudo systemctl start getty@ttyS0             # Starts getty on the ttyS0 terminal
+     sudo systemctl enable getty@ttyS0            # Specifies that getty should be run every time the OS is started
+     ```
+
+   - Debian, Ubuntu
+
+        ```sh
+        sudo stty -F /dev/ttyS0 9600        # Sets the recommended baud rate for the ttyS0 terminal at 9600 
+        sudo update-grub                    # Generates a configuration file for GRUB
+        sudo systemctl start getty@ttyS0    # Starts getty on the ttyS0 terminal
+        sudo systemctl enable getty@ttyS0   # Specifies that getty should be run every time the OS is started
+        ```
+
+   {% endlist %}
+
+1. Restart the OS.
+
+After [creating a VM from your image](upload.md#create-vm-from-user-image), you should additionally [configure it to work with the serial console](../serial-console/index.md).
+
+## Install drivers to enable compatibility with GPUs {#gpu-drivers}
+
+If working with a VM requires a [GPU](../../concepts/gpus.md), [install NVIDIA drivers](../vm-operate/install-nvidia-drivers.md) while preparing the image.
+
+## Create an image file {#create-image-file}
+
+Supported formats: Qcow2, VMDK, and VHD.
+
+We recommend that you use Qcow2 format with an optimized cluster size for faster import. To convert your image from other formats, use the `qemu-img` utility:
+
+```
+qemu-img convert -p -O qcow2 -o cluster_size=2M <name of your image file> <name of the new image file>
+```
+
+{% note info %}
+
+Don't use file compression or archiving software when preparing the image file.
+
+{% endnote %}
+

en/compute/operations/image-create/upload.md

@@ -11,53 +11,30 @@ keywords:
 __system: {"dislikeVariants":["No answer to my question","Recomendations didn't help","The content doesn't match title","Other"]}
 ---
 
-
 # Uploading a disk image to {{ yandex-cloud }}
 
-The following instructions describe how to prepare a disk image, upload the image file to {{ objstorage-full-name }}, and use it to create an [image](../../concepts/image.md) in {{ compute-name }}.
+The following instructions describe how to upload an image file with Linux OS to {{ objstorage-full-name }} and use it to create an [image](../../concepts/image.md) and VM in {{ compute-name }}.
+
+Popular virtualization systems are supported.
 
 {% note warning %}
 
 In {{ compute-name }}, you can only create images using links to files uploaded to {{ objstorage-name }}.
 
 {% endnote %}
 
-## Prepare an image {#prepare-file}
-
-Supported image formats: Qcow2, VMDK, and VHD.
-
-Boot disk images must meet the following requirements:
-
-* The OS is Linux-based.
-* The latest OS updates are installed.
-* The disk is mounted by its UUID rather than by name.
-* The Linux kernel is running with the `console=ttyS0` parameter.
-* The SSH server starts automatically at VM startup.
-* The network interface obtains the IP address via DHCP.
-* The `cloud-init` package and `virtio-net` and `virtio-blk` drivers are installed.
+## Prepare an image file {#prepare-file}
 
-Recommendations:
+{% include [image-create-requirements](../../../_includes/compute/image-create-requirements.md) %}
 
-* Optimize images before uploading them by using the `qemu-img` utility to import faster:
+For setup instructions, see [{#T}](custom-image.md).
 
-  ```
-  qemu-img convert -p -O qcow2 -o cluster_size=2M <name of your image file> <name of the new image file>
-  ```
-
-* For the image to be compatible with [GPU](../../concepts/gpus.md), [install NVIDIA drivers](../vm-operate/install-nvidia-drivers.md) while preparing the file.
+## Upload an image file to {{ objstorage-name }} {#upload-file}
 
-{% note info %}
-
-Don't use file compression or archiving software when preparing the image file.
-
-{% endnote %}
-
-## Upload an image to {{ objstorage-name }} {#upload-file}
-
-Upload your image to {{ objstorage-full-name }} and get a link to the uploaded image:
+Upload your image to {{ objstorage-name }} and get a link to the uploaded image:
 
 1. If you don't have a bucket in {{ objstorage-name }}, [create](../../../storage/operations/buckets/create.md) one.
-1. [Upload the image](../../../storage/operations/objects/upload.md) to your bucket. In {{ objstorage-name }} terms, the uploaded image is called an _object_.
+1. Upload the image to your bucket, for example, [via the management console](../../../storage/operations/objects/upload.md), using the [AWS CLI](../../../storage/tools/aws-cli.md) or [WinSCP](../../../storage/tools/winscp.md). In {{ objstorage-name }} terms, the uploaded image is called an _object_.
 1. [Get a link](../../../storage/operations/objects/link-for-download.md) to the uploaded image. Use this link when creating an image in {{ compute-name }}.
 
 ## Create an image in {{ compute-name }} {#create-image}
@@ -78,7 +55,7 @@ Create a new image from the link obtained in {{ objstorage-name }}:
 
   1. Enter the image name.
 
-      {% include [name-format](../../../_includes/name-format.md) %}
+     {% include [name-format](../../../_includes/name-format.md) %}
 
   1. If necessary, add a description of the image.
 
@@ -90,42 +67,42 @@ Create a new image from the link obtained in {{ objstorage-name }}:
 
   To create a new image via the link, use the flag `--source-uri`.
 
-  ```
-  yc compute image create --name <IMAGE-NAME> --source-uri <IMAGE-URL>
+  ```bash
+  yc compute image create --name <image-name> --source-uri <image-URL>
   ```
 
-  where:
-  - `<IMAGE-NAME>` is the name to assign to the image.
-  - `<IMAGE-URL>` is the link to the image obtained in {{ objstorage-name }}.
+  Where:
+  * `<image-name>` is the name to assign to the image.
+  * `<image-URL>` is the link to the image obtained in {{ objstorage-name }}.
 
   If necessary, add a description and specify the [family](../../concepts/image.md#family) that the image belongs to:
 
-  ```
-  yc compute image create  \
-      --name ubuntu-cosmic \
-      --description "Ubuntu Server 18.10 (Cosmic Cuttlefish)" \
-      --family ubuntu \
-      --source-uri "https://storage.yandexcloud.net/mybucket/cosmic-server-cloudimg-amd64.vmdk"
+  ```bash
+  yc compute image create \
+    --name ubuntu-cosmic \
+    --description "Ubuntu Server 18.10 (Cosmic Cuttlefish)" \
+    --family ubuntu \
+    --source-uri "https://storage.yandexcloud.net/mybucket/cosmic-server-cloudimg-amd64.vmdk"
   ```
 
   If you know the minimum requirements for the size of a disk that will be created from this image, specify the size in GB:
 
-  ```
-  yc compute image create  \
-      --name big-image \
-      --min-disk-size 20 \
-      --source-uri "https://storage.yandexcloud.net/mybucket/cosmic-server-cloudimg-amd64.vmdk"
+  ```bash
+  yc compute image create \
+    --name big-image \
+    --min-disk-size 20 \
+    --source-uri "https://storage.yandexcloud.net/mybucket/cosmic-server-cloudimg-amd64.vmdk"
   ```
 
   {% include [min-disk-size](../../_includes_service/min-disk-size.md) %}
 
 - API
 
-  To create a new image via the link, use the [Create](../../api-ref/Image/create.md) method for the `Image` resource. Pass the link to the image in the `uri` element.
+  To create a new image via the link, use the [Create](../../api-ref/Image/create.md) method for the `Image` resource. Pass the link to the image in the `uri` parameter.
 
 - Terraform
 
-  If you don't have Terraform yet, [install it and configure the {{ yandex-cloud }} provider](../../../solutions/infrastructure-management/terraform-quickstart.md#install-terraform).
+  If you don't have Terraform, [install it and configure the {{ yandex-cloud }} provider](../../../solutions/infrastructure-management/terraform-quickstart.md#install-terraform).
 
   To create an image:
 
@@ -135,9 +112,8 @@ Create a new image from the link obtained in {{ objstorage-name }}:
 
      ```
      resource "yandex_compute_image" "image-1" {
-     
-       name       = "ubuntu-cosmic"
-       os_type    = "LINUX"
+       name = "ubuntu-cosmic"
+       os_type = "LINUX"
        source_url = "<link to the image in {{ objstorage-name }}>"
      }
      ```
@@ -150,23 +126,23 @@ Create a new image from the link obtained in {{ objstorage-name }}:
 
      1. Run the check using the command:
 
-        ```
+        ```bash
         terraform plan
         ```
 
-     If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.
+       If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.
 
   1. Deploy the cloud resources.
 
-     1. If the configuration doesn't contain any errors, run the command:
+     1. Run the command:
 
-        ```
+        ```bash
         terraform apply
         ```
 
      1. Confirm that you want to create the resources.
 
-     Afterwards, all the necessary resources are created in the specified folder. You can check resource availability and their settings in [management console]({{ link-console-main }}).
+     Afterwards, all the necessary resources are created in the specified folder. You can check resource availability and their settings in the [management console]({{ link-console-main }}).
 
 {% endlist %}
 
@@ -181,3 +157,4 @@ If the image was created successfully, you can [delete the image file](../../../
 {% include notitle [Creating a VM from the prepared image](../../operations/vm-create/create-from-user-image.md#create-vm-from-image) %}
 
 For information about {{ objstorage-name }} pricing, see [{#T}](../../../storage/pricing.md).
+

en/compute/operations/index.md

@@ -78,6 +78,7 @@ __system: {"dislikeVariants":["No answer to my question","Recomendations didn't
 ## Creating new images {#image-create}
 
 * [{#T}](image-create/upload.md)
+* [{#T}](image-create/custom-image.md)
 
 ## Managing images {#image-control}