AWS

Overview

AWS is the Amazon public cloud, offering a full range of services and features across the globe in various datacenters. AWS provides businesses with a flexible, highly scalable, and low-cost way to deliver a variety of services using open standard technologies as well as proprietary solutions. This section of documentation will help you get Conduit and AWS connected to utilize the features below:

Features

  • Virtual Machine Provisioning
  • Containers
  • Backups / Snapshots
  • Resources Groups
  • Migrations
  • Auto Scaling
  • Load Balancing
  • AWS Marketplace Search and Provisioning
  • Remote Console
  • Periodic Synchronization
  • Lifecycle Management and Resize
  • Restore from Snapshots
  • EC2
  • RDS
  • S3
  • ELBs
  • ALBs
  • Route53
  • IAM Profile sync and assignment
  • Network Sync
  • Security Group Sync (selectable when provisioning, will not appear in Security Groups section)
  • Pricing Sync
  • Assign Elastic IP’s
  • Network Pools
  • MetaData Tag creation

Conduit can provide a single pane of glass and self-service portal for managing instances scattered across both AWS and private cloud offerings like VMWare and Hyper-V.

Requirements

AWS IAM Security Credentials
Access Key Secret Key Sufficient User Privileges (see MinimumIAMPolicies section for more info)
Security Group Configuration for Agent Install, Script Execution, and Remote Console Access

Typical Inbound ports open from Conduit Appliance: 22, 5985, 3389 Typical Outbound to Conduit Appliance: 80, 443

Note

These are required for Conduit agent install, communication, and remote console access for windows and linux. Other configurations, such as docker instances, will need the appropriate ports opened as well. Cloud-init Agent Install mode does not require incoming access for port 22.

Network(s)
IP assignment required for Agent install, Script Execution, and Console if the Conduit Appliance is not able to communicate with AWS instances private ip’s.

Note

Each AWS Cloud in Conduit is scoped to an AWS Region and VPC Multiple AWS Clouds can be added and even Grouped. Verify Security groups are properly configured in all Regions Conduit will scope to.

Adding an AWS Cloud

  1. Navigate to Infrastructure -> Clouds

  2. Select + Create Cloud

  3. Select AWS

  4. Enter the following:

    Name

    Name of the Cloud in Conduit

    Location

    Description field for adding notes on the cloud, such as location.

    Visibility

    For setting cloud permissions in a multi-tenant environment. Not applicable in single tenant environments.

    Region

    Select AWS Region for the Cloud

    Access Key

    Access Key ID from AWS IAM User Security Credentials.

    Secret Key

    Secret Access Key associate with the Access Key ID.

    Inventory
    Basic

    Conduit will sync information on all EC2 Instances in the selected VPC the IAM user has access to, including Name, IP Addresses, Platform Type, Power Status, and overall resources sizing for Storage, CPU and RAM, every 5 minutes. Inventoried EC2 Instances will appear as Unmanaged VM’s.

    Full

    In addition to the information synced from Basic Inventory level, Conduit will gather Resource Utilization metrics for Memory, Storage and CPU utilization per VM.

    Off

    Existing EC2 Instances will not be inventoried

    Note

    Cloud Watch must be configured in AWS for Conduit to collect Memory and Storage utilization metrics on inventoried EC2 instances.

  5. The AWS cloud is ready to be added to a group and saved. Additional configuration options available:

IMAGE TRANSFER STORE
S3 bucket for Image transfers, required for migrations into AWS.

Advanced Options

DOMAIN
Specify a default domain for instances provisioned to this Cloud.
SCALE PRIORITY
Specifies the priority with which an instance will scale into the cloud. A lower priority number means this cloud integration will take scale precedence over other cloud integrations in the group.
APPLIANCE URL
Alternate Appliance url for scenarios when the default Appliance URL (configured in admin -> settings) is not reachable or resolvable for Instances provisioned in this cloud. The Appliance URL is used for Agent install and reporting.
TIME ZONE
Configures the time zone on provisioned VM’s if necessary.
DATACENTER ID
Used for differentiating pricing among multiple datacenters. Leave blank unless prices are properly configured.
NETWORK MODE
Unmanaged or Managed
SECURITY MODE

Defines if Conduit will control local firewall of provisioned servers and hosts.

Important

When local firewall management is enabled, Conduit will automatically set an IP table rule to allow incoming connections on tcp port 22 from the Conduit Appliance.

STORAGE MODE
Single Disk, LVM or Clustered
GUIDANCE
Enable Guidance recommendations on cloud resources.
DNS INTEGRATION
Records for instances provisioned in this cloud will be added to selected DNS integration.
SERVICE REGISTRY
Services for instances provisioned in this cloud will be added to selected Service Registry integration.
CONFIG MANAGEMENT
Select a Chef, Salt, Ansible or Puppet integration to be used with this Cloud.
CMDB
Select CMDB Integration to automatically update selected CMDB.
AGENT INSTALL MODE
  • SSH / WINRM: Conduit will use SSH or WINRM for Agent install.
  • Cloud-Init (when available): Conduit will utilize Cloud-Init or Cloudbase-Init for agent install when provisioning images with Cloud-Init/Cloudbase-Init installed. Conduit will fall back on SSH or WINRM if cloud-init is not installed on the provisioned image.
API PROXY
Required when a Proxy Server blocks communication between the Conduit Appliance and the Cloud. Proxies can be added in the Infrastructure -> Networks -> Proxies tab.

Provisioning Options

PROXY
Required when a Proxy Server blocks communication between an Instance and the Conduit Appliance. Proxies can be added in the Infrastructure -> Networks -> Proxies tab.
Bypass Proxy for Appliance URL
Enable to bypass proxy settings (if added) for Instance Agent communication to the Appliance URL.
USER DATA (LINUX)
Add cloud-init user data or scripts. Assumes bash syntax.

Note

All fields and options can be edited after the Cloud is created.

Minimum AWS IAM Policies

Below are the AWS IAM Policies for EC2, RDS, and S3 covering the minimum access for Conduit applying to all resources.

See http://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies.html for more information.

EC2

{
  "Version": "2012-10-17",
  "Statement": [
   {
     "Effect": "Allow",
     "Action": [
       "ec2:AllocateAddress",
       "ec2:AssociateAddress",
       "ec2:AssignPrivateIpAddresses",
       "ec2:AttachVolume",
       "ec2:AuthorizeSecurityGroupEgress",
       "ec2:AuthorizeSecurityGroupIngress",
       "ec2:CancelExportTask",
       "ec2:CancelImportTask",
       "ec2:CopyImage",
       "ec2:CreateImage",
       "ec2:CopySnapshot",
       "ec2:CreateInstanceExportTask",
       "ec2:CreateKeyPair",
       "ec2:CreateNetworkAcl",
       "ec2:CreateNetworkAclEntry",
       "ec2:CreateNetworkInterface",
       "ec2:CreateSecurityGroup",
       "ec2:CreateSnapshot",
       "ec2:CreateTags",
       "ec2:CreateVolume",
       "ec2:DeleteKeyPair",
       "ec2:DeleteNetworkAcl",
       "ec2:DeleteNetworkAclEntry",
       "ec2:DeleteNetworkInterface",
       "ec2:DeleteSecurityGroup",
       "ec2:DeleteSnapshot",
       "ec2:DeleteTags",
       "ec2:DeleteVolume",
       "ec2:DeregisterImage",
       "ec2:DescribeAccountAttributes",
       "ec2:DescribeAddresses",
       "ec2:DescribeAvailabilityZones",
       "ec2:DescribeClassicLinkInstances",
       "ec2:DescribeConversionTasks",
       "ec2:DescribeExportTasks",
       "ec2:DescribeImageAttribute",
       "ec2:DescribeImages",
       "ec2:DescribeImportImageTasks",
       "ec2:DescribeImportSnapshotTasks",
       "ec2:DescribeInstances",
       "ec2:DescribeInstanceStatus",
       "ec2:DescribeKeyPairs",
       "ec2:DescribeNetworkAcls",
       "ec2:DescribeNetworkInterfaceAttribute",
       "ec2:DescribeNetworkInterfaces",
       "ec2:DescribeRegions",
       "ec2:DescribeSecurityGroupReferences",
       "ec2:DescribeSecurityGroups",
       "ec2:DescribeSnapshotAttribute",
       "ec2:DescribeSnapshots",
       "ec2:DescribeStaleSecurityGroups",
       "ec2:DescribeSubnets",
       "ec2:DescribeTags",
       "ec2:DescribeVolumeAttribute",
       "ec2:DescribeVolumes",
       "ec2:DescribeVolumeStatus",
       "ec2:DescribeVpcAttribute",
       "ec2:DescribeVpcClassicLink",
       "ec2:DescribeVpcClassicLinkDnsSupport",
       "ec2:DescribeVpcEndpoints",
       "ec2:DescribeVpcEndpointServices",
       "ec2:DescribeVpcs",
       "ec2:DetachNetworkInterface",
       "ec2:DetachVolume",
       "ec2:DisassociateAddress",
       "ec2:ImportImage",
       "ec2:ImportInstance",
       "ec2:ImportKeyPair",
       "ec2:ImportSnapshot",
       "ec2:ImportVolume",
       "ec2:ModifyImageAttribute",
       "ec2:ModifyInstanceAttribute",
       "ec2:ModifyNetworkInterfaceAttribute",
       "ec2:ModifySnapshotAttribute",
       "ec2:ModifyVolumeAttribute",
       "ec2:RebootInstances",
       "ec2:RegisterImage",
       "ec2:ReleaseAddress",
       "ec2:ReplaceNetworkAclAssociation",
       "ec2:ReplaceNetworkAclEntry",
       "ec2:ResetImageAttribute",
       "ec2:ResetInstanceAttribute",
       "ec2:ResetNetworkInterfaceAttribute",
       "ec2:ResetSnapshotAttribute",
       "ec2:RevokeSecurityGroupEgress",
       "ec2:RevokeSecurityGroupIngress",
       "ec2:RunInstances",
       "ec2:StartInstances",
       "ec2:StopInstances",
       "ec2:TerminateInstances",
       "ec2:UnassignPrivateIpAddresses"
     ],
     "Resource": "*"
   }
 ]
}

RDS:

{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Effect": "Allow",
     "Action": [
       "rds:AddRoleToDBCluster",
       "rds:AddTagsToResource",
       "rds:ApplyPendingMaintenanceAction",
       "rds:AuthorizeDBSecurityGroupIngress",
       "rds:CopyDBParameterGroup",
       "rds:CopyDBClusterSnapshot",
       "rds:CopyDBSnapshot",
       "rds:CreateDBCluster",
       "rds:CreateDBClusterSnapshot",
       "rds:CreateDBInstance",
       "rds:CreateDBInstanceReadReplica",
       "rds:CreateDBSecurityGroup",
       "rds:CreateDBSnapshot",
       "rds:DeleteDBCluster",
       "rds:DeleteDBInstance",
       "rds:DeleteDBSecurityGroup",
       "rds:DeleteDBSnapshot",
       "rds:DescribeAccountAttributes",
       "rds:DescribeCertificates",
       "rds:DescribeDBClusterParameterGroups",
       "rds:DescribeDBClusterParameters",
       "rds:DescribeDBClusters",
       "rds:DescribeDBClusterSnapshotAttributes",
       "rds:DescribeDBClusterSnapshots",
       "rds:DescribeDBEngineVersions",
       "rds:DescribeDBInstances",
       "rds:DescribeDBLogFiles",
       "rds:DescribeDBParameterGroups",
       "rds:DescribeDBParameters",
       "rds:DescribeDBSecurityGroups",
       "rds:DescribeDBSnapshotAttributes",
       "rds:DescribeDBSnapshots",
       "rds:DescribeDBSubnetGroups",
       "rds:DescribeEngineDefaultClusterParameters",
       "rds:DescribeEngineDefaultParameters",
       "rds:DescribeEventCategories",
       "rds:DescribeEvents",
       "rds:DescribeOptionGroupOptions",
       "rds:DescribeOptionGroups",
       "rds:DescribeOrderableDBInstanceOptions",
       "rds:ListTagsForResource",
       "rds:ModifyDBCluster",
       "rds:ModifyDBClusterParameterGroup",
       "rds:ModifyDBClusterSnapshotAttribute",
       "rds:ModifyDBInstance",
       "rds:ModifyDBParameterGroup",
       "rds:ModifyDBSnapshotAttribute",
       "rds:PromoteReadReplica",
       "rds:RebootDBInstance",
       "rds:RemoveTagsFromResource",
       "rds:RestoreDBClusterFromSnapshot",
       "rds:RestoreDBClusterToPointInTime",
       "rds:RestoreDBInstanceFromDBSnapshot",
       "rds:RestoreDBInstanceToPointInTime",
       "rds:RevokeDBSecurityGroupIngress"
     ],
     "Resource": "*"
   }
  ]
}

S3

{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Effect": "Allow",
     "Action": [
       "s3:AbortMultipartUpload",
       "s3:DeleteObject",
       "s3:DeleteObjectVersion",
       "s3:GetBucketLocation",
       "s3:GetObject",
       "s3:GetObjectVersion",
       "s3:ListBucket",
       "s3:ListBucketMultipartUploads",
       "s3:ListBucketVersions",
       "s3:ListMultipartUploadParts",
       "s3:PutObject"
     ],
     "Resource": [
       "arn:aws:s3:::bucketname",
       "arn:aws:s3:::bucketname/*"
     ]
   }
 ]
}

Resource Filter

If you need to limit actions based on filters you have to pull out the action and put it in a resource based policy since not all the actions support resource filters.

See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-supported-iam-actions-resources.html for more info on limiting resources by filter.

Resource filter example:

{
  "Effect": "Allow",
  "Action": [
   "ec2:StopInstances",
   "ec2:StartInstances"
  ],
  "Resource": *
 },
 {
  "Effect": "Allow",
  "Action": "ec2:TerminateInstances",
  "Resource": "arn:aws:ec2:us-east-1:123456789012:instance/*",
  "Condition": {
    "StringEquals": {
       "ec2:ResourceTag/purpose": "test"
     }
   }
 }

Amazon Cost and Reservation Sync

If you are enabling costing or costing and reservations sync on an amazon cloud then you will need to enable the following policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ce:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}