AWS VS OpenStack?????

Let us compare the popularity of two top cloud-computing platforms:
1) the infamous Amazon Web Services, which companies typically leverage for the speed and convenience of Amazon's global, hosted, cloud-computing infrastructure, and
2) the increasingly versatile OpenStack, which allows organizations to roll their own cloud-computing services on standard hardware.

OpenStack has similarly grown in popularity since its launch in 2010, having had a nice jump in the Spring of 2013. Some of the more notable companies contributing to OpenStack include: AT&T, MD, Canonical, Cisco, Citrix, Comcast, Cray, Dell, Dreamhost, EMC, Ericsson, Fujitsu, GoDaddy, Google, HP, Hitachi, Huawei, IBM, Intel, Juniper Networks, Mirantis, Oracle, Red Hat, SUSE Linux, VMware, and Yahoo!.
While OpenStack has a lot of diverse contributors, AWS is the fifth largest web hosting provider globally.
Worldwide Market Share by Number of Clients in 2015:

1. GoDaddy - 4.26%
2. BlueHost - 2.56%
3. Host Gator - 2.15%
4. OVH.com - 1.91%
5. Amazon Web Services - 1.81%
6. Rackspace - 1.59%
7. 1&1 - 1.54%
8. Hetzner - 1.29%
9. SoftLayer - 1.19%
10. DreamHost - 1.01%

source: http://hostadvice.com/marketshare/ (2015)

As an open-source cloud-computing protocol, OpenStack obviously can't compete on these terms with a multi-billion dollar cloud-computing and software-as-a-service company. There are a number selling points to consider, however:

• WalMart uses OpenStack to coordinate 100,000+ cores, this provided 100% uptime during Black Friday last year
• Developers gave over 300 talks at the OpenStack Summit in Tokyo this October
• Debian, Canonical, Red Hat, and SUSE Linux all support OpenStack and are active contributors
• OpenStack has enabled companies like Disney, Bloomberg, and Wells Fargo to manage their own clouds at a fraction of the cost of proprietary solutions like AWS
• OpenStack is the only solution that supports mixed hypervisor and bare metal server environments

I think these points lend themselves to the conclusion that adoption and further development in OpenStack are likely to keep pace.

Here is a subset of AWS services:

In the Compute realm we have...

• Amazon Elastic Compute Cloud (EC2) scalable virtual private servers using Xen
• Amazon Elastic MapReduce (EMR) Hadoop-based big data analytics

In Networking we have...

• Amazon Route 53 scalable DNS
• Amazon Virtual Private Cloud (VPC) isolated EC2 instances with the ability to extend corporate networks VPN
• Amazon Elastic Load Balancing (ELB)

In Content Delivery we have...

• Amazon CloudFront CDN

In Storage we have...

• Amazon Simple Storage Service (S3)
• Amazon Glacier low-cost, long-term storage for data archival
• Amazon Elastic File System (EFS) to accompany EC2

In the Database realm we have...

• Amazon DynamoDB low-latency NoSQL SSD-backed databases
• Amazon Relational Database Service (RDS) with MySQL, Oracle, SQL Server, and PostgreSQL support
• Amazon SimpleDB distributed database with EC2 and S3 interoperability, written in Erlang

In the Deployment realm we have...

• AWS Elastic Beanstalk for quick deployment and cloud app management
• AWS OpsWorks EC2 configuration services via Chef, which we discussed previously

In Management we have...

• Amazon Identity and Access Management (IAM) to authenticate into the various services
• AWS Directory Service for tying into an on-premises Microsoft Active Directory or for setting up a new stand-alone AWS directory
• Amazon CloudWatch for application and resource monitoring
• Amazon CloudHSM Hardware Security Module for data security and for meeting regulatory compliance requirements
• AWS Key Management Service (KMS) for creating and managing encryption keys

In the Application Services realm we have...

• Amazon DevPay (beta) for billing and account management
• Amazon Elastic Transcoder (ETS) for mobile video transcription from S3
• Amazon Simple Email Service (SES) for sending bulk and transactional email
• Amazon Simple Notification Service (SNS) multi-protocol application "push" notifications
• Amazon Cognito secure application-user data management and synchronization tool

In Analytics we have...

• Amazon Machine Learning for building regression models from publicly-available datasets

Here are the main components of the modular OpenStack architecture:

Compute (Nova)

• An Infrastructure as a Service (IaaS) system
• Management and automatation of pools of computer resources
• Bare metal and high-performance computing (HPC) configurations
• KVM, VMware, and Xen hypervisor virtualization
• Hyper-V and LXC containerization
• Python-based with various external libraries: Eventlet for concurrent programming, Kombu for AMQP communication, SQLAlchemy for database access, etc.
• Designed to scale horizontally on standard hardware with no proprietary hardware or software requirements
• Interoperable with legacy systems

Image Service (Glance)

• OpenStack Image Service for discovery, registration, and delivery of services for disk and server images
• Template-building from stored images
• Storage and cataloging of unlimited backups
• REST interface for querying disk image information
• Streaming of images to servers
• VMware integration, with vMotion Dynamic Resource Scheduling (DRS) and live migration of running virtual machines
• All OpenStack OS images built on virtual machines
• Maintenance of image metadata
• Creation, deletion, sharing, and duplification of images

Object Storage (Swift)

• Scalable redundant storage system
• Automatic replication of content from failed disks to other active nodes
• Suitable for inexpensive commodity hard drives and servers

Dashboard (Horizon)

• GUI for access, provision, and automation of cloud-based resources for administrators and users
• Third-party billing, monitoring, management tool integration
• Customizable (brandable) dashboard
• EC2 compatibility

Identity Service (Keystone)

• Unified authentication system across the cloud OS
• Integration with existing backend directory services such as LDAP
• Various authentication methods: username/password, token-based systems, and AWS-style logins
• Queryable, single registry of all deployed services, with programmatic determination of access for users and third-party tools

Networking (Neutron)

• Manual and automatic management of networks and IP addresses
• Distict networking models for different applications and user groups
• Flat networks (VLAN's) for separating servers and traffic.
• Static IP addresses, DHCP
• Floating IP addresses for dynamic rerouting to resources on the network
• Software-defined networking (SDN), OpenFlow, for multi-tenancy and scalability.
• Management of intrusion detection systems (IDS), load balancing, firewalls, VPN's, etc.

Block Storage (Cinder)

• Persistent block-level storage for databases and expandable file systems
• Block storage integration into OpenStack Compute and Dashboard for allocation of storage
• Various storage platforms supported: Ceph, CloudByte, Coraid, EMC (ScaleIO, VMAX and VNX), GlusterFS, Hitachi Data Systems, IBM Storage (Storwize family, SAN Volume Controller, XIV Storage System, and GPFS), Linux LIO, NetApp, Nexenta, Scality, SolidFire, HP (StoreVirtual and 3PAR StoreServ families) and Pure Storag
• Snapshot management for backing up data stored on block storage volumes
• Restoring of snapshots, use of snapshots as templates for new block storage volumes

Orchestration (Heat)

• Orchestratation of multiple composite cloud applications using templates
• OpenStack-native REST API
• CloudFormation-compatible Query API

Telemetry (Ceilometer)

• Billing system Single Point Of Contact
• Traceable, auditable delivery of counters for billing
• Counters extensible to new projects
• Independent data collection

Database (Trove)

• Database-as-a-service (DaaS) provisioning relational database engine
• DaaS non-relational database engine

Elastic Map Reduce (Sahara)

• Hadoop cluster provisioning
• Setting of parameters based on: Hadoop version, cluster topology, node hardware details, etc.
• Cluster deployment in minutes
• Scaling of already-provisioned clusters by adding and removing worker nodes on demand

Bare Metal Provisioning (Ironic)

• Provisioning of bare metal machines (as opposed to virtual machines)
• Bare-metal hypervisor API
• Plugins for interacting with bare-metal hypervisors
• PXE and IPMI simultaneous provisioning, turning machines on and off as needed
• Extensible with vendor-specific plugins for additional functionality

Multiple Tenant Cloud Messaging (Zaqar)

• Multi-tenant cloud messaging service for Web developers
• Some components inspired by Amazon's SQS, with additional semantics for event broadcasting
• Fully RESTful API for sending messages between various components of their SaaS and mobile applications
• Surfacing of events to end users and guest agents that run in the "over-cloud" layer

Shared File System Service (Manila)

• Vendor-agnostic share management API
• Create, delete, give/deny access to a share
• Support for commercial storage appliances from: EMC, NetApp, HP, IBM, Oracle, Quobyte, and Hitachi Data Systems
• Support for Red Hat's GlusterFS filesystem

DNSaaS (Designate)

• DNS as a Service

Security API (Barbican)

• REST API for secure storage, provisioning and management of secrets
• Built for use in all environments, including large ephemeral clouds

AWS Compatibility

• Interoperability with Amazon EC2 and Amazon S3
• Minimal effort to port AWS client applications to OpenStack

Multi tenancy: AWS vs OpenStack

The first conceptual difference between AWS and OpenStack is about multi-tenancy. OpenStack offers a multi-layer tenant mechanism with domain and projects. A domain is a collection of users, groups, and projects, in a way parallel to AWS’s account. LDAP groups are attached to domains. OpenStack’s Project is a container of virtual resources such as virtual machines, networks and volumes. Using projects, users can establish several isolated and independently controlled groups of resources that serve different objectives. In the Kilo release, Keystone introduced the hierarchical multi-tenancy concept, using sub-projects.
Accommodating more than one million customers, Amazon Web Services is a multi-tenant cloud by nature; however, at the account level, a single user receives a single tenant experience. Having said that, AWS offers Virtual Private Cloud, VPC, which is somewhat parallel to OpenStack’s project. Amazon’s VPC lets the user provision a logically isolated section of the Amazon Web Services (AWS) cloud where the user can launch AWS resources in a virtual network that s/he defines.  AWS’s VPC is limited to one router and one IP block; though not compulsory, this is a common practice for OpenStack projects. It is worth noting that all EC2’s virtual networking capabilities are only available using VPC.
On the other hand, unlike OpenStack, Amazon’s VPC offers extremely valuable tools that simplify the establishment of secured connectivity between VPCs and between VPC and on-premise resources. The classical use case for enterprises is running the web servers or the entire customer facing application on Amazon’s public cloud, while keeping the rest of the servers on-premise. Through its API, Amazon allows the user to establish a VPN connection and even control the customer’s and AWS’s gateways. This is extremely valuable for enterprises that chose the hybrid cloud path, especially given the fact that Amazon has integrated its VPN gateway with the market leading VPN CPEs (customer premise equipment). OpenStack’s Neutron project does offers VPN as a Service capabilities (VPNaaS); however, it is experimental and lacks the end-to-end integration that Amazon provides.

Networking: Neutron vs AWS VPC

From the network perspective, while OpenStack provides control over the L2 elements of the virtual network, AWS exposes only subnets. OpenStack Neutron’s API allows granular control of elements such as ports (the connection point for attaching a virtual server to a virtual network) and the ability to allocate VLAN IDs that correspond to VLANs present in the physical network. This is especially useful for provider networks, which are mapped to existing physical networks in the data center. Those differences are again attributed to the different concepts of AWS and OpenStack. In the private cloud, the user manages the physical networking by himself. Thus, it is crucial for virtual networking to be fully integrated with the physical data center networking. However, the public cloud is a managed service that takes all the hassle of physical network management away from the user, therefore  providing control on L2 is irrelevant to the user.
As for Layer 3 networking, conceptually, Amazon’s AWS and OpenStack’s Neutron provides comparable capabilities. Both cloud services allow creation of network subnets. OpenStack allows use of several subnets on the same virtual network, although it is not a common practice. AWS allows users to define Elastic IP addresses, which are public IP addresses reachable from the Internet. OpenStack offers a similar mechanism, the floating IP, which is part of the virtual router’s API.
Both clouds provide routing services; in AWS each VPC includes an implicit virtual router and the API allows the user to set the routing table (which contains a set of rules, called routes, that are used to determine where network traffic is directed). OpenStack’s Neutron API also allows management of the routing table;  however, it also allows management of the router entities themselves and does not limiting the number of routers per project. Moreover, a single router can be connected to more than one project.
From a network security perspective, AWS and OpenStack offer similar mechanisms. Security groups are used to inspect the traffic at the instance level. Networking ACLs and virtual firewalls are used by AWS and OpenStack respectively to inspect traffic going between subnets. There are minor nuances unique to each API; however, the general concept is very similar.

Here we compare some main components of AWS and OpenStack

Compute

 Why you need it?

To run an application you need a server with CPU, memory and storage, with or without pre-installed operating systems and applications.

	OpenStack	AWS
Definition Compute is virtual machines/servers	Instance	Instance/VM
Sizes How much memory and CPU and temporary (ephemeral) storage is assigned to the instances/VM.	Flavors: Variety of sizes: micro, small, medium, large etc.	Variety of sizes: micro, small, medium, large etc.
Operating systems offered What operating systems does the cloud offer to end-users	Whatever operating systems the cloud administrators host on the OpenStack cloud. (Red Hat certifies Microsoft Windows, RHEL and SUSE)	AMIs provided by the AWS marketplace.
Templates/images A base configuration of a virtual machine, from which other virtual machines can be created. Catalogs of virtual machine images can be created from which users can select a virtual machine.	Glance  OpenStack administrators upload images and create catalogs for users. Users can upload their own images.	(AMI) Amazon Machine Image AWS provides an online marketplace of pre-defined images. Users can upload their own images.

Networking

 Why you need it?

To network virtual servers to each other. You also need to control who can access the server. You want to protect/firewall the server especially if it is exposed to the Internet.

	OpenStack	AWS
Definition Networking provides connectivity for users to virtual machines. Connects virtual machines to one another and to external networks (the Internet).	Neutron	Networking
A private IP address internal only and non-routable to the Internet	Every virtual instance is automatically assigned a private IP address, typically using DHCP.	AWS allocates a private IP address for the instance using DHCP.
Public IP address	A floating IP is a public IP address, that you can dynamically add to a running virtual instance.	AWS public IP address is mapped to the primary private IP address.
Networking service	You can create networks and networking functions, eg. L3 forwarding, NAT, edge firewalls, and IPsec VPN.	Virtual routers or switches can be added if you use AWS VPC, a virtual public cloud.
Load Balance VM traffic	OpenStack LBaaS (Load Balancing as a Service) balances traffic from one network to application services.	ELB (Elastic Load Balancing) automatically distributes incoming application traffic across Amazon EC2 instances.
DNS. Manage the DNS entries for your virtual servers and web applications.	The OpenStack DNS project (Designate) is in “incubation” and is not part of core OpenStack (as of the April 2015 Kilo release).	Route 53 –  AWS’s DNS service.
SRIOV A method of device virtualization that provides higher I/O performance and lower CPU utilization compared to traditional implementations.	Each SR-IOV port is associated with a virtual function (VF). SR-IOV ports may be provided by Hardware-based Virtual Ethernet Bridging or they may be extended to an upstream physical switch (IEEE 802.1br).	AWS support enhanced networking capabilities using SR-IOV, provides higher packet per second (PPS) performance, lower inter-instance latencies, and very low network jitter.

Monitoring

Why you need it?

You get insight into usage patterns and utilization of the physical and virtual resources. You may want to account for individual usage and optionally bill users for their usage.

	OpenStack	AWS
Definition Monitoring provides metering and usage of the cloud.	Ceilometer	Cloudwatch
System-wide metering and usage. Option to bill users for their usage	To collect measurements of the utilization of the physical and virtual resources comprising deployed clouds. Persist data for subsequent retrieval and analysis, and trigger actions when defined criteria are met.	Monitoring service for AWS cloud resources and the applications  on AWS. Collect and track metrics, collect and monitor log files, and set alarms.

Security

Why you need it?

You need the  option of public key cryptography for SSH and password decryption. You want to firewall virtual machines to only allow certain traffic in (ingress) or out (egress).

	OpenStack	AWS
Definition Control access to your virtual machines.	Keypairs, security groups.	Keypairs, security groups.
Key pairs To login to your VM or instance, you must create a key pair. Linux: used to SSH. Windows: used to decrypt the Administrator password.	When you launch a virtual machine, you can inject a key pair, which provides SSH access to your instance.	To log in to your instance, specify the name of the key pair when you launch the instance, and provide the private key when you connect to the instance.
Assign and control access to VM instances. A security group is a named collection of network access rules that limit the traffic that access an instance. When you launch an instance, you can assign one or more security groups to it.	Supported	Supported