Thursday, May 1, 2014

AWS (Amazon Cloud) OpsWorks

Just executed a standard tutorial on running PHP applications using OpsWorks. Amazing interface and ease of use. However it lacks, ability to assign resource tags. When the stack is being created, the resource tagging should happen in a seamless way.

Sunday, April 6, 2014

Compare AWS and Azure

Feature

Amazon Web Services

Microsoft Windows Azure

Computing power

Virtual machinesElastic Compute CloudRole Instances
High Performance ComputingCluster Compute InstancesHPC Scheduler
MapReduceElastic Map ReduceHadoop on Azure
Dynamic scalingAuto ScalingAuto Scaling Application Block

Storage

Unstructured storageSimple Storage ServiceAzure Blob
Flexible entitiesSimpleDBAzure Tables
Block Level StorageElastic Block StoreAzure Drive
ArchivingAmazon Glacier
Storage GatewayAWS Storage Gateway

Databases

RDBMSRelational Database ServiceSQL Azure
NoSQLDynamoDBAzure Tables

Caching

CDNCloudFrontCDN
In-MemoryElastiCacheCache

Network

Load BalancerElastic Load BalancerFabric Controller / Traffic Manager
Hybrid CloudVirtual Private CloudAzure Connect
PeeringDirect Connect
-
DNSRoute 53
-

Messaging & Applications

Async MessagingSimple Queue ServiceAzure Queues
Push NotificationsSimple Notification ServiceService Bus
Bulk EmailSimple Email Service
WorkflowsAmazon Simple Workflow Service
SearchAmazon CloudSearch

Monitoring

Resource monitoringCloudWatchSystem Center

Securiry

Identity ManagementIdentity Access ManagementAzure Active Directory

Deployment

Resource creationCloudFormation
-
Web Application ContainerElastic BeanstalkWeb Role

Wednesday, March 12, 2014

Sample Cloud Tester Profile

  Cloud Tester
Role Definition Technical member of Cloud testing team
Pre-Requisites Past experience of performing functional testing of applications with one or more specialized areas such as performance, load/ stress and security etc.
Experience testing RESTful and SOAP based web services
Solid understanding of HTTP/S; Knowledge of JSON, XML and queue based messaging
No prior understanding of Cloud is required
Mandatory Skills Basic understanding of Cloud deployment models
Basic Understanding of Cloud provided infrastructure and services
Basic understanding of security related aspects, impact of Cloud on security and ability to create secure test data for Cloud based testing
Ability to provision hardware/ software and setup test environment in the Cloud
Working knowledge of relational databases and applicable Operating System (OS)
Ability to understand metrics such as page load times, network latency, response times etc., correlate it with requirements and report bugs wherever there are gaps
Ability to inject (large scale) data and traffic from multiple locations spread across the world (whenever required)
Optional Skills Basic understanding of Cloud Provider's costing model
Basic understanding of NoSQL databases
Knowledge of multi-tenancy architectures
Knowledge of testing for mobile enviornments and devices

Azure:
- Visual Studio based load test service
Certification (Optional) None
Supervision Close supervision required
Line reporting Reports to Test Manager
Interactions With Whom: Testing and development teams; No customer interaction

Thursday, March 6, 2014

AWS issues

- Low quality support on forums. Folks on support do not show the kind of expertise expected
- Increasing complexity will add to need to get more front line architects at greater expense
- Where do developers test without incurring significant expenses? More importantly organizations won't like unpredictability in their cost. Even free tier limits may be breached. Where are simulators? That's a big plus for Azure. May be Amazon should offer sandbox environment to large enterprises on per user basis. CloudFormation doesn't have a sandbox or simulator. What a pity!
- Some of services are launched without free tier offering. What do we do with that?
- Edicational institutions are not churning cloud developers. Amazon needs to find a way to promote cloud as a subject and get ready programmers.

Sunday, March 2, 2014

.NET AWS Cloud Development Profiles

  .NET
Cloud Developer Senior Cloud Developer Technical Lead - Cloud
Role Definition Technical member of Azure development team, writes code, unit test and perform bug fixes on Azure platform Technical member of Azure development team, contributes towards high level design of relatively less complex modules using Object oriented and Cloud design patterns; Writes code and performs unit testing and reviews work of Cloud Developers Technical leader of a team responsible for a Azure based sub-system/ module. Interacts with customer during design phases (after architecture phase) and beyond.
Pre-Requsities Good understanding of .NET platform
Knowledge of EntityFramework and Enterprise library
No prior understanding of Cloud is required
Good understanding of .NET platform
Ability to design relatively less complex modules
No prior understanding of Cloud is required
Excellent understanding of .NET platform; Minimum 6 years of experience on .NET
Ability to independently design complex sub-systems/ modules
No prior understanding of Cloud is required
Mandatory Skills Good understanding of Cloud environment and deployment models
Basic understanding of key Azure services *
Ability to program using Azure SDK for .NET
Knowledge of Management Portal; Windows OS and System Manager
Knowledge of Cross-Platform Command-Line Interface (xplat-cli) and Azure Emulators
Ability to program using OData services
Understanding of RESTful and SOAP based Web Services
In addition to Cloud Developer skills, following additional skills are required:
Deep understanding of key Azure services including aspects such as security, performance, availability etc.
Understanding of Azure design patterns and ability to apply those to the design
Understanding of monitoring capabilities available on Azure, ability to derive actions from the reports and implement them
Ability to design and program using Enterprise Library Transient Fault Handling Block
In addition to Senior Cloud Developer skills, following additional skills are required:
Ability to understand architecture documents defined by Architect(s) and follow during implementation
Ability to design complex sub-systems and technically supervise and own implementation, testing and deployment
Deep understanding of Azure security best practices and ability to design secure sub-systems **
Deep understanding of Cloud deployment models
Deep understanding of applicable OS
Ability to drive successful implementation of Continuous Integration; Derive/Take actions based on reported code quality metrics
Good understanding of integration patterns (especially between Public and On-premises applications)
Optional Skills Basic understanding of features offered by all Azure services
Ability to write Powershell scripts
Basic understanding of deployment tools such as Octopus Deploy
Use of "Visual Studio Online" service
In addition to Cloud Developer, following additional skills are required:
Basic understanding of Virtualization enviornments/  Hyper-V hypervisor and its impact on performance and security
Basic understanding of NoSQL databases
Good understanding of security related coding best practices and features offered by .NET platform; Top ten issues from OWASP; Basic understanding of OWASP .NET Cheat Sheet (https://www.owasp.org/index.php/.NET_Security_Cheat_Sheet)
Basic understanding of Azure reference architectures
In addition to skills outlined for Senior Cloud Developer, following additional skills are required:
Basic understanding of various possible use-cases in Cloud in-general and Azure in-specific
Deep understanding of Azure reference architectures
Drive to explore feasibility of using new technologies to solve specific customer problems
Basic understanding of multi-tenancy architectures
Ability to design and program using NoSQL databases
Understanding of any one Cloud agnostic library and ability to leverage similar design principles in own sub-systems
Certification (Optional) None Developing Windows Azure and Web Services Developing Windows Azure and Web Services
Supervision Close supervision required Works with average supervision Works with minimal supervision
Line reporting Reports to Project Manager Reports to Project Manager Reports to Project Manager
Interactions With Whom: Technical team; No customer interaction With Whom: Technical team; No customer interaction With Whom: Technical team; Project Manager; Customers from design phase onwards (Architects manage customer interactions during DD, Requirements and Architecture definition phases)

Friday, February 28, 2014

Proposed Java AWS Cloud Developer Profiles

 
Cloud Developer Senior Cloud Developer Technical Lead - Cloud
Role Definition Technical member of AWS development team, writes code, unit test and perform bug fixes on AWS platform Technical member of AWS development team, contributes towards high level design of relatively less complex modules using Object oriented and Cloud design patterns; Writes code and performs unit testing and reviews work of Cloud Developers Technical leader of a team responsible for a AWS based sub-system/ module. Interacts with customer during design phases (after architecture phase) and beyond.
Pre-Requisites Good understanding of Java/J2EE platform
No prior understanding of Cloud is required
Good understanding of Java/J2EE platform
Ability to design relatively less complex modules
No prior understanding of Cloud is required
Excellent understanding of Java/J2EE platform
Ability to independently design complex sub-systems/ modules
No prior understanding of Cloud is required
Mandatory Skills Good understanding of Cloud environment,  deployment models and AWS basics
Basic understanding of key AWS services *
Ability to program using AWS SDK for Java
Ability to use Command-line tools for various AWS services
Ability to program using at least one scripting language (Python or Perl)
Understanding of RESTful and SOAP based Web Services
Working knowledge of applicable OS (Windows, Linux/Unix)
Ability to use AWS Management Console for applicable services
In addition to Cloud Developer skills, following additional skills are required:
Deep understanding of key AWS services including aspects such as security, performance, availability etc.
Understanding of AWS design patterns and ability to apply those
Understanding of monitoring capabilities available on AWS, ability to derive actions from the reports and implement them
Ability to write code that handles transient failures
Basic understanding of Continuous Integration and code quality metrics
In addition to Senior Cloud Developer skills, following additional skills are required:
Ability to understand architecture documents defined by Architect(s) and follow during implementation
Ability to design complex sub-systems and technically supervise and own implementation, testing and deployment
Deep understanding of AWS security best practices and ability to design secure sub-systems **
Deep understanding of Cloud deployment models
Deep understanding of applicable OS
Ability to drive successful implementation of Continuous Integration; Derive/Take actions based on reported code quality metrics
Good understanding of integration patterns (especially between Public and On-premises applications)
Optional Skills Basic understanding of features offered by all AWS services
Ability to write shell scripts on Linux/Unix platforms
Basic understanding of deployment tools such as Chef, Puppet etc.
In addition to skills outlined for Cloud Developer, following additional skills are required:
Basic understanding of Virtualization environments/ Xen hypervisor and its impact on performance and security; Various Virtual Machine formats
Good understanding of security related coding best practices, top ten issues from OWASP and ESAPI library
Basic understanding of NoSQL databases
Basic understanding of AWS reference architectures
In addition to skills outlined for Senior Cloud Developer, following additional skills are required:
Basic understanding of various possible use-cases in Cloud in general and specifically AWS
Deep understanding of AWS reference architectures
Drive to explore feasibility of using new technologies to solve specific customer problems
Basic understanding of multi-tenancy architectures
Ability to design and program using NoSQL databases
Understanding of any one Cloud agnostic library and ability to leverage similar design principles in own sub-systems
Certification (Optional) None AWS Certified Developer - Associate AWS Certified Developer - Associate
Supervision Close supervision required Works with average supervision Works with minimal supervision
Line reporting Reports to Project Manager Reports to Project Manager Reports to Project Manager
Interactions With Whom: Technical team; No customer interaction With Whom: Technical team; No customer interaction With Whom: Technical team; Project Manager; Customers from design phase onwards (Architects manage customer interactions during DD, Requirements and Architecture definition phases)
* Elastic Compute Cloud (EC2); Elastic Block Storage (EBS); Relational Database System (RDS); Elastic Load Balancer (ELB); Simple Storage Service (S3); Identity and Access Management (AWS); ElastiCache; Simple Queue Service (SQS); Simple Notification Service (SNS); CloudFront; DynamoDB; SimpleDB
** May need understanding of one or more of these:  Multi-factor authentication (MFA); X.509 certificates; Public Key Infrastructure (PKI); Symmetric and Asymmetric encryption algorithms (AES, DES & RSA etc.); Hashing algorithms (MD5, SHA-1 & SHA-2 etc.); SSL/TLS; Claim based security/ Security Tokens; Federated identity; Stateful and stateless firewalls; Port scans; IP spoofing; IPsec; Intrusion Detection System (IDS); OS hardening; Hardware Security Module (HSM); Understanding of cheat sheets available at https://www.owasp.org/index.php/Cheat_Sheets#tab=Main

Sunday, February 23, 2014

Cloud Guidelines

Area Title Description Platform specific
Architecture, Testing HA environment, Failover Testing Have capability to artificially fail any system component. The system should recover from failure and still meet NFRs All
Design Check feasibility of abstracting the Cloud provider API Helps support Cloud agnostic development and avoid vendor lock-in; Faster time to market since developers need not learn different APIs from multiple cloud vendors All
Coding Static analysis Make sure static analysis is incorporated into build process - High quality changes are more important in cloud environment than in non-cloud environment All
Architecture Multi-cloud usage Evaluate services from multiple cloud providers and chose the one that suits the requirements the most; Architect application to be able to handle multi-cloud (Both private and public) environments All
Architecture Application bifurcation Evaluate if an application needs to be broken up to use private cloud for some functionality and public for others All
Security Use appropriate algorithms Use AES 256 bit symmetric encryption to encrypt sensitive data at rest
Use RSA 2096 bits or higher for certificates
Use SHA2-256 or higher for hash and message digest
All
Security Security boundary Divide security responsibilities clearly between cloud provider and consumer; Define trust boundary clearly too All
Security Key storage Do not store sensitive keys on your own if your cloud provider provides service for key/certificate management All
Architecture Difference between claimed and actual SLAs Ascertain any gap between claimed and actual SLAs and architect with the lower number in mind. Don't blindly rely on numbers published by Cloud provider All
Architecture Conformance to standards Check if a Cloud service meets any industry standard and give it a higher rating than the one which does not All
Deployment Repeatable and automated Make deployment completely automated All
Security Security planning Plan for handling security breaches and how do you recover from them. This plan may include notifying customers that a breach has occurred and how are they impacted because of the breach All
Architecture Software fatigue Consider automatically replacing instances (that have been running for quite some time) with fresh instances to avoid software fatigue (undetected memory leaks etc.) All
Architecture Put dynamic data near computing infrastructure and static data near users Keep dynamic data near computing instances - (For example, move data to Cloud first before processing); Move static data closer to users to avoid latency (for example through AWS CloudFront) All
Security Protect cloud credentials Rotate keys on a regular basis; Do not store keys on Cloud infrastructure or storage
Make sure certificates are renewed on an annual basis
Use multi-factor authentication wherever feasible
Use cloud infrastructure to store keys wherever such a service is available
All
Security Reduce attack surface Keep attack surface as low as possible by limiting number of ports that are open and the IP's to which those ports are open All
Security Certifications Check and make sure the Cloud provider meets the minimum set of applicable certifications (such as HIPPA, security etc.) and audit requirements All
General Pricing Always check if there is room for any price related negotiation with Cloud provider. Cloud provider may be ready to offer volume based discounts beyond published rates All
Software development Guidelines Publish cloud specific coding guidelines to the team at the start of the development to avoid any surprises later All
Architecture Tagging cloud resources See if Cloud provider allows you to tag/name resources and make use of the tags to manage cloud infrastructure more easily and efficiently. Naming convention/requirements should be established as part of Cloud deployment plan before actual roll-out All
Architecture Disable MySQL binary logs incase of large database loads MySQL Binary logging incurs significant costs and should be disabled in case of large data loads AWS
Testing Simulate customer distribution Use Cloud's datacenters spread across the world to simulate the customers spread across the globe and test for latency they are likely to experience. Having databases spread across the globe (and working as read replicas) and CDNs are some of the choices to provide better customer experience All
Architecture Handle possibility of longer disruptions Cloud provider's implementation may have bugs and the service may see long duration of interruptions much beyond committed SLAs. Sometimes, given the scale of cloud provider, the service interruption may turn grave and result in domino effect (with one service knocking off other services). Always architect with failure in mind. All
Security Consider using IDS/IPS in Cloud environments Use SNORT (an open source IDS) to protect your infrastructure; Snort is usually the first outward facing component. Snort can in turn send incoming requests to load balancers/web servers etc. Also Restful APIs should be protected by an API gateway (such as Oracle API gateway) that protects against DoS attacks, bad input etc. All