How to build your app quickly (and cheaply?) using the Cloud Intro (Admin, facilities, who I am) Agenda Overview Walk-through deployment on PaaS and IaaS Break Considerations Costs / Economics Architecture Discussion Goals for the workshop Cloud computing. WTF? WAN (Frame Relay, ISDN, etc.) Router Router LAN A LAN B For this workshop, it means: elastic compute resources, charged for like a utility Why bother? What’s a “web app”? http://www.flickr.com/photos/psd/6732568/in/set-666533/ http://www.flickr.com/photos/psd/6733371/in/set-666533/ http://www.flickr.com/photos/psd/6733187/in/set-666533/ http://www.flickr.com/photos/gopal1035/322084210/ http://www.flickr.com/photos/psd/58054545/ It’s more than static content. So that means, it’s about any and all commercial web sites, right? No. Don’t think so. Griefer alert: is this just flamebait? No. In the context of cloud computing, it makes a difference. A big difference. Apps do something. Content doesn’t. Cloud computing is about doing something. http://www.nytimes.com/2006/06/14/technology/14search.html “ Microsoft is adding 10k servers a month to their infrastructure” http://blogs.zdnet.com/microsoft/?p=1391 “ That’s one Facebook per month” http://www.datacenterknowledge.com/archives/2008/04/23/facebook-now-running-10000-web-servers/ http://www.istartedsomething.com/20080812/microsoft-enviroment-video-reveals-server-numbers-power/ Amazon talks a lot about the demand… Animoto went from 50 to 3400 Amazon virtual servers in two days http://aws.typepad.com/aws/2008/04/animoto---scali.html The New York Times converted 4TB of TIFFs to PDFs in a day. For $240 http://open.blogs.nytimes.com/2007/11/01/self-service-prorated-super-computing-fun/ http://open.blogs.nytimes.com/2008/05/21/the-new-york-times-archives-amazon-web-services-timesmachine/ http://www.roughtype.com/archives/2008/11/the_new_economi.php And Amazon shows everyone who will sit still for more than 5 seconds the following picture: http://aws.typepad.com/aws/2008/05/lots-of-bits.html Assumption: Mom+Pop ISP ™ cannot compete with this. Are Mom+Pop doomed? Ultimately, that depends on you, the developers of web apps. It’s going to depend on decisions you make about architectural and engineering concerns. Architecture is about making design choices. Engineering is about knowing your materials. Making use of the Cloud is an architectural and engineering challenge. So what are the architectural choices? What are the materials? http://rationalsecurity.typepad.com/blog/2009/01/cloud-computing-taxonomy-ontology-please-review.html http://cloudcomputing.sys-con.com/node/811519 http://www.collab-ogce.org/gce08/images/7/76/LamiaYouseff.pdf SADIST-PIMP SPI (SaaS, Paas, IaaS) http://www.mindmeister.com/maps/show_public/15936058 But wait! Once that’s sorted, you have to consider contextual dimensions… The Radeztsky Cube http://cloudforum.googlegroups.com/web/Metaverse+Decomposition.pdf http://cloudforum.googlegroups.com/web/Metaverse+Decomposition.pdf IOW, your choices are influenced by whether you are integrating established apps, or writing a green-field app…. And whether your architecture will be entirely in the public Cloud, or a mix of public and private resources. For the sake of this workshop, we’re going to refer to the SPI model… SPI Model SaaS PaaS IaaS We’re going to assume a green-field app, deployed entirely on the public Cloud… We’re going to focus on the scenario of “your code, running elsewhere” – not so much on mashups and re-use of SaaS And we’re going to examine the question of PaaS vs. IaaS for that app. Goals for the workshop OK. So what is PaaS? Platform As A Service Some examples: Google App Engine, Bungee, Force.com, Heroku Characterized by… Constraints on language and design A high level programming model A specific model of multi-tenancy Takes care of low level concerns Google App Engine That’s it. Constraints on language and design (Python + BigTable + Goog Svcs) A high level programming model (The WebApp (or other Python) framework, Datastore APIs, Memcache, etc.) A specific model of multi-tenancy (Google’s BigTable + GFS platform) Takes care of the low level concerns (Scales for you (up and down), distribution across cluster nodes, load balancing, replication of data, etc. ) Pretty cool. Infrastructure As A Service Characterized by… No constraints on language or design A high level architectural model A specific model of multi-tenancy Takes care of very few low level concerns This is a LOT more work. ;) Amazon Web Services Amazon Web Services Elastic Compute Cloud (EC2) SimpleDB Simple Storage Service (S3) Simple Queue Service (SQS) EC2 Elastic Block Store (EBS) Other stuff… Cloudfront DevPay Flexible Payments Service (FPS) Mechanical Turk Alexa By and large, “Amazon” means “EC2” EC2 is the only AWS service that one deploys to You just use the other services – whether from an EC2 instance or anywhere else on the Web is irrelevant The core “unit” of EC2 is the Amazon Machine Image – AMI An AMI is a virtual machine image – a VM A VM is just a (very large) file. Like a live ISO disk image. Typically, it is some distro of Linux. Amazon uses Xen, an open source VM system http://www.xen.org/ The key to IaaS is that you can use any app architecture you like The drawback with IaaS is that you therefore have to design your own app architecture Generally speaking, this is the same task (with the same effort) that you would need for physical hardware hosted at an ISP App servers, load balancers, databases, clusters, replication, networking… You sort it out yourself With Amazon, this begins with obtaining your credentials Then you download the command line tools and set them up… You use the tools to proceed through the AMI workflow http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ AMI ID http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ Create keypair Save the private key locally ec2-add-keypair Launch selected instance Using AMI ID and keypair name ec2-run-instances -k Returns the Instance ID Examine the running instance ec2-describe-instances http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Let’s pick that apart… http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Reservation ID http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … AWS Access Key ID AWS Access Key ID http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Security Group ID http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Instance ID http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … AMI ID http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … External DNS host name http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Internal DNS host name http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Current state of the instance http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Keypair name http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … AMI Launch Index http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Instance type http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Launch time http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Availability Zone ID Availability zones are a bit complicated… http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ Other IaaS providers may (or may not) have similar capabilities The point is that, unlike GAE, here is yet another detail that you need to think about http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ Open network access to the instance ec2-authorize -p Can include options like restricting access to a specific (public) IP address -s Eg. “-s your_public_IP_address/32”* for just your host At a minimum, you need to configure port 22 for SSH access Connect to the instance with SSH ssh –i root@ Note: with “ec2-get-console-output”, you can get (among other things) the SSH host key fingerprint, before logging on with SSH. At login, you can compare to be sure there’s no “man in the middle” Begin modifying the instance Once you’ve modified the image, you “bundle” it (which is a kind of snapshot), upload the bundle to S3, and register it with EC2 There are hundreds of publicly available AMIs for use as templates And, finally, with some set of your own AMIs, you can begin running instances of your app And that brings us to the runtime considerations… AKA – how does your app scale horizontally? A brief scalability refresher: vertical scalability is “get a bigger box” Horizontal scalability is “add more copies of the same box” To put it bluntly, if your app can't efficiently scale in a horizontal fashion, you’re wasting your time on the Cloud On GAE, under the covers tech like BigTable, GFS and the legendary Map/Reduce are taking care of this for you In an IaaS context like AWS, you have to solve the problem yourself EC2 has a SOAP API that you can invoke – whether you do so from outside EC2, or from within a running AMI is irrelevant Invoking the EC2 API to launch and terminate instances from a running instance gives you the ability to create dynamic horizontal scalability Your load balancer AMI could start extra instances of your web server AMI to meet demand… And turn them back off, when demand subsides And there are numerous ways to exploit this capability From RightScale to Scalr to Gigaspaces, Hadoop, Terracotta and countless others Combined with the other elements of AWS, like S3, SQS, EBS and SimpleDB, you can design a system that competes favorably with GAE This is what “elastic” means To repeat: this is a LOT more work. ;) No constraints on language or design (AMIs can be any Linux or Windows server platform, and your app can be anything that runs on those platforms) A high level architectural model (The core services of AWS provide a foundation, and do constrain your design – for example, you need horizontal scalability) A specific model of multi-tenancy (AMIs are securely isolated from one another, but the underlying hardware is all shared) Takes care of very few low level concerns (You roll your own) Before we wrap up “deploying to AWS”, however, let’s look at some alternatives to the command line tools from Amazon In terms of managing running instances, and the overall configuration of things, AWS provides its own Web UI https://console.aws.amazon.com/ec2/home#c=EC2&s=Home https://console.aws.amazon.com/ec2/home#c=EC2&s=Instances https://console.aws.amazon.com/ec2/home#c=EC2&s=Images https://console.aws.amazon.com/ec2/home#c=EC2&s=BundleTasks http://status.aws.amazon.com/ An ecosystem of 3 rd party providers has emerged around AWS Some of them are specialized in managing VMs http://elasticserver.com/ http://elasticserver.com/portals http://elasticserver.com/site/lamp http://elasticserver.com/site/lamp http://elasticserver.com/build-status https://elasticserver.com/ec2/ https://elasticserver.com/byo/create https://elasticserver.com/my/packages https://elasticserver.com/new-server There are other vendors offering similar services, eg. rPath And there is a spectrum between the simplicity of GAE and the complexity of AWS… Let’s take a look at Elastra, for example https://cloud.elastra.com/elastra/ https://cloud.elastra.com/elastra/ https://cloud.elastra.com/elastra/ https://cloud.elastra.com/elastra/ https://cloud.elastra.com/elastra/ https://cloud.elastra.com/elastra/ https://cloud.elastra.com/elastra/ Finally, to wrap up our quick overview of IaaS, let’s look at Rightscale… https://my.rightscale.com/dashboard https://my.rightscale.com/dashboard https://my.rightscale.com/dashboard https://my.rightscale.com/dashboard https://my.rightscale.com/dashboard https://my.rightscale.com/dashboard https://my.rightscale.com/dashboard https://my.rightscale.com/dashboard Goals for the workshop But first, a short break… (Jeopardy theme song plays) Goals for the workshop What are some of the questions you need to answer, to decide between PaaS and Iaas? How quickly do you need to go to market? How fast do you need to iterate versions of the app? What are your pre-existing platform needs, if any? What are your security, compliance, regulatory requirements? What is your capacity for system architecture and design work? SPI Model Freedom FROM Considerations Freedom TO Differentiate http://twitter.com/gblnetwkr SaaS PaaS IaaS SPI Model POWER of speed and agility POWER to control http://twitter.com/gblnetwkr SaaS PaaS IaaS Goals for the workshop What do things really cost? Simple LAMP app. 1 box as a load balancer / proxy, 4 web server boxes, and one larger box for a database server $84 / month (6 x Giant) http://laughingsquid.net/hosting/ $900 / month (1 x Biz1 + 4 x Biz1 + 1 x Enterprise II) http://order.1and1.com:80/xml/order/ServerRoot So, let’s see what the equivalent setup costs on AWS… Now, AWS bills in units of things like server hours, IOs per month, GB of storage and bandwidth actually used, etc. http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ 24 hours × 30 days = ---------------------- 720 hours in a month ---------------------- 720 hours × 5 small AWS Instances = ---------------------- 3600 hours ---------------------- + 720 hours of a large AWS instance + 5000 GB network bandwidth + 3200 GB disk space (added to the default space on the instances) + 50 mil. IORs + 30 daily backups ---------------------- roughly equivalent to 1+1 … $2233 / month http://calculator.s3.amazonaws.com/calc5.html http://calculator.s3.amazonaws.com/calc5.html Despite best efforts, this is still quite imprecise, and apples vs. oranges, but… Even more importantly… This is an incredibly stupid way to set up and use AWS $528/ month http://calculator.s3.amazonaws.com/calc5.html Still not “cheap”, though… That’s because the key to “elastic computing” isn’t being able to turn on servers at will… It’s about being able to turn them off . Let’s rewind to the an earlier part of this discussion… What’s a web app? Whatever it is, we said “it’s not static content”… And I said that makes a big difference Why is that? Well, the unwritten social contract of the ‘Net says: “you can’t turn static content off” Apps do something But they generally don’t do it 24x7 Slicing and dicing your design to strictly demarcate static content from active has always been a Good Idea ™ In a cloud computing context, it’s not just a good idea – it’s imperative What are your usage patterns? When are your peak loads, on average? What can you turn off? And you have to be clever about it… Your production usage may not even be the place you can save the most money… Consider: you want to develop the next version of your app You need resources for your developers to do that, and you need a place for them to test So, maybe you wind up renting 3 more servers from 1+1 – $300 / month But, when you examine usage patterns, you find that your developers are only using the boxes for 3 hours each day $17/ month http://calculator.s3.amazonaws.com/calc5.html Perhaps even more importantly, you’ve made no commitment here – once your dev / test phase is over, so are the costs Calculating costs is complex, and entirely context dependent There are significant potential savings, but only if you’re clever Goals for the workshop Since we worked out, sometime in the early ‘90s, what the architecture of a “client / server” system design looked like… There's been a general consensus about a sort of a canonical architecture for so-called “N-tier systems” Presentation Service Facáde Application Logic Data Persistence Web Server App Server Database Server So, we’ve talked about the need for horizontal scalability in the Cloud What does that imply? Well, as already suggested, among other things, parallelism Parallelism has significant consequences It leads one to try to avoid stateful interactions To prefer asynchronous communications (messages)… One finds oneself on the front lines of the REST War ™ – the battle of the RESTafarians vs. the established IT Universe http://www.dehora.net/journal/2008/07/25/patterns-of-web-architecture/ http://www.dehora.net/journal/2008/08/15/rest-as-an-engineering-discipline/ http://www.infoq.com/articles/webber-rest-workflow/ http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven/ http://www.redmonk.com/jgovernor/2009/02/12/the-rest-of-the-cloud/ http://delicious.com/mastermark/rest/ And it forces one to think strange things about optimal patterns of storing and accessing data Like sharding one’s data to meet resource demands http://highscalability.com/unorthodox-approach-database-design-coming-shard/ Questions like “is two-phase commit a feature? Or a bug?” begin to seem important New terms, like CAP, Paxos and BASE creep into conversations about “eventual consistency” http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1495 http://en.wikipedia.org/wiki/Paxos_algorithm http://queue.acm.org/detail.cfm?id=1394128 http://www.allthingsdistributed.com/2008/12/eventually_consistent.html This was happening anyway, driven by the clash of Web architecture with the established IT universe Cloud computing’s possibilities are accelerating the process There is an emerging consensus about what the consequences of all this are for app architectures “ The canonical cloud architecture that has evolved revolves around dynamically scalable CPUs consuming asynchronous, persistently queued events.” http://highscalability.com/canonical-cloud-architecture http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1632&categoryID=102 Use scalable ingredients Eg. Hadoop on EC2 Keep ingredients loosely coupled All communication via persistent messaging Assume constant failure Design things to persist state, restart from last known good, and continue their own tasks even if all around them fail Consider things like re-tries with exponential back-off Build IN redundancy Learn about things like the POSA Blackboard pattern, tuplespaces, and Map / Reduce Read this book! http://www.amazon.com/How-Write-Parallel-Programs-Course/dp/026203171X/ And if your needs / budget require or can accommodate it, consider RAIC Redundant Array of Independent Cloud providers http://www.jroller.com/MasterMark/entry/raic_pronounce_it_rake_please http://www.jroller.com/MasterMark/entry/raic_pronounce_it_rake_please Why bother? Web Server App Server Database Server Mainframe http://www.flickr.com/photos/dvdmerwe/231593469/ http://www.flickr.com/photos/sigkyrre/1091616886/ http://twitter.com/ gblnetwkr http://en.wikipedia.org/wiki/ Consumerization http://twitter.com/ gblnetwkr Join the conversation: http://groups.google.com/group/cloud-computing/ http://groups.google.com/group/cloudforum http://tech.groups.yahoo.com/group/cloudcomputing-tech/ … and please come talk to us, as well … http://twitter.com/mastermark http://www.jroller.com/MasterMark/ Thanks!