• Working Directory: Documents/Courses/Pluralsight-PivotalCloudFoundryDeveloper

  • Sample Code:

  • Cloud PCF: https://run.pivotal.io
  • Cloud Syslog: https://www.papertrailapp.com
  • Monitoring: newrelic.com

M01 – Introduction

  • Complete Lab M01-01 Requirements
    • Did not get Mongo working
    • Did not get Jenkins working
  • Complete Lab M01-02 Approach and Methodology
  • Watch Clip M01-01 Course Introduction
  • Complete Lab M01-03 Platform Operator Setup
  • Watch Clip M01-02 Pivotal Cloud Foundry Introduction
  • Complete Lab M01-04 Push to the Cloud
  • Watch Clip M01-03 Pivotal Cloud Foundry Introduction – Recap

M02 – Logging, Scale, and High Availability

  • Watch Clip M02-01 Logging, Scale, and HA (Part 1)
    • infrastructure is ephemeral & immutable
      • updates to systems are done by new deploy, not an in place update
      • 12 factor app is the playbook 12factor.net
    • processes
      • stateless - these can go up & down at any point
      • persist state to backend service
    • concurrency
      • scale out via process model (elastic scale)
      • consider scale for each process
    • disposability
      • maximize robustness with fast startup & graceful shutdown
    • logs
      • treat logs as event streams (STDOUT, STDERR)
    • Elastic Runtime Architecture
      • Diego
        • schedules tasks and long running processes
          • task: is guaranteed to run at MOST once
            • stage an application
          • Long run process: a web app
        • run within an immutable container
        • Containers are run within a cell
        • containers are managed by Garden
          • garden has many backends
        • rep - represents a cell
        • auction - held to bid on task or LRP
        • executor - manages container allocations in cell
        • Metron - forwards logs to loggragator
        • BBS - API to access the diego DB
        • Brain - holds auctioneer
      • Loggregator
        • Metron - runs on the cell & passes logs
        • Doppler - gathers logs from Metron & drains them to other system like splunk
        • traffic controller
          • handles requests like cf logs
        • firehose
          • websocket endpoint exposes app logs, container metrics, ER component metrics.
          • pipe into nozzles to send data to places like data dog
      • Cloud Controller API
        • exposes API for using and managing the ER
        • persists org/space/app data in CC DB
        • Blob store
          • CC persists app packates and droplets to blob store
        • CC Bridge - translates app specific messages into generic languages of tasks & LRPs
      • Routing
        • The router routes traffic to appropriate components
          • URL -> Cell
          • CF Push -> CC
          • Etc.
      • Key flows
        • Stage and Run Request Flow
          • Cloud Controller passes request to run and stage applications to the CC bridge
          • Cloud Controller -> CC-Bridge -> BBS -> Brain [Auctioneer] -> Rep [Executor] -> Container provisioned on Cell
      • Buildpacks responsible for building module
      • High Availability
        • Availability Zones
        • BOSH
          • Elastic runtime processes are monitored and automatically restarted
          • Failed VMs - BOSH is sending messages to Message bus
            • if heartbeat fails it builds a new instance
        • Failed VMs
        • self healing (brain)
  • Watch Clip M02-02 Logging, Scale, and HA (Part 2)
    • Logging for every part of your app, and every part of PCF that touches your app
    • cf logs APPNAME - shows all logs
    • cf events APPNAME - audit trail, good for troubleshooting
  • Complete Lab M02-01 Logging, Scale, and HA Lab
  • Watch Clip M02-03 Logging, Scale, and HA – Recap

M03 – Services and Manifests

  • Watch Clip M03-01 Services
    • Cloud Native Apps
    • Managed Services
    • User Provided Service Instances
      • Used to connect existing services (DB2, Oracle, in house MySQL, etc) to your PCF instances
      • Also used for App to App binding
      • Exposes itself through VCAP
    • 12 factors
      • Configuration
        • store config in the environment (store config in OS environment variables)
        • cf env - display environment variables
          • VCAP_SERVICES is where all this will show up in the ENV
      • Backing Services
        • Treat backing services as attached resources
    • Marketplace provides a variety of services
  • Complete Lab M03-01 Services
  • Watch Clip M03-02 Services - Recap
  • Watch Clip M03-03 Manifest
  • Complete Lab M03-02 Manifest Lab
  • Watch Clip M03-04 Manifest – Recap

M04 – Application Security Groups and Log Drain

  • Watch Clip M04-01 Application Security Groups
    • Application Security Group
      • virtual firewalls for ingress/egress
      • staging security groups
        • whitelist what the app should have access to when staging
          • things needed for staging, but not things like customer data
      • running security groups
        • Whitelist what things apps should have access to while running
      • space security groups
        • Whitelist what things apps should have access to while running
        • only allow access to customer DB, don’t allow access to internet
      • security group rule file
      • cf security-groups - lists all security groups
      • cf security-groups NAME - lists details of rules
      • cf staging-security-groups - list all staging groups
      • cf running-security-groups - list all running groups
  • Complete Lab M04-01 Application Security Groups Lab
  • Watch Clip M04-02 Application Security Groups - Recap
  • Watch Clip M04-03 Log Drain
    • http://12factor.net/logs
    • treat logs as event streams
    • loggregator: metron
      • executor captures STDOUT & STDERR
    • Doppler: gathers logs from metron
    • Traffic controller
      • exposes firehose of app logs, container metrics, ER component metrics
      • firehose consumed through nozzles
  • Complete Lab M04-02 Log Drain Lab
  • Watch Clip M04-04 Log Drain Lab

M05 - Blue-Green and Microservices

  • Watch Clip M05-01 Blue-Green Deployments
    • Blue-Green Deployments
      • Zero Downtime
    • Blue Green Routing
    • Implecations on App Design
      • if using serialized objects, don’t make descructive changes.
        • don’t remove fields, do have a serialVersionUID
    • Admin Processes
      • Run admin/mgmt tasks as one-off processes.
    • Database
      • Do make changes idempotent
        • copy data to a new field (don’t delete data)
      • no descructive database changes allowed
        • don’t drop a column
      • do have backwards compatible changes
        • nullable fields
  • Complete Lab M05-01 Blue-Green Deployments Lab
  • Watch Clip M05-02 Blue-Green Deployments - Recap
    • blue green is for no downtime when deploying new versions
  • Watch Clip M05-03 Microservices
    • The monolith
      • a three tiered monolith (browser, app, database)
      • not good for the cloud
      • rely on session state being passed around.
      • couples changes cycles together since the whole application is one artifact
      • services can’t scale independently
      • too many developers in one code base
      • developers struggle to understand a big code base
      • long term commitment to the tech stack
    • Microservices
      • All teams will expose data & functionality through service interfaces
      • Micrsoservices are a loosley coupled Service Oriented Architecture (SOA) with bounded contexts.
      • Microservices should know how to discover each other & connect
      • A microservice has a single responsibility
        • like linux command line items being piped together
      • bounded context
      • organize around business capabilities
      • Polyglot Persistence
        • freedom to pick the persistence solution.
        • choice of language when devloping apps.
        • device specific gateways
        • use cloud friendly protocols
          • http, rest
      • microservice benefits
        • change cycles are decoupled, enabling frequent deploys
        • allows for efficient scaling
        • developers learn a smaller codebase faster
        • fewer developers in each code base
      • microservices challenges
        • http://martinfowler.com/bliki/MicroservicePrerequisites.html
        • http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html
    • Microservices & PCF
      • microservices have significant operations overhead
        • ways that PCF fixes this
          • buildpacks help reduce this work
          • health mgmt - 4 levels of HA
          • services makes it easy to provision
          • monitoring
          • Scaling
          • Dynamic Routing
      • substantial devops skills are required to run microservices
        • ways that PCF fixes this
          • health mgmt
          • buildpacks
          • space parity & immutable infrastructure
          • polyglot persistence via service brokers
  • Watch Clip M05-04 Microservices – Recap

M06 - Application Autoscaler, Performance Monitor, and Metrics

  • Watch Clip M06-01 Application Autoscaler
  • Complete Lab M06-01 Application Autoscaler Lab
  • Watch Clip M06-02 Application Autoscaler - Recap
  • Watch Clip M06-03 Application Performance Monitor
  • Complete Lab M06-02 Application Performance Monitor Lab
  • Watch Clip M06-04 Application Performance Monitor – Recap
  • Watch ClipM06-05 Metrics
  • Complete Lab M06-08 Metrics Lab
  • Watch Clip M06-09 Metrics - Recap

M07 - Buildpacks and Service Brokers

  • Watch Clip M07-01 Buildpack
    • Buildpack API
      • bin/detect - determines whether the buildpack can stage the application
        • looks for gemfile, setup.py, etc.
      • bin/compile - build the droplet
        • building droplet to meet app needs (java vm, ruby interpreter, etc)
        • adds additional librarires as needed
      • bin/release - provides informaiton on how to run the application
        • creates the yaml config with info on how to start the application
  • Complete Lab M07-01 Buildpack Lab
  • Watch Clip M07-02 Buildpack - Recap
  • Watch Clip M07-03 Service Broker
  • Complete Lab M07-02 Service Broker Lab
  • Watch Clip M07-04 Service Broker - Recap

M08 - Continuous Delivery and Route Service

  • Watch Clip M08-01 Continuous Delivery
    • Countinuous Delivery - build software so it can be released to product at any time
      • continuous delivery != continuous deployment
      • The technical team should be ready, but business decides when to deploy
      • continuous integration
        • does the build & possibly testing
      • don’t create environment specific packages
        • don’t bake in configuration
      • externalize environment config
      • do not assume existing process is right
      • don’t use different process for different environments
      • do recreate app environments frequently
      • do ensure database changes are automated
      • do deploy less more frequently
      • do automate all testing where test failures would prevent production release from occuring
      • do try to use tools that support the process
      • blue-green strategy is a good plan
    • PCF enabling continuous delivery
      • get a new app environment in seconds
      • have complete consistency between environments
      • have a consistent api to automate deployments
      • inject environment specific configuration
      • promote apps through environments with the same process
  • Complete Lab M08-01 Continuous Delivery Lab
  • Watch Clip M08-02 Continuous Delivery – Recap
  • Watch Clip M08-03 Route Service
    • Purpose - provide transformation or processing to requests before/after they reach an application
    • Examples
      • authentication
      • rate limiting
      • logging
      • caching
    • request flow
      • client -> load balancer -> router -> route service -> load balancer -> router -> app
        • route service receivs the call from the router, does it’s bit to do what is needed, then send the request back through the load balancer & router. There are additional http headers added during this process so this extra circle only happens once. The second time through the load balancer & router the call will go to the app.
        • Service broker could provision the route service for you
        • header added:
          • X-CF-Forwarded-Url - The URL of the application route
          • X-CF-Proxy-Metadata & X-CF-Proxy-Signature - used by the router to validate the request & pass down to the application
  • Complete Lab M08-02 Route Service Lab
  • Watch Clip m8-04 Route Service - Recap

M09 - Advanced Topics

  • Tasks
    • good for one time things
      • emails, staging, db setup, etc
  • File system as a service
    • independent of container
    • uses NFS
    • creates NFS as a service for PCF
  • Container to container routing
    • after enabled apps can go direct to each other instead of having to go to the Go router
    • Application URLs are no longer public
    • Client side load balancing required
    • must be enabled by platform admin
    • must install network-policy-plugin
  • TCP routing
    • All apps exposed over http & https

      M09 – Conclusion

  • Watch Clip M09-01 Conclusion