Development, Gaming, and Hacking
Brought to you by:
Matrix Tools
(3 / 62 Articles)
View Categories

AWS Certification Notes (Chapter 2: EBS and EFS)

NOTES:


Availablity Zone = Different Data Centers broken out by Region (AZ for short and they always end with a LETTER, Regions end with a NUMBER)
IAM = Identity Management / Users and Roles at a Global Level
IAM Foundation = For Company Integration like Active Directory (SAML)
EC2 = Virtual Machine (default is Linux with a Firewalled / Dynamic / Public IP, if you need a Static IP create an "Elastic IP")
SSH into an EC2 = ssh -i {path.pem} ec2-user@{ip}
Security Groups = Control traffic rules to the EC2 (Firewall)
Time outs = Security Group configuration issue / Connect Refused = Application issue
Security Groups can reference other Security Groups, multiple instances, etc. (They are locked down to a Region)

Practice: Install Apache on an EC2

  1. SSH into the EC2 (Linux) with user "root"
  2. sudo yum update
  3. sudo yum install -y httpd.x86_64
  4. sudo systemctl start httpd.service
  5. sudo systemctl enable httpd.service (restart the service on reboot)
  6. Open Port 80 via Security Group
  7. echo "Hello World from MatrixTools-EC2-01 - $(hostname -f)" > /var/www/html/index.html

Practice: Let's Automate That!

  1. EC2 User Data can run scripts at first boot (bootstrapping)
  2. EC2 - Launch Instance - Amazon Linux 2 AMI
  3. Configure Instance Details - Advanced Details - User data
  4. Paste in the "script" from above
  5. It must start with the following line, preceeding any script:
  6. #!/bin/bash

EC2 Instances - Launch Types

  1. On-Demand
    • Pay for what you use
    • Highest cost but no upfront payment
    • No long-term commitment
    • Recommendation: for auto-scaling or short-term and un-interrupted workloads, wehre you can't predict how the application will behave
  2. Reserved
    • Up to 75% discount compared to On-demand
    • Pay upfront for what you use with long-term commitment
    • Reservation period can be 1 or 3 years
    • Reserve a specific instance type
    • Recommendation: for steady state usage applications (think database)
  3. Convertible Reserved
    • Up to 54% discount compared to On-demand
    • Can change the EC2 instance type
  4. Scheduled Reserved
    • Only launch within time window you reserve
  5. Spot
    • Best Discount: up to 90% compared to On-demand
    • You bid a price and get the instance as long as its under the price
    • Price varies based on offer and demand
    • Spot instances are reclaimed with a 2 minute notification warning when the spot price goes above your bid
    • Recommendation: for batch jobs, Big Data analysis, or workloads that are resilient to failures. NOT for critical jobs or databases
  6. Dedicated Hosts
    • Physical dedicated EC2 server / Full control and visibility
    • You bid a price and get the instance as long as its under the price
    • Allocated for your account for a 3-year period reservation
    • More expensive
    • Recommendation: for complicated licensing / regulatory or compliance needs (must keep software and data on separate machine)
  7. Dedicated Instances
    • Instances running on hardware that's dedicate to you
    • May share hardware with other instances in same account
    • No control over placement

EC2 Instances Types

  • R = applications that need a lot of RAM
  • C = applications that need a lot of CPU (databases)
  • M = applications that need balance (middle / medium)
  • I = applications that need good local I/O (databases)
  • G = applications that need good GPU (video rendering / machine learning)
  • T2/T3 = Burstable Instances ("burst credits")
  • T2/T3 Unlimited = Unlimited Bursts

EC2 AMIs (custom base images)

  • Pre-installed packages, settings, etc
  • AMI Storage lives on S3 (inexpensive, just remove old ones)
  • Public and sharable ones in the marketplace
  • Just right click on your service instance - Image - Create Image
  • Images - AMIs - right click - Launch (AMIs are region specific / cannot use same ID across regions)
  • FAQ: You can copy and share AMIs and make them public by changing the permissions. If you cannot copy an AMI a lot of times you can still Launch an instance of it and create your own AMI from that instance (billingProduct code issues)

EC2 Placement Group Strategies

  • Cluster - a low-latency group in a single Availability Zone (high performance / high risk)
  • Spread - across hardware and AZ (critical applications / low risk / limit 7 instances)
  • Partition - across partitions (hardware within a single AZ / compromise)

EC2 for Solution Architects

  • Billed by the second, t2.micro is free tier
  • Lock down port 22 except for needed SSH (chmod 0400)
  • Timeout issues - Security groups issues
  • Security Groups can reference other Security Groups

ELB (Load Balancer for EC2)

  • Classic / Application / Network
  • Built-in Health Check
  • Use the static hostname NOT the underlying IP
  • Cannot scale instantaneously - contact AWS for a "warm-up"
  • 4xx errors are client, 5xx are application, 503 means no capacity or no targets
  • If LB cannot connect to application, check security groups
  • Application (Layer 7)
    • Load balancing to multiple machines (target groups)
    • Load balancing to multiple applications on same machine (containers)
    • Load balancing based on route in URL
    • Load balancing based on hostname in URL
    • Port Mapping feature to redirect to a Dynamic Port
    • Stickiness can be enabled at the target group level by cookies
    • Supports HTTP, HTTPS, and Websockets
    • The application servers don't see the IP of the client directly, but it is placed in a header (X-Forwarded-For, X-Forwarded-Port, X-Forwarded-Proto)
  • Network (Layer 4)
    • Load balancing for TCP
    • High performance ~ millions of requests per seconds
    • Support for static IP or elastic IP (1 per AZ)
    • Less latency ~ 100 ms (vs 400 ms for ALB)
    • Use if Extreme Performance is required
    • Can see the client IP directly
    • Public facing = must attach Elastic IP – can help whitelist by clients

More Load Balancer Notes

  • App LB provide a Static DNS name
  • Network Load Balancers expose a public static IP (and can work witih TCP)
  • Use STICKINESS if Session Data is important (client gets same instance via cookie)
  • 0.0.0.0/0 means allow anyone from anywhere
  • Can also use a (LB) security group, so that traffic has to come from load balancer

Auto Scaling

  • Cooldown = period between each scale action
  • ASG = Auto Scaling Group
  • Default Termination Policy = AZ with most instances and then oldest config
  • ASG are free and also have auto-restart to KEEP running X number of instances!
  • Scale based on CloudWatch alarms (Custom Metrics with PutMetric API)
  • Automatically Register new instances to a load balancer
  • SNI = to specify SSL hostname they reach
  • ACM = AWS Certificate Manager (X.509 SSL/TLS server certificate)

Chapter 1: Servers and Load Balancers

Permanent Link
If this article helped you, or you have any thoughts on how to do this better, please click the Like button and/or leave a comment below.

Likes: [0]

Comments: [0]






(if you'd like us to reply and/or show your gravatar image)

AWS Certification Notes (Chapter 1: Servers and Load Balancers)

NOTES:


Availablity Zone = Different Data Centers broken out by Region (AZ for short and they always end with a LETTER, Regions end with a NUMBER)
IAM = Identity Management / Users and Roles at a Global Level
IAM Foundation = For Company Integration like Active Directory (SAML)
EC2 = Virtual Machine (default is Linux with a Firewalled / Dynamic / Public IP, if you need a Static IP create an "Elastic IP")
SSH into an EC2 = ssh -i {path.pem} ec2-user@{ip}
Security Groups = Control traffic rules to the EC2 (Firewall)
Time outs = Security Group configuration issue / Connect Refused = Application issue
Security Groups can reference other Security Groups, multiple instances, etc. (They are locked down to a Region)

Practice: Install Apache on an EC2

  1. SSH into the EC2 (Linux) with user "root"
  2. sudo yum update
  3. sudo yum install -y httpd.x86_64
  4. sudo systemctl start httpd.service
  5. sudo systemctl enable httpd.service (restart the service on reboot)
  6. Open Port 80 via Security Group
  7. echo "Hello World from MatrixTools-EC2-01 - $(hostname -f)" > /var/www/html/index.html

Practice: Let's Automate That!

  1. EC2 User Data can run scripts at first boot (bootstrapping)
  2. EC2 - Launch Instance - Amazon Linux 2 AMI
  3. Configure Instance Details - Advanced Details - User data
  4. Paste in the "script" from above
  5. It must start with the following line, preceeding any script:
  6. #!/bin/bash

EC2 Instances - Launch Types

  1. On-Demand
    • Pay for what you use
    • Highest cost but no upfront payment
    • No long-term commitment
    • Recommendation: for auto-scaling or short-term and un-interrupted workloads, wehre you can't predict how the application will behave
  2. Reserved
    • Up to 75% discount compared to On-demand
    • Pay upfront for what you use with long-term commitment
    • Reservation period can be 1 or 3 years
    • Reserve a specific instance type
    • Recommendation: for steady state usage applications (think database)
  3. Convertible Reserved
    • Up to 54% discount compared to On-demand
    • Can change the EC2 instance type
  4. Scheduled Reserved
    • Only launch within time window you reserve
  5. Spot
    • Best Discount: up to 90% compared to On-demand
    • You bid a price and get the instance as long as its under the price
    • Price varies based on offer and demand
    • Spot instances are reclaimed with a 2 minute notification warning when the spot price goes above your bid
    • Recommendation: for batch jobs, Big Data analysis, or workloads that are resilient to failures. NOT for critical jobs or databases
  6. Dedicated Hosts
    • Physical dedicated EC2 server / Full control and visibility
    • You bid a price and get the instance as long as its under the price
    • Allocated for your account for a 3-year period reservation
    • More expensive
    • Recommendation: for complicated licensing / regulatory or compliance needs (must keep software and data on separate machine)
  7. Dedicated Instances
    • Instances running on hardware that's dedicate to you
    • May share hardware with other instances in same account
    • No control over placement

EC2 Instances Types

  • R = applications that need a lot of RAM
  • C = applications that need a lot of CPU (databases)
  • M = applications that need balance (middle / medium)
  • I = applications that need good local I/O (databases)
  • G = applications that need good GPU (video rendering / machine learning)
  • T2/T3 = Burstable Instances ("burst credits")
  • T2/T3 Unlimited = Unlimited Bursts

EC2 AMIs (custom base images)

  • Pre-installed packages, settings, etc
  • AMI Storage lives on S3 (inexpensive, just remove old ones)
  • Public and sharable ones in the marketplace
  • Just right click on your service instance - Image - Create Image
  • Images - AMIs - right click - Launch (AMIs are region specific / cannot use same ID across regions)
  • FAQ: You can copy and share AMIs and make them public by changing the permissions. If you cannot copy an AMI a lot of times you can still Launch an instance of it and create your own AMI from that instance (billingProduct code issues)

EC2 Placement Group Strategies

  • Cluster - a low-latency group in a single Availability Zone (high performance / high risk)
  • Spread - across hardware and AZ (critical applications / low risk / limit 7 instances)
  • Partition - across partitions (hardware within a single AZ / compromise)

EC2 for Solution Architects

  • Billed by the second, t2.micro is free tier
  • Lock down port 22 except for needed SSH (chmod 0400)
  • Timeout issues - Security groups issues
  • Security Groups can reference other Security Groups

ELB (Load Balancer for EC2)

  • Classic / Application / Network
  • Built-in Health Check
  • Use the static hostname NOT the underlying IP
  • Cannot scale instantaneously - contact AWS for a "warm-up"
  • 4xx errors are client, 5xx are application, 503 means no capacity or no targets
  • If LB cannot connect to application, check security groups
  • Application (Layer 7)
    • Load balancing to multiple machines (target groups)
    • Load balancing to multiple applications on same machine (containers)
    • Load balancing based on route in URL
    • Load balancing based on hostname in URL
    • Port Mapping feature to redirect to a Dynamic Port
    • Stickiness can be enabled at the target group level by cookies
    • Supports HTTP, HTTPS, and Websockets
    • The application servers don't see the IP of the client directly, but it is placed in a header (X-Forwarded-For, X-Forwarded-Port, X-Forwarded-Proto)
  • Network (Layer 4)
    • Load balancing for TCP
    • High performance ~ millions of requests per seconds
    • Support for static IP or elastic IP (1 per AZ)
    • Less latency ~ 100 ms (vs 400 ms for ALB)
    • Use if Extreme Performance is required
    • Can see the client IP directly
    • Public facing = must attach Elastic IP – can help whitelist by clients

More Load Balancer Notes

  • App LB provide a Static DNS name
  • Network Load Balancers expose a public static IP (and can work witih TCP)
  • Use STICKINESS if Session Data is important (client gets same instance via cookie)
  • 0.0.0.0/0 means allow anyone from anywhere
  • Can also use a (LB) security group, so that traffic has to come from load balancer

Auto Scaling

  • Cooldown = period between each scale action
  • ASG = Auto Scaling Group
  • Default Termination Policy = AZ with most instances and then oldest config
  • ASG are free and also have auto-restart to KEEP running X number of instances!
  • Scale based on CloudWatch alarms (Custom Metrics with PutMetric API)
  • Automatically Register new instances to a load balancer
  • SNI = to specify SSL hostname they reach
  • ACM = AWS Certificate Manager (X.509 SSL/TLS server certificate)

Chapter 2: EBS and EFS

Permanent Link
If this article helped you, or you have any thoughts on how to do this better, please click the Like button and/or leave a comment below.

Likes: [0]

Comments: [0]






(if you'd like us to reply and/or show your gravatar image)

Azure DevOps and Automation


Category
Cloud Computing Services
Sub Category
Azure
Last Updated
6/26/2019

Azure DevOps and Automation

How I setup a FREE Automated Pipeline with Smoke Tests!

Applications Used: GitHub, Azure Pipelines, Slack and Monkey Test It for the Smoke Test.

First of all, here are the application integrations (you really only need the first integration and then any Slack notifications you would want.)
GitHub -> Azure Pipelines
GitHub -> Slack
Azure Pipelines -> Slack
MonkeyTest.it -> Slack

With all of those integrations you should be able to get feedback on every step into a Slack channel.
You will also notice that the GitHub -> Azure Pipelines integration will add an azure-pipelines.yml to the root folder of the project. This is the file that I will use to wire up Monkey Test It.

  1. Log into your Monkey Test It dashboard.
  2. Get your API key and an example of how to use it.
  3. Log into Azure.
  4. Here you can use the Pipeline Editor to update your yml file.
  5. You can use this or a Post Deployment Script to kick off the Monkey Test It API.

There are now also settings on the Website Deployment page in Visual Studio to configure a Continuous Delivery pipeline!

Other Azure DevOps and Automation Links


Do you prefer Azure or AWS?


Permanent Link
If this article helped you, or you have any thoughts on how to do this better, please click the Like button and/or leave a comment below.

Likes: [0]

Comments: [0]






(if you'd like us to reply and/or show your gravatar image)