AWS Advanced

S3

https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13886468#overview

• Attributes for objects: Key, Value, VersionID, Metadata, Sub-resources: ACL, Torrent.
• All buckets are private when created.
• Supports MFA delete.
• Read after Write consistency (POST/ PUT of new objects).
• Eventual consistency for overwrite PUT and DELETE (can take sometime to propagate)
• Control bucket access using bucket ACL or bucket policy.
• Buckets can be configured to create access logs.
• Client side encryption (Encryption-in transit) is through SSL/TLS or send encrypted data.
• Server side encryption (Encryption-at Rest)
  • S3 Managed Keys: SSE-S3 (AWS fully manages)
  • KMS Managed Keys: SSE-KMS (Customer + AWS manages)
  • Server side encryption with customer provided keys: SSE-C (Customer fully manages)
• New version of S3 object permission is not inherited, admin needs to give permission.
• When version enabled file is deleted, it will be assigned as “delete marker” and file is not actually deleted unless specifically delete the “delete marker” file.
• Cross region replication requires versioning enabled for both source and destination buckets.
  • Region must be unique.
  • Cross region replication happens after enabled and not to old objects.
  • Replication delete is prohibited and “delete marker” will be not replicated.
• Transfer acceleration tool says how fast upload works when we do  S3 Transfer acceleration.
• Lifecycle policy can be applied with versions as well.

CloudFront

• Edge locations are used for both READ and WRITE ops.
• Objects are cached with TTL.
• Explicit invalidation of cache is possible, but will be charged.
  • invalidate file or object or whole bucket (/*) is possible.
• Origin: origin where files are get to CloudFront (EC2, S3, ELB, Route53).
• Distribution: collection of edge locations.
  • Web Distribution: static (.html, .css, .php) and dynamic (live streaming) contents  over HTTP/S. Supported origin is S3 or EC2 first and after creating add more origin.
  • RTMP Distribution: streaming media files using Adobe Flash Media Servers’ RTMP protocol (helps to stream video before complete download). Supported origin is S3 only.
• Signed url/ cookies:

Storage Gateway

• Supports both virtual or physical gateway to backup data from on-prem storage solution to S3. Basically does offsite backup to S3.
• 3 Types:
  1. File Gateway (NFS & SMB)
     • file storage in S3 as objects accessed via NFS mount point.
     • ownership, permission, timestamp are stored in S3 metadata.
     • once moved, objects are managed native to S3 ops (versions).
     • 
  2. Volume Gateway (iSCSI)
     • Asynchronously stores HDD as EBS snapshots in S3.
     • Snapshots are incremental and compressed.
       • Stored volume: stores entire data set (from HDD) in GW and backed-up  from on-prem to S3.
       • 
       • Cached volume: stores frequently used data in GW and backed-up from on-prem to S3.
       • 
  3.  Tape Gateway
     • To archive data to S3.
     • VTL (Virtual Tape Library) interface helps to backup from virtual tape.
     • 

EC2

https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13886556#overview

• FIGHT-DRMCP-XZ-AU
• Root volume can’t be encrypted by DEFAULT AMI, but additional EBS volume can.
  • Can use 3rd party tool bit-locker to encrypt root volume or creating AMI in console or API.
• Termination protection is turned-OFF by default.
• When AWS terminate Spot instance, you will not be charged for partial usage. But if you terminate, you will be charged for any hr in which instance ran.
• Any SG rule changes, update reflects immediately.
• All inbound traffic is blocked by default.
• All outbound traffic is allowed by default.
• SG is stateful, meaning whenever inbound rules are added, corresponding outbound rules are added by default. No blocking allowed.
• NACL is stateless meaning, explicit inbound and outbound rules have to defined and explicit blocking is allowed.
• Have any # of EC2 instances within SG.
• More than one SG groups can be added to EC2.
• SG only allows allow rules but not deny rules.
• Removing Autoscaling Group will remove all EC2 instances.
• EBS volume must be same AZ as EC2.
  • When modify EBS volume (size or any storage type change), we needs to run Linux cmd to see the modified size in EC2 and performance of EC2 affected due change (no stop instance is required).
  • With EBS Snapshots (exist in S3), we can move EBS volume between AZs and region.
    • Create snapshots -> turn to AMI -> launch EC2 in other AZ (based on subnet).
    • Copy AMI from one region (source) to other region (destination) and spin new EC2 from that AMI.
  • Snapshots are point in time copies of volume.
  • Snapshots are incremental (only changed block will be added to S3, not whole volume).
  • On EC2 termination, other than root EBS volume persists.
  • Root volume snapshot can be taken during EC2 runs (perf get affected) or stop EC2 and then take snapshots.
  • You can create AMI from both volume and snapshot.
• 
• AMI can be selected based on
  • Region
  • OS
  • Architecture (32-bit or 64-bit)
  • Launch permission
  • Root device type
    • Instance Store (EPHEMERAL storage; volume created from template stored in S3)
      • It cannot be stopped (if hypervisor failed, you lose data)
    • EBS backed volumes (volume created from EBS snapshots)
      • It can be stopped.
    • Reboot both, data persist.
    • By default, both ROOT volumes will be deleted on terminations, but can be persisted with option to persist data.
  • Encrypted Root Device Volumes:
    • Earlier, cannot provision EC2 with encrypted root volume.
      • Process involved to achieve that:
        • provision EC2 with unencrypted root volume
        • take snapshot
        • create a copy of that snapshot with encrypt option enabled
        • create AMI from encrypted snapshots
        • launch EC2 from that AMI, we will get Encrypted Root Device Volume.
    • Now, can provision EC2 with encrypted root volume.
    • Encrypted Root Device Volume cannot be changed to unencrypted.
    • Sharing of snapshots is possible when it’s unencrypted.
• EC2 Instance Metadata
  • Info about EC2 instance.
  • http://169.254.169.254/latest/meta-data/
  • http://169.254.169.254/latest/user-data/ (returns user specific data such as 1234,john,reboot,true | 4512,richard, | 173,,,).
• Placement Groups: Grouping of EC2 instances.
  • PG name must be unique within AWS acct.
  • Only certain instance types supports (Compute, Memory , Storage Optimized, GPU).
  • AWS recommends to have homo-instance within Clustered PG.
  • Can’t merge the PGs.
  • Can’t move the existing instance to PG. Create AMI from your existing instance, then launch new instance from AMI int aPG.
  • 3 types:
    • Clustered PG: Within single AZ. Good for low N/W latency, high throughput. Only certain instances can be launched in this group. Cannot span multiple AZs.
    • Spread PG: placed on distinct underlying H/W. Good for small # of critical instances that should be separate from each other. (separate racks within AZs, to avoid failure of instance/ service when H/W fails). Can span multiple AZs.
    • Partitioned PG: AWS divides each group into logical segments called “partitions”. Good for HDFS, HBase, Cassandra apps. Each partitions within PG has its own set of racks. Each racks has its own n/w and power supply. No two partition within PG share the same racks, allowing to isolate h/w failure within applications. Can span multiple AZs.

CloudWatch

• Monitors performance of AWS services and customer’s services runs on AWS.
• Monitors events every 5 mins and can turn 1 min detailed monitoring (additional charges).
• CloudWatch Alarm can trigger notification.
  • Compute
    • Services:
      • EC2
      • Autoscaling groups
      • ELB
      • Route53 Health Checks
    • Metrics:
      • CPU
      • N/W
      • Disk
      • Status Check (hypervisor and underlying infra).
      • Cost
  • Storage and Content Delivery
    • Services:
      • EBS volumes
      • Storage Gateways
      • CloudFront

CLI

• Access key and secret ID of user will be generated and viewed once for programmatic access via CLI. If you lose, then make the current Access key and secret ID inactive and create one.
• There is a hidden dir called “.aws” (after running aws configure cmd) which stores the Access key and secret ID of user. That’s the bad practice of accessing services in AWS.
  • Instead define Role to EC2 instance and attach to it and remove whenever you need.

EFS

• EBS volume cannot be shared with more than one EC2, but EFS can share (since its common) and its easy to be elastic than EBS (it has performance impact when elasticity implemented).
• Works on NFS4 protocol.
• Pay for storage and scale petabytes.
• Support 1000s of concurrent connections.
• Span within region in multiple AZs.
• Read after write consistency.
• Lifecycle Management is supported thru different EFS class (EFS-IA), similar to S3.
• Mount the EFS on required instance and share it, we can see the changes on all EC2’s.

Databases

https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13886690#overview

• RDS has 2 key features:
  • Multi-AZ:
    • Primary and secondary DBs in each AZ with DNS redirection to secondary whenever primary fails (fail-over is automatic).
    • For disaster recovery purpose and does synchronous replication.
    • Supports SQL server, Oracle, MySQL, PostgreSQL, MariaDB.
    • Force fail-over by rebooting RDS instance.
  • Read Replica:
    • Primary and read replica DBs in each AZ.
    • For performance purpose and does asynchronous replication.
    • Primarily for READ ops.
    • Supports Oracle, MySQL, PostgreSQL, MariaDB, Aurora.
    • Must have automatic backup turned-on to deploy read replica.
    • Can have 5 RR copies of any DB.
    • Can have RR of RR (but latency will be issue).
    • Each RR has own DNS endpoint.
    • Can have RR with multi-AZ and multi-AZ source DB.
    • RR can be promoted to own DB with replication breakage.
    • Can have RR in 2nd region (from source DB in 1st region).
• RDS runs on VM (non-serverless) which AWS manages.
• Reboot with a failover option makes RDS come-up automatically after reboot.
• Aurora Serverless is serverless.
• Whenever delete RDS instance, there is an option to take snapshot or not.
• Two types of backup:
  • Automated backup:
    • Allow to recover DB at point in time within “retention period” (can be 1 to 35 days).
    • Takes full daily snapshots and store transaction log throughout the day.
    • Point-in-time recovery: when do recovery, AWS will first choose the most recent daily back up and then apply transaction log relevant to that day.
    • Enabled by default and backups stored in S3 (free storage equal to size of DB).
    • Backups takes defined window and that time I/O may be suspended and hence apps delay see latency delay in response.
    • Backups gone when delete the original RDS instance.
  • Database snapshots: 
    • Done manually and exist event after delete the original RDS instance unlike automated backups.
• When restore backup, it will be a new instance and new DNS endpoint.
• Encryption at rest is supported with KMS service.
  • All automated backups, read-replica and snapshots are encrypted as well.
  • Supports SQL server, Oracle, MySQL, PostgreSQL, MariaDB, Aurora.
• DynamoDB
  • Serverless; Stored on SSD
  • Spread 3 geographically distinct DCs.
  • Eventual Consistent Reads (default)
    • Consistency across all copies of data is usually reached within a second after write. (best read performance)
  • Strong Consistent Reads
    • Return results that reflects all writes that received a successful response prior to the read. (best write performance)
• Redshift
  • Configuration:
    • Single Node (160gb)
    • Multi Node
      • Leader (manages client connections and receives queries)
      • Compute (store data, perform queries and computations; upto 128 nodes).
  • Advanced Compression
    • Columnar data can be compressed much more than row data because similar data stored sequentially on disk.
  • When compare to traditional DB:
    • Uses multiple compression techniques.
    • No index or views usage, so less space.
    • When loading data into an empty table, Redshift automatically samples data and select appropriate compression scheme.
  • Massively Parallel Processing (MPP) can be achieved through # of nodes that can do fast query processing.
  • Backups:
    • Enabled by default with a 1-day retention period with maximum period is 35 days.
    • Maintains 3 copies of data (original, replica on compute nodes and backup in S3).
    • Asynchronous snapshots replication to S3 in another region for DR.
  • Charged for compute nodes hrs and not for leader node hrs, backup and data transfer within VPC (not outside).
  • Encryption-at Rest supported with AES-256 with KMS or own key from CloudHSM.
  • 1-AZ support (no multi-AZ).
  • Can restore snapshots to new AZ when outage.
• Aurora
  • MySQL flavor RDBMS, 5 times faster than MySQL.
  • Starts with 10GB scales in increments to 64TB (storage autoscaling).
  • Compute resources cans scale upto 32vCPUs and 244GB of memory.
  • 2 copies of data is contained in each AZ with min of 3 AZs, so 6 copies of data.
  • Creates cluster of reader and writer nodes.
  • Withstand 2 copies of data without affecting DB write availability and upto 3 copies without affecting read availability.
  • Storage is self-healing, data blocks and disks are continuously scanned for errors and repaired automatically.
  • 2 types of Aurora Replicas:
    • Aurora Replicas (current 15; fail-over supported)
    • MySQL RR (currently 5)
    • 
  • Automated backups are always enabled and doesn’t impact DB performance.
  • Taking snapshots doesn’t impact DB performance.
  • Share snapshots with other AWS accounts.
• Elasticache

Route53

https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13886776#overview

• ELBs don’t have pre-defined IPV4 address; need to resolve them using DNS name.
• Start of Authority Record (SOA) in DNS contains administrative info/ record such as server name that supplied the data for the zone, administrator name, current version and default # of seconds for TTL file on resource records.
• Name Server (NS) Records are used by Top Level Domain (TDL) servers to direct traffic to the content DNS server which contains the authoritative DNS records.

• “A” (Address) Record is a fundamental type of DNS record that maps the domain to IP address.
• Default TTL is 48 hrs. Less TTL means frequent changes of DNS records.
• CNAME (Canonical Name) resolves one domain name to another. Eg: http://m.cloudguru.com and mobile.cloudguru.com resolves to same IP address.
• Alias Records map resources record sets in hosted zone to ELB, CF distribution, S3 bucket that are configured as websites.
  • Work like CNAME: can map one DNS name (www.example.com) to another “target” DNS name (elb123.elb.amazonaws.com).
  • Key difference is CNAME can’t be used for naked domain name (zone apex record, i.e., no “www” in the domain name). Can’t have CNAME for http://acloudguru.com, it must be either an A record or an Alias.
• Common DNS types:
  • SOA Records
  • NS Records
  • A Records
  • CNAMES
  • MX Records
  • PTR Records
• Domain name registration take s 3 days to complete, but not always.
• Health Checks can be set to individual record sets (Sydney, Ohio, Ireland EC2).
  • If record set fails a health checks, it will be removed from Route53 until it passes (checks apps endpoint – .index.html).
  • Also create alarm when fails to send SNS notifications.
• Routing Policy
  • Simple Routing
  •  Weighted Routing
  • Latency-based Routing
  • Failover Routing
  • Geolocation Routing
  • Geo-proximity Routing
  • Multi-value Answer
• Simple Routing

• Weighted Routing

• Latency-based Routing
  • Route traffic based on lowest N/W latency for end user (which region has a fastest response time).
  • Need to create latency resource record set for EC2 or ELB resource in each region (Sydney, Ohio, Ireland) where resource located.
  • When Route53 receives query for the resource, it select latency resource record set which has lowest latency and then respond to particular IP address of resource from resource record set.

• Failover Routing
  • Used in Active (primary) and Passive (secondary) setup.
  • Route53 monitor health of primary site using health check.

• Geo-location Routing

• Geo-Proximity (Traffic-Flow only) Routing
  • Route53 route traffic to resources based on geographic location of users and resources.
  • Optionally, choose route more or less traffic to given resource by specifying a value, known as “bias”.
  • A bias expands or shrinks the size of geographic region from which traffic is routed to a resource.
  • To use geo-proximity, must use Route53 Traffic-Flow.

• Multi-Value Answer Routing
  • Simple Routing Policy + Health check on each record set.

VPC

https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13886986#overview

• 1 subnet = 1 AZ (1 subnet cannot span across multiple AZ).
• Default VPC
  • All subnet in default VPC have a route to internet.
  • Each EC2 has both public and private IP addresses.

• VPC peering
  • Using private IP address connect one VPC to another.
  • Instance behave as is they were on same private n/w.
  • Peering can be done with same acct VPC and other AWS acct as well.
  • Peering is STAR configuration and not TRANSITIVE.
• When create a new VPC, following will be created OOB.

• AZ will be randomized for each AWS acct.
• The first four IP addresses and the last IP address in each subnet CIDR block are not available for you to use, and cannot be assigned to an instance. For example, in a subnet with CIDR block 10.0.0.0/24, the following five IP addresses are reserved:
  • 10.0.0.0: Network address.
  • 10.0.0.1: Reserved by AWS for the VPC router.
  • 10.0.0.2: Reserved by AWS. The IP address of the DNS server is always the base of the VPC network range plus two; however, we also reserve the base of each subnet range plus two. For VPCs with multiple CIDR blocks, the IP address of the DNS server is located in the primary CIDR. For more information, see Amazon DNS Server.
  • 10.0.0.3: Reserved by AWS for future use.
  • 10.0.0.255: Network broadcast address. We do not support broadcast in a VPC, therefore we reserve this address.

  

• After adding Subnets:

• Only on IGW can be attached to VPC.
• Route-Tables
  • Subnets can communicate each other with Route Table via 10.0.0.0: Network address.

• Default Route-Table is a main Route Table and if an IGW attached to VPC, then any subnets created will be public because it will associate with main Route Table. This is a security concern. To avoid that have:
  • Main Route-Table as private.
    • 
  • Separate Route-Table as public and allow any traffic to flow through.
    • 
    • 
• Auto-assign IP address option in EC2 instance creation enabled for public subnet and not for private.

• SSH tunneling can be done using SSH to access a resource without a public IP address via a resource with a public IP address (inside of a VPC).
• https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13886952#overview
• NAT-Instances
  • Individual EC2 instances at public subnet.
  • Disable Source/ Destination checks on the instance.
  • There should be route out of the private subnet to the NAT instance, in order for this to work.
  • Autoscaling of NAT-instances can be done with multiple subnets in different AZs and script to automate failure.
  • NAT instances are behind security group.
• NAT-Gateway
  • Highly available, managed, redundant gateway  inside AZs. (with multiple EC2 instances).
  • Starts at 5Gbps and scales to 45Gbps.
  • No patching required.
  • Not associated with SG.
  • Automatically assigned to public IP address
  • Need to update route tables and no need to disable  Source/ Destination checks.
  • Multiple NAT-GW can be used in different AZ to avoid SPOF.

• NACL:
  • When create VPC, a default NACL created by default (default NACL).
  • When create subnet, default NACL will be associated.
  • A subnet can have 1 NACL, but 1 NACL can be associated with multiple subnets.
  • Want deny, then have deny rule first.
  • EC2 creates default NACL with InBound and OutBound “rules” allows all traffic.
  • Newly created NACL is everything denied by default (both InBound and OutBound Rule *).
    • Stateless: InBound and OutBound must be defined explicitly (not like SG).
    • Ephemeral ports (1024-65535): short-lived transport port.
  • NACL act first and then SG.
  • Block IP address with NACL and not with SG.
• ELB:
  • Need at least 2 public subnets to associate ELB.
• VPC Flow Logs:
  • A feature that enables you to capture information about the IP traffic going to and from network interfaces in your VPC.
  • Flow log data can be published to CloudWatch Logs and Amazon S3. After you’ve created a flow log, you can retrieve and view its data in the chosen destination.
  • Cannot tag a flow log.
  • Created at 3 levels:
    • VPC
    • Subnet
    • N/W interface
  • Filter traffic based on Accept, Reject or All.
  • Destination: CloudWatch Logs, S3
    • Destination Log Group: ForCloudWatch Logs, create log group in CloudWatch.
  • IAM Role: create new IAM role with default policy document.
  • VPC peering flow logs only works when VPCs are in same account.
  • Once configure, you cannot change the settings (eg: no change in IAM role).
  • Not all IP traffic is monitored.
    • Traffic generated by instance when they contact the Route53 but, if use own DNS server, those traffic will  be monitored.
    • Traffic generated by Windows instance for Amazon windows license activation.
    • Traffic between 169.254.169.254.
    • DHCP traffic.
    • Traffic to Reserved IP address for the default VPC router.

• Bastion Host:
  • A special computer on a network specifically designed and configured to withstand attacks and safe other n/w behind it.
  • The host has a special application (proxy server only).
  • Hardened host because of location and purpose, either at DMZ or outside of firewall and has access to untrusted n/ws.
  • Cannot use NAT Gateway as Bastion host.

• Direct Connect:
  • A cloud service solution that makes it easy to establish a dedicated network connection from on-premises to AWS.
  • Reduces network costs, increase bandwidth throughput, and provide a more consistent network experience than Internet-based connections.
  • Eg: VPN Connection keeps dropping-off, then use DX.

• VPC Endpoints:
  • Enables to privately connect VPC to supported AWS services and VPC endpoint services powered by PrivateLink without requiring an internet gateway, NAT device, VPN connection, or AWS Direct Connect connection.
  • Instances in your VPC do not require public IP addresses to communicate with resources in the service.
  • Traffic between your VPC and the other service does not leave the Amazon network.
  • Endpoints are virtual devices.
  • They are horizontally scaled, redundant, and highly available VPC components that allow communication between instances in VPC and services without imposing availability risks or bandwidth constraints on network traffic.
  • Two types:
    • Interface EP
    • 
    • Gateway EP
      • similar to NAT Gateway.
      • Supports S3 and DynamoDB.
      • 

ELB

https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13888022#overview

• Application and Classic LB comes with DNS name and not with IP address. Network LB won’t get static IP address.
• Instances are monitored using health-check by ELB (states: InService or OutofService).
• Classic LB:
  • HTTP 504 Gateway Time-out Error  is due to web server or DB layers has some issues and request failed within idle timeout period.
  • Supports X-Forwaded-For-Header (forward IP address of user) and sticky session features.

• Target Group is a group consist of targets (types: Instance, IP ranges, Lambda functions) and target can be selected based on RULES you set and route the request based on decision.

• HA Architecture:

SQS

SWF

SNS

API Gateway

Kinesis

Cognito

https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13888022#overview

Lambda

• Services That Invoke Lambda Functions Synchronously

1. Elastic Load Balancing (Application Load Balancer)
2. Cognito
3. Lex
4. Alexa
5. API Gateway
6. CloudFront (Lambda@Edge)
7. Kinesis Data Firehose

• Services That Invoke Lambda Functions Asynchronously

1. Simple Storage Service
2. Simple Notification Service
3. Simple Email Service
4. CloudFormation
5. CloudWatch Logs
6. CloudWatch Events
7. CodeCommit
8. Config

• https://docs.aws.amazon.com/lambda/latest/dg/lambda-services.html#intro-core-components-event-sources

https://www.udemy.com/aws-certified-solutions-architect-associate/learn/lecture/13888044#overview