Cloud Tech - On demand computing power, on demand storage and in-build services for IT solution.
----------------------------------------------------------------------------------------------------------------------------
Cloud Types - Infrastructure As a Service ( Iaas ) - AWS EC2
- Software As a Services ( SaaS ) - Most of Inbuilt AWS Services
- Platform As a Service ( PaaS ) - AWS Beanstalk
----------------------------------------------------------------------------------------------------------------------------
AWS Infrastructure - Data Centers - Physical locations of AWS infrastructure
- Regions - Has multiple availability zones, Some regions has limited services
- Edge Locations - Users access AWS services via these locations
- Availability Zones - Set of data centers with redundant power and storage
----------------------------------------------------------------------------------------------------------------------------
AWS Global Services - Identity and Access Management ( IAM, IAM MFA, IAM Federation - User, Groups, Roles, Active Directory Integration, Never use ROOT other than initial setup )
- CloudFront ( CDN )
- Route 53 ( DNS )
- Firewall ( WAF )
----------------------------------------------------------------------------------------------------------------------------
AWS Regional Service - Computing Service ( EC2 - Infrastructure as a Service )
- App service ( Beanstalk - Platform as a Service )
- Lamda ( Function as a Service )
----------------------------------------------------------------------------------------------------------------------------
IAM - User, Groups, Roles, Password Policies, Multi-factor Authentication, One IAM account per physical user, One Role per Application , Policy attach to the group.
----------------------------------------------------------------------------------------------------------------------------
EC2 - ( Elastic Virtual Machines Elastic Block Storage, Elastic Load Balancer, Auto Scale Groups )
----------------------------------------------------------------------------------------------------------------------------
EC2 SSH command - ssh -i testkey.pem ec2-user@13.212.19.241
ssh -i testkey.pem ec2-user@ec2-13-212-220-134.ap-southeast-1.compute.amazonaws.com
----------------------------------------------------------------------------------------------------------------------------
EC2 IP Types - Public IP
- Private IP
- Elastic IP ( Private IP will not change after EC2 restart )
Note : Inbound rule 0.0.0.0/0 accepts any connection from any device
----------------------------------------------------------------------------------------------------------------------------
EC2 Apache Installation - yum install -y httpd
- systemctl start httpd.service
- systemctl enable httpd.service
----------------------------------------------------------------------------------------------------------------------------
EC2 Boostrap - EC2 User data scripts
----------------------------------------------------------------------------------------------------------------------------
EC2 Instant Types - On demand ( Per second price , highest price, good for short term elastic loads )
- Reserved ( Pay upfront with long term commitment with reserved instant type. Ex: DB setups )
- Reserved - convertible ( Can change instance type ) - Reserved - scheduled convertible
----------------------------------------------------------------------------------------------------------------------------
EC2 Spot Instances - Most cost efficient , Will terminate if your max price < spot price. Good for Batch jobs, Data analytics, Image processing which are resilience to failures
----------------------------------------------------------------------------------------------------------------------------
EC2 Spot Block Instances - Spot instances in a time frame without interruptions
----------------------------------------------------------------------------------------------------------------------------
EC2 Spot Request Type - Onetime / Persistence to renew based on current price
----------------------------------------------------------------------------------------------------------------------------
EC2 Spot Fleet - Set of spot instances and it will automatically request lowest spot instances.
----------------------------------------------------------------------------------------------------------------------------
EC2 Dedicated Hosts - Dedicated hardware and controls with 3 year reservations. Expensive.
----------------------------------------------------------------------------------------------------------------------------
EC2 Instance Types - R for RAM
- C for Compute, M for Medium,
- G for Graphics and ML
- I for DBs
- T1, T2 to handle any load
----------------------------------------------------------------------------------------------------------------------------
EC2 AMI - Customization images of given OS ( Required packages, Security, Faster boot-time without EC2 User data, Monitoring and other features like AD integrations ) , Can be stored in S3, Can share with other AWS account, Sharing AMI with account doesn't affect ownership of it, Sharing it with another region then they become owners of it. To copy an AMI, original owner should provide access for it. If you copy shared AMI, then you will be the owner of shared AMI. AMI can be stored in S3 or EBS. You can share the encrypted AMI with its keys only. Cannot share Windows or AWS Marketplace AMI which are having billing product code. Instead you can launch and create image from it and share.
----------------------------------------------------------------------------------------------------------------------------
EC2 Placement Group - Cluster - Clusters instances in same Availability Zone and same Hardware. Great network performances. Ex: Big data jobs.
- Spread - Spread instances across hardware platform under different Availability Zones. Good for critical applications
- Partitions - Spread instances across partitions in Availability Zone. Good for Big data applications, Kafka
----------------------------------------------------------------------------------------------------------------------------
ENI - Virtual network card, has primary IPV4 and secondary IPV4s. Has one public IP, one MAC, One or more security groups, Specific to single Availability Zones. We can move ENI from one instance to another in case of fail-over.
----------------------------------------------------------------------------------------------------------------------------
EC2 Hibernates - RAM will stored in encrypted EBS. Can be used to reduce startup time. Support C,M and R families, RAM < 150 GB, Support most of VM types, Root EBS need to encrypted, Hibernate time < 60 days
----------------------------------------------------------------------------------------------------------------------------
AWS scalability - Vertical scalability - ( Increasing the size of instance, t2.micro -> t2.large, Ex : Non distributed systems like Databases )
- Horizontal scalability - ( Elasticity, Distributed systems to increases number of instances, Ex : web application )
----------------------------------------------------------------------------------------------------------------------------
AWS High-availability - Distribute the system at least in 2 data centers ( AWS zones )
- Active high-availability - Horizontal scaling with LB, Auto scaling groups
- Passive high-availability - Multi-zone RDS
----------------------------------------------------------------------------------------------------------------------------
AWS Load balancing- Routes user traffic to different downstream nodes.
- Provide single access point ( DNS )
- Handle downstream failures using health checks on ports.
- Heath check url : /helth , checks HTTP status 200 in every 5 seconds
- Support SSL implementation
- Enforce stickiness with cookies
- Support high availability across zones
----------------------------------------------------------------------------------------------------------------------------
AWS Load Balancer Types - Classic load balancer ( 2009, HTTP, HTTPS, TCP )
- Network load balancer ( 2017, TCP, TLS (secure TCP), UDP )
- Application load balancer ( 2016, HTTP, HTTPS, Websockets )
----------------------------------------------------------------------------------------------------------------------------
ELB security group - From user to Load balancer (TCP 80, 443 from any where 0.0.0.0/0 )
- From Load balancer to EC2 ( TCP 80, from load balancer source )
----------------------------------------------------------------------------------------------------------------------------
ELB Error Codes - 4xx client related
- 5xx application related , 503 - not registered target
----------------------------------------------------------------------------------------------------------------------------
ELB Monitoring - ELB access logs, CloudWatch metrics for connection counts, aggregated reports
----------------------------------------------------------------------------------------------------------------------------
TCP/ IP and OSI Layer Representations -
| TCP/IP |
OSI Layers |
Examples |
| |
|
|
| Application Layer |
Application Layer |
- End User
HTTP, SSH, DNS, FTP, DHCP
|
| Presentation Layer |
- Data representations
SSL, JPEG, MPEG, SSH
|
| Session Layer |
- Host to Host port Connectivities
API's , Sockets
|
| Transport Layer |
Transport Layer |
- End to End Connection
TCP / UDP
|
| Internet Layer |
Network Layer |
- Packets
IP , IPSec
|
| Network Layer |
Data Link |
- Frames
Switch, Bridge, Ethernet
|
| |
Physical |
- Physical structure
- Binary Transmission
|
----------------------------------------------------------------------------------------------------------------------------
Classic Load Balancer - Can load balance EC2 nodes with security groups
----------------------------------------------------------------------------------------------------------------------------
Application Load Balancer - Routing based on paths in URL
- Routing based on Query strings
- Routing based on hostname in URL
- Great for microservice based solutions
- Target groups - ECS, EC2, Lamda
- True client ip injected to request header -X-Forwarded-For,
- Calling port has been injected to request header - X-Forwareded-Port
- Calling protocol injected to request header - X-Forwarded-Proto
- Can be added multiple target groups to route requests
- Latency less than 400ms
----------------------------------------------------------------------------------------------------------------------------
Network Load Balancer - For TCP and UDP
- Can handle millions of request per seconds
- Latency less than 100ms
- Has one static IP per Availability Zone
----------------------------------------------------------------------------------------------------------------------------
Elastic Load Balancer Stickiness - Applicable to Classic and Application Load Balancers
- Need to be handled by Target groups
----------------------------------------------------------------------------------------------------------------------------
Elastic Cross Load balancing - In Classic LB it is disabled by default, can be enabled without fee
- In Application LB it is always activated, no fees will be charged
- In Network LB, it is disabled by default, fees will be charged if activated
----------------------------------------------------------------------------------------------------------------------------
Elastic Load Balancing Secured - SSL is previous version, TLS is newer version
- Certificates issued by Certificate Authorities
Ex : GoDaddy, Digicert, Symentic, GlobalSig,etc.
- X.509 certificates will be used
- AWS certificate manager can be used
----------------------------------------------------------------------------------------------------------------------------
Server Name Indication ( SNI ) - This to load multiple web sites from single web server
- Only works with ALB, NLB and ClouFront
- Multiple certificates can be attached to ALB, NLB
- Multiple target groups need to be configured
----------------------------------------------------------------------------------------------------------------------------
ELB Connection Draining - Connection timeouts for draining instances
- New connection will routed to other instances.
- De-registration delay 300 seconds by default
----------------------------------------------------------------------------------------------------------------------------
Auto Scaling Groups - Scale in / out instances based on traffic
- Register new instances
- If instance terminated, it will create new instance automatically
- CloudWatch alarms for target average CPU usage, Number of requests, Network In / Out
- Have to create security groups, target groups, application load balances, auto scaling templates
----------------------------------------------------------------------------------------------------------------------------
Auto Scaling Policies - Target Tracking Scaling - Ex: Track average CPU is around 40%
Step Scaling - Ex : If CloudWatch CPU > 70% , then add 2 units
Ex : If CloudWatch CPU < 70% , then remove 1 units
Scheduled Actions - Based user patterns. Ex : Weekdays
----------------------------------------------------------------------------------------------------------------------------
ASG Scaling cool-down - Prevents adding or removing new instances/changes before reflect previous changes , default period 300 seconds, Can reduce cool-down period to reflect the change immediately.
----------------------------------------------------------------------------------------------------------------------------
ASG Scaling policies - Once add scaling policy, it will display in CloudWatch
----------------------------------------------------------------------------------------------------------------------------
ASG Default Termination Policy - Select Availability Zone with Maximum instances
- Delete instance with oldest configurations
----------------------------------------------------------------------------------------------------------------------------
ASG Life Cycle Hooks - This an install or remove specific modules during the scale in and out.
----------------------------------------------------------------------------------------------------------------------------
EBS - Elastic Block Storage is a network drive, Once terminate EC2 instance it will lost its root volume. Unexpected terminations can be done from AWS end. We can attache EBS instead of root volume.
- Network drive is not a physical drive. Similar to USB linked to network. There will be a latency.
- Based on Availability Zone, Cannot move to another AZ
- Billing is based on volume size
- Can change volume time to time
----------------------------------------------------------------------------------------------------------------------------
EBS Types - GP2 - SSD
Average cost, balance prices with performance, uses as boot volumes, Good for test and dev works, 1 GB to 16 TB, 3 IOS per GB, Max 16000 IOPS, Zone Specific
- IOI - SSD, high cost, high performance and low latency. Good for Databases, uses as boot volumes, 4 GB to 16 TB, Provisioned IOPS, Can set IOPS, Max 64000 IOPS, IOPS:Volume=50:1
- STI - HDD, low cost, frequently access and high throughput. Good for big data,Logs,Kafka, 500 GB to 16 TB, Not for boot volumes, Max IOPS 500, 500 Mbps throughput, Throughput optimized HDD
- SCI - HDD , low cost, low latency, Good for infrequent access. Good for big data, 500 GB to 16 TB, Max IOPS 250 Mbps throughput, Cold HDD
----------------------------------------------------------------------------------------------------------------------------
EBS Creation Commands -
1. List block storage : lsblk
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdb 202:16 0 2G 0 disk
2. Check file system : sudo file -s /dev/xvdb :
/dev/xvdb: data
This says no file system created so far
3. Create file system : sudo mkfs -t ext4 /dev/xvdb
It will format and create new file system on given storage
4. Create mount folder : sudo mkdir /data
5. Mount the volume : sudo mount /dev/xvdb /data
6. Check mount points : lsblk
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdb 202:16 0 2G 0 disk /data
7. Set auto mount in every reboot : sudo cp /etc/fstab /etc/fstab-orginal
----------------------------------------------------------------------------------------------------------------------------
EBS Snapshots - Stored in S3, Uses IOs while backup and having a downtime is recommended, Volumes need to be warmed-up once restored ( using dd or fio commands ), Can be automate using AWS life cycle manager
----------------------------------------------------------------------------------------------------------------------------
RDS Rest Encryption - Encrypting master and replicas using AES-256
- If master node not encrypted, replicas cannot be encrypt
- If master node not encrypted, replicas cannot be encrypt
- Manual work
----------------------------------------------------------------------------------------------------------------------------
RDS In-flight Encryption - Encryption at data transmission time
- All clients have to use SSL connectivity
- SSL certificates will use
----------------------------------------------------------------------------------------------------------------------------
Encrypt RDS Snapshot - Create snapshot of the database
- Copy the snapshot and enable encryption
- Restore the database from snapshot
- Migrate client apps
----------------------------------------------------------------------------------------------------------------------------
RDS Security - It has default VPC and not exposed to public by default
- Can connect using user/passwd or IAM based authentication
- Auth token has 15 minute life time
- Auth token can be obtained from RDS service
- Client connectivity must be SSL
----------------------------------------------------------------------------------------------------------------------------
RDS Aurora - AWS Optimized DB 5 times faster than MySQL and 3 times faster than Postgres
- Support 15 replicas while MySQL support only 5
- Support 15 replicas while MySQL support only 5
- Default HA
- Maintains 6 copies across 3 availability zones
- 4 to 3 instances will use as write instances
- Shared volume across 3 AZ
- Less than 30 second fail-over
- Support cross region replications
- Has writer endpoint ( Master ), reader endpoints ( Read replicas )
- Automatic backups, recovery, industry compliance, Patches with zero downtime
- Can create with MySQL or Postgres compatibility modes
- There will be 2 endpoints for read and write
- DB size automatically increased from 10 GB to 64 TB
----------------------------------------------------------------------------------------------------------------------------
RDS Aurora Server-less - Automated database instantiations
- Scaling based on load
- Good for unpredictable loads
- Pay per seconds
----------------------------------------------------------------------------------------------------------------------------
RDS Aurora Cross Region Read Replicas - Good for disaster recoveries
----------------------------------------------------------------------------------------------------------------------------
RDS Aurora Global Database - Has one region of read and writes, other 5 regions are readers
----------------------------------------------------------------------------------------------------------------------------
AWS Elasticache - In memory db based on Radis or Memcache
- High performance , low latency
- Sharding ( Data partitioning in multiple nodes ) for write scaling
- Read replicas for read scaling
- Multi AZ with fail-overs
- AWS manage backups, fail-overs, os patches
- First query Elasticash, If no results, then read RDS and update Elasticache
- Both support SSL not support IAM authentication
----------------------------------------------------------------------------------------------------------------------------
AWS Elasticache Cases - Share status like user sessions
- Reduce DB load
- Radis supports Multi AZ fail-overs, AOF persistence, Backups and Restorations, Can use as message broker, By default no Redis AUTH for commications
- Memcached is a multi-thread, no backups and based on shardings
----------------------------------------------------------------------------------------------------------------------------
AWS Default Ports - FTP: 21
- SSH: 22
- SFTP: 22
- HTTP: 80
- HTTPS: 443
----------------------------------------------------------------------------------------------------------------------------
AWS Route 53 - Is Managed DNS that has rules and records to help clients to find the server location via domain name
----------------------------------------------------------------------------------------------------------------------------
Route 53 Records - A : Hostname to IPV4
- AAAA : Hostname to IPV6
- CNAME : Hostname to Hostname and it is not free
- Alias : Hostname to AWS resource and it is free
----------------------------------------------------------------------------------------------------------------------------
Route 53 Usage - For public domains resolutions and private domain resolutions ( Inside VPC )
----------------------------------------------------------------------------------------------------------------------------
Route 53 Features - Load balancing,
- Routing policies - Fail overs, Geo-Locations, latency
----------------------------------------------------------------------------------------------------------------------------
Route 53 DNS setup - Create hosted zone using Domain name. Ex: myname.com
- Create A name record. Ex: myapp.myname.com with IPV4 address.
- Check it using nslookup myapp.myname.com
- Create EC2 instances in different regions
- Create public internet load balancer link EC2 instances.
- Define TTL
- Create CNAME with load balancer
----------------------------------------------------------------------------------------------------------------------------
DNS general - Web browser will cache DNS request and response for TTL duration
- Low TTL value makes high traffic to Route 53
- CNAME can be used with non root domains : Ex : myapp.myname.com
- CNAME will do hostname to hostname mapping
- Alias points hostname to AWS resource and supports both root and non root domains
- Alias supports native health checks and it is free
----------------------------------------------------------------------------------------------------------------------------
Route 53 policies - Simple : Add multiple IP addresses for single A record
- Weighted : Traffic routed based on defined percentages against TTL
: Can be defined regions, Can be associated with health checks
- Latency : Redirected to the server with least latency evaluated in regions
- Health Check : default 30 seconds and 3 attempts
: Cost will be added for frequent health checks
: Supports HTTP, TCP, HTTPS health checks
- Fail-over : Based on health checks and route to DR setup not to primary setup
- Geo-locations : Based on user location like countries
- Multi-Value : Can be used when we required to route traffic to multiple resources
: Can be linked with health checks
----------------------------------------------------------------------------------------------------------------------------
Route 53 as Registrar - Works like GoDaddy, Google Domains
----------------------------------------------------------------------------------------------------------------------------
Route 53 to Integrate 3rd party domains - Create Hosted Zone in Route 53
- Update NS records of 3rd party web server to use Route 53 name servers, Name servers can be found in your Hosted Zones
----------------------------------------------------------------------------------------------------------------------------
AWS Solution Excellency- Cost : Instance scaling, CPU type, Reservations
- Performance : Vertical scaling, ELB, ASG
- Reliability : Multi ELB, ASG
- Security : Security Groups
- Operation Excellency : ASG
----------------------------------------------------------------------------------------------------------------------------
Golden AMI - Provides all information required to launch EC2 instances, It has latest security patches
----------------------------------------------------------------------------------------------------------------------------
AWS Beanstalk - One layer abstraction to EC2 and provide environment for the deployment
----------------------------------------------------------------------------------------------------------------------------
AWS S3 Buckets - Has global unique name
- Regionally located
- Can store objects in directories
- Buckets have a key. Ex : s3://mybucket/myfolder/mytest.txt , myfolder/mytest.txt is a key
- Max allowed object size - 5 TB
- Multipart upload for more than 5 GB data
- Can use pre-sign URL to access objects
- Can enable versioning like 1,2,3 for same object uploads
- Once delete, can be restored
- Can deploy static web sites - Enable website hosting using index.html and error.html
----------------------------------------------------------------------------------------------------------------------------
AWS S3 Naming - No uppercase, No underscore, No IP, 3 to 63 Characters , Start with a lowercase or number
----------------------------------------------------------------------------------------------------------------------------
AWS S3 Encryption Types - By S3 , AES-256, Must set header for Encryption using HTTP/S
- By AWS KMS, Provide audit trails, Must set header for Encryption using HTTP/S, KMS will protect the Key
- By custom keys, Must be HTTPS calls
- By client side encryption and decryption, Encrypt first and upload
----------------------------------------------------------------------------------------------------------------------------
AWS S3 Security - User based IAM Policies
- Bucked policies - Block public access, cross account access
- Support VPC Endpoints ( without www )
- Audit API / Access Logs
- MFA based deletions
- Pre-signed URLs valid for specific time
----------------------------------------------------------------------------------------------------------------------------
AWS S3 Bucket Policies - Generate S3 bucket policy using policy generator
----------------------------------------------------------------------------------------------------------------------------
AWS CLI - aws --version
- aws configure ( It will create config file and credential file in ~/.aws )
- Never put personal AWS credentials in EC2
- aws configure in EC2 to store only the region
- Personal credentials are intended to use from personal computer
- IAM roles attached to EC2 instances comes with policy authorizing what EC2 can do
- ssh -i testkey.pem ec2-user@13.212.44.205
- To avoid aws configuration in EC2, attach roles to EC2 and attach policies to roles
- It will take few minutes to propagate policy changes
----------------------------------------------------------------------------------------------------------------------------
AWS S3 bucket operations in CLI - aws s3 ls
- aws s3 ls s3://<unique-name>
- aws s3 mb s3://<unique-name>
- aws s3 rb s3://<unique-name>
- aws s3 cp hi.txt s3://<unique-name>
----------------------------------------------------------------------------------------------------------------------------
AWS Policy Generations - Create / add / remove policies using policy creator
- Use AWS policy simulator to verify the functionality
----------------------------------------------------------------------------------------------------------------------------
AWS EC2 Metadata - http://169.254.169.254/latest/meta-data
- Provides information about EC2
- Can read IAM role name not the policy
- curl http://169.254.169.254/latest/meta-data/local-ipv4
- curl http://169.254.169.254/latest/meta-data/iam/security-credentials/myec2s3view/
{
"Code" : "Success",
"LastUpdated" : "2021-01-12T02:10:17Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "ASIAYZDZKN633CCTNH7Y",
"SecretAccessKey" : "4thYiQVwduH1PvwGzYCgbXI8ky490QbsHusv67cw",
"Token" : "--- short lived token --- ",
"Expiration" : "2021-01-12T08:33:57Z"
}
----------------------------------------------------------------------------------------------------------------------------
AWS SDK Exponential Back-off - Re-try on API failure for rate limited API
----------------------------------------------------------------------------------------------------------------------------
MFA Based S3 Delete - S3 versioning is required
- Only root account can enable/disable MFA-delete via CLI
- Root user -> security credentials -> MFA devices
- Enable CLI for MFA based S3 delete setup
- Get Root access key and secret key
- aws configure --profile root-mfa-s3-delete
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]: ap-southeast-1
Default output format [None]:
- aws s3 ls
2021-01-10 22:31:14 elasticbeanstalk-ap-southeast-1-603697344439
2021-01-15 21:02:15 mfa-delete-s3
- aws s3 ls --profile root-mfa-s3-delete
2021-01-10 22:31:14 elasticbeanstalk-ap-southeast-1-603697344439
2021-01-15 21:02:15 mfa-delete-s3
- Enable MFA based delete via CLI only
- aws s3api put-bucket-versioning --bucket mfa-delete-s3 --versioning-configuration Status=Enabled,MFADelete=Enabled --mfa "arn:aws:iam::603697344439:mfa/root-account-mfa-device 149189" --profile root-mfa-s3-delete
- Then add items to bucket "mfa-delete-s3" and delete them.
- It will not allow to delete a version as it enable MFA
- Disable MFA based delete via CLI only
- aws s3api put-bucket-versioning --bucket mfa-delete-s3 --versioning-configuration Status=Enabled,MFADelete=Enabled --mfa "arn:aws:iam::603697344439:mfa/root-account-mfa-device 196789" --profile root-mfa-s3-delete
- It will allow to delete a versioned item
- Go to Root account and remove the previously generate access keys
----------------------------------------------------------------------------------------------------------------------------
S3 Access Logs - All request to S3 will be logged can be analyse later based on tool such as Athena
- Separate logging bucket will be added to collect logs
- Logging bucket and App bucket should not be the same due to logging loops
- Security rules will be update automatically
----------------------------------------------------------------------------------------------------------------------------
S3 Replications - Supports Cross Regions and Same Regions Async replications
- Enable versioning before enabling replications
- Buckets can be in different accounts
- Have to provide IAM rules to first S3 bucket to copy second S3 bucket
- Cross region replication Use Cases ( lower latency access , compliance, replication among account )
- Same region replication Use Cases ( live replications )
- Only new objects will be replicated
- Delete with version id - delete from source and not replicate
- Delete without version id - update with delete marker and not replicate
- No chain replication
----------------------------------------------------------------------------------------------------------------------------
S3 pre-signed URLs - Has specific expiry time
- Use case - download / upload for logged-in/not-logged-in users
- aws s3 presign s3://mybucket/myobject --expires-in 3600 --region my-region
----------------------------------------------------------------------------------------------------------------------------
S3 Storage Classes - Standard General Purpose
- High durability, availability across AZs ( Big Data, Mobile Apps, Content delivery )
- Standard Infrequent Access
- High durability, availability across AZs, Pay less ( DR setups, Backpus )
- One-Zone Infrequent Access
- Low latency, high performance, Data loss possibility ( Secondary backups )
- Intelligent Tiering
- Automatically move objects between two tires
- Low latency high throughput, Retrieval fees
- Resilience against disasters.
- Glacier ( Alternation for Magnetic Tapes )
- Low cost, high data retention period, Retrieval fees
- Each Glacier called "Archive" can be upto 40 TB
- File retrieval time is high, Minimum storage duration 90 days
- First restore before view the backup files
- Glacier Deep Archive
- Cheapest, Minimum storage duration 180 days, Retrieval fees
- First restore before view the backup files
----------------------------------------------------------------------------------------------------------------------------
S3 Life Cycle Rules - Can move objects between Storage classes
- Ex : Move objects to Standard IA after 60 days of creation
- Ex : Move Glacier after 6 month of creation
- Ex : Delete access logs after 365 days
- Ex : Can enable wild card rules.
----------------------------------------------------------------------------------------------------------------------------
S3 Baseline Performance - Auto scaling
- Latency 100-200ms
- 3500 PUT/POST/DELETE, 5000 GET/HEAD requests per second
- KMS , KMS quota will impact on performance
----------------------------------------------------------------------------------------------------------------------------
S3 Performance Optimizing - Multipart parallel upload
- Transfer acceleration via Edge location ( upload only )
- Byte range parallel fetches
----------------------------------------------------------------------------------------------------------------------------
S3 select and Glacier select - Read less data using SQL
----------------------------------------------------------------------------------------------------------------------------
S3 Event Notification - Ex: S3:Ojectcreatd, S3:Objectremoved, etc..
Use case: Create thumbnails of upload images
Events can be configured with SNS, Lambda, SQS
----------------------------------------------------------------------------------------------------------------------------
S3 Athena - Server-less service to analyse S3 files
- SQL to query. Use JDBC/ ODBC
- Charge per query and amount of data scanned
- Usae cases ( BI, Analytics, Reporting, Analyse ELB logs, Cloudwatch logs )
----------------------------------------------------------------------------------------------------------------------------
S3 Object Lock and Glacier Vault Lock - Write once read many
- Block on version deletion
----------------------------------------------------------------------------------------------------------------------------
AWS CloudFront - Content Delivery Network
- Improve read performance by caching at edge locations for TTL
- Support by DDoS protection, AWS WAF,
- Can handle HTTPS internal and external web traffic
- Edge locations can protect S3, EC2, ALB by communication over private network using OAI ( Origin Access Identity )
- Support Geo Restrictions - Use cases : Copyright Laws
----------------------------------------------------------------------------------------------------------------------------
AWS CloudFront Origins - S3 bucket
- Custom HTTP origins ( ELB, EC2 instance, S3 web site )
----------------------------------------------------------------------------------------------------------------------------
No comments:
Post a Comment