Unveiling the Power of AWS: How to Create an S3 Bucket and Establish CLI-based Connectivity with an EC2 Instance
What is S3:
AWS S3 (Simple Storage Service) is a cloud storage service provided by Amazon Web Services. Think of it as a gigantic virtual storage container where you can securely store and retrieve any amount of data from anywhere on the web.
An S3 bucket is like a folder or directory that you create within S3 to store your files. It acts as a central repository for your data, whether it's images, documents, videos, or any other type of file. Each bucket has a unique name that you choose, and it can be accessed over the internet using a URL.
S3 buckets offer several advantages. Firstly, they provide durability, which means your data is highly protected and won't get lost easily. AWS replicates your files across multiple servers and data centers, ensuring data redundancy and minimizing the risk of data loss.
Secondly, S3 buckets are highly scalable. You can start with a small bucket and expand it as your storage needs grow. There are no limits on the number of objects you can store in a bucket, allowing you to store vast amounts of data effortlessly.
Moreover, S3 buckets offer easy accessibility and integration. You can access your bucket programmatically using AWS SDKs or through a web-based interface. S3 integrates seamlessly with other AWS services, making it a versatile storage solution for various applications and workflows.
To interact with an S3 bucket, you can use the AWS Management Console, AWS CLI (Command Line Interface), or AWS SDKs. These tools allow you to perform operations like creating buckets, uploading and downloading files, setting access permissions, enabling versioning, and configuring lifecycle policies.
Overall, AWS S3 buckets are a reliable and flexible storage solution for individuals and businesses alike. They provide secure, scalable, and cost-effective storage for your data, making it easily accessible whenever you need it.
How to create an S3 bucket and connect it with the EC2 instance:
Once signed in, navigate to the S3 service by searching for "S3" in the AWS Management Console search bar or by locating it under the "Storage" section.
In the S3 dashboard, click on the "Create bucket" button.
Enter a unique and meaningful name for your bucket. Bucket names must be globally unique across all existing bucket names in AWS.
For basic usage, you can skip this step and proceed with the default settings.
Disable the bucket versioning.
Click on the "Create" button to proceed.
The S3 bucket is successfully created, Next, we connect it with the EC2 instance through CLi
In the EC2 dashboard, click on the "Launch Instance" button to start the instance creation process.
Next, to access S3, Go to the IAM and create a user or select an existing user and give the USER S3 full access permission, Generate an Access key for CLI and get the Access ID and Secret key
connect to your EC2 instance
Now, update and install AWS CLI in your system
Next, run the command aws configure - and give the Access ID and Secret Key, for a user to get access to the console and S3
Once, connected run the command aws s3 ls for the bucket list
Next, let's create a folder and inside the folder create a file to test and copy to S3
aws s3 cp my_folder s3://s3_bucket_name --recursive
The file is uploaded to s3
The best practice is to sync s3 and the system with the sync command, so that both the service stay updated with each other.
aws s3 sync . s3://s3_bucket_name
That's it! You have successfully launched an EC2 instance and connected it to your S3 bucket. Now
let us see commonly used commands for different uses as per requirements for S3:
1- To exclude specific files or directories while using the aws s3 sync
aws s3 sync your-local-directory s3://your-bucket-name --exclude "*.txt"
2- You can use more complex patterns or specify multiple exclusions by separating them with spaces.
aws s3 sync your-local-directory s3://your-bucket-name --exclude ".txt" --exclude "subdir/" --exclude "file.jpg"
3- To include specific files or directories while using the aws s3 sync
aws s3 sync your-local-directory s3://your-bucket-name --include "*.jpg"
4- You can use more complex patterns or specify multiple inclusions by separating them with spaces.
aws s3 sync your-local-directory s3://your-bucket-name --include ".jpg" --include "subdir/" --include "file.txt"
5- Both exclude and include specific files or directories while using the aws s3 sync
aws s3 sync your-local-directory s3://your-bucket-name --exclude "*" --include ".jpg" --include "subdir/*"
In the above command, replace your-local-directory with the path to your local directory that you want to sync with the S3 bucket, and your-bucket-name with the name of your S3 bucket.
6- List all buckets or objects in a bucket.
Example: aws s3 ls or aws s3 ls s3://your-bucket-name
7- Creates a new bucket.
Example: aws s3 mb s3://your-bucket-name
8- Copies files or directories between the local machine and S3 bucket or between S3 buckets.
Example: aws s3 cp your-file.txt s3://your-bucket-name or
aws s3 cp s3://source-bucket/file.txt s3://destination-bucket/
9- Syncs the contents of a local directory with an S3 bucket or between S3 buckets, recursively.
Example: aws s3 sync your-local-directory s3://your-bucket-name or
aws s3 sync s3://source-bucket s3://destination-bucket
10- Deletes objects or buckets.
Example: aws s3 rm s3://your-bucket-name/object.txt or
aws s3 rb s3://your-bucket-name (to delete an empty bucket)
11- Moves objects between locations within an S3 bucket or between S3 buckets.
Example: aws s3 mv s3://source-bucket/object.txt s3://destination-bucket/
Thank you for taking the time to read my blog! I truly appreciate your interest and support. I hope you found the content valuable and enjoyable. Stay tuned for more articles coming soon. Your readership means a lot to me. Thank you once again, and have a fantastic day!