Storage and Database Marvels: A Recap of AWS re:Invent 2023’s Game-Changing Announcements
AWS storage and database services are utilized by millions of customers to revolutionise their businesses, enhance flexibility, lower expenses, and expedite the pace of innovation. In this blog post, I will take you through AWS re:Invent 2023 Storage and Database impactful updates:
Amazon FSx and EFS
· Amazon FSx for NetApp ONTAP now enables users to create Multi-AZ file systems in Shared VPCs from participant accounts, making it even easier for their organization to decentralize how they administer network and storage resources.
· Introduced on-demand data replication for Amazon FSx for OpenZFS, enabling users to transfer incremental point-in-time snapshots of their volumes easily and efficiently between file systems.
· Introduced Amazon EFS Archive, it is a new storage class that is cost-optimized for long-lived file data that is accessed a few times a year or less. With EFS Archive, users now have a cost-effective way to retain even coldest data so that it’s always available to power new business insights.
· Amazon EFS now supports up to 250,000 read IOPS and up to 50,000 write IOPS per file system, making it easier to power IOPS-intensive file workloads on AWS.
· Amazon FSx for NetApp ONTAP, now allows users to create, manage, and back up FlexGroup volumes using the AWS Management Console, FSx CLI, and AWS SDK, making it even easier to manage and protect their data at-scale.
· Users can create FSx for ONTAP scale-out file systems that deliver up to 36 GB/s of throughput and 1.2 million IOPS — 9x higher performance than before. FSx for ONTAP is the first and only storage service that allows users to launch and run fully managed, fully featured ONTAP file systems in the cloud.
· Amazon EFS Replication now supports failback, making it easier and more cost-effective to synchronize changes between EFS file systems after Disaster Recovery and other failover events.
Amazon S3
· Users can now use Mountpoint for Amazon S3 to access objects stored in the new Amazon S3 Express One Zone storage class using file system operations. The new S3 Express One Zone storage class is purpose-built to deliver the fastest cloud object storage for performance-critical applications that demand consistent single-digit millisecond request latency.
· Users can now use Amazon Athena to query data stored in the Amazon S3 Express One Zone storage class for up to 2.1x faster query performance than S3 Standard.
· Introduced Amazon S3 Express One Zone storage class, it is purpose-built to deliver the fastest cloud object storage for performance-critical applications that demand consistent single-digit millisecond request latency. S3 Express One Zone can improve data access speeds by 10x and reduce request costs by 50% compared to S3 Standard and scales to process millions of requests per minute for most frequently accessed datasets.
· AWS Command Line Interface and Python SDK now automatically use the AWS Common Runtime to accelerate data transfer between Amazon S3 and Amazon EC2 Trn1, P4d, and P5 instances.
· Mountpoint for Amazon S3 CSI driver is now generally available. With the new Mountpoint for Amazon S3 Container Storage Interface driver, Kubernetes applications can access S3 objects through a file system interface, achieving high aggregate throughput without any changes to application.
· Introduced Amazon S3 Access Grants integration with identity providers to simplify data lake permissions.
AWS Backup
· AWS Backup introduced the general availability of restore testing, a new capability within AWS Backup that helps perform automated and periodic restore tests of supported AWS resources that have been backed up.
· AWS Backup now support for EBS Snapshots Archive, allowing customers to automatically move EBS Snapshots created by AWS Backup to EBS Snapshots Archive.
AWS DRS
· AWS Elastic Disaster Recovery now allows users to automate validations when launching EC2 instances for recovery and drills. This saves time used to perform tedious manual verifications, as AWS DRS empowers users to define custom automated validations tailored to their specific needs.
· AWS Elastic Disaster Recovery introduced general availability in the AWS GovCloud (US) Regions.
Amazon RDS
· AWS introduced the general availability of Amazon RDS for Db2, which makes it easy to set up, operate, and scale Db2 databases in the cloud.
· AWS introduced Amazon Aurora Limitless Database, which enables users to scale their Amazon Aurora clusters to millions of write transactions per second and manage petabytes of data.
Amazon ElastiCache
· AWS Introduced the general availability of Amazon ElastiCache Serverless, a new serverless option for Amazon ElastiCache that simplifies cache management and instantly scales to support the most demanding applications.
Amazon DynamoDB
· Amazon DynamoDB now supports zero-ETL integration with Amazon Redshift, enabling customers to run high performance analytics on their DynamoDB data.
· Amazon DynamoDB zero-ETL integration with Amazon OpenSearch Service provides customers advanced search capabilities, such as full-text and vector search, on their Amazon DynamoDB data.
Amazon DocumentDB
· Amazon DocumentDB (with MongoDB compatibility) now supports vector search, a new capability that enables users to store, index, and search millions of vectors with millisecond response times.
Amazon OpenSearch
· AWS introduced the general availability of vector engine for Amazon OpenSearch Serverless. Vector engine for OpenSearch Serverless is a simple, scalable, and high-performing vector database which makes it easier for developers to build machine learning –augmented search experiences and generative artificial intelligence applications without having to manage the underlying vector database infrastructure.
Others
· AWS introduced the general availability of Amazon Neptune Analytics, a new analytics database engine. Neptune Analytics makes it faster for data scientists and application developers to get insights and find trends by analyzing graph data with tens of billions of connections in seconds.
· Amazon Redshift introduces Amazon Q generative SQL in Amazon Redshift Query Editor, an out-of-the-box web-based SQL editor for Redshift, to simplify query authoring and increase productivity by allowing users to express queries in natural language and receive SQL code recommendations.
Features or Services In Preview
· [EC2] — AWS introduced EC2 U7i, next-generation High Memory instances. U7i instances offer up to 32TiB of DDR5 memory enabling customers to scale transaction processing throughput in a fast-growing data environment. U7i instances are part of AWS 7th generation and are powered by custom fourth generation Intel Xeon Scalable Processors (Sapphire Rapids) delivering up to 125% more compute performance over existing U-1 instances.
· [Amazon Redshift ML] — Amazon Redshift enhances Redshift ML to support large language models (LLM). Amazon Redshift ML enables customers to create, train, and deploy machine learning models using familiar SQL commands.
· [Amazon Redshift] — Amazon Redshift announces Multidimensional Data Layouts, a new powerful table sorting mechanism that improves performance of repetitive queries.
· [Amazon DataZone] -AWS introduced the preview of a new generative AI-based capability in Amazon DataZone to improve data discovery, data understanding and data usage by enriching the business data catalog.
· [zero-ETL integration] — AWS introduced Amazon RDS for MySQL zero-ETL integration with Amazon Redshift that allow users to access transactional data from Amazon RDS for MySQL to run analytics and machine learning (ML) on petabytes of data in Amazon Redshift.
· [zero-ETL integration] -Introduced Amazon Aurora zero-ETL integration with Amazon Redshift which enables near real-time analytics and machine learning (ML) using Amazon Redshift on petabytes of transactional data from Amazon Aurora.
· [Amazon MemoryDB] — Amazon MemoryDB for Redis now supports vector search in preview, a new capability that enables users to store, index, and search vectors. MemoryDB is a database that combines in-memory performance with multi-AZ durability.
Please feel free to write @ piyush.jalan93@gmail.com for any queries on AWS Storage and Database updates & stay tuned for next write-up.
Thank you!
