Amazon DynamoDB references
Amazon DynamoDB is one of my favourite databases.
It’s a pain during the early stages of a project because you have to design your data model based on your queries, which is a good exercise to do anyway. If you are working on a new project without clear query patterns it can be hard to come up with an efficient data model upfront.
My favourite feature of DynamoDB is its predictability and constant performance. I know what to expect from it, and I can plan around it. Constraints are good for performance and reliability.
This article lists resources diving into how DynamoDB works under the hood.
AWS docs
Publications
Abstract: NoSQL cloud database services, like Amazon DynamoDB, are popular for their simple key-value operations, unbounded scalability and predictable low-latency. Atomic transactions, while popular in relational databases, carry the specter of complexity and low performance, especially when used for workloads with high contention. Transactions often have been viewed as inherently incompatible with NoSQL stores, and the few commercial services that combine both come with limitations. This talk examines the tension between transactions and non-relational databases, and it recounts my journey of adding transactions to DynamoDB. I conclude that atomic transactions with full ACID properties can be supported without unduly compromising on performance, availability, self-management, or scalability.
- Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service (USENIX ATC 2022)
- Read PDF or watch video presentation
- Lessons learned from 10 years of DynamoDB by Somu Perianayagam, Akshat Vig
- Summary by Marc Brooker - The DynamoDB paper
- Summary by Murat Demirbas
Abstract: Amazon DynamoDB is a NoSQL cloud database service that provides consistent performance at any scale. Hundreds of thousands of customers rely on DynamoDB for its fundamental properties: consistent performance, availability, durability, and a fully managed serverless experience. In 2021, during the 66 hour Amazon Prime Day shopping event, Amazon systems including Alexa, the Amazon.com sites, and Amazon fulfillment centers, made trillions of API calls to DynamoDB, peaking at 89.2 million requests per second, while experiencing high availability with single-digit millisecond performance. Since its launch in 2012, DynamoDB’s design and implementation have evolved in response to our experiences operating it. The system has successfully dealt with issues related to fairness, traffic imbalance across partitions, monitoring, and automated system operations without impacting availability or performance. Reliability is essential, as even the slightest disruption can significantly impact customers. This paper presents our experience operating DynamoDB at a massive scale and how the architecture continues to evolve to meet the ever-increasing demands of customer workloads.
Abstract: NoSQL cloud database services are popular for their simple key-value operations, high availability, high scalability, and predictable performance. These characteristics are generally considered to be at odds with support for transactions that permit atomic and serializable updates to partitioned data. This paper explains how transactions were added to Amazon DynamoDB using a timestamp ordering protocol while exploiting the semantics of a key-value store to achieve low latency for both transactional and non-transactional operations. The results of experiments against a production implementation demonstrate that distributed transactions with full ACID properties can be supported without compromising on performance, availability, or scale.
Videos and talks
Search the AWS Events channel: https://www.youtube.com/@AWSEventsChannel/search?query=dynamodb
- AWS re:Invent 2018: Amazon DynamoDB Under the Hood: How We Built a Hyper-Scale Database (DAT321) by Jaso Sorenson
- AWS re:Invent 2019: Scale fearlessly with Amazon DynamoDB adaptive capacity (DAT304) by Kai Zhao
- AWS re:Invent 2020: Amazon DynamoDB advanced design patterns – Part 1 by Rich Houlihan
- AWS re:Invent 2020: Amazon DynamoDB advanced design patterns – Part 2 by Rich Houlihan
- AWS re:Invent 2021 - DynamoDB deep dive: Advanced design patterns by Rich Houlihan
- AWS re:Invent 2023 - Building highly resilient applications with Amazon DynamoDB (DAT333) by Jeff Duffy, Tom Skinner, Richard Edwards III
- AWS re:Invent 2024 - Data modeling core concepts for Amazon DynamoDB (DAT305) by Jason Hunter, Sean Shriver
- AWS re:Invent 2024 - Advanced data modeling with Amazon DynamoDB (DAT404) by Alex Debrie
- AWS re:Invent 2023 - Dive deep into Amazon DynamoDB (DAT330) by Amrith Kumar
- AWS re:Invent 2024 - Dive deep into Amazon DynamoDB (DAT406) by Amrith Kumar
- Details from the USENIX ATC 2022 paper.
- AWS re:Invent 2024 - An insider’s look into architecture choices for Amazon DynamoDB (DAT419) by Amrith Kumar, Joseph Idziorek
- AWS re:Invent 2024 - Multi-Region strong consistency with Amazon DynamoDB global tables (DAT425-NEW) by Jeff Duffy, Somu Perianayagam
Other
- Amazon DynamoDB – a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications by Werner Vogels, on the day of DynamoDB announcement day in 2012.
- DynamoDB’s Best Feature: Predictability by Marc Brooker
- Dynamo, DynamoDB, and Aurora DSQL by Marc Brooker