AWS S3 Annotations: 1GB of mutable metadata per object

Key Takeaways

• S3 Annotations allow up to 1000 mutable annotations per object, with a combined 1GB capacity, compared to the previous 2KB limit for user-defined metadata
• Annotations are queryable through S3 Metadata Tables and integrate with Athena, Redshift, and Iceberg-compatible engines
• Unlike traditional S3 metadata, annotations can be modified without re-uploading the entire object

AWS has launched Amazon S3 Annotations, a feature that lets teams attach up to 1GB of structured metadata directly to S3 objects. The annotations are mutable, queryable, and independent of the underlying object. That last part matters: you can update metadata without touching the object itself.

S3 has supported tags and user-defined metadata for years, but both came with tight limits. Tags cap at 10 per object. User-defined metadata maxes out at 2KB. Neither could be modified without re-uploading the entire object. For teams managing billions of objects across compliance, analytics, and AI workloads, these constraints forced the creation of separate metadata databases.

What are S3 Annotations exactly?

Annotations are structured data in JSON, XML, or YAML format that attach to S3 objects. Each object can hold up to 1000 annotations with a combined size of 1GB. Daniel Abib, senior specialist solutions architect at AWS, framed the problem in the announcement: "While these capabilities work well for their intended purposes, they have limitations when you need to attach much richer context without building and maintaining separate metadata systems."

The key distinction from existing S3 metadata: annotations are mutable. Traditional object metadata required a copy-on-write pattern. To change a single tag, you'd read the full object, then write it back with new metadata. For a 100GB video file, that's expensive and slow.

How do annotations become queryable?

Annotations integrate with S3 Metadata Tables, a feature AWS released earlier. When you enable annotation tables on a bucket, every annotation flows into a managed Apache Iceberg table. Mai-Lan Tomsen Bukovec, VP at AWS, explained: "You can query across all your objects with Amazon Athena, Amazon Redshift or any Iceberg-compatible engine, and agents can discover annotations in natural language through the S3 Tables MCP server."

This is where the AI angle comes in. AWS is positioning annotations as context for agentic workflows. An AI agent processing a data lake can query annotations to understand what objects contain, their compliance status, or their processing history, without opening the objects themselves.

What's the catch?

Storage and API costs. Annotations are billed at S3 Standard rates regardless of the underlying object's storage tier. If you have a petabyte of data in S3 Glacier Deep Archive, the annotations still cost Standard rates. When objects are copied, annotations copy too, and each annotation is a separate PUT request.

Corey Quinn, chief cloud economist at The Duckbill Group, offered a characteristically pointed take: "S3 gets an object store in the object store, and now you can bolt a full gigabyte of 'context' onto each object, because the four existing metadata mechanisms weren't confusing enough. Every announcement these days whispers from behind you 'agentic workflows,' which is Seattle for 'your AI will generate the data, then pay Athena to read it back.'"

Why the community response has been positive

Despite the cost considerations, Reddit's r/aws community has reacted favorably. User ReturnOfNogginboink summarized the sentiment: "The really important part here is that annotations can be modified. Unlike object metadata, which requires you to read the full object out of S3, and rewrite it to S3 with new metadata. This is the big deal here. This is going to unlock all kinds of new workflow possibilities."

For teams that have built and maintained their own metadata databases alongside S3, annotations offer consolidation. One system instead of two. One query language instead of joining S3 inventories with external databases.

Who should care about this?

AWS highlights media, financial services, and life sciences as target verticals. The common thread: industries that store massive datasets with complex compliance, lineage, and classification requirements. A media company tracking rights and licensing for millions of assets. A financial firm maintaining audit trails on transaction records. A pharma company documenting provenance of research datasets.

Engineering teams building ML pipelines also have a clear use case. Training datasets need metadata about their contents, preprocessing steps, and quality metrics. Previously, that metadata lived in a separate system. Now it can live with the data.

S3 Annotations are now generally available in all AWS regions.

Frequently Asked Questions

What's the difference between S3 Annotations and S3 tags?

S3 tags are limited to 10 per object and are immutable without re-uploading. Annotations allow up to 1000 per object, support 1GB total size, and can be modified independently of the object.

Can I query S3 Annotations with SQL?

Yes. When you enable annotation tables on a bucket, annotations flow into Apache Iceberg tables that you can query with Amazon Athena, Redshift, or any Iceberg-compatible engine.

How are S3 Annotations billed?

Annotations are billed at S3 Standard storage rates regardless of the object's storage class. Copying objects includes copying annotations, with each annotation counted as a separate PUT request.

What formats do S3 Annotations support?

Annotations can be written in JSON, XML, or YAML format.

Do S3 Annotations work with existing S3 objects?

Yes. Annotations can be added to existing objects without modifying or re-uploading the objects themselves.

Source: InfoQ