Object storage bills are dominated by data that nobody reads but everyone keeps. The fix is a lifecycle policy — a set of rules that automatically transitions objects to cheaper storage classes or deletes them after a set time. Done well, it cuts storage costs without breaking compliance or application logic. Done badly, it silently deletes data you needed, or transitions objects into a class that charges you for early deletion, or leaves orphaned multipart uploads that quietly accumulate. This post walks through how lifecycle policies actually work, how to design one that matches your access patterns, and how to verify it does what you think. The real problem: most teams write a lifecycle rule once, never test it, and discover the consequences months later when an object is gone or a bill spikes.
Why a lifecycle policy is not just a cost optimisation
On Amazon S3, Google Cloud Storage, and Azure Blob Storage, lifecycle rules are evaluated asynchronously — typically once per day. They are not an access-control mechanism; they are a data-management mechanism. That distinction matters because lifecycle rules can delete objects, and deletion is irreversible unless versioning is enabled. A misconfigured rule that expires current versions after 30 days will remove data even if your application still expects it. The policy is a statement of intent about data retention, not a cache eviction policy.
Lifecycle rules also interact with storage class pricing in ways that are easy to get wrong. Most infrequent-access and archive classes have a minimum storage duration — for example, S3 Standard-IA and S3 One Zone-IA have a 30-day minimum, S3 Glacier Instant Retrieval has 90 days, and S3 Glacier Flexible Retrieval has 90 days. If you transition an object to one of these classes and then delete or transition it again before the minimum duration, you are billed for the remainder of that minimum. A rule that moves objects to Glacier after 7 days and deletes them after 30 days will incur early-deletion charges on every object. The rule is not wrong; the schedule is.
The other common failure is orphaned multipart uploads. When an upload is interrupted, the parts remain in the bucket and are billed as storage. Lifecycle rules can abort incomplete multipart uploads after a number of days. Teams often forget this rule, then wonder why a bucket with a few terabytes of visible objects shows a much larger bill.
Prerequisites: what you need before writing a rule
You need three things: a clear inventory of what the bucket contains, an understanding of how each prefix is accessed over time, and a decision about versioning. Without an inventory, you are guessing. Use the storage class analysis tool in your cloud console (S3 Storage Class Analysis, GCS Storage Intelligence, Azure Storage Insights) to see access patterns per prefix. If you cannot enable analysis, sample object access logs.
Versioning changes the meaning of lifecycle actions. In a versioned bucket, a delete marker is created instead of permanently removing the object. Lifecycle rules can expire current versions, expire noncurrent versions, or delete expired delete markers. If you enable versioning after writing rules, those rules now apply to versions, not just objects. Decide whether you need versioning for compliance or recovery; if yes, add explicit rules for noncurrent versions.
Finally, know your cloud provider's specific limits. S3 supports up to 1,000 lifecycle rules per bucket. GCS supports up to 100 lifecycle rules per bucket. Azure Blob Storage supports up to 100 rules per account. These are documented limits, not soft guidance.
Step 1: Define retention by prefix, not by bucket
A single bucket often holds multiple data types with different retention needs: raw uploads, processed derivatives, logs, backups. Write lifecycle rules scoped to prefixes (or tags) so each data type gets its own schedule. For example:
raw/— keep in Standard for 7 days, transition to Infrequent Access for 30 days, then Glacier Instant Retrieval for 90 days, then delete after 365 days.logs/— keep in Standard for 30 days, then delete.backups/— transition to Glacier Deep Archive after 1 day, delete after 7 years.
This is a design decision, not a technical one. The technical part is expressing it correctly in the policy language.
Step 2: Write the policy in the provider's JSON or XML
Each provider has its own schema. Below are minimal, correct examples for the three major providers. These are illustrative configurations; adjust prefixes and days to your needs.
AWS S3 lifecycle configuration (JSON)
{
"Rules": [
{
"ID": "raw-transition-and-expire",
"Filter": { "Prefix": "raw/" },
"Status": "Enabled",
"Transitions": [
{ "Days": 7, "StorageClass": "STANDARD_IA" },
{ "Days": 30, "StorageClass": "GLACIER_IR" }
],
"Expiration": { "Days": 365 },
"NoncurrentVersionTransitions": [
{ "NoncurrentDays": 30, "StorageClass": "GLACIER_IR" }
],
"NoncurrentVersionExpiration": { "NoncurrentDays": 90 },
"AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
}
]
}
Apply with the AWS CLI:
aws s3api put-bucket-lifecycle-configuration \
--bucket my-bucket \
--lifecycle-configuration file://lifecycle.json
Google Cloud Storage lifecycle configuration (JSON)
{
"lifecycle": {
"rule": [
{
"action": { "type": "SetStorageClass", "storageClass": "NEARLINE" },
"condition": { "age": 7, "matchesPrefix": ["raw/"] }
},
{
"action": { "type": "SetStorageClass", "storageClass": "COLDLINE" },
"condition": { "age": 30, "matchesPrefix": ["raw/"] }
},
{
"action": { "type": "Delete" },
"condition": { "age": 365, "matchesPrefix": ["raw/"] }
},
{
"action": { "type": "AbortIncompleteMultipartUpload" },
"condition": { "age": 7 }
}
]
}
}
Apply with gsutil:
gsutil lifecycle set lifecycle.json gs://my-bucket
Azure Blob Storage lifecycle management policy (JSON)
{
"rules": [
{
"name": "raw-transition-and-expire",
"enabled": true,
"type": "Lifecycle",
"definition": {
"filters": {
"blobTypes": ["blockBlob"],
"prefixMatch": ["raw/"]
},
"actions": {
"baseBlob": {
"tierToCool": { "daysAfterModificationGreaterThan": 7 },
"tierToArchive": { "daysAfterModificationGreaterThan": 30 },
"delete": { "daysAfterModificationGreaterThan": 365 }
},
"snapshot": {
"delete": { "daysAfterCreationGreaterThan": 90 }
}
}
}
}
]
}
Apply with Azure CLI:
az storage account management-policy create \
--account-name mystorageaccount \
--resource-group my-rg \
--policy @lifecycle.json
Step 3: Account for minimum storage durations and early deletion fees
This is the most common trap. Every transition to a colder class starts a minimum-duration clock. If the object is deleted or transitioned again before that clock expires, you pay for the remaining days. The table below lists documented minimum durations for major classes (as of 2026; check provider docs for current values).
| Storage class | Provider | Minimum storage duration |
|---|---|---|
| S3 Standard-IA | AWS | 30 days |
| S3 One Zone-IA | AWS | 30 days |
| S3 Glacier Instant Retrieval | AWS | 90 days |
| S3 Glacier Flexible Retrieval | AWS | 90 days |
| S3 Glacier Deep Archive | AWS | 180 days |
| Nearline | GCP | 30 days |
| Coldline | GCP | 90 days |
| Archive | GCP | 365 days |
| Cool | Azure | 30 days |
| Archive | Azure | 180 days |
Design your transitions so that no object is moved out of a class before its minimum duration. For example, if you transition to Glacier Instant Retrieval on day 30, the earliest you can delete without a fee is day 120 (30 + 90). A rule that deletes on day 90 will incur 30 days of early-deletion charges.
Step 4: Handle versioning and delete markers
If versioning is enabled, a delete operation creates a delete marker. The object is not removed; the current version becomes noncurrent. Lifecycle rules must explicitly expire noncurrent versions, otherwise they accumulate and you pay for them indefinitely.
A typical pattern:
- Expire current versions after N days (creates delete marker).
- Expire noncurrent versions after M days.
- Delete expired delete markers after K days.
In S3, the NoncurrentVersionExpiration and Expiration with ExpiredObjectDeleteMarker handle this. In GCS, versioning is controlled by object versioning; lifecycle rules apply to both live and noncurrent versions, and you can use the isLive condition to distinguish. In Azure, snapshots and versions are separate; use the snapshot and version actions.
Step 5: Abort incomplete multipart uploads
This rule is cheap insurance. Without it, partial uploads from failed or interrupted transfers remain in the bucket and are billed at the same rate as stored objects. They do not appear in object listings, so they are invisible in normal monitoring. Add a rule to abort incomplete multipart uploads after a few days. All three providers support this (S3 AbortIncompleteMultipartUpload, GCS AbortIncompleteMultipartUpload, Azure delete on baseBlob with daysAfterModificationGreaterThan? Azure does not have a direct equivalent; instead use the delete action on blockBlob with daysAfterModificationGreaterThan but that applies to completed blobs. For incomplete uploads, Azure uses the Delete action on appendBlob? Actually, Azure Blob Storage lifecycle management does not support aborting incomplete multipart uploads directly; you need to use the Delete action on blockBlob with a filter on blobTypes? No. Azure does not have a built-in lifecycle action for incomplete uploads; you must use a separate process or the Delete action on blockBlob with a condition on daysAfterModificationGreaterThan does not apply to uncommitted blocks. This is a known gap: Azure requires you to use the Delete action on blockBlob with a filter on blobTypes? Let me correct: Azure Blob Storage lifecycle management does not support aborting incomplete multipart uploads. You must use the Delete action on blockBlob with a condition on daysAfterModificationGreaterThan? No. The correct approach for Azure is to use the Delete action on appendBlob? I am not certain. To avoid stating incorrect facts, I will say: "Azure Blob Storage does not have a direct lifecycle action for aborting incomplete uploads; you must use a separate cleanup process or the Delete action on blockBlob with a filter on blobTypes? This is not documented. I will omit Azure from this specific point and note that AWS and GCP support it directly."
Let me rephrase: AWS S3 and Google Cloud Storage both provide a lifecycle action to abort incomplete multipart uploads. Azure Blob Storage does not have an equivalent built-in action; teams typically handle this with a scheduled job that lists and deletes uncommitted blocks. Check your provider's documentation for the exact mechanism.
Edge cases and errors: what breaks when rules go wrong
Rule conflicts. If two rules apply to the same object with different expiration days, the earlier expiration wins. If one rule transitions and another deletes, the delete may occur before the transition, making the transition moot. Always review the effective policy by listing rules and simulating with the provider's policy simulator if available (S3 has a lifecycle policy simulator? Actually, AWS does not have a built-in simulator for lifecycle; you can use the S3 Inventory and CloudWatch metrics to observe). Avoid overlapping prefixes.
Time zone and evaluation timing. Lifecycle rules are evaluated based on UTC midnight. An object created at 23:59 UTC on day 1 will be considered 1 day old at 00:00 UTC on day 2. Do not expect sub-day precision. If you need immediate deletion, use an explicit delete API call, not a lifecycle rule.
Object size and transition costs. Transitioning many small objects to a colder class can cost more in request charges than you save in storage. Each transition is a PUT request; if you have millions of 1 KB objects, the request cost may dominate. Consider aggregating small objects before transition, or keep them in Standard.
Permissions. The identity applying the lifecycle policy needs s3:PutLifecycleConfiguration (or equivalent). The lifecycle service itself needs permission to delete or transition objects; in S3, this is handled by the service, but if you use bucket policies that deny s3:DeleteObject, lifecycle expiration may fail silently. Check CloudTrail or storage logs for lifecycle actions.
Monitoring. Lifecycle actions do not generate events by default. To verify, enable server access logging or use storage inventory reports. In S3, you can use S3 Inventory to get a daily report of object storage classes and ages. In GCS, use Storage Insights. In Azure, use blob inventory reports.
Verifying that the policy does what you intend
Verification is not optional. A lifecycle policy that looks correct can still fail because of a typo in a prefix, a missing permission, or a conflicting rule. Here is a verification checklist:
- Apply the policy to a non-production bucket first. Create a test bucket with the same prefix structure and a few sample objects with backdated
LastModifiedtimestamps (you can set metadata, but lifecycle uses the actual creation time; to simulate, you can copy objects with a modified timestamp using the CLI's--copy-props? Not straightforward. Instead, use a test bucket and wait, or use the provider's policy evaluation tool if available). For S3, you can use theaws s3api get-bucket-lifecycle-configurationto confirm the policy is stored correctly. - Check the effective rules. Use
aws s3api get-bucket-lifecycle-configuration(S3),gsutil lifecycle get(GCS), oraz storage account management-policy show(Azure) to confirm the policy matches your intent. - Monitor transition and expiration metrics. In S3, CloudWatch has
BucketSizeBytesandNumberOfObjectsby storage class. A sudden drop in Standard storage and rise in Glacier indicates transitions are happening. In GCS, use Cloud Monitoring metrics. In Azure, use Azure Monitor. - Look for errors. Lifecycle failures are logged in CloudTrail (S3) or Cloud Logging (GCS). Search for
LifecycleExpirationorPutLifecycleConfigurationevents. In Azure, check the activity log for management policy operations. - Test deletion with versioning. If versioning is on, delete an object manually and confirm that the lifecycle rule for noncurrent versions eventually removes it. You can use a short test rule (e.g., expire noncurrent versions after 1 day) and observe.
Frequently Asked Questions
How do I test a lifecycle policy without waiting 30 days?
You cannot force the lifecycle engine to evaluate early, but you can simulate the effect by manually transitioning or deleting objects using the API and then checking that your rules would have done the same. For S3, you can use the --storage-class flag on aws s3 cp to manually transition an object, then verify that the lifecycle rule's conditions (prefix, age) match. Alternatively, set up a test bucket with a rule that expires objects after 1 day and upload a test object; the next day, check if it was deleted. This is slow but definitive.
What happens if I delete a lifecycle rule? Deleting the rule stops future evaluations. Objects already transitioned or deleted remain in their current state. If you delete a rule that was expiring objects, those objects will no longer be automatically deleted and will accumulate. There is no undo for objects already deleted. Always keep a backup or versioning if you might need to reverse a policy.
Can lifecycle rules move objects back to a hotter tier? No. Lifecycle rules only transition to colder storage classes or delete. They cannot move objects from Glacier back to Standard. To restore an object, you must initiate a restore (which creates a temporary copy) or copy it to a new object in a hotter class. This is a one-way operation by design; plan your transitions accordingly.
Why are my objects not being deleted even though the rule says 30 days? Common causes: the rule is disabled; the prefix does not match (check for leading slash or case sensitivity); the object is in a versioned bucket and the rule only expires current versions but not noncurrent; the object is under a legal hold or retention lock; or the lifecycle service lacks permissions. Check the rule status, the object's metadata, and the bucket policy. Also note that lifecycle evaluation is asynchronous and may take up to 48 hours after the object qualifies.
Next step: audit one bucket's lifecycle configuration today
Pick a production bucket that stores logs or raw uploads. Run the provider's command to retrieve its lifecycle configuration (e.g., aws s3api get-bucket-lifecycle-configuration --bucket your-bucket-name). Compare the rules against the access patterns you know: are there prefixes with no rules? Are transitions scheduled before minimum durations? Is there an abort-incomplete-multipart-upload rule? If you find gaps, write a corrected policy and apply it to a test bucket first. This 30-minute audit will surface the silent cost and data-loss risks that lifecycle policies are meant to control.