Monday, March 16, 2026

Investigating S3 Cost Growth in AWS – How We Identified the Issue and What We Found

S3 Investigation Summary

We started this analysis because S3 cost was not reducing as expected, even after cleanup activities.

Cost signal

Recent S3 monthly cost stayed around:

  • Oct 2025: ~$5.8k

  • Nov 2025: ~$5.8k

  • Dec 2025: ~$4.7k

  • Jan 2026: ~$5.3k

  • Feb 2026: ~$5.9k

So S3 was still running at roughly $5k–$6k/month.

How we validated it

We did not rely only on folder view in S3 console.
We validated using:

  • bucket size metrics

  • Storage Lens

  • storage class split

  • prefix drill-down

This confirmed:

  • bucket size is around 260–270 TB

  • most of the data is still in Standard

  • storage is concentrated under a non-prod backup prefix

Key finding

This is not mainly a versioning issue or another-region issue.
The main problem is:

  • large backup data retained in Standard

  • under a non-prod backup path

  • with likely retention / lifecycle gap

Incomplete multipart upload issue

One additional contributor may be incomplete multipart uploads.

In this case, large SQL backup files are uploaded to S3 in parts.
If a backup/upload job fails in the middle and the upload is not completed or aborted, S3 keeps the uploaded parts.

That means:

  • no final usable backup object

  • but storage is still consumed

  • and cost continues in Standard storage

This usually happens when:

  • backup job fails midway

  • retry starts a new upload

  • old partial upload is not cleaned up

Fix direction

Main actions:

  • review retention for non-prod backup data

  • apply / correct lifecycle rules for affected prefixes

  • move older backups from Standard to Glacier / archive

  • enable abort incomplete multipart uploads

  • validate whether old backup copies can be deleted

Expected saving

Based on current S3 run-rate, expected saving is roughly:

  •  up to ~$3k/month, if retention can be reduced further

Short root cause statement

Root cause: S3 growth is mainly driven by large non-prod backup data remaining in Standard storage longer than required. In addition, failed multipart backup uploads may be leaving orphaned uploaded parts in S3, adding to storage without creating final usable backup files.

No comments:

Post a Comment

Investigating S3 Cost Growth in AWS – How We Identified the Issue and What We Found

S3 Investigation Summary We started this analysis because S3 cost was not reducing as expected , even after cleanup activities. Cost signal ...