S3 Investigation Summary
We started this analysis because S3 cost was not reducing as expected, even after cleanup activities.
Cost signal
Recent S3 monthly cost stayed around:
-
Oct 2025: ~$5.8k
-
Nov 2025: ~$5.8k
-
Dec 2025: ~$4.7k
-
Jan 2026: ~$5.3k
-
Feb 2026: ~$5.9k
So S3 was still running at roughly $5k–$6k/month.
How we validated it
We did not rely only on folder view in S3 console.
We validated using:
-
bucket size metrics
-
Storage Lens
-
storage class split
-
prefix drill-down
This confirmed:
-
bucket size is around 260–270 TB
-
most of the data is still in Standard
-
storage is concentrated under a non-prod backup prefix
Key finding
This is not mainly a versioning issue or another-region issue.
The main problem is:
-
large backup data retained in Standard
-
under a non-prod backup path
-
with likely retention / lifecycle gap
Incomplete multipart upload issue
One additional contributor may be incomplete multipart uploads.
In this case, large SQL backup files are uploaded to S3 in parts.
If a backup/upload job fails in the middle and the upload is not completed or aborted, S3 keeps the uploaded parts.
That means:
-
no final usable backup object
-
but storage is still consumed
-
and cost continues in Standard storage
This usually happens when:
-
backup job fails midway
-
retry starts a new upload
-
old partial upload is not cleaned up
Fix direction
Main actions:
-
review retention for non-prod backup data
-
apply / correct lifecycle rules for affected prefixes
-
move older backups from Standard to Glacier / archive
-
enable abort incomplete multipart uploads
-
validate whether old backup copies can be deleted
Expected saving
Based on current S3 run-rate, expected saving is roughly:
-
up to ~$3k/month, if retention can be reduced further
Short root cause statement
Root cause: S3 growth is mainly driven by large non-prod backup data remaining in Standard storage longer than required. In addition, failed multipart backup uploads may be leaving orphaned uploaded parts in S3, adding to storage without creating final usable backup files.
