Whitepaper 'FinOps and cost management for Kubernetes'
Please consider giving OptScale a Star on GitHub, it is 100% open source. It would increase its visibility to others and expedite product development. Thank you!
Webinar 'FinOps and cloud cost optimization for ML/AI workloads.' Register here →
Ebook 'From FinOps to proven cloud cost management & optimization strategies'
OptScale — FinOps
FinOps overview
Cost optimization:
AWS
MS Azure
Google Cloud
Alibaba Cloud
Kubernetes
OptScale — MLOps
ML/AI Profiling
ML/AI Optimization
Big Data Profiling
OPTSCALE PRICING
Acura — Cloud migration
Overview
Database replatforming
Migration to:
AWS
MS Azure
Google Cloud
Alibaba Cloud
VMWare
OpenStack
KVM
Public Cloud
Migration from:
On-premise
Acura — DR & cloud backup
Overview
Migration to:
AWS
MS Azure
Google Cloud
Alibaba Cloud
VMWare
OpenStack
KVM

How to save on Alibaba Cloud OSS by avoiding non-actual artifacts in your bucket

In one of our previous articles we described the quickest way to save on Amazon S3. Let’s discover how to reduce costs on Alibaba Cloud OSS by avoiding incomplete multipart uploads and non-actual artifacts in the bucket.

Problem description: Incomplete multipart uploads in Alibaba Cloud

How-to-save-quickly-on-Alibaba-Cloud-OSS

Using Alibaba Cloud Object Storage Service (OSS) you can store and protect any amount of data. Millions of companies derive benefit from cost-efficiency, scalability, reliability and performance of this cloud storage.

Alibaba Cloud provides customers with several storage options for various use cases, like Object Storage Service (OSS), Apsara File Storage NAS (NAS), Elastic Block Storage (EBS) and Storage Capacity Unit (SCU). OSS got widespread adoption, so it could be assumed, if your company uses Alibaba Cloud, most likely, it uses OSS.

The artifacts are usually uploaded to OSS, during a CI/CD workflow. A proper artifact retention policy is aimed to avoid storage of non-actual artifacts. But let’s talk about another simple and extremely efficient way of resource optimization, which is applied to non-actual artifacts in your Alicloud OSS buckets. Retention for incomplete multipart upload (MPU) objects helps get rid of non-actual artifacts and prevent paying for idle cloud resources.

The part of the data which has already been uploaded to OSS will still be stored there, even if that specific upload session was not completed in a right way. Some may argue that upload failures are not a typical case during an R&D process to take into consideration. But the analysis and expertise shows that a bucket used for storing artifacts, which was created over one year, usually contains up to 20 percent of incomplete MPU objects of total storage.

How to check incomplete MPU in Alibaba Cloud OSS?

The Alicloud user interface shows incomplete MPU objects right in the bucket interface.

Navigate to bucket page – https://oss.console.aliyun.com/bucket/<region>/<bucket_name>/object

You will see files which are stored in the current bucket. On this page you will see how many incomplete MPU you currently have right now. Button called “Parts”.

incomplete MPU button parts

For checking how much storage you spend on an incomplete MPU you can open “parts” and click to “Start Statistics“. It will show you the total size for every incomplete MPU.

incomplete MPU start statistics button

If you see that incomplete multipart uploads bytes are a significant part of the total storage, you should go and optimize it.

Free cloud cost optimization. Lifetime

How to optimize incomplete MPU storage in Alibaba Cloud OSS?

The best way to ensure that a specific bucket won’t have incomplete MPU storage issue is to set Lifecycle Policy for it – https://www.alibabacloud.com/help/en/doc-detail/31904.htm

OSS Lifecycle management is a very powerful though complicated process, but creation of MPU policy is quite short and simple. 

You can use cloud shell console to do that (check the Cloud Shell icon in the top menu of the Alibaba Cloud console), as well as using alicloud Command Line Interface (aliyun).

To set a MPU retentions, do the following:

  1. Ensure that you have aliyun installed and configured.
  2. Prepare the following mpu-retention.xml file:

<?xml version=”1.0″ encoding=”UTF-8″?>

  <LifecycleConfiguration>

      <Rule>

          <ID>MPU Retention</ID>

          <Prefix></Prefix>

          <Status>Enabled</Status>

          <AbortMultipartUpload>

              <Days>7</Days>

          </AbortMultipartUpload>

      </Rule>

  </LifecycleConfiguration>

It describes the policy to abort all incomplete upload parts which were not active within the last 7 days.

  1. Run the following command to apply the MPU retention policy to your-bucket:

aliyun oss lifecycle -e <<bucket-endpoint>> –method put oss://<<your-bucket>> mpu-retention.xml

Now you have set the MPU retention rule for your bucket and may be sure that you won’t see the issue again. Please note that the lifecycle rule takes effect 48 hours after it is configured or modified.

Overlooked resources are contributed to a company cloud bill, and users don’t even expect that they’re paying for them.
💡 Find the ways of identifying and cleaning up orphaned snapshots to keep MS Azure and Alibaba Cloud costs under control → https://hystax.com/finops-best-practices-how-to-find-and-cleanup-orphaned-and-unused-snapshots-in-microsoft-azure-and-alibaba-cloud

Enter your email to be notified about new and relevant content.

Thank you for joining us!

We hope you'll find it usefull

You can unsubscribe from these communications at any time. Privacy Policy

News & Reports

FinOps and MLOps

A full description of OptScale as a FinOps and MLOps open source platform to optimize cloud workload performance and infrastructure cost. Cloud cost optimization, VM rightsizing, PaaS instrumentation, S3 duplicate finder, RI/SP usage, anomaly detection, + AI developer tools for optimal cloud utilization.

FinOps, cloud cost optimization and security

Discover our best practices: 

  • How to release Elastic IPs on Amazon EC2
  • Detect incorrectly stopped MS Azure VMs
  • Reduce your AWS bill by eliminating orphaned and unused disk snapshots
  • And much more deep insights

FinOps and cloud cost optimization for ML/AI workloads

Join our live demo on 27th 
March and discover how OptScale allows running ML/AI or any type of workload with optimal performance and infrastructure cost.