Whitepaper 'FinOps and cost management for Kubernetes'
OptScale is fully available as an open source solution under Apache 2.0 on GitHub
Ebook 'From FinOps to proven cloud cost management & optimization strategies'

How to save on Alibaba Cloud OSS by avoiding non-actual artifacts in your bucket

Share on linkedin
Share on twitter
Share on facebook

In one of our previous articles we described the quickest way to save on Amazon S3. Let’s discover how to reduce costs on Alibaba Cloud OSS by avoiding incomplete multipart uploads and non-actual artifacts in the bucket.

Problem description: Incomplete multipart uploads in Alibaba Cloud

How-to-save-quickly-on-Alibaba-Cloud-OSS

Using Alibaba Cloud Object Storage Service (OSS) you can store and protect any amount of data. Millions of companies derive benefit from cost-efficiency, scalability, reliability and performance of this cloud storage.

Alibaba Cloud provides customers with several storage options for various use cases, like Object Storage Service (OSS), Apsara File Storage NAS (NAS), Elastic Block Storage (EBS) and Storage Capacity Unit (SCU). OSS got widespread adoption, so it could be assumed, if your company uses Alibaba Cloud, most likely, it uses OSS.

The artifacts are usually uploaded to OSS, during a CI/CD workflow. A proper artifact retention policy is aimed to avoid storage of non-actual artifacts. But let’s talk about another simple and extremely efficient way of resource optimization, which is applied to non-actual artifacts in your Alicloud OSS buckets. Retention for incomplete multipart upload (MPU) objects helps get rid of non-actual artifacts and prevent paying for idle cloud resources.

The part of the data which has already been uploaded to OSS will still be stored there, even if that specific upload session was not completed in a right way. Some may argue that upload failures are not a typical case during an R&D process to take into consideration. But the analysis and expertise shows that a bucket used for storing artifacts, which was created over one year, usually contains up to 20 percent of incomplete MPU objects of total storage.

How to check incomplete MPU in Alibaba Cloud OSS?

The Alicloud user interface shows incomplete MPU objects right in the bucket interface.

Navigate to bucket page – https://oss.console.aliyun.com/bucket/<region>/<bucket_name>/object

You will see files which are stored in the current bucket. On this page you will see how many incomplete MPU you currently have right now. Button called “Parts”.

incomplete MPU button parts

For checking how much storage you spend on an incomplete MPU you can open “parts” and click to “Start Statistics“. It will show you the total size for every incomplete MPU.

incomplete MPU start statistics button

If you see that incomplete multipart uploads bytes are a significant part of the total storage, you should go and optimize it.

Free cloud cost optimization. Lifetime

How to optimize incomplete MPU storage in Alibaba Cloud OSS?

The best way to ensure that a specific bucket won’t have incomplete MPU storage issue is to set Lifecycle Policy for it – https://www.alibabacloud.com/help/en/doc-detail/31904.htm

OSS Lifecycle management is a very powerful though complicated process, but creation of MPU policy is quite short and simple. 

You can use cloud shell console to do that (check the Cloud Shell icon in the top menu of the Alibaba Cloud console), as well as using alicloud Command Line Interface (aliyun).

To set a MPU retentions, do the following:

  1. Ensure that you have aliyun installed and configured.
  2. Prepare the following mpu-retention.xml file:

<?xml version=”1.0″ encoding=”UTF-8″?>

  <LifecycleConfiguration>

      <Rule>

          <ID>MPU Retention</ID>

          <Prefix></Prefix>

          <Status>Enabled</Status>

          <AbortMultipartUpload>

              <Days>7</Days>

          </AbortMultipartUpload>

      </Rule>

  </LifecycleConfiguration>

It describes the policy to abort all incomplete upload parts which were not active within the last 7 days.

  1. Run the following command to apply the MPU retention policy to your-bucket:

aliyun oss lifecycle -e <<bucket-endpoint>> –method put oss://<<your-bucket>> mpu-retention.xml

Now you have set the MPU retention rule for your bucket and may be sure that you won’t see the issue again. Please note that the lifecycle rule takes effect 48 hours after it is configured or modified.

OptScale, a FinOps and cloud cost management platform, is available on cloud marketplaces

AWS-logo
azure
Alibaba Cloud marketplace
Digital-ocean

Overlooked resources are contributed to a company cloud bill, and users don’t even expect that they’re paying for them.
💡 Find the ways of identifying and cleaning up orphaned snapshots to keep MS Azure and Alibaba Cloud costs under control → https://hystax.com/finops-best-practices-how-to-find-and-cleanup-orphaned-and-unused-snapshots-in-microsoft-azure-and-alibaba-cloud

Enter your email to be notified about new and relevant content.

Thank you for joining us!

We hope you'll find it usefull

You can unsubscribe from these communications at any time. Privacy Policy

News & Reports

FinOps & Test Environment Management

A full description of OptScale as a FinOps and Test Environment Management platform to organize shared IT environment usage, optimize & forecast Kubernetes and cloud costs

From FinOps to proven cloud cost management & optimization strategies

This ebook covers the implementation of basic FinOps principles to shed light on alternative ways of conducting cloud cost optimization

Engage your engineers in FinOps and cloud cost saving

Discover how OptScale helps companies quickly increase FinOps adoption by engaging engineers in FinOps enablement and cloud cost savings