Session 3: Cloud Storage

Introduction

Central to using cloud for nearly all services is storing data. Cloud storage is quite different from what most are used to related to saving a file to your disk or USB removable media or even our HPC. During our workshop on creating a VM we didn't use cloud storage, we simply create a VM "virtual disk" that is attached to the VM just like your hard drive is attached to your own computer. However there are disadvantages to this : 1. the main OS disk is typically deleted when the VM is deleted, although you can create a 'durable' disk to share 1. the data on the main OS disk is tied to that Virtual Machine and hence that operating system, that is, it's typically inaccessible from other cloud services 1. it is limited in size. The largest of virtual disks are around 1 TB. Azure Cloud storage accounts are limited to 5 TB and you may have multiple storage accounts.
1. You can only move data to/from a virtual or shared disk storage using a virtual machine 1. Most importantly virtual disks very expensive compared to cloud storage

Cloud companies think of "storage" as anything that save files, or perhaps more importantly anything they can market to you as something to save files.

Readings

• Storage as a Service from "Cloud Computing for Science and Engineering"
• Optional: this is long (27 minutes) but a good basic introduction to Azure storage:
  Azure Training: Explore Azure Storage services ( free training from Microsoft Learn)
• Table of Azure Storage Product Offerings
• Azure Documentation: Introduction to the core Azure Storage services

• Slides/Lecture: Azure Cloud Storage for Researchers with links for details on each slide

• optional Understanding block blobs, append blobs, and page blobs

Activities

• Download and install the Azure Cloud Storage Explorer See the "Download now" button at the top of that page. You may review the content of the page
• complete exercises in Using Azure Cloud Storage to create and use storage
• Azure Storage Pricing Exercise

Meeting September 24 2:00-3:30pm

• About Projects, Mahmoud Parvizi
• Discussion and Review of previous sessions:
  • Using the Portal
  • Creating and Using Virtual Machines
• What is cloud storage?
  • concept review: cloud storage vs VM disks
  • discuss exercises to be worked on next week
• Review of Broad Cloud Concepts:
  • On-Demand, Compute, Storage, Identity Management
• Discussion : future activities and needs

Optional Activity:

Python And Cloud Storage

For Intermediate Python users, and if you have time and interest, consider this tutorial from Azure: Quickstart: Manage blobs with Python v12 SDK

• Requirements:
• use the blob storage account you created in the exercise above
• familiarity with Azure portal
• Python installed on your computer (suggest python 3.6 minimal)
• familiarity with the terminal and command line