You are on page 1of 5

Hello, everyone. My name is Flavio Pereira.

I'm part of the technical enablement at


Oracle Cloud Infrastructure. And this is the portion 3 of the data migration. We're
talking about data transfer services and Storage Gateway. So on this part, we're
going to cover the online transport, we're going to cover pretty much the Storage
Gateway side.

So for online transport, there's a couple of ways you can do in order to move data
from your on-premises environment to the Cloud. When you're talking about online
transport, it relies over to internet connectivity, or bandwidth that you might
have on your on-premises environment to allow a connection from your location to
the cloud. So there's three options you can use. You can use VPN over internet. You
have a VPN connectivity from your on-premises to OCI. And then you can send data
through that VPN connection.

You can also use FastConnect, which allows you to give a dedicated link from your
location to OCI, and goes to the high speed. You can get from 1 gigabit per second
all the way to 10 gigabits per second on a network.

Another option to use is your Storage Gateway. So a Storage Gateway is considered


online transport because Storage Gateway is going to use your internet connectivity
to connect your on-premises environment to an object storage, and then allow you to
copy data from your on-premises environment to that Storage Gateway and then
synchronize the data to the object storage in OCI.

So what exactly is Storage Gateway? We're going to cover the Storage Gateway here.
So the Storage Gateway is an appliance that you can install in your on-premises
environment, and then have your applications connecting to that appliance, and then
sending data over to NFS file system.

So this appliance is a virtual machine. You can install that on any virtual machine
you might have inside of your on-premises environment. This is a Docker container.
You can easily install that on top of Linux environment and then bring that up on
your environment.

So then the Storage Gateway will have connectivity to the object storage, which is
the OCI object storage that's up and running on your account. So that makes things
easier to copy over. So if your application is copying the data inside of the
object storage to NFSv4, the Storage Gateway will recognize when the data landed on
specific multipoint. And then it will start copying the data from that mount point
to the object storage.

So what's the main use case for the Storage Gateway? Well, the main case is the two
of them is the hybrid cloud. You have on-premise application using the storage in
OCI.

And there's a few hybrid environments that we can walk through and cover some of
those details here. Another one is one-time data migration. So if you just need to
do migration of the data from your on-premises to OCI, that data can take that over
internet connectivity. So you might want to use Storage Gateway just to start
copying the data to that NFS mounting point, and then allow that to become
synchronized over to the OCI object storage.

So those are the two main use cases. But as you can see on the slide, there's other
things. We're going to talk to like cloud tiering, backups, archives. So I'm going
to go over some of those [AUDIO OUT].

So this is an option that you can use for hybrid cloud, what we call tiering using
cloud sync feature. So that's going to give you the ability to connect to any local
mass that you have on your on-premises environment. And cloud sync feature will
keep monitoring and watching any change or any change on that specific file system
or that specific folder. And then storage gateway will recognize that and start
copying your data from your NAS to your Storage Gateway, and then from there is
going to send that over to the object storage. So it's a really cool feature if you
have multiple NAS storages around your on-premises environment or it there is a
machine where you want to, every time some application copies the data to that
specific file system, you wanted the cloud sync to detect that and then start
copying the data from that NAS to the Storage Gateway. And then, later on, that's
going to be copied over to the object storage. So you can run multiple synch jobs
in parallel. You can have multiple sync jobs watching your storage, your NAS, and
then leverage that feature to start copying the data to your OCI.

Another option is using for content repositories. So you can just have Storage
Gateway on your on-site. And then all the data that's actually put on the Storage
Gateway, we send it over to the object storage. And then, from there, you can
access using pre-authenticated URL for all those files.

So if you're just looking to host some files inside of your on-premises


environment, like files or even reports or images that you want to create as a
content repository, that's a great use case to use object storage and Storage
Gateway for.

Another one is backup copies. So usually, traditionally, for on-premises


environment, they use this 3-2-1 strategy for backups. What exactly that is-- so at
least three copies of the data. That's one of the requirements. And then you have
to use two types of different storage. And at least one copy has to be put it
offsite, offsite of your location. So you can use cloud environment for that.

You have three copies. You can have one copy on your on-premises environment, and
two of the copies can be placed on the cloud environment. And you use two different
types of storage-- so cloud storage, or object storage, or archive storage counts
as a different type of storage for that. And then one of the copies will be placed
offsite. You can even put it on different regions inside of OCI. So you might have
copies across regions on your cloud environment.

So 3-2-1 strategies use tapes vaulted on-premises, which sometimes is cheap, but
actually keeping it vaulted remotely, that's where the cost will go up. So if you
use cloud storage for that, that's cheap. That's a better way to keep your data
secure. There's all the encryption for the object storage too, the REST encryption
that we provide to you. So you have all that data, your backups, and all the data
you put in there on a security way.

Another option is using for cold disaster recovery. So you might want to create a
standby environment in a cloud, in OCI in this case. And then if there is something
wrong, the Storage Gateway is copying your data from your on-premises to the object
storage. And then you can easily access that data from the object storage to
recreate your environment in a Cloud environment. So it's easy to use that for cold
disaster recovery.

So here's some of the options for tuning and pinning for frequently used data. So
as you know, Storage Gateway will synchronize the data from the Storage Gateway
appliance all the way to your OCI object storage. So if your application or user is
accessing files, you need to retrieve some of the files, the file got retrieved
from the OCI object storage to the Storage Gateway. If these files are frequently
retrieved type of data, Storage Gateway will recognize that and cache that file
locally. So that way you don't have to always request the data from the object
storage to be placed in the Storage gateway to be able to access your application.
So that's when we have a cache locally inside of of your storage gateway.
And you can also do some "Pin" on some files. So if you have a file that you know
your application will access-- every kind is going to be frequently accessed by
applications and users, you can "Pin" those files, and then they'll have a copy--
always going to have a copy locally inside of your Storage Gateway, so the
application doesn't have to retrieve the data from the OCI Object Storage to access
that. So this is a really cool-- those are really cool options for-- access will be
sent over to the Cloud, and then if the application is to retrieve and get access
to those files, so you don't have to wait for first all the bandwidth internet
connectivity to get access to those files, as you have a copy locally inside of
your Storage Gateway.

For Storage Gateway performance-- and that's one thing you need to understand-- so
with Storage Gateway, it's going to use your internet connectivity. It depends on
your bandwidth, and it can sustain some of the uploads and download speed here. So
as you can see on this slide, we have 10 gigabytes per second on the FastConnect.
And then if you have files that range from 10 gigabytes to 50 gigabytes, the upload
speed is going to be from 450 to 500 megabytes per second, and download is going to
be 700 to 750 megabytes per second. So you have to understand that it depends on
your internet connectivity, how you connected from your own premises to OCI, and
then the size of the file is really important too.

So Storage Gateway will perform better if you have bigger files than actually
putting a lot of small files inside of the storage. So here's some of the limits on
the Storage Gateway resources. You have to understand that the Storage Gateway
doesn't exceed 10 file systems per Storage Gateway. So if you need more than 10,
the recommendation is to spin up another Storage Gateway-- a second Storage Gateway
to bypass this limit. Another one is the local storage for file system cache, which
is important to understand as you're going to cache some of the files locally on
the Storage Gateway. We recommend that you have 500 gigabytes on the space for your
local-- for your Storage Gateway installation, then, to support the file system
cache.

The other one is the limit of number for files that you're going to put in on the
cache. So the amount is 20,000. So you can have 20,000 files on the cache. So just
keep aware of that number as well. And of course, to improve all the performers--
like I mentioned in a slide before-- try to compress the files, zip the files. Even
if it's small, just combine them, zip them, so it'd be better-- it'll reduce the
number of objects on the namespace so it's easy for Storage Gateway to synchronize
data from the Storage Gateway to the object storage.

Here are some of the Storage Gateway FAQ questions and some of the common questions
that customers will ask when they're using Storage Gateway. And one common question
is, if you can use Storage Gateway as a general purpose network attached storage
server? And the answer is no. It's not meant to be a NAS inside of your
environment, it's actually just to synchronize the data from your NAS environment
to Storage Gateway and send that over to the object storage. It's not meant to be a
NAS storage server per say.

And how much does Storage Gateway cost? So it's free with the OCI subscription. So
if you have and OCI subscription, Storage Gateway is free. You can just download
the Dockerfile, install on the Linux machine, and then bring it up to Storage
Gateway as well. Another question is, do I need to use Object Storage REST APIs to
use Storage Gateway? No, you don't have to use that. Storage Gateway will perform
all the tasks for you behind the scenes, so all you have to understand is how you
mount the Storage Gateway as an NFS file system. And then all the communication
from Storage Gateway to object storage will be executed to the REST API, but then
Storage Gateway will translate that for you. OK?

There's more information on FAQ if you click on the link. You're going to find all
the information there. All right. So let me do a quick Storage Gateway demo and
show you how you can create a file system on the Storage Gateway map to your OCI
storage bucket and then how you can map to your VM. All right. So this is my OCI
account. I'm logging in my OCI. And then I have a Storage Gateway installed in my
OCI account that's going to copy files from my virtual machines to the Storage
Gateway, and then sending that over to the OCI object storage.

So if you go on the compute instances, I do have VM, which is an Oracle Linux VM.
And then after installing Oracle Linux, I downloaded the Docker container for
Storage Gateway and then I installed a Storage Gateway here on that VM. So once you
install, once you go through the command line, install the Storage Gateway using
the Docker container, then you will be able to access a web page. And that web
page, you're going to walk through it to set up the Storage Gateway to talk to your
object storage.

So I have that configured here. I set up the Storage Gateway web page, and then I'm
accessing through a local host here on this port. And then I'm going to log in with
my username and password, which is the admin. And then once I do, this is the
homepage of the Storage Gateway. So I do have two file systems that I created here
already. I have the FSDATA and the SYNC one. But if you want to create a new file
system, all you have to do is click Create File System. And you've got to give it a
name for the file system. For example, TEST-FS.

And you can select if you want that file system to be connected to a standard OCI
object storage or archive OCI storage. So it depends on your use case or what
you're trying to achieve here. So if you want to just archive data, you can connect
that to the archive bucket. If you want to do just standard object storage, you can
select the first one. Then you have to enter the API ending point of the OCI object
storage. If you don't know, that's fine.

If you click here on API Endpoints, that's going to redirect you to the
documentation of the object storage, and then going to provide you the Storage
Gateway API endpoint right here. So it depends on your region. And then you can get
the region you want to create your object storage, and then use one of those API
endpoints there. So once you do it-- let's just pick one here. For example, let's
do phoenix-1. Just going to copy that. Back here.

Once they're entered, the API endpoint's going to ask me to enter the compartment
OCID, the tenancy OCID, the user OCID, the public key's finger print, and the
private key. So this is actually how the Storage Gateway will authenticate to your
account. You're going to specify the compartment where the OCI bucket will be
created, then the tenancy OCID, the user OCID that will make the authentication,
and then you have to create the finger print here-- provide the public finger print
here on that option. That's going to come up with the private key that you're going
to generate as well.

So if you created a private key already for your dynamic group or for your OCI CLI,
that's exactly the same steps that you're going to use here. So once you do that,
there's a couple of advanced options for file system configuration. So you can
define who can actually access the NFS mounting point. So you might have "I want to
do specific to a specific subnet address" or "I'm going to give access to
everyone." Some of the export options as well-- concurrent uploads and sync policy.
So this is actually a couple things you can do to make the performance a little bit
better on the Storage Gateway as well.

So once you have that, you save. And once you save, then it's going to start
creating the file system. So once you do that, what's going to happen is-- I'm
going to just go back here to my FSDATA-- you're going to have information like
that, where you have the Mount command that you can use to start mounting the NFS
file system on your local environment. So let's just show that real quick, how that
works.

So if I have here my terminal and I have a dev instance that I'm up and running
here, and on that dev instance, I have a mounting point called SGW-DATA. So there's
no files here. So what I'm going to do, I'm going to mount-- let's just get the
command here real quick. So I'm going to mount the NFS file system . So 172.16.2.11
is the IP address of my Storage Gateway. And I do have the FSDATA, right? This is
the file system name that I created here on Storage Gateway. And this is the mount
point that I want to use, so SGW-DATA.

So before I do the mount, let me just go back here to the OCI account and show you
the object storage. There's an object storage associated to this FSDATA that I
created on the Storage Gateway, which is that bucket here. And then you can see
that bucket-- there is one file saying the version of my Storage Gateway. So this
bucket is associated to the Storage Gateway. So every file I copied over will show
up here on the bucket. So let's do a quick test.

Let's just mount this here. Then, now, if you go to the SGW-DATA-- so if I create a
file to say touch flavio.txt. And then we're going to see that file will show up
here in a minute. So let's just go back here in the Metrics. Just go back to the
Objects. And there you go. So you can see, we already synchronize the file from
Storage Gateway to my object storage, so I can see the file here. Of course, this
is a small file. There's nothing on it. But once you start copying directories or
zip files over to the mount point, all the data will start to synchronize over
there.

So if you go back to the Storage Gateway page, you can see some activity
information. So that will show you the activity of synchronized files here. As it's
just a new one, I don't have any complete uploads, just going to refresh and get
the data from there. You can change some of the settings as well. So after the
fact-- so if you want to do some of the "change the compartment" or "change the
object storage," you can do that too. The only thing you have to do, you have to
disconnect the file, and then there's options to change that after you disconnect.

Yeah. So this is how the Storage Gateway works going forward. So if you want to use
the Cloud Sync option-- which is synchronize the data from a specific NAS, and then
once that happens, you start copying data over to the OCI object storage-- on the
Cloud Sync, you can start creating the job, and then once you do it, you provide
the source and the target path. So where exactly is the source and the target, and
then Storage Gateway will monitor that. And then we'll start synchronize data from
one location to another one. OK?

Yeah. So this is a quick demo of how the Storage Gateway works. And I hope that
helps you to actually start moving the data from your location to your OCI. This is
one of the options to use to start moving your data. Yeah. Thanks for watching.

You might also like