WEBVTT

00:07.020 --> 00:07.800

Let's face it.

00:07.830 --> 00:08.820

Nothing is perfect.

00:09.540 --> 00:13.110

The better we architect our systems, though, the more near perfect they become.

00:13.470 --> 00:17.190

But even so, someday something is likely to go wrong, despite our best effort.

00:18.090 --> 00:23.340

Part of preparing for the unexpected is regularly backing up our data to help us
recover from eventual

00:23.340 --> 00:23.970

failures.

00:25.600 --> 00:31.630

Snapshot Lifecycle Management, or SLM, as will refer to it in this course, helps us
fully automate

00:31.630 --> 00:34.570

backups of our Elasticsearch clusters and indices.

00:36.210 --> 00:42.150

We can set up a policies to instruct asylum when to back up, how often and how long
to keep each snapshot

00:42.150 --> 00:44.070

around before automatically deleting it.

00:45.970 --> 00:51.880

To experiment with snapshot lifecycle management will need to set up a repository
where the snapshots

00:51.880 --> 00:52.540

can be stored.

00:53.020 --> 00:59.800

Configure the repository in our Elasticsearch cluster defined the SLM policy to
automate snapshot creation

00:59.800 --> 01:05.050

and deletion and test the policy to see if we registered it correctly and it works
as expected.

01:05.860 --> 01:08.080

The steps will take our easy to understand points.

01:08.080 --> 01:10.000

We break them down into their basic actions.

01:11.610 --> 01:16.440

A repository is simply a place to save files and directories, just as you would on
your local hard
01:16.440 --> 01:16.860
drive.

01:17.550 --> 01:20.910

Elasticsearch uses repositories to stored snapshots.

01:21.980 --> 01:25.640

The first type we'll explore is the shared file system repository.

01:26.240 --> 01:29.630

In our exercise, since we're working with a single node, this will be easy.

01:30.050 --> 01:34.190

However, when multiple nodes are involved, we would have to configure them to
access the same file

01:34.190 --> 01:36.680

system, possibly located on another server.

01:38.030 --> 01:44.360

The second repository type we'll explore relies on cloud data storage services that
use service specific

01:44.360 --> 01:51.740

plug ins to connect to services like West three, Microsoft Azure Object Storage or
Google Cloud Storage

01:51.740 --> 01:53.270

Repository Checks.

01:54.990 --> 01:57.600

After picking the repository that best fits our needs.

01:57.870 --> 01:59.790

It's time to let Elasticsearch know about it.

02:00.420 --> 02:05.250

If we used a shared file system, we need to add a configuration line to the
Elasticsearch acquire HTML

02:05.250 --> 02:05.670

file.

02:05.790 --> 02:09.930

As we'll see in the exercises for cloud based storage repositories.

02:10.200 --> 02:13.290

We'll need to install the required repository plug in on each node.

02:13.920 --> 02:19.020

Elasticsearch needs to log into these cloud services, so we will also need to add
the required secret

02:19.020 --> 02:20.340

keys to its key store.

02:20.820 --> 02:23.010

This will be fully explained in the exercises.
02:24.410 --> 02:29.900
At this point, all prerequisites are met, and we can define the policy to instruct
SLM on how to automate

02:29.900 --> 02:36.260

back ups with the following parameters schedule what frequency and time to snapshot
our data.

02:36.620 --> 02:41.030

You can make this as frequent as you require without worrying too much about
storage constraints.

02:41.480 --> 02:46.190

Snapshots are incremental, meaning that only the differences between the last
snapshot and the current

02:46.190 --> 02:47.750

snapshot need to be stored.

02:48.380 --> 02:53.300

If almost nothing changed between yesterday's backup in today's data, then the
snapshot will require

02:53.300 --> 02:58.580

negligible storage space, meaning that even if you have gigabytes of data, the
snapshot might require

02:58.580 --> 03:00.050

just a few megabytes or less.

03:01.250 --> 03:06.800

The name defines the pattern he use when naming the snapshots repository specifies
where to store our

03:06.800 --> 03:07.580

snapshots.

03:08.240 --> 03:14.180

Config dot indices is the list of indices to include and retention is an optional
parameter we can use

03:14.180 --> 03:17.510

to define when SLM can delete some of the snapshots.

03:18.170 --> 03:21.800

We can specify three options here expire after.

03:22.010 --> 03:23.690

This is a time based expiration.

03:23.810 --> 03:30.050

For example, a snapshot created on January 1st with an expire after set to 30 days
will be eligible

03:30.050 --> 03:32.330

for deletion after January 31st.

03:33.550 --> 03:39.280

Men count tells Elasticsearch to keep at least this number of snapshots, even if
they're all expired.

03:40.000 --> 03:44.500

And Max Count tells Elasticsearch to never keep more than this number of snapshots.

03:45.280 --> 03:51.940

For example, if we have 100 snapshots and only one is expired, but Max Count is set
to 50, then 50

03:51.940 --> 03:55.390

of the oldest snapshots will be deleted, even the unexpired ones.

03:57.620 --> 04:00.590

With our policy finally defined, we can display its status.

04:01.250 --> 04:03.520

This will list policy details and settings.

04:03.530 --> 04:06.980

Show us how many snapshot attempts were successful and how many failed.

04:07.460 --> 04:11.270

When SLM is scheduled to run the next time and other useful information as well.

04:13.010 --> 04:16.130

So with the theoretical part out of the way, we can finally learn by doing.

04:16.430 --> 04:20.060

Since we're experimenting with a single note here, things will be pretty
straightforward.

04:22.840 --> 04:27.760

For the repository, we'll just use a directory located on the same machine where
Elasticsearch is running.

04:28.000 --> 04:30.730

So let's start by creating a mount shared ESR directory.

04:31.600 --> 04:39.010

Pseudo mkdir bash p slash mmt slash shared slash s.

04:40.740 --> 04:41.190

All right.

04:41.430 --> 04:46.140

Now, in a multi node, Elasticsearch cluster, you would then have to mount your
actual shared storage

04:46.140 --> 04:47.910

on each node using the directory mount.

04:47.910 --> 04:48.240

Shared.

04:48.240 --> 04:49.950

Yes, as a mount point at this point.

04:50.580 --> 04:56.190

So if you're using NFS besides entering the appropriate mount command such as Sudo
mount, whatever

04:56.190 --> 05:01.770

your NFS server is to this directory, you would also add relevant entries to your
ETSI FS tab file

05:01.770 --> 05:06.030

as well so that the NFS shares are automatically mounted each time the servers boot
up.

05:06.360 --> 05:10.440

But again, for the sake of example, we're just going to store this on our local
file system in a directory

05:10.440 --> 05:13.020

under Mount Shared S and not a shared file system.

05:13.830 --> 05:17.220

First, let's make the Elasticsearch username and group the owners of Mount shared.

05:17.220 --> 05:24.020

Yes, and then give the user full read and write permissions to do that will say
sudo change own dash.

05:24.240 --> 05:24.690

Ah.

05:25.830 --> 05:28.130

Elasticsearch calling Elasticsearch.

05:30.490 --> 05:30.910

Slash.

05:30.910 --> 05:31.780

Empty slash.

05:31.780 --> 05:32.590

Shared slash.

05:32.590 --> 05:33.130

Yes.

05:34.460 --> 05:42.170

And Sudo change mod 75 oh slash I'm in t slash shared slash s.

05:43.710 --> 05:44.130

All right.

05:45.210 --> 05:50.910

Next will either line path dot repo amount shared yes to Elasticsearch start way so
that the service

05:50.910 --> 05:53.250

knows the location of its allocated repository.

05:53.850 --> 05:58.410

Note that on production systems we should add this line to all master and all data
nodes as well.
05:58.860 --> 06:02.700
So pseudo dash s2 elasticsearch.

06:05.970 --> 06:12.360

And now we're Elasticsearch and we can say echo single quote path dot repo.

06:15.210 --> 06:17.970

Colon square bracket.

06:18.180 --> 06:19.410

Double cooked slash.

06:19.410 --> 06:20.850

Empty slash.

06:20.850 --> 06:21.930

Shared slash.

06:21.930 --> 06:22.830

Yes.

06:24.480 --> 06:25.350

Square bracket.

06:26.440 --> 06:27.070

Single quote.

06:28.780 --> 06:32.980

And we're going to send that off to Etsy, slash Elasticsearch.

06:34.660 --> 06:35.770

Elasticsearch.

06:37.320 --> 06:38.190

Not why IMO.

06:38.820 --> 06:43.170

So this is just going to append that line path dot repo amount share to yes to the
end of our Elasticsearch

06:43.170 --> 06:44.160

dot line ML file.

06:45.370 --> 06:45.670

Great.

06:47.880 --> 06:48.180

All right.

06:48.180 --> 06:52.590

Now, in every know where we make this change, we need to restart Elasticsearch to
apply that new setting.

06:52.590 --> 07:01.420

So let's get out of the Elasticsearch account here and we can now say sudo system
control restart Elasticsearch

07:01.590 --> 07:02.520

dots service.

07:10.020 --> 07:11.220

Give that a few seconds.

07:18.380 --> 07:21.770

At this point, we can now register the repository in our cluster like so.

07:22.040 --> 07:22.550

Curl.

07:22.580 --> 07:22.850

Dash.

07:22.850 --> 07:23.060

Dash.

07:23.060 --> 07:23.690

Location.

07:24.290 --> 07:24.590

Dash.

07:24.590 --> 07:24.920

Dash.

07:24.920 --> 07:25.970

Request puts.

07:27.050 --> 07:29.270

Let's do a backslash there to get a new line.

07:30.770 --> 07:32.240

HTTP colon, slash slash.

07:32.240 --> 07:39.410

Look localhost on port 9200 slash underscore snapshot slash backup.

07:39.770 --> 07:41.300

Underscore repository.

07:45.210 --> 07:47.760

And single, quote, another backslash for another new line.

07:48.390 --> 07:51.870

We're going to get the header line because our special version of Curl takes care
of that for us.

07:51.870 --> 07:57.810

So we're just going to jump straight to dash dash data, dash raw, single quote,
curly bracket.

07:58.080 --> 07:59.820

And now we have the body of our request here.

08:00.450 --> 08:03.000

We're going to specify a type of force.

08:03.150 --> 08:11.130

So as in file system, comma and the following settings within this block, we will
have a location.

08:12.580 --> 08:16.000

That is set to slash and anti slash shared slash.

08:16.030 --> 08:16.600

Yes.

08:17.110 --> 08:20.200

And under there will have a backup underscore repository.

08:21.500 --> 08:26.210

Double Coke and close off all of our curly brackets and single quote there.

08:26.210 --> 08:29.170

And that should send it off and looks like it went through.

08:29.180 --> 08:33.920

We have an acknowledgement, okay, with the next command, we can list all the
registered repositories

08:33.920 --> 08:35.720

to make sure everything is configured properly.

08:35.900 --> 08:43.250

So we'll just say curl local host 1800 underscore snapshot pretty.

08:47.180 --> 08:50.360

And there you can see that it did take we do have that now shared.

08:50.360 --> 08:52.700

Yes back up repository set up successfully.

08:53.090 --> 08:55.520

We can now define our first asylum policy.

08:55.550 --> 08:58.130

Let's go through the details of this next action as we go through it.

08:58.280 --> 09:01.430

We'll start off by just kicking off the request with Colonel.

09:01.490 --> 09:02.620

That's just location.

09:03.640 --> 09:03.860

Dash.

09:03.860 --> 09:04.430

Request.

09:04.520 --> 09:06.380

Put backslash.

09:07.400 --> 09:08.030

HTP.

09:08.030 --> 09:08.540

Colon slash.

09:08.540 --> 09:08.810

Slash.

09:08.810 --> 09:09.770

Local host.

09:11.150 --> 09:12.770

1800 slash.

09:12.770 --> 09:13.310

Underscore.

09:13.430 --> 09:15.860

Slim slash policy.

09:16.280 --> 09:17.600

Slash backup.

09:18.140 --> 09:19.460

Underscore policy.

09:20.450 --> 09:22.670

Underscore daily single quote.

09:23.030 --> 09:23.900

That's the format there.

09:23.930 --> 09:24.500

Double check it.

09:25.850 --> 09:29.060

We'll skip the header line because our curl command actually does that for us.

09:29.060 --> 09:30.050

And we'll go straight to dash.

09:30.050 --> 09:30.650

Dash data.

09:30.650 --> 09:31.460

Dash raw.

09:33.310 --> 09:34.570

Single curly bracket.

09:34.570 --> 09:36.370

And here comes the body of the request.

09:36.820 --> 09:37.870

We'll start with schedule.

09:39.680 --> 09:40.250

And so that.

09:40.250 --> 09:42.530

20033.
09:42.590 --> 09:43.220
Star.

09:43.250 --> 09:43.740

Star.

09:43.760 --> 09:44.600

Question mark.

09:45.200 --> 09:46.310

Double quote, comma.

09:46.640 --> 09:47.540

So what does that mean?

09:48.440 --> 09:52.070

Well, we were instructing it to run every day at 3:03 a.m..

09:52.280 --> 09:54.590

This is specified with a cron expression here.

09:54.920 --> 10:02.480

And the format here is second minute our day of month, month, day of week and year.

10:02.480 --> 10:03.620

Where the year option.

10:07.040 --> 10:08.810

Where the year parameter is optional.

10:10.270 --> 10:10.600

All right.

10:10.630 --> 10:17.110

Next line name is going to be in plenty brackets.

10:17.260 --> 10:17.860

Back up.

10:19.320 --> 10:19.890

Dash.

10:21.570 --> 10:28.470

Curly bracket now slash DX curly bracket, pointy bracket and the quote comma.

10:29.400 --> 10:34.410

So all the snapshot names will start with a fixed string back up dash and the date
will be appended

10:34.410 --> 10:34.710

to it.

10:34.740 --> 10:35.610

That's what we're saying here.

10:36.450 --> 10:40.290

A random string of characters will be added at the end to ensure that each name is
unique.
10:40.500 --> 10:42.510
It's usually a good idea to use date math.

10:42.630 --> 10:47.160

This helps us to easily spot the date and time of each object, since the resulting
names could end

10:47.160 --> 10:48.000

up looking like clusters.

10:48.000 --> 10:51.120

Six Dash 20 20.0 3.23.

10:51.120 --> 10:52.020

Underscore 15.

10:52.020 --> 10:52.940

Colon 16.

10:52.950 --> 10:56.310

So that would be like easy to read and figure out what that file represents.

10:57.920 --> 11:04.430

I still say repository and start that up to back up underscore repository.

11:06.440 --> 11:10.460

They will store the snapshots in the repository that we previously registered in
our cluster.

11:11.870 --> 11:12.260

All right.

11:12.260 --> 11:14.930

Next, we're going to say config.

11:17.330 --> 11:18.830

And that's going to be a new block here.

11:19.910 --> 11:21.170

And we'll say indices.

11:22.770 --> 11:27.000

And set that to an inside square brackets, a star character.

11:28.500 --> 11:33.870

So by using the special asterisks, wildcard characters star there, we include all
the indices in our

11:33.870 --> 11:38.580

cluster as part of the back up close off that block there.

11:39.800 --> 11:41.210

And we'll say retention.

11:44.870 --> 11:51.530

Again a new block expire underscore after and we'll set that to 60 days.

11:53.490 --> 11:58.380

So we're just instructing SLM to periodically remove all snapshots that are older
than 60 days here.

11:59.370 --> 12:00.780

Now we can just close everything off.

12:02.440 --> 12:03.130

Like so.

12:05.130 --> 12:06.120

And it went through.

12:06.360 --> 12:06.690

Great.

12:07.170 --> 12:09.780

So let's check out the status of our newly created policy now.

12:10.200 --> 12:12.720

We can say Colonel, local host.

12:12.780 --> 12:13.340

Coordinator.

12:14.310 --> 12:14.730

Slash.

12:14.730 --> 12:15.180

Underscore.

12:15.270 --> 12:15.720

Slim.

12:16.320 --> 12:17.370

Slash policy.

12:18.150 --> 12:19.050

Slash backup.

12:19.710 --> 12:20.910

Underscore policy.

12:21.240 --> 12:22.200

Underscore daily.

12:23.390 --> 12:25.730

And question mark human to make a human readable.

12:38.100 --> 12:39.240

And looks like it's in there.

12:39.570 --> 12:41.120

Let's go for pretty format, actually.

12:44.790 --> 12:45.360

Even better.

12:46.080 --> 12:46.320

All right.
12:46.320 --> 12:49.950
So it looks like that took we have all the parameters in there successfully that we
specified.

12:49.950 --> 12:51.570

Looks like it's all in there correctly.

12:52.730 --> 12:57.260

So besides confirming the policy settings we just defined, we can also see when
this will run the next

12:57.260 --> 12:59.030

time with next execution.

12:59.690 --> 13:01.290

How many snapshots have been taken?

13:01.310 --> 13:02.060

How many have failed?

13:02.060 --> 13:02.690

And so on.

13:03.350 --> 13:07.550

Of course, we may not be able to wait until the next scheduled run, since we are
testing and experimenting

13:07.550 --> 13:11.060

so we can execute the policy immediately by using the following command.

13:11.810 --> 13:12.260

Curl.

13:13.070 --> 13:13.520

Dash.

13:13.520 --> 13:14.210

Location.

13:14.780 --> 13:15.050

Dash.

13:15.050 --> 13:15.260

Dash.

13:15.260 --> 13:15.830

Request.

13:16.160 --> 13:16.700

Post.

13:19.580 --> 13:22.440

Actually p local hosted NATO.

13:22.940 --> 13:23.350

Oops.

13:25.140 --> 13:28.140

Underscore asylum policy.
13:29.430 --> 13:30.450
Slash back up.

13:31.470 --> 13:36.900

Policy daily and underscore execute to actually make it do it.

13:37.230 --> 13:39.210

Now, let's see what that does.

13:45.680 --> 13:48.830

Well, it came back with a snapshot name and what had actually saved it off, too.

13:48.860 --> 13:50.000

So it looks like it ran.

13:50.210 --> 13:53.300

Let's check how much data storage is now used by the repository.

13:55.160 --> 13:57.080

Let's switch back to our Elasticsearch user.

14:02.630 --> 14:04.070

And check out our amount shared.

14:04.070 --> 14:04.270

Yes.

14:04.310 --> 14:04.850

Directory.

14:08.500 --> 14:12.210

And we can say, do you dash h dash?

14:12.220 --> 14:14.680

Max death equals one.

14:16.620 --> 14:19.740

And looks like we have 20 megabytes of stuff in our backup repository.

14:19.980 --> 14:25.440

So for our first run, this should be similar in size to what was actually used by
our indices by checking

14:25.440 --> 14:26.680

the status of the policy again.

14:26.700 --> 14:31.470

He'll show us a new field last success indicating the snapshot that we just took
earlier if we were

14:31.470 --> 14:33.420

to keep running this over and over and over again.

14:33.900 --> 14:34.680

So there you have it.

14:37.260 --> 14:41.220

Next, let's set up asylum within a US S-3 repository.
14:41.880 --> 14:46.230
Amazon Web Services is very popular in the corporate world, so it's useful to go
through an example

14:46.230 --> 14:49.920

where we use in a US S3 bucket to store snapshots.

14:50.520 --> 14:54.300

Now the following steps are going to require basic knowledge about S3 and IAM.

14:54.300 --> 14:59.400

Since we need to configure the bucket and secure login mechanisms beforehand,
you're going to need

14:59.400 --> 15:03.720

an account to actually go through this exercise and have some familiarity with us.

15:04.110 --> 15:08.490

If that doesn't describe you and you're not comfortable with giving us a credit
card number because

15:08.490 --> 15:13.950

the service could cost real money, just watch and you don't have to actually follow
along in this exercise

15:13.950 --> 15:14.650

if you don't want to.

15:14.670 --> 15:15.330

That's okay.

15:16.170 --> 15:20.640

But first, to be able to work with an S3 bucket S3 is Amazon Simple Storage
Service.

15:21.180 --> 15:27.450

Elasticsearch will require a plug in which can easily be installed first, will go
to the user share

15:27.450 --> 15:31.170

Elasticsearch directory and from here we'll say sudo bin.

15:31.650 --> 15:36.150

Slash Elasticsearch dash plug in install.

15:37.100 --> 15:39.800

Repository Dash S3.

15:41.300 --> 15:41.990

Just like that.

15:44.200 --> 15:45.880

And it should just do that automatically.

15:49.340 --> 15:51.230

We will continue and let it do what it wants to do.

15:51.620 --> 15:51.970

All right.

15:51.990 --> 15:53.300

And we're good now.

15:53.300 --> 16:00.350

We just need to restart the service to actually activate that plug in pseudo system
control, restart

16:01.820 --> 16:04.520

the Elasticsearch start service.

16:08.440 --> 16:10.600

And that will take a few seconds and we're done.

16:10.630 --> 16:10.960

Great.

16:11.620 --> 16:11.890

All right.

16:11.890 --> 16:17.140

So next, Elasticsearch needs to be able to log into the services offered by USB
three in a secure manner.

16:17.620 --> 16:23.770

So to do this, we need to log into our US account and create an IAM user with the
necessary S3 permissions

16:23.770 --> 16:25.360

before continuing with this lessons.

16:26.260 --> 16:30.550

Now again, you're going to need a basic knowledge of IWC and NATO Lewis account to
do this, which

16:30.550 --> 16:33.100

also involves having a credit card on file with IWC.

16:33.100 --> 16:37.540

So again, if you're not comfortable with that or you don't want to spend money,
just watch.

16:37.570 --> 16:37.870

Okay?

16:39.190 --> 16:44.050

So if you do want to go follow along anyway and set up an AWB account, you can
follow the official

16:44.050 --> 16:45.670

guides out there if you don't already have one.

16:45.670 --> 16:46.480

It's pretty simple.

16:47.080 --> 16:52.420

You will be asked to enter a payment method, but this particular exercise actually
does follow under
16:52.420 --> 16:53.950
the IWC free tier.

16:53.950 --> 16:58.960

So if you were to set up a brand new IWC account, you could actually get through
this without paying

16:58.960 --> 17:01.450

any money but still be careful.

17:02.230 --> 17:04.150

So I'm going to log into the IWC console.

17:04.150 --> 17:05.200

I've actually already done that.

17:06.280 --> 17:11.080

So to create the credentials we need, we're going to go to our IWC console and go
into IAM.

17:13.100 --> 17:17.210

Identity and access management and from here will create a new user that we can use
for this purpose.

17:17.330 --> 17:23.180

So we'll click on users and add a user and we need to give this user a name.

17:23.180 --> 17:26.900

Let's call it Elasticsearch Dash S3.

17:29.450 --> 17:30.230

Boxer Tron.

17:30.230 --> 17:30.680

Here we go.

17:31.850 --> 17:37.160

And now we need to make this programmatic access because we are going to be using
this from a program,

17:39.020 --> 17:39.890

go to permissions.

17:39.890 --> 17:46.910

And from here, we're going to attach existing policies directly and we're going to
search for Amazon

17:46.910 --> 17:53.240

S3 full access and go ahead and select that and move on to the tag screen.

17:54.290 --> 18:00.750

We don't need any review what we've done, so we're creating a new user,
Elasticsearch S3 with Amazon,

18:00.750 --> 18:03.980

this three full access policy, go ahead and create that user.

18:05.180 --> 18:09.830

And now we have the access key ID and the secret access key for this new user.

18:10.130 --> 18:14.210

Now as soon as you leave the screen, you will never be able to get that secret
access key again.

18:14.510 --> 18:18.800

So if you are following along, be sure to show that and copy it off somewhere safe.

18:19.040 --> 18:23.630

Same thing with the access kid, so make sure you have those squirreled away
somewhere so we can now

18:23.630 --> 18:27.290

enter that into Elasticsearch so it can communicate with our S3 buckets.

18:28.640 --> 18:33.080

Once you've configured the IAM user and you have your keys available, let's add
them to Elasticsearch

18:33.080 --> 18:33.800

is key store.

18:35.300 --> 18:38.730

So this would need to be repeated for each note if we were on a production cluster.

18:38.750 --> 18:40.790

But for this note, let's just say pseudo.

18:42.000 --> 18:47.100

Note that I'm in the user share Elasticsearch directory here bin slash
Elasticsearch.

18:47.670 --> 18:52.260

Dash key store add as three dot client.

18:53.460 --> 18:53.910

Whoops.

18:54.180 --> 18:55.130

Help if I spelled that right.

18:55.140 --> 19:00.750

Client dot default dot access underscore key.

19:01.260 --> 19:01.550

Okay.

19:02.550 --> 19:04.740

And it will prompt you for the access key.

19:05.190 --> 19:09.300

I already have one, but I'm going to go ahead and overwrite that and I'm going to
copy that.

19:11.360 --> 19:12.590

From the one that I just made.

19:13.530 --> 19:15.030

And right click to paste that in.

19:16.710 --> 19:18.660

And I'm going to do the same thing for the secret key.

19:20.110 --> 19:23.800

So we'll say pseudo bean slash elasticsearch.

19:25.180 --> 19:32.020

Dash key store ad as three dot client dot defaults dot secret to underscore key.

19:34.260 --> 19:35.790

Make sure that's all spelled properly.

19:36.840 --> 19:37.890

And same deal.

19:38.160 --> 19:41.640

I'm going to overwrite that and select the new one.

19:42.450 --> 19:42.930

Copy it.

19:45.630 --> 19:49.710

And right click to paste that and take that in.

19:50.520 --> 19:55.110

We can now instruct the S3 client to reload its security settings and pick up the
new logging credentials

19:55.110 --> 19:56.340

and make sure they look okay.

19:56.880 --> 19:57.210

Curl.

19:57.210 --> 19:57.450

Dash.

19:57.450 --> 19:57.690

Dash.

19:57.690 --> 19:58.350

Location.

19:58.980 --> 19:59.250

Dash.

19:59.250 --> 19:59.490

Dash.

19:59.490 --> 20:00.020

Request.

20:00.030 --> 20:00.540

Post.

20:03.310 --> 20:06.140

HTP local hosts.
20:06.640 --> 20:09.490
9200 Underscore Nodes.

20:10.660 --> 20:13.780

Reload secure settings.

20:15.640 --> 20:16.630

In pretty format.

20:18.740 --> 20:19.940

Looks like it worked.

20:20.450 --> 20:22.120

Successful as one failed to zero.

20:22.130 --> 20:23.060

That's all a good sign.

20:23.900 --> 20:24.260

All right.

20:24.260 --> 20:27.380

So at this point, we can register the cloud storage repository.

20:28.070 --> 20:32.360

The only mandatory parameter for registering an S three repository is its bucket
name.

20:32.870 --> 20:34.190

So let's just do that.

20:35.150 --> 20:35.630

Curl.

20:36.110 --> 20:36.530

Dash.

20:36.530 --> 20:37.180

Location.

20:37.190 --> 20:37.490

Dash.

20:37.490 --> 20:37.970

Dash.

20:38.330 --> 20:39.020

Request.

20:40.050 --> 20:40.530

Put.

20:41.900 --> 20:42.620

HDP.

20:43.630 --> 20:45.670

Local host 9200.
20:46.630 --> 20:48.130
Underscore snapshots.

20:49.300 --> 20:50.260

Slash backup.

20:50.950 --> 20:52.390

Underscore repository.

20:53.930 --> 20:55.130

Underscore is three.

20:58.360 --> 21:00.520

And again we'll skip the header because we're cheating.

21:00.830 --> 21:01.720

Dated Ashura.

21:04.440 --> 21:05.430

Kick off that block.

21:06.750 --> 21:10.650

The tape will be as three and the settings.

21:12.880 --> 21:14.440

We'll just consist of a bucket name.

21:16.240 --> 21:18.910

And we'll call her a bucket elastic dash slam.

21:20.770 --> 21:21.760

Close all that off.

21:26.350 --> 21:30.140

And not surprisingly, it gave me an error because that bucket doesn't actually
exist yet.

21:30.220 --> 21:31.560

So back to the console.

21:32.970 --> 21:35.610

So let's just go to S3 within the console here.

21:37.920 --> 21:44.310

And from here we'll make a new bucket, create bucket, and you will need to give
this some globally

21:44.310 --> 21:45.300

unique name.

21:45.310 --> 21:49.290

So I'm going to just call this elastic dash and slam a dash f Kane.

21:49.290 --> 21:52.380

But you cannot use that same name because it has to be unique.

21:52.380 --> 21:56.790

So substitute your own name, organization, just something to make that special.
21:57.420 --> 21:59.760
And that should be pretty much all that we need.

22:00.780 --> 22:03.780

The default settings should work for us and let's create bucket.

22:06.660 --> 22:07.080

All right.

22:07.080 --> 22:11.160

So an elastic asylum if Cain is sitting there waiting for us to receive data.

22:13.890 --> 22:14.970

And let's try that again.

22:18.320 --> 22:22.640

Instead of the classic slam, it's going to be elastic asylum F Cain or whatever you
named yours to.

22:22.640 --> 22:26.600

It has had to be a unique name and that time it worked.

22:26.990 --> 22:27.590

Very cool.

22:28.490 --> 22:33.290

So if we now list all registered repositories, we should see both our shared
filesystem repository

22:33.290 --> 22:35.330

and our cloud storage repository.

22:36.020 --> 22:36.890

Let's say curl.

22:39.670 --> 22:41.650

Local host, 1900.

22:42.960 --> 22:44.700

Slash underscore snapshot.

22:45.950 --> 22:46.310

Pretty.

22:49.180 --> 22:50.170

And there they both are.

22:50.440 --> 22:50.770

Cool.

22:51.760 --> 22:53.980

Now let's define our second asylum policy.

22:54.010 --> 22:58.750

The policy settings will be similar to before, but will now target our brand new as
three repository

22:59.050 --> 23:00.820

as a destination for a snapshot.
23:00.820 --> 23:02.830
So I'll plod through this as quickly as I can.

23:03.250 --> 23:03.610

Curl.

23:03.610 --> 23:04.390

Location.

23:04.390 --> 23:05.740

Request put.

23:08.140 --> 23:18.130

HTP local host coordinator a100 slash underscore SLM policy backup policy daily.

23:19.670 --> 23:20.570

As three.

23:23.400 --> 23:24.120

Skip the header.

23:24.150 --> 23:25.000

Go straight to the data.

23:25.020 --> 23:25.740

Rob Lock.

23:28.480 --> 23:32.830

We'll start with a schedule indicating that we'll run this at 3:30 a.m..

23:35.330 --> 23:36.170

Just like before.

23:37.700 --> 23:40.910

Our name again will just be back up dash timestamp.

23:48.940 --> 23:49.870

Just like that.

23:51.810 --> 23:53.070

Our repository.

23:54.440 --> 24:04.940

Will be set up to our new one backup underscore repository, underscore S3 and our
config block will

24:04.940 --> 24:07.100

just tell us which indices we want to back up.

24:07.190 --> 24:08.600

And again, we'll say all of them.

24:17.030 --> 24:17.540

Okay.

24:19.760 --> 24:23.060

And the retention will make it a little bit more complicated this time.
24:25.070 --> 24:26.030
Make sure you spell it right.

24:29.270 --> 24:29.600

Again.

24:29.600 --> 24:31.070

We'll have expire after.

24:32.360 --> 24:33.920

And set that to 60 days.

24:34.850 --> 24:38.450

But we'll also set a min count in a max count just to make things interesting.

24:46.100 --> 24:49.990

Close everything off and hopefully I don't have any typos.

24:50.000 --> 24:50.810

We'll see if it works.

24:51.470 --> 24:51.800

All right.

24:52.140 --> 24:52.850

And it's true.

24:53.630 --> 24:56.500

So now we can fire off a snapshot operation like so.

24:57.500 --> 25:01.310

Curl location, request post.

25:03.470 --> 25:10.460

HTP local host 900 underscore a solemn slash policy.

25:11.300 --> 25:14.050

Slash backup policy.

25:14.240 --> 25:15.560

Daily as three.

25:15.620 --> 25:17.480

What's the name of our new policy that we just created?

25:18.740 --> 25:19.250

Underscore.

25:19.250 --> 25:19.960

Execute.

25:20.940 --> 25:21.770

See what happens.

25:26.860 --> 25:28.090

Looks like it worked.

25:28.180 --> 25:30.160

So the confirmation is displayed immediately.
25:30.370 --> 25:33.010
Now, this might sometimes give the illusion that the job is done.

25:33.490 --> 25:37.510

However, when a lot of data has to be uploaded, the transfer might continue in the
background for

25:37.510 --> 25:38.320

a long time.

25:39.040 --> 25:43.620

But if we go back to our IWC three bucket console, we might be able to see some
data coming in.

25:43.630 --> 25:44.380

Let's take a look.

25:47.200 --> 25:48.910

So there's that new bucket we just made.

25:51.840 --> 25:53.130

And yeah, there's data in it.

25:53.670 --> 25:54.270

Really cool.

25:56.450 --> 26:00.500

And you can also check this using the IWC command line interface if you want it to
as well, instead

26:00.500 --> 26:03.200

of going through the user interface on the web console here too.

26:03.200 --> 26:04.610

But it looks like it worked.

26:04.610 --> 26:05.240

That's awesome.

26:05.780 --> 26:07.370

Congratulations on all your hard work.

26:07.760 --> 26:12.260

You're now armed with the knowledge to create awesome systems that automatically
back up important data

26:12.500 --> 26:14.720

and can save the day when disaster strikes.