Uploading data from Computer to server using command line 2020

HOW TO UPLOAD DATA FROM COMPUTER TO SERVER USING COMMAND LINE

There are situations where it is not possible or not convenient to use the Upload Form to add
new Layers to GeoNode via the web interface. As an instance:
● The dataset is simply too big to be uploaded through a web interface.
● We would like to import some data from the mass storage programmatically.
● We would like to import some tables from a DataBase.
● We need to process the data first and, maybe, transform it to another format.
This section will walk you through the various options available to load data into your GeoNode
from GeoServer, from the command-line or programmatically.
Management Command importlayers
The geonode.layers Django app includes 2 management commands that you can use to load or
configure data in your GeoNode.
Both of them can be invoked by using the manage.py script.
First of all let’s take a look at the –help option of the importlayers management command in
order to inspect all the command options and features.

Run

DJANGO_SETTINGS_MODULE=geonode.settings python manage.py importlayers --help

This will produce output that looks like the following
usage: manage.py importlayers [-h] [--version] [-v {0,1,2,3}]
[--settings SETTINGS] [--pythonpath PYTHONPATH]
[--traceback] [--no-color] [-u USER] [-i] [-o]
[-k KEYWORDS] [-l LICENSE] [-c CATEGORY]
[-r REGIONS] [-n LAYERNAME] [-t TITLE]
[-a ABSTRACT] [-d DATE] [-p] [-m] [-C CHARSET]
[path [path ...]]
Brings a data file or a directory full of data files into a GeoNode site.
Layers are added to the Django database, the GeoServer configuration, and the
pycsw metadata index.
positional arguments:
path path [path...]
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v {0,1,2,3}, --verbosity {0,1,2,3}
Verbosity level; 0=minimal output, 1=normal output,
2=verbose output, 3=very verbose output

1
 Uploading data from Computer to server using command line 2020

--settings SETTINGS The Python path to a settings module, e.g.

"myproject.settings.main". If this isn't provided, the
DJANGO_SETTINGS_MODULE environment variable will be
Used.
--pythonpath PYTHONPATH
A directory to add to the Python path, e.g.
"/home/djangoprojects/myproject".
--traceback Raise on CommandError exceptions
--no-color Don't colorize the command output.
-u USER, --user USER Name of the user account which should own the imported
Layers
-i, --ignore-errors Stop after any errors are encountered.
-o, --overwrite Overwrite existing layers if discovered (defaults
False)
-k KEYWORDS, --keywords KEYWORDS
The default keywords, separated by comma, for the
imported layer(s). Will be the same for all imported
layers if multiple imports are done in one command
-l LICENSE, --license LICENSE
The license for the imported layer(s). Will be the
same for all imported layers if multiple imports are
done in one command
-c CATEGORY, --category CATEGORY
The category for the imported layer(s). Will be the
same for all imported layers if multiple imports are
done in one command
-r REGIONS, --regions REGIONS
The default regions, separated by comma, for the
imported layer(s). Will be the same for all imported
layers if multiple imports are done in one command
-n LAYERNAME, --name LAYERNAME
The name for the imported layer(s). Can not be used
with multiple imports
-t TITLE, --title TITLE
The title for the imported layer(s). Will be the same
for all imported layers if multiple imports are done
in one command
-a ABSTRACT, --abstract ABSTRACT
The abstract for the imported layer(s). Will be the
same forall imported layers if multiple imports are
donein one command
-d DATE, --date DATE The date and time for the imported layer(s). Will be
the same for all imported layers if multiple imports
are done in one command. Use quotes to specify both

2
 Uploading data from Computer to server using command line 2020

the date and time in the format 'YYYY-MM-DD HH:MM:SS'.

-p, --private Make layer viewable only to owner
-m, --metadata_uploaded_preserve
Force metadata XML to be preserved
-C CHARSET, --charset CHARSET

Specify the charset of the data

While the description of most of the options should be self explanatory, its worth reviewing
some of the key options a bit more in details.
● The -i option will force the command to stop when it first encounters an error.
Without this option specified, the process will skip over errors that have layers and
continue loading the other layers.
● The -o option specifies that layers with the same name as the base name will be
loaded and overwrite the existing layer.
● The -u option specifies which will be the user that owns the imported layers. The
same user will be the point of contact and the metadata author as well for that layer
● The -k option is used to add keywords for all of the layers imported.
● The -C option specifies the character encoding of the data.

The import layers management command is invoked by specifying options as described above
and specifying the path to a single layer file or to a directory that contains multiple files. For
purposes of this exercise, lets use the default set of testing layers that ship with geonode. You
can replace this path with the directory to your own shapefiles.
NB: Navigate to your project file where you can access the manage.py file and then run the
following command

DJANGO_SETTINGS_MODULE=geonode.settings python manage.py importlayers -v 3

/home/massoudhamad/Desktop/FinlandTrip2501-2304/trees/
This command will produce the following output to your terminal
Verifying that GeoNode is running …
Found 1 potential layers.
/home/massoudhamad/.virtualenvs/resilience-academy-geonode/local/lib/python2.7/site-
packages/owslib/iso.py:117: FutureWarning: the .identification and .serviceidentification
properties will merge into .identification being a list of properties. This is currently
implemented in .identificationinfo. Please see
https://github.com/geopython/OWSLib/issues/38 for more information FutureWarning)
/home/massoudhamad/.virtualenvs/resilience-academy-geonode/local/lib/python2.7/site-
packages/owslib/iso.py:495: FutureWarning: The .keywords and .keywords2 properties will
merge into the .keywords property in the future, with .keywords becoming a list of
MD_Keywords instances. This is currently implemented in .keywords2. Please see
https://github.com/geopython/OWSLib/issues/301 for more information
FutureWarning)
>>> Step 2. Make sure we are not trying to overwrite a existing resource named [trees] with the
wrong type

3
 Uploading data from Computer to server using command line 2020

>>> Step 3. Identifying if [trees] is vector or raster and gathering extra files
>>> Step 4. Starting upload of [trees] to GeoServer…
/opt/geonode/geonode/geoserver/helpers.py:1417: FutureWarning: The behavior of this
method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test
instead.
if store:
>>> Step 5. Generating the metadata for [trees] after successful import to GeoSever
>>> Step 6. Making sure [trees] has a valid projection
>>> Step 7. Creating style for [trees]
>>> Step 8. Creating Django record for [trees]
Found geoserver resource for this layer: trees
/home/massoudhamad/.virtualenvs/resilience-academy-geonode/local/lib/python2.7/site-
packages/geoserver/style.py:83: FutureWarning: The behavior of this
method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test
instead.
if not user_style:
/home/massoudhamad/.virtualenvs/resilience-academy-geonode/local/lib/python2.7/site-
packages/geoserver/style.py:87: FutureWarning: The behavior of this method will change in
future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
if user_style:
... Creating Default Resource Links for Layer [geonode:trees]
-- Resource Links[Prune old links]...
-- Resource Links[Prune old links]...done!
-- Resource Links[Compute parameters for the new links]...
-- Resource Links[Create Raw Data download link]...
-- Resource Links[Create Raw Data download link]...done!
-- Resource Links[Set download links for WMS, WCS or WFS and KML]...
-- Resource Links[Set download links for WMS, WCS or WFS and KML]...done!
-- Resource Links[Legend link]...
-- Resource Links[Legend link]...done!
-- Resource Links[Thumbnail link]...
-- Resource Links[Thumbnail link]...done!
-- Resource Links[OWS Links]...
-- Resource Links[OWS Links]...done!
[created] Layer for '/home/massoudhamad/Desktop/FinlandTrip2501-2304/trees/trees.shp'
(1/1)
Detailed report of failures:
Finished processing 1 layers in 3.0 seconds.
1 Created layers
0 Updated layers
0 Skipped layers
0 Failed layers
3.000000 seconds per layer

4
 Uploading data from Computer to server using command line 2020

If you encounter errors while running this command, you can use the -v option to increase the
verbosity of the output so you can debug the problem.
The verbosity level can be set from 0-3 with 0 being the default.

Management Command updatelayers

While it is possible to import layers directly from your servers filesystem into your GeoNode,
you may have an existing GeoServer that already has data in it, or you may want to configure
data from a GeoServer which is not directly supported by uploading data.
GeoServer supports a wide range of data formats and connections to database, and while many
of them are not supported as GeoNode upload formats, if they can be configured in GeoServer,
you can add them to your GeoNode by following the procedure described below.
GeoServer supports 3 types of data: Raster, Vector, Databases and Cascaded.
Run:
python manage.py updatelayers -w geonode -f planet_satellite_images_2018
---End---

Contacts:
resilienceacademytz@gmail.com ,
https://resilienceacademy.ac.tz
https://geonode.resilienceacademy.ac.tz