423923, 1201 PM ‘Welcome — Data Science for Energy System Modeling Welcome Contents * Python * Installing the package manager conda * Managing environments with conda ‘* Speeding things up with Mamba ‘* Python environment for this course: ‘esm-2023, ‘© What is JupyterLab? ‘© Markdown Syntax Welcome to the website accompanying the course Data Science for Energy System Modelling. This course is being developed by Dr. Fabian Neumann and offered as part of the curriculum of the Department of Digital Transformation of Energy Systems at TU Berlin On this website you will find practical introductions to many Python packages that are useful for dealing with energy data and building energy system models. Course materials other than practical introductions to Python packages for students at TU Berlin are provided on ISIS. The course covers tutorials and examples for getting started with Python, nunpy, matplotlib , pandas, geopandas , cartopy, rasterio, atlite, networkx, pyomo, pypsa, plotly, hyplot , and streanlit . Topics covered include * time series analysis (e.g. wind and solar production) ‘* tabular data (e.g. LNG terminals) ‘* geographical data (e.g, location of power plants) * data visualisation * converting weather data to renewable generation * land eligibility analysis (e.g. where to build wind turbines) * optimisation Skip to main content hntps:tneum github ioldata-sconce-for-esminiro html Mocid=IwAR 18Sc pIN4105OTG2MKYEFe2IF¢mH2t-obiDMAHMgOKHnPwSuzifAGcSlaspeedl... 17412973, 12.01 PM Wieleome— Data Seence for Energy System Modeling ‘* power flow modelling (linearised) capacity expansion planning ‘* sector-coupling * interactive visualisation and dashboarding Python @ Note This section is adapted from another course called Earth and Environmental Data Science: Python Environments Python and nearly all of the software packages in the scientific python ecosystem are open- source. Coordinating the compatibility between these different packages and their multiple versions used to be a nightmare! Fortunately, the problem is solved by using a Python distribution and/or package manager. You should use a package manager! Installing the package manager conda Anaconda Python distribution The easiest way to set up a full-stack scientific Python deployment is to use a Python distribution. This is an installation of Python with a set of curated packages which are guaranteed to work together. For instance, you can install on your computer the popular Anaconda Python Distribution. Follow the link above to obtain a one-click installers for your operating system. For Linux and MacOS users, you can access the command line by opening the terminal program. For Windows users, you should first install Anaconda (described above) or Miniconda (described below), which gives you access to the “Anaconda Prompt” desktop application Skip to main content hitps:/ineum github oldata-science-fr-esmvinra htmbcls=IWAR 18Sc1piN4 1OSOTG2MKYEFe2tFamH2+-obDhAHMGOKHnPweuxtQeslispesdi.. 217423923, 1201 PM ‘Welcome — Data Science for Energy System Modeling From the Anaconda Prompt, you should be able to run _conda and other shell commands. Lightweight alternatives: micromamba and miniconda If you don’t want to download a large file like the Anaconda Python Distribution (ca. 800 MB), there are lightweight alternative installation methods (like micromamba and miniconda 1, Micromamba Installation 2. Miniconda Installation Managing environments with conda Python coupled with a package manager provides a way to make isolated, reproducible environments where you have fine-tuned control over all packages and configuration. To create a conda environment, you execute the following command: conda create --name my_environment python=3.11 numpy To use this environment, simply “activate” it by executing; conda activate my_environment You should now see the string (my_environnent) prepended to your prompt. Now, if you execute any Python-related tool from the command line, it will frst search in your environment. To install additional packages into your environment: conda install Some packages are community-maintained (e.g. conda-forge ) and require you to specify a different “channel”: conda install -c conda-forge Skip to main content hitps:/ineum github oldata-science-fr-esmvintra htmbcls=IwAR18Sc1piN4 1OSOTG2MKYEFe2tFamH2+-obJDhAHMGOKHnPweuZtAGeslispesdi.. 317423923, 1201 PM ‘Welcome — Data Science for Energy System Modeling You can deactivate your environment by typing: conda deactivate To see all the environments on your system: conda info --envs To get a complete summary of all the packages installed in your environment, run conda list If you want to permanently remove an environment and delete all the data associated with it: conda env remove --nane my environment --all A conda environment can also be defined through an environment.yan1 file. With that file, a new environment with the exact configuration can be installed by executing conda env create -f my_environment.yml Below we will see an example of an environment file. For extensive documentation on using environments, please see the conda documentation, Speeding things up with Mamba In order to put together an actual Python environment from your package specifications, conda has to solve a difficult puzzle, to ensure that the combination of packages is mutually compatible. Each package specified has certain dependencies on other packages. Moreover, each version of one package requires certain minimum versions of other packages. Other packages in your environment may have different or incompatible versions. The default implementation of conda can be very slow. Fortunately, there is a much faster alternative called mamba. To install it, just run: Skip to main content hnps:tneum github ioldata-sconce-for-esmviniro html Mecid=IwAR 18Sc pIN4 1OSOTG2MKYEFe2IFamH2t-obiDnAHMgOKHnPwSuzifAGcSlaspeedl... 47423923, 1201 PM Welcome — Data Science for Energy System Modeling conda install -n base -c conda-forge mamba Now you can install environments and packages as before, but using the manba command instead of conda . Everything will be faster. Python environment for this course: esm-2023 The latest environment specification for this course lives at https://raw.githubusercontent.com/fneum/data-science-for-esm/main/environmentyaml Copy and paste the following environment.yml file somewhere on your local hard drive: name: esm-2023 channels: ~ conda-forge dependencies: ~ python>=3.11 - pip # main packages ~ nunpy - scipy ~ pandas>=1.4 ~ geopandas>=0.11.0 - xarray ~ networkx ~ yam. - pyomo - netedfa ~ pypsa>=0.22.1 ~ atlite>=0.2.10 = powerplantmatching>: + pasterio!=1.2.10 # interactive python ~ ipython - jupyterlab # reading excel tables - tabula-py ~ xird ~ Lem - pytables - pyxlsb = openpyx1 Skip to main content hnps:tneum github ioldata-scence-foresminira html Mecid=IwAR 18Sc p1N4 1OSOTG2MKYEFe2IF¢mH2+-obiDnAHMgOKHnPwéuzifAGcSlaspeedl... SIT423923, 12001 PM = fiona = shapely>2 + proj = geopy ~ pyense = cartopy * descartes = country_converter # plotting - matplotlib>=3.5 - plotly - hvplot = holoviews = geoviews = graphviz # solvers ~ coincbe ~ glpk # publishing - jupyter-book - ghp-import Create this environment using mamba mamba env create -f path/to/environment.yml Activate this environment conda activate esm-2023 Welcome — Data Science for Energy System Modeling This environment should be sufficient for all of your work in this course, What is JupyterLab? JupyterLab will be our primary method for interacting with the computer. JupyterLab contains a complete environment for interactive data science which runs in your web browser. JupyterLab has excellent documentation. Rather than repeat that documentation here, we point you to their docs. The following pages are particularly relevant: |nps:itneum github ioldata-sconce-foresmviniro html Mecié=IwAR 18Sc fp1N4 1O50TG2MKYEFe21F¢mH2-obiDhAHMgOKHnPuSuzifAGcSixspoed Skip to main content or412928, 1201 PM Wieleome— Data Seence for Energy System Modeling * Working with Files The Text Editor ‘+ Notebooks © Terminals ‘* Managing Kernels and Terminals Markdown Syntax Throughout the course, you might want to write rich text documents using Markdown. This is also very common in Jupyter Notebooks. Here are some useful references on Markdown syntax * Markdown Guide / Basic Syntax ‘* Official Markdown Documentation hnps:tneum github ioldata-sconce-foresmviniro html Mocid=IWwAR 18Sc pIN4 1OSOTG2MKYEFe21F¢mH24-obiDnAHMgOKHnPwBuzifAGcSlaspeedi... 77