QE Data Engineer in 2020 . ence EYEE e o rrr) / \ / — e © © e | Basic Terminal usage How does the computer work? e How does the Internet work? G ‘Git — Version control Data structures & algorithms ~~ { S S REST Gitis used for tracking changes in source code and coordinating work among programmers. n your day 10 day work you willuse Git server asa service lke GitHub, GitLab or Bitbucket. , Learn how to write clean, extensibile code. Spend some time Understanding programming paradigms and best practices. Get familar with an IDE of code editor lke VSCode. Q Unit testing S N : @ ~ Integration testing ~~~ @ Functional testing ~~~- [mcs en) XN Understand ne Entity Relationship (ER) model and normalisation, Learn how to Q design databases and model data. Understand scaling patterns. “XN CAE tISorea ~~ @ ‘OLTP vs OLAP Horizontal vs vertical scaling Understand the difference between Document, Wide column, ‘Graph and Key-value NoSOL databases. We recommend S ‘mastering one database from each category. ‘Amazon Aurora Q © MongoDB ‘Apache Cassandra xz, “® “4 y [Neos ’ ‘Most modern data processing frameworks are based on Apache @ _Hiactoop and MapReduce to some extent, Understanding these pens Hadooe ‘concepts can help you learn modem frameworks much quicker. HOFS @ MapReduce Managed Hadoop ‘Amazon EMR S Google Dataproc Hybrid frameworks are able to process both batch and streaming ‘data, Batch data processing is often done by analytical data ‘warehouse applications, See Data warehouses for more, es ee eo (Apache Spark) © a © 3 RabbitQ Apache ActiveMQ } Apache Airflow S (Ctentering dat ipsines SET - y Aieeercemionsecounates = ~ y >~+ He Infrastructure orchestration ee <=, xc @ © "AWS ClovdFarmation GitHub Actions ‘Active Directory S ‘Azure Active Directory Q Legal compliance Encryption Key management Data governance & integrity