Divija

hadoopdivija@gmail.com,
Big Data Software Engineer | Software Engineer

EXPERIENCE SUMMARY

• Over 5 years of professional IT experience in designing and developing software applications Including 3 years Big Data
technologies and 2+ Java/J2EE with strong understanding of software development life cycle
• Excellent understanding of Hadoop architecture and underlying components/frameworks including distributed storage (HDFS),
resource management (YARN) and MapReduce programming paradigm
• In-depth understanding of Spark architecture including Spark Core, Spark SQL, DataFrames/Dataset API’s, Spark Streaming,
Spark MLib
• Programmed in Scala, Java and scripting languages like Python
• Extending Hive and Pig core functionality by using custom User Defined Function.
• Knowledge of AWS cloud services, data modeling on NoSQL databases like MongoDB, HBase and Cassandra
• Specialization in server side Java & J2EE technologies.
• Experience in designing the User Interface using HTML, CSS, JavaScript
• Experience in Web/Ecommerce development using Java Frameworks

TECHNICAL SKILLS

Big Data Ecosystem Cloudera, Hortonworks, Hadoop, HDFS, MapReduce, Pig, Hive, Sqoop, Flume,
ZooKeeper, Oozie, HBase, Spark, Spark Streaming, Strom, Solr, Kafka, Ambari
Languages Scala, Java, Python, C#, SQL, PL/SQL, ASP.Net, VB.Net C, C++
Databases/ NoSQL Databases Oracle, MS SQL, MySQL, PostgreSQL, TeraData, MongoDB, Cassandra, HBase,
DynamoDB
Operating Systems Windows, Unix/Linux, Mac
Frameworks Hibernate, Spring MVC, Spring IOC
UI Technologies HTML5, CSS, Java Script, Angular, Bootstrap
SCM/Integration/Build Tools GIT, GitHub, SVN, Bit Bucket, Jenkins, Docker, SBT, Maven, Gradle
Java/J2EE H JSP, Servlets, JDBC, EJB
Design Methodologies Agile, Waterfall
IDE’s Eclipse, My Eclipse, IntelliJ
Loggers Framework Log4J
AWS Services EMR, EC2, RDS, S3, IAM, EBS, Security Groups, Auto Scaling, SQS, ELB
Design Patterns Singleton, Factory, DAO
Webservices SOAP and Restful services
Utilities ElasticSearch, Logstash, Kibana

EDUCATION

• Master of Computer Science – Software Engineering Arizona State University (ASU) Dec’ 2015
• Bachelor of Technology – Electronics and Communications, GITAM University May’ 2013

PROFESSIONAL EXPERIENCE

AT&T, GA March 2016 - present

Role: Big Data Software Engineer

Responsibilities:

• Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig,
HBase, Zookeeper and Sqoop
• Designed and Implemented data ingestion, aggregation, integration and analytics using Scala/Python Spark, MapReduce, Pig,
Hive
• Streamlined data from RDBMS (Oracle, MySql, MS SQL and Teradata) to HDFS using Scala, Akka based data ingestor
• Created Hive Managed and External tables defined using both static and dynamic partitions, compressions for data backups
• Optimization techniques like joins, partitioning, bucketing, Hives SerDe’s for data imports from XML and JSON files
• Worked with various file formats and compressions including RCFile, Parquet, ORCFile, Avro, GZip and Snappy
• Worked on optimization and performance tuning of MapReduce, Hive & Pig Jobs.
• Responsible for developing data pipelines using Sqoop, MapReduce and Hive to extract data from machine logs and store
results for downstream consumption using Scala/Python scripts
• Developed data pipelines using Flume, Pig and MapReduce to ingest asset/contract expenditure related data for analysis and
reporting
• Used Spark RDD, map-side joins for parsing EXCEL, CSV, XML and JSON data
• Developer Pig and Hive UDF’s to analyze/transform data in HDFS
• Programmed Inventory analysis tool using MapReduce jobs and Hive scripts to parse raw data into ElasticSearch indexes
• Used Logstash for ingesting data from flat files, MySQL, ORACLE and Postgresql to Elasticsearch
• Worked on Kibana to generate customer facing dashboards, coded Groovy scripts for complicated aggregations
• Orchestrated and Implemented Data Validation library using Scala, Akka Streams and E;asticSearch for validating large
datasets from flat files, HDFS
• Experience data processing like collecting, aggregating, moving from various sources using Apache Flume and Kafka
• Responsible for Continuous Integration (CI) and Continuous Delivery (CD) process implementation using Jenkins along with
Shell scripts to automate routine jobs
• Integrated Build Process through Jenkins to various SCM tools like Git for automated deployements
• Experience with container based deployments using Docker, working with Docker images, Docker HUB and Docker
registries.

Environment: Hadoop, HDFS, MapReduce, Spark, Spark SQL, Hive, Pig, Sqoop, Flume, Oozie, HBase, Java, Scala, Python, Perl,
SQL, Oracle, TeraData, MySQL, Elasticsearch, Logstash, Kibana, XML, JSON, Javascript, AngularJS, GIT, Jenkins, Docker, SBT

NTT DATA, INDIA Nov’ 2013 – Jul’ 2014

Role: Software Engineer

Responsibilities:
• Developed Stored procedures in SQL Server to perform data validation on scanned files before updating the database tables
• Implemented role based access control for the syndicates selection listed under Chaucer
• Monitored Monthly data merge between offshore business to Charisma database
• Performed upgrades to various policy wizards as per client change requests
• Migrated underwriting wizard related scripts from VB to C#
• Developed WebForms using C# and ASP.NET for new modules
• Developed Stored Procedures and Triggers for maintaining and interacting with the database
• Developed and consumed Web Services
• Have worked on VB6 for reporting modules

Environment: C#, VB.NET, ASP.NET, Web Services, WCF, LINQ, IIS, SOAP, XML, Ajax, XSL, XSLT, Web Services, JavaScript,
HTML/ HTML5, CSS3, VB Script, VISUAL STUDIO 2010, WINDOWS ⅞, SQL SERVER 2008

ValueLabs LLP Aug’ 2012 – Oct’ 2013

Role: Software Engineer

Responsibilities:
• Involved in all phases of the SDLC using Agile Methodology, designed use case diagrams, class diagrams, and sequence diagrams
as a part of Design Phase
• Developed application service components and configured beans using Spring IOC framework
• Implemented RESTful network communication protocol to consume a web service, getting JSON response and parsing them to get
the required information
• Involved in server-side validation based on business rules by using hibernate validator
• Extensively used JavaScript to provide dynamic User Interface and for the client side validations
• Involved in developing server pages using Servlets and JSPs
• Backend application layer is implemented using Spring and deployed the entire project onto the Tomcat Server
• Extensively used SQL concepts to Insert, Retrieve, Update the information in the database
• Used MySQL Database for storing and retrieving the data
• Used GIT as the configuration management tool to manage the code repository

Environment: Java 7, J2EE, HTML, Spring IOC, RESTful, Log4J, GIT, Eclipse, Tomcat Server, MySQL