SECTION-1

LOGS / MONITORING

- What components/entities should we monitor?

Ans – During the ETL process monitoring, we must be aware of the Source Component, Destination
component and Loader Component.

 Source Component – we must ensure that we have received the proper import feed from
the source. If the import feed received is not proper or not in the time bound, then thre is a
chance of mismatch in the output file.

 Destination Component – Here, we must ensure that, data sink is proper and no error during
execution.

 Loader Component – During this phase, we must be proactively checking for the moving of
data from source DB to the target DB.

Once, we ensure that the File has been moved to the target place we can say that the process has
been completed.

- What will you report on each component/entity?

Ans - From receiving of the import feed to generating the output feed and sending this output feed
to the target DB, we must be proactively checking the different phases.

 Source Component – For the source component, we must be looking for the input feed from
the source machine. For example, if the source machine is supposed to share the file format
and layout, we must ensure that the received input is proper. Again, we must ensure that
the received inputs are placed in proper place with proper extension.

If we found any deviation in the input feed, we must report this to the source contact and
should inform about the deviation.

 Transformation components – During this phase, we must carefully monitor the transmission
process. i.e. – If the available input needs to be transmitted, we must ensure that It has been
transmitted successfully. We must check the logs to make sure that the process was successful.
 If the logs say, the transaction is not proper ( job failure ) , then we have to find out the RCA
on this and should report to the respective team.

- How will you structure the log payload? and why?

Ans - we can use automated processing to investigate log files, or we can use algorithms to
categorize, index and search through log files based on specific parameters (by date, user,
number, etc.)

Structured logging uses a defined format to add important details to logs and make it easier
to interact with them in different ways.

The default layout for many types of application logs is a plain text layout, which is easily
readable for humans but difficult to interact with for machines.

Structured logging takes plain text application logs and converts them into a set of data
points that can be more easily analysed by a machine .

- How will you report errors?

Ans- Errors or any kind of incident should be reported with proper RCA. If any error is found during
the execution, then we must start finding RCA for this.

Once, we find our RCA and scenarios, we must create a case or Jira id for this with steps to
reproduce and error logs.

Also, if required, we must provide the dump file and screenshot for the observed issue.

- What type of tools/services will you use? and how?

Ans- for monitoring purpose, we should use the monitoring tool like, control-m, App Dynamic,
Nagios. And for ticketing purpose, we can use JIRA, Bugzilla, Salesforce.

For monitoring Purpose, we should set the tool (configure the parameter) in a such a way that, we
must receive alert when the job deviates from its original behaviour. (like job failure, long running,
source sharing issue, etc). Once we received the alerts, we can start looking into that.

For ticketing purpose, we must raise the case with proper description and must aware about the
SLA.
SECTION-2

NETWORKING

- What happened?

Ans- A server connection timeout means that a server is taking too long to reply to a data request
made from another device.

Timeouts are not a reply message: they show up when there is not a reply and a server request is
not fulfilled in a mentioned length of time.

Time out may occur because of –

1) Server Doesn’t Exist

2) Network Infrastructure
3) Network Hardware Issue
4) Data Request Error

- How will you troubleshoot that issue?

Ans – To troubleshoot such timeout connection errors, we must ensure which category it belongs to.

After the findings, we can try to simulate the same again and must be checking for the proper
connections and securities.

- How can you prevent it from ever happening again?

Ans- We must ensure that the setup is proper and customer’s IP is being whitelisted for the
operations.

Also, we must inform custimer about the guideline and scope ( like firewall setting and home-
network setup ) for the server.

- What tools are available to you to help troubleshoot the issue?

Ans- Putty, Ping Utility, Subnet IP calculator

 - What could the customer have done to prevent this issue?

Ans - Customer must be aware of the protocols, guidelines and scope of the features.

If customer is facing the issue, he/she must be sure that, he has proper internet connection ( like
setup to the authenticate network, processing the proper data, Hardware connection are proper,
connecting via proper license).

SECTION-4

DATABASE/ARCHITECTURE

- Replicating the data

Before proceeding for the data replication, we must ensure that the imports are completed in the
source DB.

During the data replication, we should inform the customer that, replication has been initiated so we
can request them not use the app until the operation is completed. ( Non-business hour is most
suitable time )

Once the data got replicated to the target server, we should perform the cache refresh activities on
the target Application server to make sure of get rid from the old database.

We must ensure that the replicated db must have M-view so that DML operation is there.

- Data-integrity

It is the most important part from both point of view.

Data integrity provides a stability and assurance of the data.

During this operation, we must take a back up of the existing DB and should start the exporting all
the required data and compare with the received input feeds to make sure both are on sink.

If we found some data discrepancy, immediate action should be taken.

 - Database/API architecture (depending on the source)

Ans- An architectural framework must be created that enables the exclusive functionality of the API

To achieve the best, the components must be organized in below layers

- Security layer
- Cache storage layer
- Representation layer
- Organization layer

- Source & destination limits/restrictions

Ans – There is always a scope and limit for both source and destination end.

 The source can have DML operations whereas the destination has only read operation.

 Before output file generates on destination, the source database must completed all it’s
insert/update/delete operation.

- Performance & scalability

Ans- To increase the performance of the db and make the search operation fast, we should use
Indexing technique. Also we should try to process the straight query. The procedure and functions
should also contains proper table name and conditions.

For the database Scalability we should use the shared-disk technique.

- Security & Monitoring

Ans- Database security methods.

Physical database security

Web applications and firewalls
Database encryption
Manage passwords and permissions
Isolate sensitive databases
Change management
Database auditing
database monitoring includes: -

Query details (top CPU, slow running, and most frequent)

Session details (current user connections and locks)

Scheduled jobs

Replication details

Database performance (buffer, cache, connection, lock, and latch)

SECTION-5

TROUBLESHOOTING

a. What will be your initial questions (within the 1st email/response)

Ans- Hi X,

Could you please share the details steps followed and the actual error/ warning you are getting
along with the screenshot?

Thanks in Advance.

 Also, I would change the status of the tickets to work in progress and responsibility party as
customer.

b. What will be your follow-up questions within the 2nd email/response, based on the
customer’s answers? (assume that you’ve got all of your questions answered)

Ans- Hi X,

Thanks for the shared details. The process and steps look fine. The error message you received says
that, there is some network error during the transaction, and it caused the error for sync.

We are investigating more on this and will reach out to you soon.
 2. How will your questions change if only 1 connector failed to sync?

Ans - Hi X,

Hope you are doing good. After out investigation and analysis, we have found that connector “A”
was not running hence it failed to connect sync the data. So, we believe for all connector except ‘A’
were synced and updated.

Could you please retry the same activities again, as we can see that connector “A” is running now?

Please confirm once, post your activities.

SECTION-6

EMAIL COMMUNICATION

Hi Team,

Thanks for your response on the provided workaround.

As we can see, this requirement needs a model changes on both database and application level, we
have initiated the process. We believe, we will be completing this soon, and this feature would be
available on next release (schedule at end of this quarter).

Hence, requesting you to please proceed with the provided solutions until we share the next release.

We will provide the extended support for the workaround to make this happen in time.

-Regards,
XXXXXXXXXX