Professional Documents
Culture Documents
Cisco Webex integration with Cloud Access Security Broker (CASB) / Data Leakage Protection (DLP)
Cloud access security broker (CASB) is an on-premises or a cloud-based software that sits between cloud service users and cloud
applications. It monitors all activity and enforces security policies. A CASB can offer a variety of services, including, but not limited
to monitoring user activity, warning administrators about potentially hazardous actions, enforcing security policy compliance, and
automatically preventing malware.
Cisco Cloudlock is the cloud-native CASB and Cloud Cybersecurity Platform that helps accelerate use of the cloud, including the
apps you buy and build. Cisco Cloudlock secures your cloud users, data, and apps across Software-as-a-Service (SAAS), Platform-
as-a-Service (PAAS), Infrastructure-as-a-Service (IAAS), and orchestrates security across your existing security investments.
Data loss prevention software detects potential data breaches/data ex-filtration transmissions and prevents them by monitoring,
detecting and blocking sensitive data while in-use (endpoint actions), in-motion (network traffic), and at-rest (data storage).
© 2019 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 1 of 89
Cisco dCloud
eDiscovery
eDiscovery search and extraction tool—while standard Cisco Webex Teams customers have access to only 90 days of content,
Pro Pack customers can access unlimited data within Webex Teams spaces. Use email addresses, space IDs, keywords, and
specific time limits to narrow the search. For more information, see Ensure Regulatory Compliance of Cisco Webex Teams
Content.
End-to-End Encryption
End-to-End encryption is based on the model of a centralized Key Management Server (KMS). Components of the Cisco Webex
solution (i.e. clients) request key material through a secure channel from a centralized KMS.
The figure bellow shows the separation of key management infrastructure from the component concerned with data storage. The
client participating in a Webex communication retrieves keys from the KMS, encrypts data (messages or files), and sends the
encrypted information to be stored in the content server. Similarly, for data retrieval a client needs authorization to retrieve the
required keys from the KMS. Upon receiving keys from the KMS, the client can request the encrypted data from the content store
and decrypt it.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 2 of 89
Cisco dCloud
The architecture of Cisco Webex allows customers to either use the Cisco provided KMS in the cloud (default) or to deploy their own
instance of the KMS in a customer owned datacenter (Cisco Hybrid Data Security – HDS, enhanced feature available as part of the
Cisco Webex Pro Pack). By deploying a separate instance of KMS in the customer environment, the customer’s encryption keys for
their Webex organization are now located and owned by the customer. This provides an additional level of security and control to
the customer.
While increasing security with Cisco Hybrid Data Security, seamless business to business communication in Cisco Webex is
maintained through the concept of KMS Federation. Example shows two enterprise organizations with locally deployed KMS
instances being securely federated.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 3 of 89
Cisco dCloud
• Requirements
• Topology
• Session Users
• Get Started
• Scenario 1: Data Leakage Protection (DLP) Cisco Cloudlock Integration with Cisco Webex Teams
• Scenario 3: Install Cisco Webex HDS Configuration Utility & Enable HDS
• Appendix
Requirements
Table 1 outlines the requirements for this preconfigured demonstration.
Table 1. Requirements
Required Optional
• Database – PostgreSQL
This lab will explain in detail how to deploy and configure all components required for Cisco HDS.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 4 of 89
Cisco dCloud
Topology
This content includes preconfigured users and components to illustrate the scripted scenarios and features of the solution. Most
components are fully configurable with predefined administrative user accounts. You can see the IP address and user account
credentials to use to access a component by clicking the component icon in the Topology menu of your active session and in the
scenario steps that require their use.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 5 of 89
Cisco dCloud
Session Users
A Cisco Webex organization is created with a DNS domain at the start of the exercise. Please refer to the Cisco dCloud portal for
details. To login to Cisco Webex use the following email addresses and passwords.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 6 of 89
Cisco dCloud
Get Started
BEFORE PRESENTING
Cisco dCloud strongly recommends that you perform the tasks in this document with an active session before presenting in front
of a live audience. This will allow you to become familiar with the structure of the document and content.
It may be necessary to schedule a new session after following this guide in order to reset the environment to its original
configuration.
Follow the steps to schedule a session of the content and configure your presentation environment.
For best performance, connect to the workstation with Cisco AnyConnect VPN [Show Me How] and the local RDP client on
your laptop [Show Me How]
NOTE: You can also connect to the workstation using the Cisco dCloud Remote Desktop client [Show Me How]. The dCloud Remote
Desktop client works best for accessing an active session with minimal interaction. However, many users experience connection and
performance issues with this method.
NOTE: In case you are running through this exercise at your own time, there are two PostgreSQL servers provided as part of the
topology. While postgre2.dcloud.cisco.com has been completely preconfigured for you, postgre1.dcloud.cisco.com is a blank Linux
server that allows you to go through all install and configuration steps yourself.
The Cisco HDS components do provide extensive logging information that enables customers to monitor and maintain the different
services and their communication with the Cisco Control Hub. It is required to provide a Syslog destination as part of the Cisco HDS
configuration. A very basic solution can be a Syslog daemon running on a Linux or similar server. While this will allow the HDS
components to send messages which are stored by syslog in a flat file it doesn’t provide any search or alerting capabilities for specific
events or messages. For a production deployment it is recommended to evaluate a more feature rich redundant logging infrastructure
with alerting and dashboard capabilities. In the optional section of this lab guide an example deployment is provided based on Elastic
Search. Basic syslog server is configured for you on the syslog.dcloud.cisco.com Linux machine. Please refer to the appendix of this
document for the steps required to configure a basic syslog server on Linux.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 7 of 89
Cisco dCloud
A Cisco HDS clusters requires a single x.509 digital certificate for authentication against the Cisco Webex Control Hub Service. The
certificate needs to be trusted by the Certificate list and needs to have a friendly name of “kms-private-key”. No additional subject
alternate names are required. For the purpose on of this exercise free 90-day certificates are utilized. The process to issue Let’s
Encrypt certificates includes creating a certificate signing request (CSR) and use Let’s Encrypt tools to send the CSR to the Let’s
Encrypt service and get the certificate issued are documented in the appendix of this document.
NOTE: Because of the 90 days limited lifetime of the let’s encrypt certificate it is NOT recommended to use this for a production
deployment of Cisco HDS.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 8 of 89
Cisco dCloud
Scenario 1. Data Loss Prevention (DLP) Cisco Cloudlock integration with Cisco
Webex Teams
NOTE: The Cloudlock and eDisovery scenarios of the lab guide are completely independent of the Hybrid Data Security scenarios.
If you would like to complete the HDS scenarios only, you can skip directly to the scenario and start the lab.
Cisco Cloudlock currently does not provide an API or any easy process to clean up an existing configuration – a functionality not
required for regular operations. For this exercise, a manual cleanup is required.
Open a web browser using incognito/private mode and navigate to the Cloudlock instance
(https://demo.cloudlockng.com/gate/login), from the dropdown list select Webex Teams and click GO. (For most of the
Cloudlock lab you can use your own web browser, however, you can also connect to workstation 1 (198.18.1.36) with
username / password dcloud\cholland / C1sco12345)
NOTE: To prevent issues due to browser caching with logins, it’s always best to use an incognito/private browser.
Navigate to Settings and on the Platform tab, verify if there is an existing integration.
If the Status reads Not Authorized or Needs authorization, you can skip to Step 10. If it reads Authorized then continue on
to the next step.
Click Edit.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 9 of 89
Cisco dCloud
On the Configure Platform dialog, check Delete old incidents and click Revoke Authorization
In the Revoke confirmation dialog select OK. After clicking OK the Status changes to Needs authorization.
For every policy listed (except the default Blacklisted IPs policy), use the dropdown list in the Status column and choose Delete
Policy and click Delete Policy.
For the default Blacklisted IPs policy, verify that the Status is listed as Inactive. If it is not, change it to Inactive now. When
finished your policy list show look like the Figure 3.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 10 of 89
Cisco dCloud
Full administrators can assign the compliance officer role to any person within their organization. Full administrators can't assign the
compliance officer role to themselves, another full administrator must assign the role to them.
Using the same browser, open a new tab and navigate to the Webex Control hub (http://admin.webex.com). Enter the email
cholland@ssXXX.dc-YY.com and click Sign In. If you are not using the same browser for Cloudlock then enter dCloud123!
as the password.
To integrate Cisco Cloudlock with Webex Teams a user with compliance officer privileges is required. In this exercise you will promote
Anita Perez to compliance officer for the organization.
By selecting this role, we are giving this user permissions to do DLP, eDiscovery, and archival function. We will use the event API
provided by Cisco Webex Teams to give these permissions.
In the next step of this exercise Cloudlock is used to request these permissions.
NOTE: For this lab exercise, please select the Full Administrator privileges for Anita. This role is currently required for the
Cloudlock integration to work.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 11 of 89
Cisco dCloud
After you see the message User Successfully Updated, click User.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 12 of 89
Cisco dCloud
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 13 of 89
Cisco dCloud
Now you will authorize Cloudlock to Webex Teams. The authorization has to be completed by a compliance officer to obtain the
correct permissions to delete messages within Webex Teams. Because of this you will have to log out of Cloudlock with Charles
(who is not a compliance officer), close the browser (to clear the cache), open the browser again (using incognito/private mode),
and log back into Cloudlock using Anita’s account. Anita’s account has already been added to Cloudlock as an admin so she can
browse to the platforms page.
Reopen the browser in incognito/private mode and log into Cloudlock (https://demo.cloudlockng.com/) using Webex Teams as
Anita aperez@ssXXX.dc-YY.com / dCloud123!
Click Accept.
Click Settings.
Next to Webex Teams and under the Actions column click Authorize.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 14 of 89
Cisco dCloud
On the next page a prompt for authorization appears to confirm Cloudlock to have access to specific functions in Cisco
Webex, click Accept.
The configuration dialog allows you to apply policies to different scopes, leave the scope Monitor and files of all users. Other
options will be explored in a later step of the exercise. Click Close.
NOTE: The Charles and Anita user accounts have been pre-configured with limited roles in the Cloudlock instance for this lab. In
the figure bellow you can see the Superadmin view. As you will notice, Manage Users and Manage Roles are not available for
Charles or Anita. This restriction was applied to keep the integrity of the base lab. These roles will not be needed for this lab.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 15 of 89
Cisco dCloud
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 16 of 89
Cisco dCloud
Cloudlock provides a significant number of predefined policies, such as: Social Security Numbers, Country specific ID numbers,
SWIFT number, etc.
To create a policy to monitor and issue actions related to elements in Cisco Webex Teams, navigate to Policies, select Add a
Policy, and choose Add Predefined Policy.
Choose Credit Card Number from the Predefined Policies option (use the search function to find it), select Critical as the
policy Security Level and click Configure Policy.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 17 of 89
Cisco dCloud
NOTE: By default, Cloudlock is configured to catch the most exact matches, which will result in fewer incidents. In this
configuration Cloudlock may not catch all instances that your company requires. In this case, you will change the search tolerance
level to lenient which will result in more matches/incidents. The lenient tolerance level works for this lab, but you will need to
determine which tolerance level works best for your company.
Figure 3. Tolerance
Feel free to review the Content, Context, Summary tabs by clicking on each tab. As you saw earlier, the Content and
Context tabs have individual configurations pages such as Threshold, Tolerance, Exposure, etc. When you are finished
looking, click Save All Changes at the bottom right of the screen.
Next, you will create response actions to the policy defined in the previous step.
On the Policies page, click Create from the Response Actions column for the credit card policy you just created.
There are two types of specific response actions possible: Global and Platform.
Recommended action(s) are to notify the user and/or compliance officer either via the Webex Teams platform or by email.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 18 of 89
Cisco dCloud
Most times, for Webex Teams, a recommended action would be to delete the message or file that violates a policy as well as notify
the user and compliance officer(s).
• Notify the compliance officer (Anita Perez) that the user violated the credit card policy.
• Notify the user that a credit card number most not be shared via Webex Teams.
Drag all the Platform Specific actions to the dotted box to the right.
For the action of Notify the Admin via Message you need to specify Anita’s Webex Teams email address
(aperez@ssXXX.dc-YY.com) as the owner.
NOTE: In the search box there may be many of the same email addresses for the users. This is due to the emails being used over
and over again. Just pick any of the emails for aperez.
For the message to be sent to the admins enter: {{user}} has violated the Webex Teams security policy {{policy}}.
For the message sent to the user enter: You have violated the company’s Webex Teams security policy {{policy}}. The
offending message has been deleted. The incident has been logged and reported to the compliance officer.
After you are finished configuring the response actions, click the last Save button. (You might need to scroll down the page to
see it.)
NOTE: Each section has its own Save button and those can be clicked as well.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 19 of 89
Cisco dCloud
Navigate back to the main policies page by clicking the Policies link at the top or the main tab on the left. Verify the response
actions have been correctly created.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 20 of 89
Cisco dCloud
In this exercise you will create a new custom policy, based on a regular expression. This can be utilized for non-normalized
searches.
On the Policies page, click Add a Policy and choose Build your own.
Choose the option Custom Regex for the Policy Type, Critical for the Severity Level and enter a name, such as
Confidential content, for the Policy Name. Then click Configure Policy.
Next, you will need to create a regex that will match the word confidential with a lowercase or uppercase C, to support both
cases, use [cC]onfidential in the Flag content that matches this Regular Expression box. Keep the rest of the
configuration options at default. Similar to the Credit Card policy, this custom policy is applied to all Cisco Webex users and
spaces, for now.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 21 of 89
Cisco dCloud
Create the same Response Actions as you did with the Credit Card Policy.
Drag all the Platform Specific actions to the dotted box to the right.
For the action of Notify the Admin via Message you need to specify Anita’s Webex Teams email address
(aperez@ssXXX.dc-YY.com) as the owner.
For the message to be sent to the admins enter: {{user}} has violated the Webex Teams security policy {{policy}}.
For the message sent to the user enter: You have violated the company’s Webex Teams security policy {{policy}}. The
offending message has been deleted. The incident has been logged and reported to the compliance officer.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 22 of 89
Cisco dCloud
After you are finished configuring the response actions, click the last Save button.
Figure 4. Policies
To test the policies you created you will need to create some messages in Cisco Webex Teams.
If not already, connect to Workstation 1 (198.18.1.36) with username / password dcloud\cholland / C1sco12345, start the
Cisco Webex Teams client and login with cholland@ssXXX.dc-YY.com / dCloud123!.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 23 of 89
Cisco dCloud
On workstation 1 open the Webex Teams clients and create a 1:1 space ( > Contact a Person) with Kellie Melby. Next,
send a message with the word confidential in the text.
Create a Space with Kellie and Monica Cheng called Stock Price, send a message that states: with this confidential
information you will know when to buy or sell.
Type in one of the spaces or a new one a credit card number like the following message: This is my Credit Card number
5307700612341234.
Send a few more messages in the 1-to-1 conversation and in the Stock Price space. Create some more spaces and send
more messages if you would like.
Observe the Cisco Webex Teams client to see Cloudlock DLP policies actions
After a few minutes you will start seeing action being taken on all the messages that match the policies you created earlier.
NOTE: If you enabled HDS earlier the lab it can take Cloudlock takes at least 5 minutes to start working. Once it does start working
the messages should be removed much faster (less than a minute).
1. Go to one of the spaces you sent confidential or a credit card number. Notice that the message you had sent was deleted by
Anita Perez who is the compliance officer.
2. Notice there is now a new space created between Charles and the Security Center bot.
3. Select that space and you will see for every violation a separate message will appear. This is the message you configured to
send to the user when a policy occurred. You will have a separate message for every violation.
4. Connect to Workstation 2 (198.18.1.37) and login with username / password dcloud\aperez / C1sco12345.
5. Open the Webex Teams client and login with aperez@ssXXX.dc-YY.com / dCloud123!
6. You see that as the compliance officer, Anita was notified by the Security Center bot that Charles Holland violated security
policies.
Next, you will test the Confidential policy with a file. Cloudlock can inspect content and/or filenames of different files. By default, it
inspects both content and/or filenames based on the two policies that you created.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 24 of 89
Cisco dCloud
7. On the Workstation 1 desktop there is a file called presentation.pptx and it contain a slide that have the confidential word.
Upload it to your Stock Price space.
Once the file has been uploaded, after a few minutes it is removed by the compliance officer (Anita Perez).
The file was removed because the policy applied to anyone that uploads a file labeled confidential. In the next step, you will
change the confidential policy only to apply to spaces with external participants.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 25 of 89
Cisco dCloud
8. Return to the web browser and login to Cloudlock (if not already) as Charles or Anita and edit the policy for the word
Confidential. In the Detection Criteria Column, click Edit.
9. Navigate to Context > Exposure and click Webex Teams to expand it.
Cloudlock provides multiple options to select for which objects in Webex Teams a policy applies or which objects are specifically
excluded from a policy.
10. In this example select Shared with any external user and then click Save All Changes.
11. On Workstation 1 create a new space. Invite Kellie and obiwan@identitylab12.ciscolabs.com. This space allows business
to business collaboration with external participants.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 26 of 89
Cisco dCloud
Users are notified if a space contains external participants by the yellow icon [ ] in the bottom right corner shown in the above
figure.
12. Since the policy has changed and now only applies to spaces with external participants, send a message from Charles to Kellie
in their 1:1 space with the word confidential in the message. The message will no longer be deleted or raise a policy violation.
13. Send a similar message into the space with external participants.
Because it is an external user the policy is applied to this space. The content is deleted, and as before, the user (Charles) and
the compliance office (Anita) get a notification.
Go back to Cloudlock admin interface and check the reports of the incidents that occurred during our tests.
1. Click on Dashboard.
Since there is only a single integration in Cloudlock, only the Cisco Webex Teams application is displayed.
2. Click Webex Teams within the Platform Statistics box to view incidents within Webex Teams.
3. Click cholland@ssXXX.dc-YY.com in the Users with most incidents box to view incidents for a user.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 27 of 89
Cisco dCloud
5. You can see all the information that was logged such as which Webex Teams space was affected, who sent it, what was
matched, etc. Figure 27 shows confidential material policy violation and details about that event are displayed. This refers to the
upload of the PowerPoint file and shows that the word confidential was found on slide 2 which triggered the incident. Click the
tabs at the top (Access Control, Incident History, Incident Notes) of the incident to view more information.
Figure 1. Incident
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 28 of 89
Cisco dCloud
6. Go back to the dashboard and click on Webex Teams. You see all the incidents for the Webex application.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 29 of 89
Cisco dCloud
NOTE: This section assumes you completed the previous Cloudlock section that setup the Cloudlock integration and also
configured Anita as a compliance officer. In addition, the last section also setup test messages that were sent in order to violate
policy configuration.
If, a company requires details for legal proceedings, the compliance officer can access the Cisco Webex Teams eDiscovery Search
and Extraction tool from Cisco Webex Control Hub. These tools can generate reports that contain all of the conversations held
in Cisco Webex spaces and any files shared within those spaces.
Administrators can limit the number of messages and files that are kept by configuring a data retention policy. When the data retention
threshold period is met, aging content is purged.
A compliance officer can search Cisco Webex Teams for space, IDs, keywords, and any email address of a user in an organization.
Reports are generated from the e-Discovery console and can be downloaded in JSON format files. JSON files can be viewed in a
human readable format in the Firebox browser or other tools.
Click here for a reference script to convert the json file to concordance format (common format in legal discovery).
1. In the browser, Anita is logged into Cloudlock as an active user. Open a new tab and navigate to the Webex Control Hub
(https://admin.webex.com). If you already have a browser open before, and Anita isn’t the active user, please close the
browser and relaunch to clear the cache.
3. Click Accept.
4. From the Control Hub page navigate to Troubleshooting > Status and click View eDiscovery. The eDiscovery option is only
available if the user has the compliance officer role assigned to them.
6. For this exercise search for content created by cholland@ssXXX.dc-YY.com. Leave the time range as it is and enter the
word confidential. Click Search.
NOTE: Messages were sent with the word confidential in the previous section.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 30 of 89
Cisco dCloud
You will see a message stating that Cisco Webex Teams is conducting a search based on your criteria.
After few minutes, a report is provided on those search parameters with a summary of the number of spaces, messages, and files
that match. An estimated size of the total report is specified.
7. Give the report a meaningful name and description and then click Generate Report.
The console displays a prompt that the report is being generated, the browser will ask for permission to send notification. Please
acknowledge, you will see a popup notification once the report has been created.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 31 of 89
Cisco dCloud
8. If the page does not refresh after seeing the completion message, then click the Search tab and click back to Reports.
9. Download [ ] the zip file with all the information. The report, in zip file format, is available on the desktop.
10. Right-click on the file and use 7-zip to extract it to a new directory.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 32 of 89
Cisco dCloud
11. Open the folder it will show a directory per space where the search word occurred.
NOTE: Inside each directory is a JSON file and a folder. The later contains all the files that exist in each space.
Each JSON file describes all the messages and all the participants of each room where the search word appears.
The JSON file can be opened by double clicking with notepad++ (already install in the desktops of the workstation). Notepad++
also has a plugin for JSON. Alternatively, there are plenty of online translators, for example, https://jsoneditoronline.org/. Copy-
paste from Notepad++ to the right side of the web page, examine the translated readable output.
Notice that there are nine objects (more or less, depending on the number of messages that matched the search criteria in each
space).
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 33 of 89
Cisco dCloud
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 34 of 89
Cisco dCloud
In the Array, number 0 is always the last message in that space. The content of the message is in the actor array in the object
array under the displayName field.
More information about the format and different fields in the JSON file can be found at:
https://collaborationhelp.cisco.com/article/en-us/nr70c1m
NOTE: Additional 3rd party eDiscovery platforms, like Actiance, allow customers to conduct advance searches and enhanced
visualization as well as reports of the incidents.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 35 of 89
Cisco dCloud
For this deployment, we will use the syslog server to deploy Docker. This deployment will allow us to create the configuration ISO
for the HDS environment.
NOTE: The Docker image for creation and maintenance of the Cisco HDS ISO configuration image is intended to be installed on an
administration workstation. Because of some implications of running all components of this lab virtual, for this exercise the Docker
environment is run on a Linux host. While this works, there are some security and usage implications not recommended for production
deployment.
NOTE: All YUM and Docker commands require to be run as root. If not signed in as root all commands need to be prefixed with
sudo to switch to root privileges.
Configuration for the Cisco HDS nodes is deployed through a virtual ISO file mounted on each HDS node in the VMware ESXi
environment. Cisco provides a configuration utility to create and update the ISO configuration image as a Docker container. The next
steps show how to pull and deploy the Docker image to create the Cisco HDS ISO configuration.
The required parameters specified can be found in the Cisco HDS deployment guide:
http://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cloudCollaboration/spark/hybridservices/datasecurity/cmgt_b_hybrid-data-
security/cmgt_b_hybrid-data-security_chapter_01.html.
Connect to Workstation 1 (198.18.1.36) if not already using username / password: dcloud\cholland / C1sco12345. Once
connected, on the desktop, launch a new instance of PuTTY.
Connect to syslog.dcloud.cisco.com (also a saved session) and login with Username: root and Password: C1sco12345.
Login to the Docker hub to access the Cisco HDS configuration Docker image using the following command:
Use the following command to pull the Cisco HDS configuration Docker image:
Note: This might take a moment to download. Wait for the Pull complete message for each instance.
Run the Cisco HDS configuration Docker container using the following command:
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 36 of 89
Cisco dCloud
On Workstation 1 open Firefox and login into the Cisco Webex Control Hub management interface:
http://admin.webex.com with Userid: cholland@ssXXX.dc-YY.comand Password: dCloud123!.
Click Accept if prompted at the first logon to confirm the terms and conditions of the service.
NOTE; The setup process might take a moment before the next screen opens.
Before the Hybrid Data Security nodes can be registered in the administration portal the software has to be downloaded and
deployed. Select the radio button next to No, I need to install and configure the software.
Click Next.
Click OK to save the VMware ESXi OVA template to the local disk. It will save to the desktop by default.
Click Ok on the Register Hybrid Data Security Node window in the Webex Control Hub.
Using Firefox, open a new browser tab and navigate to the syslog host where the HDS config Docker is already installed:
http://198.18.135.60:8080.
Click Log in [ ]. If you are using the same browser as you logged into with Control Hub you won’t need to login again,
however, if you aren’t, then login with the credentials Userid: cholland@ssXXX.dc-YY.com and Password: dCloud123!.
After successfully logging in, the browser will show an error Unable to connect. This error is caused by the fact that the Docker
image is running on a Linux host and not on a local workstation. Please change the URL from 127.0.0.1 to 198.18.135.60 and
press Return.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 37 of 89
Cisco dCloud
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 38 of 89
Cisco dCloud
First, the configuration tool asks if a new configuration is to be created or an existing configuration ISO file should be updated.
Select No since this is a new install and there is no existing ISO file. On the main screen of the Cisco Hybrid Data Security
configuration tool the required steps are displayed to create the configuration ISO file.
Click Continue.
IMPORTANT: It is extremely important that when an ISO file already exists, that an update of the configuration parameters is
specified. Not doing so can destroy access to existing data encrypted with on premise KMS keys!
Browse to the Desktop where the certificate was copied for you.
Click Continue.
If you receive a Warning, check the box for I understand and wish to continue, then click Continue.
Next, configure the database parameters used by the Cisco Webex HDS nodes to connect to the PostgreSQL database.
The parameters are:
c. Username: hdsuser
d. Password: Cisco,123
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 39 of 89
Cisco dCloud
IMPORTANT: If you choose to install and configure the database server yourself please make sure that you enter
postgre1.dcloud.cisco.com:5432 to point HDS to the correct database host.
Specify the Syslog destination used by Cisco HDS nodes for logging messages. Syslog URL: udp://198.18.135.60:514.
Click Continue.
Configure the Key Access Level. This configuration will define to what extent a Cisco HDS Key Management Server will share
keys with entities outside your organization. Currently, there is only the option available to share keys with other entities.
NOTE: Cisco’s policy is not to allow customers to share any key material beyond their own organization. This will imply that Webex
Teams spaces with users outside a user’s own organization (“KMS federation”) are not allowed.
Select the Service Account and Select Cloud Access radio button.
Click Continue.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 40 of 89
Cisco dCloud
NOTE: Components deployed on premise as part of Cisco Hybrid Data Security require multiple service accounts and password
be created. It is the customer administrator’s responsibility to change these passwords periodically.
Click Continue.
This concludes the configuration of the ISO file for Cisco HDS nodes. After 10-15 seconds click the Download ISO link and
save the file, config-drive.iso, to the desktop which is the default location.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 41 of 89
Cisco dCloud
The dCloud lab infrastructure for this lab provides a standalone VMware ESXi host used to deploy the Cisco HDS OVA template. To
activate Cisco HDS a minimum of 2 nodes is required. For production deployments, a minimum of 3 nodes is recommended.
From Workstation1 launch the VMware vSphere client via the icon on the Desktop or taskbar [ ].
c. Password: C1sco12345
Navigate to the Desktop folder (default download location) and select the hds.ova file.
NOTE: The location of the hds.ova file depends on browser used, alternatively check the Downloads.
Click Next. The file is validated on selecting next and can take a moment before moving to the next screen.
On the Name and Location screen enter hds01 as the name for virtual machine.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 42 of 89
Cisco dCloud
Click Next.
On the Deployment Configuration screen keep 2 CPU selected on the Configuration dropdown list.
CAUTION: For production deployments, the default, Thick Provisioned Lazy Zeroed, should be chosen. Thin Provisioning should
be used with caution and for purposes in this lab to save disk space on the SAN! While VMware states that there is no performance
penalty between Thick or Thin provisioned virtual machines, other implications such as running out of space on a ESXi datastore
with thin provisioned machines must be taken into account and properly monitored and planned for.
Click Next.
On the Storage screen, click on VM Datastore instance for the destination storage location.
Click Next.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 43 of 89
Cisco dCloud
CAUTION: For production deployments, Thin Provisioning should be used with caution! While VMware states that there is no
performance penalty between Thick or Thin provisioned virtual machines, other implications such as running out of space on a ESXi
datastore with thin provisioned machines must be taken into account and properly monitored and planned for.
Click Next.
Click Next.
Wait for the VM deployment to finish. Upon successful deployment, the message Completed Successfully appears.
Click Close.
Repeat the virtual machine deployment steps to deploy a second Cisco HDS node called hds02.
After deploying the second HDS node, in the vSphere client, click Inventory, then ESXi server 198.18.133.31 and finally the
Virtual Machines tab.
Highlight both nodes (Shift or Ctrl click), hds01 and hds02, and right-click on the selection.
From the shortcut menu, select Power > Power On. Alternatively select both nodes and press Ctrl+B.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 44 of 89
Cisco dCloud
Open the Console for the hds01 (node hds02 when repeating) virtual machine from the VMware vSphere Client by clicking
on the console button [ ] after selecting the virtual machine or by right-clicking on virtual machine and from the shortcut
menu selecting Open Console.
When prompted with ciscoecp_<designator> login: login with Username: admin and Password: cisco.
NOTE: It can take 3 to 5 min for the login prompt to display. Wait for the correct prompt before attempting to login. Until then, the
prompt will display localhost login: and the screen will refresh a number of times.
Next, the system prompts you to change the default password. Enter the current password cisco.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 45 of 89
Cisco dCloud
The Cisco HDS virtual machine console displays a security disclaimer. Confirm with by selecting OK.
NOTE: To navigate through the following screens click the mouse once in the screen and then use the Up/Down and Right/Left
arrows and press the Enter key to make a selection. To release the mouse press Ctrl+Alt.
To configure the basic parameters (hostname, IP address, DNS, NTP) select Edit Configuration from the menu.
Select Yes to acknowledge the notification: Changes to this node will end active calls currently using this node.
Select Static for the IP address configuration of the Cisco HDS node.
Table 4.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 46 of 89
Cisco dCloud
For this exercise arrow down and delete the other four NTP servers.
IMPORTANT: For production deployments, more than one NTP servers are required to ensure that the Cisco HDS nodes have a
time synchronization source. A valid time source is required for the nodes to validate certificates used for encrypted communication.
Confirm that all parameters have been correctly configured and then select Save Changes & Reboot.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 47 of 89
Cisco dCloud
Acknowledge the notification that changes to the node can result in reconfiguration of components by selecting Done.
Upload Cisco HDS configuration ISO image into the VMware Datastore
From the VMware vSphere client select the ESXi server 198.18.133.31.
Under Datastores, right-click on the VM Datastore option and select Browse Datastore….
Click the Upload Files button [ ] and then select the option Upload File….
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 48 of 89
Cisco dCloud
In the Upload Items dialog, navigate to the Desktop, select the config-drive.iso file, and click Open.
From the VMware vSphere client select the ESXi server 198.18.133.31 navigate to the Virtual Machines tab.
Highlight both nodes, hds01 and hds02, and right-click on the selection.
From the shortcut menu, select Power > Shut Down Guest. Alternatively select both nodes and press Ctrl+D.
Click Yes to confirm guest shutdown and wait for the VMs to power off.
Next, begin to mount the Cisco HDS configuration ISO image within the virtual machine. Right-click hds01 (repeat the same
steps for hds02) virtual machine and from the shortcut menu select Edit Settings….
To confirm the warning about limited editing from vSphere Client click OK.
Under Device Type select the radio button for Datastore ISO File and click Browse….
Click OK.
Under Device Status check the boxes for Connect at power on.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 49 of 89
Cisco dCloud
Click OK.
Repeat the steps for the hds02 node to connect the ISO to it.
Open the tab in the Firefox browser you had opened to the Control Hub earlier (https://admin.webex.com).
Navigate to Services and then on the Hybrid Data Security card click Set Up.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 50 of 89
Cisco dCloud
With both Cisco HDS nodes running, select Yes, I’m ready to register my Hybrid Data Security Node and click Next.
On the Register Hybrid Data Security Node screen enter the following parameters:
Click Next.
The following confirmation screen shows the previously entered parameters. Select Go to Node. This action redirects the
browser to the web interface of the Cisco HDS node and start the cloud registration process.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 51 of 89
Cisco dCloud
Since the web interface on the Cisco HDS node is using a self-signed certificate, the browser is not able to verify the certificate
and displays the security exception shown in Figure 3. Select Advanced and Add Exception in the next dialog shown in
Figure 4.
Figure 3. Advance
On your browser, follow the respective prompts to accept the security exception. Click Confirm Security Exception.
On the registration confirmation dialog check: Allow Access to the Hybrid Data Security Node.
Click Continue.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 52 of 89
Cisco dCloud
Congratulations you have registered your first Cisco Hybrid Data Security Node!
Return back to the Webex Control Hub and click Add Resource from the Hybrid Data Security Services page. Repeat the
steps 1-13 for node hds02 using the parameters below. Close the browser tab after the second HDS node has registered
successfully.
The service shows the status Operational under the HDS Cluster list select dcloud.cisco.com.
NOTE: The status can also be: Impaired Service (Directly after registering.) or Not Operational (While the new nodes are being
setup).
The dialog box shows the configuration parameters and status of the Cisco HDS cluster. Select Open nodes list on the fly-out
window.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 53 of 89
Cisco dCloud
The newly registered Cisco HDS nodes verify the software version deployed from the OVA template against the latest available
software version released to the cloud. The HDS nodes will automatically go through the upgrade process if needed. If an
upgrade is available, you will see a message at the top of the page to Install now… Then you would click Upgrade Now from
the confirmation dialog box. HDS nodes are automatically upgraded and installed,
The Node List page displays the upgrade progress for both HDS nodes. It will take approximately 5-10 minutes for the upgrade
process, wait for the updates to complete before proceeding to the next step.
The Status changes to Running when the Upgrade is complete. Don’t continue until both have a status.
After the HDS nodes have been upgraded, in the Control Hub navigate to Services > Hybrid Data Security card and click
Edit settings.
NOTE: In the General section an email address can be specified that receives notifications about the status of the Cisco HDS
nodes. This is not part of this lab exercise.
Under Service Status the Webex Organization can be enabled to start using the Cisco Hybrid Data Security node for issuing
keys to users. Click on Start Trial.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 54 of 89
Cisco dCloud
NOTE: For this lab exercise Cisco HDS will be enabled using 2 nodes. For production deployments, at least 3 nodes are suggested.
Observe that the Service Status now shows Trial Mode is enabled and the FQDN of the Cisco HDS Cluster.
IMPORTANT: This lab uses a trial Webex organization. Under no circumstances should the trial organization be moved to production.
Please DO NOT PRESS the button!
Currently there are no users enabled for Cisco HDS (trial). Click the Add Users link. (See the screen above.)
Add the following email addresses to the Cisco HDS trial (press enter after typing the email address to add the second
address):
cholland@ssXXX.dc-YY.com
aperez@ssXXX.dc-YY.com
The Service Status page now confirms that two users are active for use with Cisco HDS.
Now all new Webex Teams spaces created by the users activated for Cisco HDS will be provided with encryption keys by the
locally deployed HDS nodes. Existing spaces that the user initiated before deployment and activation of Cisco HDS will continue
to use the Cisco Cloud Key Management Server – existing keys are NOT being migrated from the cloud to the Cisco HDS nodes.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 55 of 89
Cisco dCloud
NOTE: If the DLP (Cloudlock) and eDiscovery parts of this lab have already been completed, logout and restart the Cisco Webex
Teams client on ALL workstations by clicking on the avatar and selecting Sign out. Then exit the Webex Teams client.
Login to Webex Teams with Username: cholland@ssXXX.dc-YY.com and password: dCloud123!. Follow the above note
and make sure to log off and exit the Webex Teams client if you already completed the DLP (Cloudlock) and/or eDiscovery
modules of the lab. You will also need to logout/login to all Webex Teams clients including Anita’s on workstation 2
(198.18.1.37, dcloud\aperez / C1sco12345) then client login of username: aperez@ssXXX.dc-YY.com and password:
dCloud123!. Failure to log out and back will cause the next steps to fail. If this is the first time logging into the Webex Teams
clients, then you can continue with the exercise.
From the main screen click on the button and Contact a Person.
Type a message and send to Anita. Since Charles and Anita are enabled for HDS the end-to-end encryption keys for this new
space are provided by the local KMS.
Replace <bearer from developer.webex.com goes here> with the access token you obtained from the developer.webex.com
website. Make sure there is a single blank between the word Bearer and the token.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 56 of 89
Cisco dCloud
Change to the directory: cd c:\Users\cholland\desktop and run the command python get_uuids.py.
Keep this window open and available, the UUIDs are required in later steps.
NOTE: Cisco Webex is obfuscating real user IDs and PII information as much as possible. To identify KMS activities of a particular
user it is required to know the users UUID.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 57 of 89
Cisco dCloud
From a new PuTTY session (right-click the top bar of the window of the existing session and choose New Session from the
menu) to the syslog server (root/C1sco12345) (don’t close the existing one that is running the Docker container) and run the
following command:
cat /var/log/messages | grep <UUID of cholland>
Observe that similar messages (as shown in the example in Figure 76) are displayed. The example shows several requests to
KMS. Hostnames (hds01 and hds02). It also shows that both nodes are being used simultaneously and are load balancing. The
first line shows a KMS:REQUEST with a method: create and the UUID associated with Charles Holland. Bottom KMS:REQUEST
with method: retrieve is a request from the client for the created key.
Figure 5. Output
On workstation 2 (RDP to 198.18.1.37. Login with Username: dcloud\aperez and Password: C1sco12345).
Logon to Webex Teams with aperez@ssXXX.dc-YY.com / dCloud123!. Then check the 1:1 conversation between Charles
and Anita and send a reply.
In the PuTTY window on workstation 1, run the command: cat /var/log/messages | grep <UUID of Anita>.
Observe that this time the example is slightly different since the key was created by the initial request of Charles when he
established the 1:1 conversation with Anita we only see a KMS:REQUEST with method: retrieve when Anita opens the
conversation.
Figure 6. KMS:Request
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 58 of 89
Cisco dCloud
INFORMATION: The concept of KMS Federation is introduced, that concept being the ability to share keys between different Cisco
Cloud KMS, the.
Federation in this context doesn’t share the pain and misery that the Collaboration industry has lived through with protocols like SIP
or XMPP. There are no additional configuration steps required to allow business to business communication in environments that
use an on premise KMS. The certificate deployed with KMS is used to establish a mutually authenticated connection between the
on-premise deployment and the Cisco Collaboration Cloud. This allows multiple customer deployed KMS instances to seamlessly
and securely federate.
In this exercise, Charles and Anita are both users that have been enabled for KMS. Next is an example where a user is part of the
conversation that either uses the Cisco Cloud KMS or a customer that has also deployed Cisco Webex HDS and runs its own KMS.
NOTE: For continuous monitoring of all KMS related messages open a new terminal window (PuTTY) to the syslog machine, use
the following command and keep window open: tail -f /var/log/messages|awk '$5 == "kms:" {print $0}'
Return to Workstation 1 and create a new Webex Teams team using a Name of your choosing.
Select the new team, click Team members, and add Anita and Kellie to the team.
Click on Spaces within the team and write something into the General space of that newly created team.
On Workstation 3 (RDP to 198.18.1.38. Login with Username: dcloud\kmelby and Password: C1sco12345). Launch the
Webex Teams client and login with kmelby@ssXXX.dc-YY.com / dCloud123!.
Check the team that you created in the previous step, of which Kellie is a member, and write something into the General space.
User Kellie is part of same Webex organization as Charles and Anita. Kellie has not been enabled for the Webex HDS trial hence
the user is still using the Cisco Cloud KMS. Inviting Kellie to a team/space that was created by a HDS enabled user (Charles)
shows that keys are being shared between the local KMS and users based on the cloud KMS.
Check the syslog server with the following command: cat /var/messages | grep <UUID of Kellie>.
NOTE: Similar to the previous example, the UUID of Kellie is requesting keys from the local KMS via the federation link with the
cloud KMS on which Kellie is hosted.
Return to Workstation 1, create a new Webex Teams team and add the following members: Anita and
luke@identitylab12.ciscolabs.com. The second member is an external user in a Webex Org that is also enabled for
HDS/KMS. Creation of this space will force the two KMS to establish a federated connection.
NOTE: The UUID below represents the KMS deployed in Webex Org. identitylab12.ciscolabs.com.
This is not the best way to grep the information we need – check further how to best get the messages we need for this piece.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 59 of 89
Cisco dCloud
Look for the message EPHEMERAL_KEY_COLLECTION – first section of messages in Figure 79 and Figure 80 shows the
establishment of KMS Federation.
Use this command to display messages that relate to the user Luke, in Org. identitylab12.ciscolabs.com, retrieving keys from
the local KMS.
cat /var/log/messages | grep 7b97da7e-0855-4241-a720-7c56e52ef46e | more
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 60 of 89
Cisco dCloud
CAUTION: Cisco does not specifically recommend or require the use of Elastic Stack. There are many other logging solutions that
can receive standard syslog messages available that can be utilized. The example provided in this scenario is not intended for
production use Instead this scenario shall only give an overview of the benefits of using an enhanced logging solution. This
information is provided as is with no warranty or support by Cisco. Use at your own risk.
Configure ELK
For this exercise the components of Elastic Stack (Logstash, Elasticsearch, and Kibana) are deployed in a single Docker container
without redundancy or high availability.
Note: Docker and the ELK container have already been installed for you on the ELK machine. For your reference, please refer to
the appendix of this document on how to install Docker on a Linux host and prepare the machine for ELK.
For Elastic Search to correctly parse the log messages received from KMS nodes some additional configuration needs to be added
to the standard Docker image retrieved from the repository. Docker provides a build option where existing images can be modified
with additional files or other configuration parameters (required files have been already downloaded from the GitHub repository in
the previous chapter).
NOTE: Configuration files and scripts required for this scenario are hosted in the following GitHub repository
https://github.com/tobiasneumann42/ciscohds_elk.
To pull the content to the local machine git command must be installed. The following steps to install git are for documentation
purposes only – This has been done for you!
Using PuTTY, open a new terminal session to the elk Linux host (elk.dcloud.cisco.com and root/C1sco12345). To clone
the latest version from the GitHub repository run the following command from the directory /root:
From the PuTTY connection to the ELK server enter cd /root/ciscohds_elk and run the command:
Docker allows existing images, downloaded from the repository, to be modified based on definitions made through an input file
called Dockerfile. The above command will take the information from the Dockerfile and update the image from the repository
with exercise specific configuration information and create a new image called elkhds.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 61 of 89
Cisco dCloud
After the Docker container is built from the previous step, start the elkhds Docker container with the following command:
docker run -p 5601:5601 -p 9200:9200 -p 9300:9300 -p 514-515:514-515/udp -p 5044:5044 -t --name elk01 elkhds
The command, specifies the ports, instructs Docker which ports from the container to expose on the host system (see Table 5).
The name, parameter, is just a reference for the instance and, elkhds, refers to the image that was created in the previous step
based on the provided Dockerfile.
TCP 5601 Kibana Web front end and visualization component of ELK
UDP 514-515 Logstash Syslog receiver ports. (514 standard syslog, 515 KMS)
Congratulations! You have successfully installed an ELK stack. In the next exercise, the required configuration changes are applied
to make use of this new component.
For HDS nodes to use the Elastic Stack for logging, the configuration of the ISO image needs to be modified accordingly. Original
image is configured to send all syslog messages to the 198.18.136.60 (syslog.dcloud.cisco.com) UDP port 514. To use ELK, the
HDS nodes need to send syslog messages to 198.18.136.61 (elk.dlcoud.cisco.com).
On the syslog machine, if not still running, launch the Cisco Webex HDS configuration tool with the following command:
docker run -p 8080:8080 --rm -it --name ciscohds ciscosparkhds/hds-setup:stable
In the browser, browse to 198.18.135.60:8080 (refer to the lab section Configuring Cisco Webex HDS for details – remember
to replace 127.0.0.1 with 198.18.135.60 after authentication).
On the ISO Import screen, to change an existing configuration ISO you must select Yes for import and Browse… and select
the existing file config-drive.iso from the desktop. Then click Continue.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 62 of 89
Cisco dCloud
IMPORTANT: This step is VERY important. Not selecting the existing ISO and instead creating a new file for an active HDS
deployment can render the existing keys in the database unusable. User data encrypted with these keys will be lost.
On the X.509 certificate configuration screen, select Yes to continue using the existing certificate chain and private key. Click
Continue.
Confirm the information on the Database Credentials screen and click Continue. (password is Cisco,123)
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 63 of 89
Cisco dCloud
On the System Logs configuration screen, change the Syslog URL to udp://198.18.135.61:515. Changing the Syslog URL
instructs the HDS nodes to send log traffic to the elk host using UDP on port 515. Then click Continue.
NOTE: Make sure you change the port as it is not the default syslog port that elk is listening on for KMS messages.
Keep Service Account and Select Cloud Access selected and click Continue.
From the VMware vSphere client select the ESXi server 198.18.133.31.
Under Datastores, right-click the VM Datastore and from the shortcut menu select Browse Datastore….
Click on the Upload Files button [ } and then select the option Upload File….
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 64 of 89
Cisco dCloud
In the Upload Items dialog, navigate to the location, c:\users\cholland\desktop, where you previously downloaded the
config-drive(1).iso file to and click Open and then Yes.
To activate the new configuration, shutdown the hds01 (hds02 when repeating) virtual machine. Right-click on hds01 (hds02
when repeating) and from the shortcut menu select Power > Shutdown Guest.
Once shutdown is complete, edit the virtual machine properties by, right-clicking on the VM and selecting Edit Settings. Then
click OK.
Click on CD/DVD drive 1. Select the Datastore ISO File and click Browse….
Select the newly uploaded ISO from the VM Datastore, config-drive(1).iso. Make sure the Connect a power on check box is
selected.
Click OK.
Check that the Cisco Webex Control Hub that hds01 is in operation. Then repeat steps 1-13 for hds02.
The script used to update the UUID information with user readable data against the Cisco Webex REST API requires an access
token. For a continuous running script it is recommended to register an application on developer.webex.com. The provided
parameters allow an application to use a Refresh Token to create new Access Tokens before expiration. In this example, the script
is run using the Access Token that was obtained earlier from the developer.webex.com site.
Go to developer.webex.com and copy the access token (as explained in the earlier module to gather UUIDs).
Open Windows PowerShell [ ] and enter the following commands, this will provide the epoch time value.
NOTE: Enter each line one at a time and not all at once.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 65 of 89
Cisco dCloud
Open Notepad++, create a new file and enter the following information:
Launch a new PuTTY terminal connection to the elk host and issue the following commands:
mkdir /tmp/scripts
cp /root/ciscohds_elk/tmp/scripts/webex_proc.py /tmp/scripts/
In a Windows Command window, change the path to cd c:\users\cholland\desktop and issue the following command to
copy the Webex API access token to the elk host.
pscp webex_teams_proc.token root@elk:/tmp/scripts/webex_teams_proc.token
On the ELK server, change the directory to cd /tmp/scripts and launch python3.6 webex_proc.py.
Leave the script running. It will update the user information in the elastic backend on a continuous basis. The script operation
can be monitored by creating a new PuTTY terminal window to the elk host and the issuing the following command:
tail -f /var/log/webex_proc.log.
To create continuous traffic against the local KMS, a script is provided that creates new Webex Teams spaces periodically, adds all
users to it and sends a message to the space. All clients (workstations) retrieve a key for every new space created to decrypt space
title and content.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 66 of 89
Cisco dCloud
On Workstation 1 open a new Windows Command window (click the button in the task bar and type cmd).
From the path cd c:\users\cholland\desktop, copy the webex_hds_traffic.py script file to the desktop with the following
command:
pscp root@elk:/root/ciscohds_elk/webex_hds_traffic.py webex_hds_traffic.py
Using Notepad++, edit the script on the Desktop, webex_hds_traffic.py, to add the Bearer token from
developer.webex.com.
Run the script with C:\users\cholland\Desktop> python webex_hds_traffic.py and keep it running in the background (do not
close the window).
In the following, from Workstation 1, you will navigate to the Kibana web portal and familiarize yourself with the navigation of the site
and interpret some of the messages sent by the HDS nodes.
From Workstation 1, using Firefox, open a new tab and browse to the following URL: http://elk.dcloud.cisco.com:5601
When connecting for the first time, select @timestamp from the Time Filter field name and then click Create to configure a
default index that is used to store information.
Wait a couple of minutes and press the refresh index icon (circular arrows). This will update the field index mapping in elk.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 67 of 89
Cisco dCloud
In the main Discover dialog screen of Kibana, all messages are displayed including a graphical representation of messages
received in the selected timeframe. Information can be searched using full text from the query field.
Figure 2. Discover
Kibana allows search queries, visualizations, and dashboards to be configured. In the following steps this will be explored. You will
use the commands bellow to copy the predefined .json file to Workstation 1.
Return to the Kibana web portal and navigate to Management and click on Saved Objects.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 68 of 89
Cisco dCloud
From the next screen select Import. Select the hdselk.json file copied in the previous step from the desktop.
Click Yes, overwrite all to confirm the overwriting of any existing objects.
On the Kibana main search page click Discover. In the search field enter Charles.
In the previous scenario, using a standard syslog server, the log messages only contained the userID in UUID format. Using
the python script running in the background, the log entries now also contain detailed user information in human readable format.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 69 of 89
Cisco dCloud
Select the triangle at the start of the message. Parsing patterns for HDS added to the ELK infrastructure creates index fields
for a specific part of the log messages. These can be utilized for graphical representation or mathematical computations.
Next use the objects imported in the previous step into Kibana. Click Discover and then Select Open.
The page shows the available stored queries, click on KMS Health.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 70 of 89
Cisco dCloud
Cisco Webex HDS nodes send continues health status messages including current memory utilization. The preconfigured
search shows all related health status information.
This time select webexhds_KMS_key_requests. This output provides HDS administrators with direct visibility on who is
requesting keys from on-premise KMS.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 71 of 89
Cisco dCloud
Pulling it all together, let’s visualize the information returned by the queries and create a nice dashboard. In Kibana select
Dashboard and HDS_dash01.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 72 of 89
Cisco dCloud
Under Saved Sessions, double click on postgre1 to launch the SSH session. Alternatively, connect to the IP Address
198.18.135.58.
Install the yum repository for PostgreSQL version 9.6 required for Cisco HDS using the following command:
Install PostgreSQL version 9.6 and required dependencies using the following command:
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 73 of 89
Cisco dCloud
Enable PostgreSQL to automatically start on system boot using the following two commands:
systemctl start postgresql-9.6
systemctl enable postgresql-9.6
Create a new PostgreSQL user for use with HDS deployment using the following command:
CREATE USER hdsuser WITH PASSWORD 'Cisco,123';
Create a new PostgreSQL database for use with HDS deployment using the following command:
CREATE DATABASE hdsdb OWNER hdsuser;
Assign the required privileges to the database as well as the user created in the previous steps:
GRANT ALL PRIVILEGES ON DATABASE hdsdb to hdsuser;
ALTER ROLE hdsuser WITH SUPERUSER;
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 74 of 89
Cisco dCloud
Add the following line to allow access from external hosts in the IP range 198.18.0.0/16 to the database with md5
authentication.
# IPv4 local network connections for HDS nodes:
host all all 198.18.0.0/16 md5
CAUTION: This rather large range is specific to the setup in the Cisco dCloud environment. For production deployments, only the
specific Cisco Webex HDS nodes and systems required to monitor database operations should be allowed to communicate with the
database.
Save changes and exit from Nano editor with Cntrl-X and Y.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 75 of 89
Cisco dCloud
By default, PostgreSQL only listens to SQL requests on localhost (127.0.0.1). For external database connections, as required
by Cisco HDS, PostgreSQL has to be configured to listen on other configured IP address(s). In the section CONNECTION AND
AUTHENTICATION change the following:
IMPORTANT: Make sure the hash # at the start of the lines is removed.
Restart PostgreSQL to activate the changes by entering in the following command: service postgresql-9.6 restart.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 76 of 89
Cisco dCloud
Check the status of PostgreSQL with the command: systemctl status postgresql-9.6.service.
NOTE: Installation of the monitoring software is not mandatory. The software is used in this lab to demonstrate some of the inner
workings between the Cisco HDS nodes and the database.
Us the following command to install a web server which is required for PostgreSQL Admin: yum –y install httpd.
NOTE: Because of an intermittent issue with yum –y install commands, the installation might not work. If the installation fails, please
retry using the command without the –y option and manual confirm “y” when prompted.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 77 of 89
Cisco dCloud
Enable httpd to automatically start on system boot with the following commands:
systemctl start httpd.service
systemctl enable httpd.service
PHP is required for PostgreSQL Admin. Using the following command, install the required PHP components:
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 78 of 89
Cisco dCloud
To access the PostgreSQL Admin webpage the default configuration must be modified. Open the phpPgAdmin.conf file using
the command: nano /etc/httpd/conf.d/phpPgAdmin.conf and change the following parameters:
b. Under <IfModule !mod_authz_core.c> change Deny from all to Allow from all
Open the config.inc.php config file using the command: nano /etc/phpPgAdmin/config.inc.php
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 79 of 89
Cisco dCloud
Restart the PostgreSQL server using the command: systemctl restart postgresql-9.6.service
Tell the httpd service to reload configuration with the following command: systemctl reload httpd.service
Select the database host postgre1 from the panel on the left and logon with the credentials for the database admin created
earlier in the PostgreSQL install section Username: hdsuser Password: Cisco,123.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 80 of 89
Cisco dCloud
After successfully logging in, the pane on the left displays details for the available databases. While the hdsdb has been created
from the PostgreSQL CLI tool, it has not been populated yet. This will automatically occur on the first Cisco HDS being registered
to the cloud and activated.
This concludes the installation and configuration of the PostgreSQL database and the monitoring components.
For the CentOS Linux host syslog.dcloud.cisco.com in this lab the Syslog daemon has already been installed. You don’t need to
install the syslog server, however, you will need to follow these steps to configure the process.
In the event that a Syslog server needs to be installed on Linux (CentOS 7), the following command can be utilized:
Connect to syslog.dcloud.cisco.com and login with Username: root and Password: C1sco12345.
Using the nano editor open the rsyslog.conf file: nano /etc/rsyslog.conf
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 81 of 89
Cisco dCloud
Verify within the syslog.conf configuration file that UDP and/or TCP service is enabled by removing the hash ‘#’ sign from the
front of the lines of code under # Provides UDP syslog reception and # Provides TCP syslog reception.
Cisco HDS machines can use either protocol (UDP or TCP) for sending logging messages.
Restart the Syslog daemon by entering in the following command: systemctl restart rsyslog.service.
Verify that Syslog is listening to the correct protocols and ports by entering in the following command: netstat -antup |
grep 514.
Do not close the syslog session as it will be used in the next scenario.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 82 of 89
Cisco dCloud
For the purpose of this exercise, Let’s Encrypt is used as the issuer for these free, valid for 90 days, certificates.
Continuing in the PuTTy session connected to the syslog server, run the following command: yum clean all.
To begin, it is necessary (in addition to installing the required tools) to add the required repository using the following
command: yum -y install epel-release
Install the Let’s Encrypt tools using the following command: yum –y install letsencrypt
Change the last line (subjectAltName) of the configuration file to match the domain of your pod.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 83 of 89
Cisco dCloud
Create the certificate signing request using openssl using the following commands:
CAUTION: A file with the commands for openssl and letsencrypt, can be found on the desktop of Workstation1. Please modify
the commands to match your POD’s domain then copy and paste them into the PuTTY terminal window on the Linux host syslog.
If the commands are not modified, using another domain will cause issues.
openssl req \
-new -newkey rsa:2048 -sha256 -nodes \
-keyout privkey1.pem -out signreq.der -outform der \
-subj "/C=UK/ST=Some State/L=Some Place/O=<your-POD-domain>/emailAddress=webmaster@hds.<your-POD-
domain>/CN=hds.<your-POD-domain>" \
-reqexts SAN \
-config openssl.cnf
Issue the certificate request from the Let’s Encrypt’s Staging server using the following commands:
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 84 of 89
Cisco dCloud
CAUTION: Let’s Encrypt production servers only allow a limited number of requests. To ensure that their servers are not blocking,
please use the staging server first and upon successful response repeat the request with the production server.
letsencrypt certonly \
--standalone \
--preferred-challenges http \
--server https://acme-staging.api.letsencrypt.org/directory --text \
--config-dir letsencrypt/etc --logs-dir letsencrypt/log \
--work-dir letsencrypt/lib --email "webmaster@hds.<your-POD-domain>" \
--csr "signreq.der"
Upon the successful completion of the challenge handshake please remove the files created before running the command
against the Let’s encrypt production environment: rm –f 000*
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 85 of 89
Cisco dCloud
Now enter the following commands to issue the certificate request against the Let’s Encrypt production server:
letsencrypt certonly \
--standalone \
--preferred-challenges http \
--server https://acme-v01.api.letsencrypt.org/directory --text \
--config-dir letsencrypt/etc --logs-dir letsencrypt/log \
--work-dir letsencrypt/lib --email "webmaster@hds.<your-POD-domain>" \
--csr "signreq.der"
Note: The Screenshot shows that Let’s encrypt is verifying the certificate request by sending an http request to the FQDN for which
the certificate is requested. This requires the host to be accessible from the internet via the requested FQDN.
Now that there are three files, the certificate, the chain of root CAs, and private key files, the components need to be converted
into a single pfx file. Use the following openssl command to convert the components:
openssl pkcs12 -export -out hds.pfx -inkey privkey1.pem -in 0000_cert.pem -certfile 0000_chain.pem -
certfile 0001_chain.pem -name kms-private-key
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 86 of 89
Cisco dCloud
To create the configuration for the Cisco HDS nodes the hds.pfx certificate file needs to be copied to Workstation1. To do
this, first open a command prompt window on Workstation1 and change directory to the downloads folder using this
command
cd c:\users\cholland\downloads.
Use the PuTTY Secure Copy command (PSCP) to initiate the copy: pscp root@syslog:letsencrypt/hds.pfx hds.pfx
Return to the PuTTy window connected to the syslog server. At the [root@syslog letsencrypt]# prompt, return to the root
directory by issuing the command cd /root.
Start the Docker daemon using the following command: systemctl start docker.
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 87 of 89
Cisco dCloud
This command downloads a test image and runs it in a container. When the container runs, it prints an informational message
and exits. You might need to scroll up again to see the following output.
NOTE: To start the Docker daemon enter the command: systemctl start docker.
NOTE: To enable the automatic start of Docker on boot issue the following command: systemctl enable docker.
NOTE: To verify if Docker is working correctly issue the command: docker run hello-world.
To increase the virtual memory configuration for Linux, required for ELK to run, issue the command: sudo sysctl -w
vm.max_map_count=262144.
To make this change permanent edit /etc/sysctl.conf. Enter the command: nano /etc/sysctl.conf.
Verify that the setting has been correctly applied by issuing the command: sysctl vm.max_map_count.
Pull the latest version of elk from Docker registry by issuing the command: docker pull sebp/elk:553
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 88 of 89
Cisco dCloud
© 2016 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.
Lab Guide v1.1 Page 89 of 89