RAPID Module Deployment

RAPID Module Deployment

Contents
RAPID Deployment...................................................................................................................1
About RAPID Module......................................................................................................... 1
About RAPID Analysis Submissions.....................................................................................2
Configure RAPID............................................................................................................... 4
Start RAPID....................................................................................................................... 7
Use the RAPID Python SDK Client...................................................................................... 8
RAPID HTTP API..............................................................................................................10
RAPID Monitoring Services................................................................................................ 11
RAPID Environment File....................................................................................................12
RAPID Docker Compose YAML File...................................................................................14

VMware, Inc. ii
RAPID Module Deployment

RAPID Module Deployment

This document describes the deployment of the RAPID Module (Rapid API for Detection).

About RAPID Module

RAPID Module (Rapid API for Detection) provides access to the most advanced malware (sandboxing)
and static analysis in the industry with simple, powerful, and well-documented APIs and integration
options. Use RAPID to streamline submissions and provide a quick response to malicious content. The
RAPID Module can be embedded in your appliance or infrastructure.
Some of the many benefits include:

• Access to entire analysis pipeline

# Sandboxing for malware analysis

# File/document prefilter, etc.

# URL malicious/benign listings and rate limiting

# Static file analysis

# Regular enhancement from VMware engineering

• Fast protect for malicious items

• Simple, fast, and powerful integration

• Delivery as an upgradeable package

• Extensible and configurable

RAPID Module and docker

The RAPID Module is deployed as a docker container (https://www.docker.com/) so that you can run
the module anywhere that a docker environment is available. Since docker is available on several
different platforms this means that the RAPID Module can be run in any of the available docker
environments. This can be Linux, Windows, or any other platform that can support a running docker
environment.
See the complete list of supported platforms (https://docs.docker.com/ee/supported-platforms/
#supported-platforms) for docker environments.
The following versions of docker are the minimum requirements for the RAPID Module:

VMware, Inc. 1
RAPID Module Deployment

• docker: 18.09.1 (https://docs.docker.com/engine/release-notes/#18091)

• docker-compose: 1.21.2 (https://docs.docker.com/release-notes/docker-compose/#1212)

Container Sizing
VMware has tested a RAPID instance as follows:

• Approximately 250 submissions/min

• 16000+ files
• 13000+ URLs
• asdk_api_container memory limit: 800MB
• analysis_completion container memory limit: 400MB
• malscape_completion container memory limit: 400MB
• 8 Physical cores available

Tests were a combination of malicious and benign samples, some results from static and others from
dynamic analysis.
For a scenario like this, VMware recommends at least 8 cores and 4GB RAM. You can alter these
parameters in the .env file.

About RAPID Analysis Submissions

Submit a file or URL for analysis
The API call /analysis/submit allows files or URLs to be submitted to RAPID for analysis. RAPID will
use multiple detection components (such as sandboxing, static file analysis, URL blocking, etc.) to
determine one of the following outcomes:

• benign — No malicious behavior was detected for the file/URL.

• malicious — The file/URL is was determined to be malicious.

• skipped — RAPID was not able to analyze the submission (see Skipped result).

• queued — RAPID requires more time to analyze the file (see Asynchronous processing of
submissions).

VMware, Inc. 2
RAPID Module Deployment

Asynchronous processing of submissions

After submitting a file or URL for analysis, RAPID may respond with a result 'queued' along with an
identifier for the submission ('submission_uuid'). In this case RAPID is processing the submission
asynchronously and /analysis/wait_completed API call should be used to get the results for these
submissions.
/analysis/wait_completed is a blocking API call; it will stream submission responses as they
arrive from RAPID. These responses will be for submissions that resulted in a 'queued' result. When
submitting a file or URL, a parameter 'client_id' may be provided. /analysis/wait_completed, called
with the same client_id, will stream responses for submissions that used the same client ID. It is
possible to run multiple parallel calls to /analysis/wait_completed if they use different client IDs.
Submission responses from /analysis/wait_completed contain a field 'last_event_id'. This is a
unique identifier for each submission response. Providing 'last_event_id' parameter to /analysis/
wait_completed will stream all submission responses with a 'last_event_id' after the provided
'last_event_id'. If 'last_event_id' parameter is 0, then only newly received submission responses will
be returned.

Skipped result
If RAPID was unable to analyze the submitted file or URL then it will response with result 'skipped',
along with a reason that it was unable to analyze the file or URL (for example, 'The submitted URL is
missing a URI scheme').

Metadata and follow-up analysis

Some detection components will yield additional metadata (along with the result). This may be an
AV classification (for example, 'trojan') or information about the components that contributed to the
outcome. If an outcome has been decided, but there may be additional metadata to come from a
slower component, then RAPID will provide a 'follow_up_submission_uuid'. This will link to a further
more detailed analysis of this file or URL that may yield additional valuable metadata about the analysis.

Detailed analysis reports

Where dynamic analysis was used, there may be a very detailed report available for this analysis. In this
case the RAPID response will include the metadata 'analyst_api_task_uuid' (or it may be provided
in a 'follow-up' analysis), which can then be used in the API call /analysis/get_analysis_report to
retrieve this detailed analysis report.

Query file by hash

The API call /analysis/query_file_hash allows the client to get an analysis result for a file which is
already known by RAPID service and/or VMware NSX Network Detection and Response. Instead of
submitting a file, the hash of the file is provided (md5, sha1, or sha256). If the file is known, the result
will be returned in the same way as a regular file analysis submission. If the file is unknown, a 'skipped'
response is returned.

VMware, Inc. 3
RAPID Module Deployment

RAPID prefilter submission

The API call /prefilter/submit allows RAPID to be used as a prefilter, where it will provide an
immediate response that the file or URL is either 'benign' or 'needs_more_analysis'. If the response
is 'benign' then RAPID will guarantee to respond with 'benign' even for a full analysis submission. If
the response is 'needs_more_analysis' then more analysis is required to determine if the file or URL
is 'benign' or 'malicious'. This additional analysis can be done with a full analysis submission through
RAPID, or using another malware analysis system.

Configure RAPID
Prerequisites
Docker
The RAPID service runs on the enterprise (https://www.docker.com/enterprise-edition) as
well as the community (https://www.docker.com/community-edition) editions of docker.
To get started, follow the steps on the Docker website (https://docs.docker.com/install/)
to install the docker-engine. Then follow the steps on the Compose website (https://
docs.docker.com/compose/install/) to install docker-compose.
.env
The .env file is used to define the license and execution mode of the RAPID service. This
file must be located in the directory from which you run the docker-compose command.
See RAPID Environment File on page 12 for more information and a complete list of
configuration parameters found in the .env file.
docker-compose.yml
The docker-compose.yml file is used to define the available services, networks, and
volumes that the docker services will use. This file must be located in the directory from
which you run the docker-compose command.
See RAPID Docker Compose YAML File on page 14 for more information and a complete
example of the docker-compose.yml file.
Configuration Steps

The RAPID Module is deployed as a set of Docker containers.

Procedure

Step 1: Install docker

Download either the community (https://www.docker.com/community-edition) or enterprise (https://
www.docker.com/enterprise-edition) edition. Install it following the instructions on the docker
website. Then test your installation:

VMware, Inc. 4
RAPID Module Deployment

lastline@lastline:~$ docker run hello-world

Step 2: Create a RAPID working directory

Use the mkdir command to create your RAPID work space, for example ~/dev/RAPID.

lastline@lastline:~$ mkdir ~/dev/RAPID

Step 3: Download services files into the working directory

Download the services files into your RAPID working directory. You can copy the .env and docker-
compose.yml files from this document.

Step 4: Acquire a license for the RAPID Module

Contact VMware Support (https://my.vmware.com/group/vmware/get-help) for the license key and
token you need to run RAPID. Edit the .env file to add the key and token you received:

...
LASTLINE_RAPID_API_KEY=20CHARLICENSEKEYXXXX
LASTLINE_RAPID_API_TOKEN=16CHARSECRETKEYX
...

Step 5: Select the running mode

Edit the .env file to select the running mode. Select one of static-analysis on page 12,
dynamic-analysis on page 12, or static-dynamic on page 12.

...
ANALYST_SDK_RUNNING_MODE=selected_mode
...

Note: The selected running mode must match the capabilities of your license. See RAPID
Environment File on page 12 for more information and a complete list of configuration
parameters found in the .env file.

Step 6: Define the services you want to run

Edit the docker-compose.yml file to select the services you want to run. By default all services are
started, but some may not be required for your system. These services can optionally be removed.
You can also modify the default ports for the RAPID Module.
See RAPID Docker Compose YAML File on page 14 for more information and a complete example
of the docker-compose.yml file.

Step 7: Login to the RAPID docker registry

VMware, Inc. 5
RAPID Module Deployment

Use your VMware credentials to authenticate to the docker registry (https://docker-

rapid.lastline.com).

lastline@lastline:~$ sudo docker login https://docker-rapid.lastline.com

Note: You only have to perform this step once, not for each update.
These instructions show running docker configuration commands as root (using sudo).
While this works, it is generally recommended that you create a docker group and add
yourself to that group to avoid using the root user for configuration.

Step 8: Fetch the docker images

Once you have authenticated successfully to the RAPID docker registry, download all of the images
(https://docs.docker.com/compose/reference/pull/) declared in the provided .env and docker-
compose.yml files.

lastline@lastline:~$ cd ~/dev/RAPID/
lastline@lastline:~$ sudo docker-compose pull
Pulling analyst-sdk-llmonitoring ... done
Pulling analyst-sdk-health-checker ... done
Pulling avbd ... done
Pulling rabbitmq ... done
Pulling analyst-sdk-malscape-completion ... done
Pulling memcached ... done
Pulling analyst-sdk-analysis-completion ... done
Pulling analyst-sdk-api ... done
Pulling statsd ... done

Note: By default, this will fetch all images with the version defined in the .env file. To fetch
a specific version of the images, edit the .env file to define the release you want
downloaded.

...
IMAGE_RELEASE_TAG=version-to-fetch
...

Related Topics
RAPID Environment File on page#12
RAPID Docker Compose YAML File on page#14

VMware, Inc. 6
RAPID Module Deployment

Start RAPID
Configuration Steps

Once the configuration is complete for your RAPID implementation, start the docker services.

Procedure

Step 1: Start the docker services

Change directories into your RAPID work space and then start the docker services.

lastline@lastline:~$ cd ~/dev/RAPID/
lastline@lastline:~$ sudo docker-compose up

This command starts the all RAPID services selected in docker-compose.yml, including optional
monitoring and visualization/search containers.
To launch the services in the background, hiding any console output, you can run docker in
"detached mode":

lastline@lastline:~$ sudo docker-compose up --detach

Creating network "docker_analyst-sdk-network" with driver "bridge"
Creating docker_memcached_1 ... done
Creating docker_rabbitmq_1 ... done
Creating docker_avbd_1 ... done
Creating docker_analyst-sdk-llmonitoring_1 ... done
Creating docker_analyst-sdk-malscape-completion_1 ... done
Creating docker_analyst-sdk-analysis-completion_1 ... done
Creating docker_analyst-sdk-api_1 ... done

Step 2: Stop the docker services

Terminate all docker containers with the docker-compose down command.

lastline@lastline:~$ docker-compose down

If you want to stop a specific service, use the docker-compose ps command to get the container
names and then shutdown the selected service with the docker stop command:

lastline@lastline:~$ docker-compose ps
Name Command State
Ports
---------------------------------------------------------------------------------------------
docker_analyst-sdk-analysis- /usr/bin/python /sbin/entr Up
completion_1 ...

VMware, Inc. 7
RAPID Module Deployment

docker_analyst-sdk-api_1 /usr/bin/python /sbin/entr Up

0.0.0.0:9090->9090/tcp
...
docker_analyst-sdk- /usr/bin/python /usr/bin/l Up
llmonitoring_1 ...
docker_analyst-sdk-malscape- /usr/bin/python /sbin/entr Up
completion_1 ...
docker_memcached_1 docker-entrypoint.sh Up 11211/tcp
memcached
docker_rabbitmq_1 docker-entrypoint.sh rabbi Up 15671/tcp,
15672/tcp,
... 25672/tcp,
4369/tcp,
5671/tcp,
5672/tcp
lastline@lastline:~$ sudo docker stop docker_rabbitmq_1

Use the RAPID Python SDK Client

Prerequisites
Install the python client from a host running the Analyst SDK. Download the library code in a Debian
Software Package file from http://127.0.0.1:9090/documentation/analyst-sdk-api-client.deb. This
link is local to the host that is running the RAPID Module and is only available if the analyst-sdk-api
docker container is up.
On Debian-based (https://www.debian.org/) systems this can be installed with dpkg -i analyst-sdk-
api-client.deb.
Configuration Steps

You can interact with the Analyst SDK API using the python client. Use the following steps to submit a
request:

Procedure

Step 1: Submit an analysis request

Run the example script:

import StringIO
from analyst_sdk_api_client import analysis
client = analysis.AnalystSDKClient(base_url='http://127.0.0.1:9090/')

# To do an URL submission
url = analysis.URLSubject('http://example.com')
client.submit(url)

VMware, Inc. 8
RAPID Module Deployment

# To do a File submission
fl = analysis.FileSubject(
StringIO.StringIO('WScript.Echo("Hello World!")'),
filename='test.vbs'
)
client.submit(fl)

Step Result
If the result is already known, it is returned immediately:

{'submission_uuid': '56d49dd8dd5f48ccb1f1b48c2711b97a', 'result': 'benign'}

Otherwise you will get information that the analysis was queued:

{'submission_uuid': '832d519632b847088dba5fbe0c02262a', 'result': 'queued'}

If the result is a follow-up analysis, you will receive metadata with the created result:

{
'submission_uuid': 'f7ffa0913b2c4872b9e896b695dbb228',
'result': 'malicious',
'metadata': {'follow_up_submission_uuid': '67450657d3c4426cbc9c9cbc4f74e8a4'}
}

Step 2: Wait for completion of results

To wait for completion of results, either queued or follow-up analysis results, use the
wait_completed call from the client:

from analyst_sdk_api_client import analysis

client = analysis.AnalystSDKClient(base_url='http://127.0.0.1:9090/')
for res in client.wait_completed():
print res

Step Result
You will get results immediately as they are pushed from the client. If the result is for a follow-up
analysis, you should get something like:

{
'submission_uuid': '67450657d3c4426cbc9c9cbc4f74e8a4',
'result': 'malicious',
'last_event_id': 1,
'metadata': {
'is_follow_up_to_submission_uuid:': 'f7ffa0913b2c4872b9e896b695dbb228',
'analyst_api_score': 100,
'analyst_api_task_uuid': '72ccc347238a001004917ac9e82046da'
}
}

VMware, Inc. 9
RAPID Module Deployment

If the result is for a queued analysis, you should see something like:

{
'submission_uuid': 'bc267f84c8f247a08a7594093da0f4ed',
'result': 'benign',
'last_event_id': 2,
'metadata': {
'analyst_api_score': 0,
'analyst_api_task_uuid': '72ccc347238a001004917ac9e82046da'
}
}

Each result returned by wait_completed always includes an last_event_id. This ID identifies each
event. The wait_completed call also accepts the last_event_id as parameter. This allows you
to request events that were previously returned by wait_completed. To use this ID, supply the
last_event_id for the last event that was successfully processed:

from analyst_sdk_api_client import analysis

client = analysis.AnalystSDKClient(base_url='http://127.0.0.1:9090/')
for res in client.wait_completed(last_event_id=1):
print res

The client will stream all events after this last_event_id.

RAPID HTTP API

HTTP API reference
The complete RAPID API documentation is available from the service itself at http://127.0.0.1:9090/
apidocs/#/analysis (http://127.0.0.1:9090/apidocs/#/analysis).

Note: This is a local link (127.0.0.1 IP address) to the host that is running the RAPID Module.

Web example
Once the RAPID Module has started, you can access its API at http://127.0.0.1:9090/analysis. For
example:

curl -N "http://127.0.0.1:9090/analysis/wait_completed.json?
client_id=analyst_sdk&last_event_id=1" -H "accept: application/json"

Note: These examples all use local links (127.0.0.1 IP address) to the host that is running the RAPID
Module.

VMware, Inc. 10
RAPID Module Deployment

The RAPID Module provides rapid responses to the submission of file and URLs. Its response will
indicate that the entity being analyzed is either malicious, benign, or queued for analysis.

Swagger example
Swagger (https://swagger.io/) format documentation is available at http://127.0.0.1:9090/apidocs/
#/analysis. This documentation provides details of available RAPID resources and methods, the
submission parameters, and return codes. You can also interact with the methods using the [Try it out]
button.

RAPID Monitoring Services

You can monitor the RAPID Module containers with the following application. It can be activated with
the default docker-compose.yml file.

VMware, Inc. 11
RAPID Module Deployment

Grafana Interface
http://127.0.0.1:8081/ — If enabled in the docker-compose.yml file, this endpoint hosts an optional
Grafana (https://grafana.com/) interface for visualizing/searching detailed stats reported by the
services.
Grafana requires a separate connection. The default login is admin/admin.

RAPID Environment File

The most important configuration parameters in the .env file are the following:

• LASTLINE_RAPID_API_KEY — The VMware license key to use for the service.

• LASTLINE_RAPID_API_TOKEN — The VMware license secret to use for the service.

• ANALYST_SDK_RUNNING_MODE — The running mode of the analysis service, defining which

components are available.
Available modes are:

# static-analysis — Enable only components performing local static analysis for detection.

# dynamic-analysis — Enable only components performing deep analysis on a remote VMware

NSX Network Detection and Response analysis system. This includes dynamic analysis as well
as reputation-based detection.

# static-dynamic — Enable all components described in static-analysis and dynamic-

analysis.

Important: The running mode must match the services allowed by your license otherwise the
requested service will not operate.

.env
The following is the default .env file:

# File that hold common environment variables that are shared by different docker containers

# RAPID license for running the service and communicating with the Lastline backend
LASTLINE_RAPID_API_KEY=
LASTLINE_RAPID_API_TOKEN=

# The running mode used for the service; possible values are
# - static-dynamic: Static and dynamic analysis components are enabled,

VMware, Inc. 12
RAPID Module Deployment

# - dynamic-analysis: Only dynamic analysis components are enabled,

# - static-analysis: Only static analysis components are enabled.
# For details, refer to the documentation
ANALYST_SDK_RUNNING_MODE=static-dynamic

# Specify how components interact with Lastline backend services when running in static mode.
# Note: This variable will be ignored for other modes; possible values are-
# - codehash: Upload hashes of the file contents to Lastline for analysis. If the file is known
# to Lastline then the pre-computed analysis result is used.
# - full: Upload the whole file to services for analysis
# - off: Do not interact with any hosted services
STATIC_MODE_UPLOAD_TYPE=codehash

# To ensure submitted files are deleted after analysis whenever they are uploaded to Lastline
# backend services, set confidential_analysis to True
LASTLINE_ANALYST_CONFIDENTIAL_ANALYSIS=False

# For connecting the service with a Lastline Enterprise or Detonator on-premises Manager appliance,
# use the following URL, replacing the FQDN of the installed Manager:
# LASTLINE_ANALYST_API_URL=https://log.<fqdn>/analysis

# Configure logging by the analysis daemons (e.g., via syslog). By default, the service logs to a
# local syslog service as well as to the Lastline cloud backend. Leaving the SYSLOG_TYPE value
# empty, the local reporting is disabled
SYSLOG_TYPE=local
SYSLOG_LEVEL=warning

# If syslog is configured to report locally (as specified as default above), specify the address
# where syslog data is reported to. This can be a unix socket on a dedicated reporting container,
# but could also be bound to an arbitrary service unix socket endpoint mounted into the docker
# containers (such as /dev/log on Linux or /var/run/syslog on Mac OS)
#
# NOTE: The below value is mounted into the docker containers by default. Thus, even if logging is
# disabled, configure a valid service name here, or remove the default mount in all enabled
# services docker containers (see docker-compose.yml)
SYSLOG_LOCAL_ADDRESS=/dev/log
# To log to a remote server instead (e.g., a logstash instance running in another container called
# "logstash" and using TCP port 5000), use
#
# - SYSLOG_TYPE=remote
# - SYSLOG_REMOTE_ADDRESS=logstash:5000
#
# Note that you can also influence the verbosity of the logging (at a rather steep cost of
# performance):
#
# - SYSLOG_LEVEL=debug
#
# To explicitly disable syslog, do not specify a value for this setting:
#
# - SYSLOG_TYPE=

# The image tag pulled from the Lastline repository and executed

VMware, Inc. 13
RAPID Module Deployment

IMAGE_NAME_BASE=docker-rapid.lastline.com/
IMAGE_RELEASE_TAG=rapid-release-2.3.0

# How many analysis container instances should be instantiated. This container will be
# performing analysis for all submissions in 'queued' state.
NBR_ANALYSIS_INSTANCES=2
# How many processes for running the API should be launched. Increasing this number increases
# the memory requirement of the API container but improves throughput of the API.
UWSGI_NUM_PROCESS=2

# Memory limits used for each docker container used

#
# Define how much memory we allow each container to use. If the container needs more memory
# than the limit, the container will be killed and restarted again, up to the number of
# times defined by the restart policy for the service in the docker-compose.yml file. If it
# fails more than that, the container is not restarted anymore
MEM_LIMIT_ANALYST_SDK_API=1000M
MEM_LIMIT_ANALYST_SDK_ANALYSIS_COMPLETION=1000M
MEM_LIMIT_ANALYST_SDK_HEALTH_CHECKER=200M
MEM_LIMIT_ANALYST_SDK_LLMONITORING=400M
MEM_LIMIT_ANALYST_SDK_MALSCAPE_COMPLETION=400M
MEM_LIMIT_ANALYST_SDK_MALSCAPE_RATE_LIMITER=200M
MEM_LIMIT_ANALYST_SDK_TEST=800M
MEM_LIMIT_AVBD=1200M
MEM_LIMIT_RABBITMQ=
MEM_LIMIT_MEMCACHED=1250M
MEM_LIMIT_STATSD=1000M

# Internal memory limits for applications inside a container.

# Some services are able to provide specific memory limits for their applications. These
# limits should be lower than their container memory limits
# memcached limit is a plain integer in megabytes.
MEM_LIMIT_INTERNAL_MEMCACHED=1000

RAPID Docker Compose YAML File

The docker-compose.yml file defines all of the available services for your RAPID deployment. Your
deployment may not require all the listed services.
Under normal operational circumstances this file won't need modifications. However, services that are
not required can be removed to prevent docker from starting an unnecessary container. Each service
entry in the YAML file that contains a required comment describing the context in which the service is
needed.
The following example indicates that the analyst-sdk-api service must be started in all deployment
scenarios.

services:
analyst-sdk-api:

VMware, Inc. 14
RAPID Module Deployment

# required: yes
...

The following example indicates that the avbd service only needs to be started if one of the two
"running modes" providing static analysis are enabled.
The entries for the service will also define optional containers providing advanced logging and/or
monitoring services. These are labeled as optional and are not required to be started.

services:
...
avbd:
# required: only for running mode "static-only" and "static-dynamic"
...

The docker-compose.yml file should be edited to accommodate your deployment scenario by removing
any unwanted services.
The docker-compose template and any optional components (for example, monitoring) that are
mentioned in the template file should only be considered for general guidance for launching the
analysis components inside of docker. The actual deployment environment, availability of external
monitoring systems, or scalability requirements may necessitate adjusting how the RAPID containers
and their dependencies are launched in a production environment. VMware will maintain and provide
support only for the primary, required components. You must maintain any additional and optional
components that you define to be launched.
For SSL termination, a default SSL certificate is provided by VMware. We strongly recommend you
update the docker-compose.yml file with your own certificate. To do so, uncomment the following lines
and add the path to your custom SSL certificate, replacing <path to certificate> and <path to certificate
key>.

#volumes:
# - "<path to certificate>:/etc/ssl/certs/server.crt:ro"
# - "<path to certificate key>:/etc/ssl/private/server.key:ro"

docker-compose.yml
The following is the default docker-compose.yml file:

version: "2.4"

volumes:
analyst-sdk-data:
driver: local
elasticsearch-data:
driver: local
avbd-data:
driver: local
graphite-whisper-data:
driver: local
grafana-data:

VMware, Inc. 15
RAPID Module Deployment

driver: local

networks:
analyst-sdk-network:
driver: bridge

services:
analyst-sdk-api:
# required: yes
# NOTE: This service runs as root but it drops privileges once it starts up
image: ${IMAGE_NAME_BASE}nginx:${IMAGE_RELEASE_TAG}
depends_on:
- analyst-sdk-api-backend
mem_limit: ${MEM_LIMIT_ANALYST_SDK_API}
# Uncomment the following lines and add the path to the custom SSL
# certificate to use when accessing through https.
#volumes:
# - "<path to certificate>:/etc/ssl/certs/server.crt:ro"
# - "<path to certificate key>:/etc/ssl/private/server.key:ro"
networks:
- analyst-sdk-network
ports:
- '9090:80/tcp'
- '9443:443/tcp'
env_file:
- ./.env
restart: always
analyst-sdk-api-backend:
# required: yes
image: ${IMAGE_NAME_BASE}analyst-sdk:${IMAGE_RELEASE_TAG}
depends_on:
- memcached
- rabbitmq
- analyst-sdk-analysis-completion
- llmonitoring
# remove these lines if no dynamic analysis is enabled in the "running mode"
- analyst-sdk-malscape-completion
- analyst-sdk-submission-rate-limiter
# remove this line if no static analysis is enabled in the "running mode"
- avbd
mem_limit: ${MEM_LIMIT_ANALYST_SDK_API}
volumes:
- "analyst-sdk-data:/analyst_sdk/:rw"
- "${SYSLOG_LOCAL_ADDRESS}:${SYSLOG_LOCAL_ADDRESS}:rw"
networks:
- analyst-sdk-network
env_file:
- ./.env
environment:
# What service this container is running. This is needed so the
# analyst-sdk image can know what service should be started by the
# container, as we can have different services that can be launched

VMware, Inc. 16
RAPID Module Deployment

# by the image.
- SERVICE_RUNNING=analyst-sdk-api
- SERVICE_UID=1000
restart: always
analyst-sdk-analysis-completion:
# required: yes
image: ${IMAGE_NAME_BASE}analyst-sdk:${IMAGE_RELEASE_TAG}
depends_on:
- memcached
- rabbitmq
- llmonitoring
# remove this line if no dynamic analysis is enabled in the "running mode"
- analyst-sdk-submission-rate-limiter
mem_limit: ${MEM_LIMIT_ANALYST_SDK_ANALYSIS_COMPLETION}
volumes:
- "analyst-sdk-data:/analyst_sdk/:rw"
- "${SYSLOG_LOCAL_ADDRESS}:${SYSLOG_LOCAL_ADDRESS}:rw"
networks:
- analyst-sdk-network
env_file:
- ./.env
environment:
# What service this container is running. This is needed so the
# analyst-sdk image can know what service should be started by the
# container, as we can have different services that can be launched
# by the image.
- SERVICE_RUNNING=analysis-completion
- SERVICE_UID=1000
restart: always
scale: ${NBR_ANALYSIS_INSTANCES}
analyst-sdk-health-checker:
# required: yes
image: ${IMAGE_NAME_BASE}analyst-sdk:${IMAGE_RELEASE_TAG}
depends_on:
- llmonitoring
mem_limit: ${MEM_LIMIT_ANALYST_SDK_HEALTH_CHECKER}
volumes:
- "analyst-sdk-data:/analyst_sdk/:rw"
- "${SYSLOG_LOCAL_ADDRESS}:${SYSLOG_LOCAL_ADDRESS}:rw"
networks:
- analyst-sdk-network
env_file:
- ./.env
environment:
- SERVICE_RUNNING=health-checker
- SERVICE_UID=1000
restart: always
llmonitoring:
# required: yes
image: ${IMAGE_NAME_BASE}llmonitoring-daemon:${IMAGE_RELEASE_TAG}
volumes:
- "analyst-sdk-data:/analyst_sdk/:rw"

VMware, Inc. 17
RAPID Module Deployment

- "${SYSLOG_LOCAL_ADDRESS}:${SYSLOG_LOCAL_ADDRESS}:rw"
mem_limit: ${MEM_LIMIT_ANALYST_SDK_LLMONITORING}
networks:
- analyst-sdk-network
env_file:
- ./.env
environment:
SERVICE_UID: 1000
TILLER_YAML: |
---
global:
receiver_address: tcp://0.0.0.0:11302
upload_interval_seconds: '300'
uploaders:
http:
key: '${LASTLINE_RAPID_API_KEY}'
appliance_secret_token: '${LASTLINE_RAPID_API_TOKEN}'
targets: [
{
name: 'http-analyst-sdk',
url: 'https://monitoring-rapid.lastline.com/appliance/monitoring/report_rapid'
}
]
carbon:
prefix: 'lastline.analyst_sdk'
identifier: '${LASTLINE_RAPID_API_KEY}'
targets: [
{
name: 'carbon-analyst-sdk',
host: 'statsd',
port: '2003',
}
]
...
restart: always
analyst-sdk-malscape-completion:
# required: only for running mode "dynamic-only" and "static-dynamic"
image: ${IMAGE_NAME_BASE}analyst-sdk:${IMAGE_RELEASE_TAG}
depends_on:
- rabbitmq
- llmonitoring
mem_limit: ${MEM_LIMIT_ANALYST_SDK_MALSCAPE_COMPLETION}
volumes:
- "analyst-sdk-data:/analyst_sdk/:rw"
- "${SYSLOG_LOCAL_ADDRESS}:${SYSLOG_LOCAL_ADDRESS}:rw"
networks:
- analyst-sdk-network
env_file:
- ./.env
environment:
# What service this container is running. This is needed so the
# analyst-sdk image can know what service should be started by the

VMware, Inc. 18
RAPID Module Deployment

# container, as we can have different services that can be launched

# by the image.
- SERVICE_RUNNING=malscape-completion
- SERVICE_UID=1000
restart: always
analyst-sdk-submission-rate-limiter:
# required: only for running mode "dynamic-only" and "static-dynamic"
image: ${IMAGE_NAME_BASE}analyst-sdk:${IMAGE_RELEASE_TAG}
depends_on:
- llmonitoring
mem_limit: ${MEM_LIMIT_ANALYST_SDK_MALSCAPE_RATE_LIMITER}
volumes:
- "${SYSLOG_LOCAL_ADDRESS}:${SYSLOG_LOCAL_ADDRESS}:rw"
networks:
- analyst-sdk-network
env_file:
- ./.env
environment:
# What service this container is running. This is needed so the
# analyst-sdk image can know what service should be started by the
# container, as we can have different services that can be launched
# by the image.
- SERVICE_RUNNING=submission-rate-limiter
- SERVICE_UID=1000
restart: always
avbd:
# required: only for running mode "static-only" and "static-dynamic"
image: ${IMAGE_NAME_BASE}avbd-scan:${IMAGE_RELEASE_TAG}
networks:
- analyst-sdk-network
mem_limit: ${MEM_LIMIT_AVBD}
env_file:
- ./.env
volumes:
- "avbd-data:/var/lib/avbd/:rw"
environment:
- UPDATE_AUTH=${LASTLINE_RAPID_API_KEY}:${LASTLINE_RAPID_API_TOKEN}
- SERVICE_UID=1000
- USER_GROUP_ID=1000:1000
restart: always
rabbitmq:
# required: yes
# NOTE: This service runs as root but it drops privileges once it starts up
image: ${IMAGE_NAME_BASE}rabbitmq:${IMAGE_RELEASE_TAG}
mem_limit: ${MEM_LIMIT_RABBITMQ}
networks:
- analyst-sdk-network
env_file:
- ./.env
environment:
- RABBITMQ_DEFAULT_USER=analyst-sdk
- RABBITMQ_DEFAULT_PASS=analyst-sdk

VMware, Inc. 19
RAPID Module Deployment

- RABBITMQ_DEFAULT_VHOST=analyst-sdk
restart: always
memcached:
# required: yes
image: ${IMAGE_NAME_BASE}memcached:${IMAGE_RELEASE_TAG}
mem_limit: ${MEM_LIMIT_MEMCACHED}
user: '1000'
networks:
- analyst-sdk-network
env_file:
- ./.env
entrypoint:
- memcached
- -m ${MEM_LIMIT_INTERNAL_MEMCACHED}
restart: always
statsd:
# required: no, optional monitoring component for graphite metrics.
# NOTE: This service runs as root since the statsd components need to access some paths
# in the container that need root access.
image: ${IMAGE_NAME_BASE}analyst-sdk-statsd:${IMAGE_RELEASE_TAG}
mem_limit: ${MEM_LIMIT_STATSD}
networks:
- analyst-sdk-network
ports:
- '8081:80/tcp'
volumes:
- "graphite-whisper-data:/opt/graphite/storage/whisper"
- "grafana-data:/opt/grafana/data"
env_file:
- ./.env
restart: always

Release History
Release Description
RAPID-2.3 Added the Lastline Analyst API hash query to analysis pipeline for codehash static mode
analysis. Allowed password candidates to be sent with URL submissions. Added a rate
limiting feature to manage submission rate to the Lastline backend when required.
RAPID-2.2 Improved codehash static mode analysis. Improved full analysis of archive file
submissions. Fixed errors arising from the inclusion of Unicode characters in password
candidates submission metadata.
RAPID-2.1 Added 'codehash' static file backend upload mode. Added archive handling for prefilter
submissions. Changed docker-compose.yml to use TCP sockets for llmonitoring.
Added additional metadata for file submissions.
RAPID-2.0 Initial release version

VMware, Inc. 20
RAPID Module Deployment

VMware, Inc.
3401 Hillview Ave.
Palo Alto, CA 94304
www.vmware.com

Copyright © 2023 VMware, Inc. All rights reserved. Copyright and trademark information.

VMware, Inc. 21