Restricted

Enterprise File Broker

Design
[Project code] [Enter Project Name Here]

Version [Enter document version number here]

Version Status [Draft]
Author: [Enter author name here]
Based on EFB design template: v1.20
Owner of EFB design template: Vincent Loach <vincent.loach@bt.com>

SCOPE
This document describes the detailed configuration design for the EFB transport.

PURPOSE
The purpose of this document is to provide a view of the configuration required
across the multiple touch points involved to enable the implementation of file(s)
transportation.
The solution described in this document meets the requirements specified by the
Project. The level of detail in this document will allow the various areas involved
including respective teams to base their firewall rules, user account creation and
the implementation of the configuration into Test and Production.
It is vitally important that this document is reviewed by everyone attached to the
project. The final version of this document defines what will be implemented into
live. If anything in this document does not match what is actually required, it will
cause significant delays to your deployment, as proper Change Management
processes will be applied to any subsequent changes. If the EFB operations team
cannot deploy your transports live during the allocated change window, you may
well be required to fund an additional Work Request to make those changes at a
later date.

1
 Restricted

Template Amendment History

Date of
Version Reason of change Author Issue
Ameen
0.1 Amendments made after pilot run 05/2013
Mohammed
1. Numbering value added to section headings
2. Draft emailing list removed
Ameen
0.2 3. Public key updated 5/2013
Mohammed
4. Document history grid added
5. Internet facing address added
1. Section 1 – overview – describe the file content data type Ameen 09/09/20
0.3
being transferred per transport Mohammed 13
1. Diagrams incorporated with EFB components
2. organised the control tables
3. included two new variations for EFB transport on flow 4 and Ameen 31/10/20
0.4
5 Mohammed 13
4. user creation section 8 rendered to a template for raising
account by the designer.
Ameen 29/11/20
0.5 - User creation (unix )caveat
Mohammed 13
- CMO DMZ host removed ( FMO DMZ is now operational).
Ameen 23/01/20
0.6 Section 8 – account creation details updated, tailored for the
Mohammed 14
FMO environment.
Ameen 08/05/20
0.7 - Test Environment details updated to reflect FMO
Mohammed 14
Document History moved to the top of the template.
Ameen 12/11/20
0.8 Caveats added to section 5 – Risk and assumptions
Mohammed 14
Section 9 - User account creation template updated
Cleaned up text.
2016-
1.0 Added all relevant public keys to EFB Public Keys section. Vincent Loach
10-07
Changed management details.
Added notes about directories and overwrites to Risks and 2016-
1.1 Vincent Loach
Assumptions. 12-14
Added new internal EFB “MSG” host IP address.
2017-
1.2 Added column to indicate which internal EFB “MSG” server to Vincent Loach
02-08
deploy MSG config files to.
Added BT formatting; Emphasis on Encryption; Process added.
Jared Watson 2018-
1.3 Added more detailed test system IP addresses. Appendix B
Vincent Loach 01-22
added.
Cleaned up Style usage so that Table of Contents makes more 2018-
1.4 Vincent Loach
sense. 03-06
Added new BT datacentre IP addresses and public key.
2018-
1.5 Updated directory quantity limit notes to cover archive Vincent Loach
09-19
directories.
Updated with new Denali BT server details. 2018-
1.6 Vincent Loach
Added transition paragraph to “Risk and Assumptions”. 11-28
Added note about new umask (027). 2019-
1.7 Vincent Loach
Added line to make explicit we are using SFTP. 01-04
BT Font embedded. Removed all references to T-Systems/FMO
in live.
2019-
1.8 Removed EFB PGP encryption keys as encryption should now be Vincent Loach
03-18
end-to-end.
Numerous small tweaks and fixes.
Added text to bring attention to using efb-internalhub1 for EFB 2019-
1.8.1 Vincent Loach
drops. 06-05
2019-
1.9 Updated with new Denali BT test server details Mohit Raj
08-02
2020-
1.10 Updated structure of “4 File Details” post collection actions. Vincent Loach
02-06
1.11 Added Glossary. Increased red text. Updated new BT logo. Arun Gowda 2020-
Removed explicit BT references. Vincent 03-02

2
 Loach
Added note about annual review and possible deactivation. 2020-
1.12 Vincent Loach
Cleaned up headers and footers. 03-12
2020-
1.13 Added optional support email address table. Vincent Loach
04-09
2020-
1.14 Removed optionality of long term support information. Vincent Loach
10-15
Updated template to conform to document guidelines. Added:
2020-
1.15 more text about file permissions; more front page text; PGP Vincent Loach
11-12
public key section.
Added note on possible additional WR if design does not work as
2021-
1.16 written. Added test system public keys. Added optional text for Vincent Loach
04-29
when LTS details are only of people.
2021-
1.17 Clarified language in a few places in Appendix B. Vincent Loach
07-22
Added “Error Handling” and “SPLUNK logs” section to end of 2021-
1.18 Vincent Loach
Appendix A. 09-14
Added elements from GCP template to create single unified
template document. 2022-
1.19 Vincent Loach
Expanded descriptive text for test server FQDNs and IP 04-28
addresses.
Cleaned up “Emailing and Sign Off List”. Removed Appendix A 2022-
1.20 Vincent Loach
(for the moment). 10-17

3
 Document History
Issued Date Details

0.1 2020-01-06 Draft

0.2 Review

0.3 Approval

Related Documentation Version Date

EFB Request Forms

Emailing and Sign Off List

Role Email Address Approval/Sign Off
Solutions Architect
Project Manager
Shadow Designer
Server DA System 1
Server DA System 2
Server DA System 3
EFB implementation team BTEFBSupport@cognizant.c
Only required for EE EFB
om
EFB-GCP implementation team DEDSDev@TechMahind Only required for GCP EFB
ra.com
EFB-GCP Support team deds.support@bt.com Only required for GCP EFB
IIP Platform Lead Designer ian.3.clark@bt.com Only required for GCP EFB

(Add/delete Server DAs as required, one per system EFB will interact with.) (Select
implementation team depending on EFB instance.)
Long term support contacts
System Email Address (and/or NG Autofix queue if internal)
System 1
System 2
System 3

4
 Restricted

Table of Contents
1. Overview.......................................................................................................................... 6
2. EFB Transport Flow........................................................................................................ 6
Flow 1 – $Source to $Dest – Internal Only..........................................................................6
Flow 2 – $Source to $Dest – External to Internal................................................................6
Flow 3 – $Source to $Dest – Internal to External................................................................6
Flow 4 - $Source to $Dest – Drop........................................................................................7
Flow 5 – $Source to $Dest – Pickup....................................................................................7
Flow 6 – BTW SDEDS to GCP – EFB acts as Client to Pull from BTW SDEDS................7
3. Environments.................................................................................................................. 8
Production Source/Target and EE EFB Details................................................................10
Production Source/Target and EFB Details......................................................................11
Flow 1.....................................................................................................................................12
Flow 2.....................................................................................................................................12
Flow 3.....................................................................................................................................13
Flow 4.....................................................................................................................................13
Flow 5.....................................................................................................................................14
Flow 6.....................................................................................................................................14
Test Source/Target and Test EFB Details.........................................................................15
Flow 1.....................................................................................................................................15
4. File Details..................................................................................................................... 16
5. Risk and Assumptions................................................................................................. 17
6. Encryption of data – Ensure you have read this section...........................................18
7. Configuration Attachments.......................................................................................... 18
8. EFB Public Keys........................................................................................................... 20
9. User account creation template...................................................................................21
10. Glossary........................................................................................................................ 22
Appendix A : Process Overview........................................................................................ 23
Appendix B – Guide to how EFB operates........................................................................24
For all connections to your server.................................................................................24
Files from BT for Third Party.......................................................................................... 24
Files for BT from Third Party.......................................................................................... 25
Pull by BT from Third Party...................................................................................................25
Push to BT by Third Party......................................................................................................26
Overwrites and Uniqueness........................................................................................... 27
Runtime and Delivery Order........................................................................................... 27
Error Handling................................................................................................................. 27
SPLUNK logs................................................................................................................... 28

5
 Restricted

1. Overview
Select one initial text paragraph from two below, depending on EFB instance:

This document explains how EFB will use the Secure File Transfer Protocol (SFTP) to transfer files between systems.

This document explains how EFB will use SFTP to receive the <whatever> files from <whatever> server and use GCP
protocol to upload the <whatever> files to Google Cloud. The Secure File Transfer Protocol (SFTP) will be mainly used to
transfer files between other systems.

Change this in the unlikely event we are using FTPS. This line makes explicit that SFTP is being used, as a small number of
third parties have become ‘confused’ about this.

Text in Red serves as guidance to completion. Please delete before issuing the document.

Text in black should form part of your document. Please do not delete.

Give a brief business background of the project/EFB request concerning the EFB Impact. Please also state the transports
required (Describing the type of data being transferred. I.e CDRs, Provisioning requests, Queries, Credit Card details etc. ) .

2. EFB Transport Flow

The below diagrams are a few sample flows of transports. Illustrate you EFB transport diagrammatically, insert, delete and
Edit as appropriate.

The below is the high level visual representation of the Transports requirements as part of this
project.
Flow 1 – $Source to $Dest – Internal Only

EFB

$Source Internal $Dest

HUB

Pull Push
Flow 2 – $Source to $Dest – External to Internal

EFB

$Source Internal $Dest

DMZ
HUB
Pull

Pull Push
Flow 3 – $Source to $Dest – Internal to External

EFB

$Source Internal $Dest

HUB DMZ

Push
6
 Restricted

Pull Push

Flow 4 - $Source to $Dest – Drop

EFB

$Source DMZ Internal $Dest

HUB
Pull

Drop

$Source will drop the files on to DMZ

Flow 5 – $Source to $Dest – Pickup

EFB

$Source Internal $Dest

HUB DMZ

Push

Pull Pickup

$Source will pick up the files from the DMZ.

Flow 6 – BTW SDEDS to GCP – EFB acts as Client to Pull from BTW SDEDS

$Source EFB

(Greenside / Redside)

SFTP Pull GCP Push

7
 Restricted

3. Environments
Delete and Edit details as appropriate

The information provided in this section is relevant to the interested parties below. The details
provided in this section will allow the individual areas to do the necessary work for allowing
connectivity and accessibility.
 Source Server DA (Also see EFB public key section)
 Target Server DA (Also see EFB public key section)
 Firewall Team (if any new IP addresses are formed then the EFB design team will need to be
notified)
 Integration Testing (informational)
 EFB operations (informational)
Please note that any Unix user account creation must be submitted via the I.T shop prior to
submitting this design. The order ID reference generated when requesting accounts via the I.T shop
will need to be captured in section 8 of this document (User account creation section).

Summary of actions – REQUESTER PLEASE READ THIS

1. Firewall teams to implement firewalls as per design flow 1, flow 2, flow 3, flow 4 and flow
5. This also includes firewall opening for flow 1.
Any new NAT’ed addresses wil need to be communicated back to the EFB design team
2. Design authorities for the external server will need to allow for the DMZ EFB to connect to
their server to allow the pull activity to occur.

Firewall rules need to be opened on the following flows

Production

Flow Firewall

Flow 1 The firewall is required / is not required

Flow 2 The firewall is required / is not required

Flow 3 The firewall is required / is not required

Flow 4 The firewall is required / is not required

Flow 5 The firewall is required / is not required

Testing

Flow Firewall

Flow 1 The firewall is required / is not required

8
 Restricted

Flow 2 The firewall is required / is not required

Flow 3 The firewall is required / is not required

Flow 4 The firewall is required / is not required

Flow 5 The firewall is required / is not required

9
 Restricted

Select EFB type from two options below. Remove other.

Production Source/Target and EE EFB Details
EFB
MSG HUB DMZ
$MSG- $DMZ- 213.121.34.20
NAT- Internet- 213.121.34.21
n/a
Facing- Facing- 212.140.88.96
address address 212.140.88.97
10.45.16.20 10.45.72.31
$MSG-IP- 10.45.16.21 $DMZ-IP- 10.45.72.32
Address 10.45.48.11 Address 10.45.76.35
10.45.48.12 10.45.76.36
blp13782001.eezone.bt.c
om
blp13782002.eezone.bt.c blp13782009.eezone.bt.com
$MSG- om blp13782010.eezone.bt.com
$DMZ-Host
Host blp13782005.eezone.bt.c blp13782011.eezone.bt.com
om blp13782012.eezone.bt.com
blp13782006.eezone.bt.c
om

Any external system connecting to the EFB DMZ should use FQDN: efb.bt.com
Any internal system connecting to the EFB DMZ should use FQDN: efb-dmzhub.eezone.bt.com
Any internal system connecting to EFB MSG should use the FQDN: efb-internalhub.eezone.bt.com
These FQDNs will then route traffic to the currently live host.
Any external third party should use all four $DMZ-Internet-Facing-address for their own firewall
modifications. EFB uses multiple DMZ hosts for load balancing and disaster recovery. The FQDN
efb.bt.com will resolve to the current live IP address in the global DNS. It has a low TTL (10 minutes) to
allow for Disaster Recovery usage.
For establishing the firewall connectivity between EFB and other internal BT servers, internal servers
need to add the below EFB SDIP aliases into their interconnects. Once the SDIP interconnects are
approved at internal server end, the same needs to be communicated back to EFB Design team - then we
can raise the Fireflow request to open the firewalls from our MSG server to internal BT server.

EFB SDIP’s:
ee_d_efb_prod_app_app13782
ee_d_efb_dr_app_app13782

10
 Restricted

Production Source/Target and EFB Details

EFB -GCP

EFB-GCP - 193.113.10.71
Internet-Facing-
address 193.113.11.167

EFB-GCP - IP- 10.54.39.29

Address 10.52.39.27

EFB-GCP Host (
btwholesalebigdata.sdeds.bt.com
Internet Facing)

sdeds.btwholesalebigdata.intra.bt.com
EFB-GCP Host

Any external system connecting to the EFB should use: btwholesalebigdata.sdeds.bt.com

Any internal system connecting to the EFB should use: sdeds.btwholesalebigdata.intra.bt.com

These FQDNs will then route traffic to the currently live host.
Any external third party should use all EFB-GCP-Internet-Facing-address for their own firewall
modifications. EFB uses multiple hosts for load balancing and disaster recovery. The FQDN
btwholesalebigdata.sdeds.bt.com will resolve to the current live IP address in the global DNS.
For establishing the firewall connectivity between EFB and other internal BT servers, internal servers
need to add the below EFB SDIP aliases into their interconnects. Once the SDIP interconnects are
approved at internal server end, then they need to raise the Fireflow request to open the firewalls from
our EFB server to internal BT server. The same needs to be communicated back to EFB Design team.

EFB SDIP’s:
sdeds_btw_ec_bigdata_servers_app10765

11
 Restricted

Flow 1
EFB
Source Destination
MSG HUB
NAT
<data $MSG-NAT- <data
IP Address Facing IP Address
here> Facing-address here>
address
Local IP $MSG-IP-
<data <data
Host Address Address Host
here> here>
$Source $Dest
<data <data
User ID User ID
here> here>
Host
$MSG-Host <data
Directory
<data here>
Directory
here> <data
Temp Dir
here>
Comments This is an internal transfer, so no need for DMZ.

Flow 2
EFB
Source DMZ MSG HUB Destination
$DMZ-
Internet Internet-
IP Address <data here> n/a IP Address <data here>
Facing Facing-
address
$DMZ-IP- $MSG-IP-
IP Address
$Source Host <data here> Address Address $Dest Host <data here>

Host $DMZ-Host $MSG-Host

User ID <data here> User ID <data here>
Directory <data here>
NAT Facing
Directory <data here> N/A N/A
address Temp Dir <data here>

Comments [ Insert comment here ]

12
 Restricted

Flow 3
EFB
MSG DMZ
Source HUB Destination
$DMZ-
Internet
IP Address <data here> N/A Internet- IP Address <data here>
Facing
Facing-address
$MSG-IP- $DMZ-IP-
IP Address
Host <data here> Address Address Host <data here>
$Source $Dest
Host $MSG-Host $DMZ Host Pickup DMZ
User ID <data here> <data here>
User ID
NAT Facing Pickup DMZ
Directory <data here> N/A <data here>
address Directory

Comments [ Insert comment here ]

Flow 4
EFB
Source DMZ MSG HUB Destination
$DMZ-
<data Internet Internet-
IP Address n/a IP Address <data here>
here> Facing Facing-
address
$DMZ-IP- $MSG-IP-
<data IP Address
Host Address Address Host <data here>
here>
$Source $Dest
Drop DMZ <data Host $DMZ-Host $MSG-Host
User ID <data here>
User ID here>
Drop DMZ <data
Temp-Directory <data here>
Directory here> NAT
Drop DMZ Facing N/A N/A
temp address Directory <data here>
Directory

Comments [ Insert comment here ]

13
 Restricted

Flow 5
EFB
MSG DMZ
Source HUB Destination
$DMZ-
Internet
IP Address <data here> N/A Internet- IP Address <data here>
Facing
Facing-address
$MSG-IP- $DMZ-IP-
IP Address
Host <data here> Address Address Host <data here>
$Source $Dest
Host $MSG-Host $DMZ Host Pickup DMZ
User ID <data here> <data here>
User ID
NAT Facing Pickup DMZ
Directory <data here> N/A <data here>
address Directory

Comments [ Insert comment here ]

Flow 6
EFB
Source Destination

IP 10.54.39.29 Local IP <data

URL
Address 10.52.39.27 Address here>

sdeds.btwholesalebig <data
Host data.intra.bt.com Username
here>
$Source $Dest <data
User ID gsftpgcp1 Bucket name
Host here>
<data
Directory
Director /DLM/out here>
y Temp Dir (if <data
any) here>
Comment
.
s

14
 Restricted

Test Source/Target and Test EFB Details

This contains the EE EFB test server details.
Please remove the respective TEST DMZ columns as per your design. This is going to be one or the
other. Third Party initiating or EFB initiating.
Briefly describe the reasons for test and what needs to be established through testing.
EFB TEST
Test MSG HUB Test DMZ
BF1: 193.113.10.244
$DMZ-Internet- BF2: 193.113.10.248
Facing-address (Third parties will need to allow this
(EFB initiating) IP address through their firewall for
connections from EFB test DMZ.)
BF1: 193.113.10.212
$DMZ-Internet-
BF2: 193.113.10.220
Facing-address
(Third parties will need to connect to
(Third party
this IP address when interacting with
initiating)
EFB test DMZ.
BF1 - 10.45.21.79 $DMZ-IP- BF1 - 10.45.26.39
$MSG-IP-Address
BF2 - 10.45.21.80 address BF2 - 10.45.26.50

BF1 -
blt13782005.eezone.bt.com $DMZ- HOST BF1 - blt13825001.eezone.bt.com
$MSG-Host
BF2 - BF2 - blt13825019.eezone.bt.com
blt13782006.eezone.bt.com

Any external system connecting to the test EFB DMZ should connect to the IP addresses listed under $DMZ-
Internet-Facing-address (Third party initiating).
Any external system expecting connections from the test EFB DMZ should expect connections from the IP
addresses listed under $DMZ-Internet-Facing-address (EFB initiating).
Any internal system connecting to test EFB MSG should use the $MSG-Host for BF1.
There are no global DNS entries for the test EFB DMZ. The “ eezone.bt.com ” addresses are for internal use within
BT.
There are two test environments for EFB now - BF1 (005efbrn1t) and BF2 (005efbrn2t).
At the moment we will be using only BF1 “005efbrn1t” for testing as default username.

Flow 1
EFB
Source DMZ MSG HUB Destination
External $DMZ- Internal
Internet Internet-
IP Address <data here> n/a IP Address <data here>
Facing Facing-
address
Host <data here> $DMZ-IP- $MSG-IP- Host <data here>
IP Address
Address Address
Host $DMZ-Host $MSG-Host

15
 Restricted

User ID <data here> User ID <data here>

Temp Dir <data here> Temp Dir <data here>
Directory <data here> Directory <data here>

Comments

16
 Restricted

4. File Details

File sizes and

Transport Filenames quantities Schedule Polling Post collection action

Flow 1
$Source to Delete
$Dest

Flow 2
$Source to Rename with default extension .pro
$Dest

Flow 3
$Source to Rename to X_%{FF}
$Dest

Flow 4 Archive to
/c1/exbvar/o2o/ftp/xfrefb/archive
$Source to with default filename
$Dest %{FF}.%{YY}%{MM}%{DD}

Flow 5 Archive to
/c1/exbvar/o2o/ftp/xfrefb/archive
$Source to with filename
$Dest %{FF}.delivered

Where a filename is modified on rename/archive, the following parameters are supported:

%{FF} the filename
%{DD} the day
%{MM} the month (as digits)
%{YY} the year (2 digit) eg. 19
%{CC} the century (2 digit) eg. 20
%{RANDOM} random integer between 1 and 32767, inclusive
GCP EFB only below
In case of CFT Push, use the below table.

File sizes and

Transport Filenames quantities Schedule Polling Run Remote

Flow 1
$Source to NA NA NA NA NA
$Dest

17
 Restricted

5. Risk and Assumptions

Highlight any risks you may feel there are to a particular EFB transport.

I.e. The file size being transported is at the upper limit of the EFB transport capability. Although EFB
should be able to manage this, it should be tested alongside the current production EFB transport to
ensure there is no impact to resource and existing EFB jobs being transported at the same schedule.

 DAs for servers EFB will deliver into to ensure that delivery directories and final directories all exist.
EFB or the Ops team that install EFB transports in live or test at BT will not create missing
directories on remote servers. Delivery directories and final directories should be on the same file
system such that the SFTP rename command is an atomic file system action (as per POSIX).
 DAs for all remote servers to ensure that user accounts specified exist on remote servers, and that
the public key enclosed in this document is authorized for that user. The user will need suitable
access to all remote directories to read, write, modify, move and delete files. If EFB is to pull a file
from a remote server it will need suitable read, write, modify, move and delete (if applicable)
privileges at both the file and directory level.
 EFB expects files in source directories to be ready for transport as soon as they are visible (i.e. a file
that matches “Filename” in the above table appears). DAs to ensure that their systems only place
completed files in these directories. Two mechanisms are clearly available:
1) They may use temporary files with names that do not match “Filename” in the above table, and
move the temporary file to a matching “Filename” when it is ready for transportation.
2) They may use the “temp” delivery directory one level below the final directory and move the file
up one level when it is complete.
 EFB will maintain file privileges end-to-end, as far as is possible. DAs for source and remote servers
to ensure that file privileges make sense end-to-end. The umask of the remote server should not
constrain the uploaded files unnecessarily. The umask of the internal EFB service is now 027. It is
a BT security idiom that - as standard - “other” should not have any access to files. It is therefore
very likely that (in Unix terms) files will be uploaded with a permission set of 640 (or “rw-r-----” if
you prefer).
It is possible for EFB to issue a “chmod” command to the remote server if something like 664 or
666 permissions are preferred and this is requested. (This requires the remote SFTP
implementation to service the chmod command correctly!). This needs requesting during the
design process.
The remote user that EFB uploads files as should possess sensible group membership to allow file
processing to continue in an optimal manner.
 The final destination directory for files will need to be kept to a reasonable size. File deliveries slow
down significantly if the destination directory contains many files (e.g the number of files in the
destination directory is measured in thousands.) It is recommended that the delivery directory is
just that: a place where files are put for moving onwards to the actual live system that consumes
them. It is recommended that files are not left in the delivery directory permanently. The same file
quantity considerations apply to any archive directory EFB is required to use. The archive directory
where EFB places files should be regularly cleared down to ensure it does not contain thousands
and thousands of files.
 The default state for EFB is to assume that files should be unique, and that attempting to overwrite
files is an error condition. This can be changed if required.
 If files are to be archived, and the archived files are persistent in the archive, but a file could be
presented more than once in the source directory, EFB is built to not overwrite the previously
archived file. This is considered an error. If you think a file could be presented more than once at

18
 Restricted

source (and is also to be archived) we recommend the addition of the date and a random number to
the archive filename, %{CC}%{YY}%{MM}%{DD}_%{RANDOM}_%{FF} or similar, to prevent this
error.
 At the start of each new calendar year, the EFB design team will review aggregate transport
logging for the previous year. Any transport that has processed zero files in the entire previous
calendar year will be deactivated. If any transport in this design may ‘hibernate’ for an entire
calendar year and encounter zero files to move, then that must be clearly mentioned in the
“Comments” field for the transport in this design document.
 If the long term support contact(s) in this design are for a specific person or persons at an
organisation, rather than a generic support team contact (e.g. support email address, helpdesk, or
NG Autofix queue) then this is a risk. As EFB transports often run for years, it is always possible for
a person to leave an organisation, giving us no way of contacting that organisation if there are
problems interacting with their server. The EFB operations team reserve the right to temporarily
deactivate any transport where our contact details for that server are no longer valid. We will then
wait for the owners of the server to contact us to diagnose the problem.
 EFB does not keep any archived copies of any files it successfully transfers to the final destination.
After a file has been successfully delivered, it is deleted from the EFB servers. The EFB servers do
not have large amounts of disk space for archives of past files. Do not request redeliveries from the
EFB team – this is simply not possible. (Files that have failed to be delivered are kept until deliveries
can be reattempted.)
6. Encryption of data – Ensure you have read this section
 Data protection and security is a key part of the role that we all have across our company. We all
have our part to play in this and here is where you come in. Any data we transmit must be done in
line with our policies and guidelines. We appreciate the vast majority of BT employees act
professionally and in line with BT’s Values but if you behave in a way that’s inconsistent with these
policies or The Way We Work, BT may take disciplinary action depending on local legislation and
regulation.  
https://intra.bt.com/bt/security/policy/everyone/secpol4/Pages/index.aspx
 All data leaving our company and arriving from a third party must be encrypted
 The project or requestor must have ensured that data is assessed against the data policy and
security guidelines of BT.
 It is the responsibility of the requester working with Security to ensure that the correct level of
encryption is undertaken
 Data should be encrypted between source and destination and EFB should not be used in any case
where this is available or could be made available.
Data Specifications (GCP EFB only)

Data Classification Data Description

7. Configuration Attachments
EFB operations can implement the attached configuration scripts into production.

19
 Restricted

Please add the configuration filename as greyed out readable text in the design document. This
allows it to be found by the SharePoint search capability. The configuration file itself – as an
enclosed object – cannot be found by name . This text assists greatly if/when we have to locate a
design document years down the line. Delete this paragraph in the distributed design.
Efb-internalhub2 is the preferred location for simple pullpush transports. IF YOUR TRANSPORT
INVOLVES ANOTHER SYSTEM PUSHING FILES TO EFB THEN IT MUST GO ON efb-internalhub 1
INSTEAD, AS THAT IS WHERE efb-internalhub.eezone.bt.com RESOLVES TO!

Internal Server for

EFB Tracker MSG config
Transport Configuration File Reference Number deployment

Flow 1 CONFIG_FILENAME_SEARCH_FODDER
efb-internalhub2
$Source to $Dest <attach configuration file >

Flow 2 CONFIG_FILENAME_SEARCH_FODDER
efb-internalhub2
$Source to $Dest <attach configuration file >

Flow 3 CONFIG_FILENAME_SEARCH_FODDER
efb-internalhub2
$Source to $Dest <attach configuration file >

Flow 4 CONFIG_FILENAME_SEARCH_FODDER
efb-internalhub2
$Source to $Dest <attach configuration file >

Flow 5 CONFIG_FILENAME_SEARCH_FODDER
efb-internalhub2
$Source to $Dest <attach configuration file >

20
 Restricted

8. EFB Public Keys

DAs for all production servers to ensure this standard SSH/SFTP EFB public key is authorised for the
user EFB will be connecting as:

Production EFB Public SSH/SFTP Key

Optional: if test transports are being implemented: When test transports are used, DAs for test
servers to ensure these SSH/SFTP EFB public keys are authorised for the user EFB will be
connecting as:

Test EFB Public SSH/SFTP Key

External connection DMZ key Internal connection MSG key

For
historical reasons, test EFB uses different public keys for internal and external connections.
Optional: some third parties may be getting/putting files on the production DMZ. EFB Operations
team to ensure that this public key is authorised for the user third party is connecting in as
(005efbxxxxx).

Production THIRD PARTY Public SSH/SFTP Key

<insert third party SSH/SFTP public key here>

Optional: some third parties may be getting/putting files on the test DMZ. EFB test team to ensure
that this public key is authorised for the user third party is connecting in as (005efbxxxxx).

Test THIRD PARTY Public SSH/SFTP Key

<insert third party SSH/SFTP public key here>

Optional: if PGP encryption is being used. This is a nice-to-have, and should only delay document
completion and release if the project agrees that they want this to happen.
As a service we will gather all relevant PGP public keys into the EFB design document, where readily
available.

PGP Public Keys

<insert other system’s PGP public key here>

GCP KeyFile
This GCP service account keyfile path to be used before connecting to the mentioned Bucket:
Remove this if not GCP EFB.

21
 Restricted

/path/to/gcp/private/keyfile

9. User account creation template

To be updated for BT methodology. Remove this section completely if no new user is required.

22
 Restricted

10. Glossary

Design Authority. A person expected be thoroughly knowledgeable about a system,

and to be able to fulfil certain duties:
i) They can sanity check all EFB design aspects for suitability and provide sign-
off of the design..
ii) They can organise the creation of things such as users and directories
DA iii) They can organise for firewalls to be open so that EFB can connect to their
system.
iv) They can organise for the EFB public key to be authorised for any user EFB
will connect inbound as.
They can do these tasks themselves or (more likely) know which operational team they
should assign to the task.

From DeMilitarized Zone. Servers placed in network zone that can connect to the
internet, and be connected to from the internet. At BT, they cannot then connect
inbound into the internal network. To get/put files from/to them, connections must
DMZ originate from an internal host. EFB has several servers in the DMZ that provide
interfaces to the internet.
Any external systems which need connectivity using EFB will use the EFB DMZ servers.
You can check the flow diagram in section 2 for more information.

DNS Domain Name Server. Translates FQDNs to IP addresses.

Fully Qualified Domain Name. The name of a host with all elements defined so there is
FQDN no ambiguity. The hostname suffolk is ambiguous, but the FQDNs suffolk.orange.co.uk
and suffolk.uk.tmo are not.

IP Internet Protocol.

From Messaging hub. An early version of EFB ran on servers known as “messaging
hubs”. Although that is not really the case anymore, and “internal hub” is arguably a
more technically correct description, use of MSG persists. It also provides a nice three
MSG letter initialism that works well with the initialism DMZ.
Any Internal systems to BT which needs to connect to EFB should use EFB MSG server
to connect hence named as EFB MSG. You can check the flow diagram in section 2 for
more information.

NAT Network Address Translation.

POSIX Portable Operating System Interface.

SDIP Secured Domain Interconnect Policy.

Time To Live. How long a server’s FQDN maps to its IP address in the global DNS
TTL
system.

23
 Restricted

Appendix A – Guide to how EFB operates

This is mostly for the benefit of external third parties who may not have had interactions with EFB before.
However, most of it also applies to internal BT systems that are now interacting with EFB (although the
servers interacted with will obviously be the internal EFB servers).
For all connections to your server.

For SFTP
You tell us the SFTP hostname (or IP address) to use over the internet. You provide is with the username to
login with. We provide you with the public key counterpart to the private key we will use. You add this
public key to the authorized keys of the username we will login with.
We tell you the IP addresses our connection will appear to come from (see Internet-Facing-address above)
so you can open a hole in your firewall (if necessary) to allow our incoming connection.
You tell us the directory to use. Obviously the directory needs to exist and the user we will login as needs to
have sensible access privileges to this directory (see below for more on this). We will use the SFTP “cd”
command to change into the directory path provided so what is given to us should be a sensible path we
can use with a “cd” command invocation. Due to a small code deficiency we either need a full absolute path
(starts with “/”) or we can also get or put files just from the directory we are placed in when we login (a
path of “.”). We cannot use paths that start with “~”.
It is recommended that the directory path has no spaces or unusual characters in it. Upper and lower case
letters, numbers, underscores, dots and hyphens are recommended as the characters to use.
We agree on the filenames to use. Again, it is recommended that the filenames do not contain spaces or
other unusual characters. Upper and lower case letters, numbers, underscores, dots and hyphens are
recommended as the characters to use.
Our system then initiates an SFTP connections, logs in and moves to the relevant directory.

For GCP
You need to share the google project that the destination bucket sits under. Also the bucket name with the
paths where the files needs to be stored by EFB.
Google Project should have the specified bucket and path so the using gsutil tool we could connect to your
bucket and place all the files.
Files from BT for Third Party
Files are uploaded with an extension consisting of “.inprogress” followed by a 13 to 15 digit number (Unix
epoch time in seconds and the Unix process ID for those that are interested.) The file is then renamed to the
final filename without this extensions at the conclusion of the upload.
You can provide us with a temporary directory to upload into. Files are still uploaded with the above
extension, then moved to the final destination directory at the conclusion of the upload without the
extension.
Depending on how your software is setup to monitor the delivery directory one of these two methods may
serve you better. We default to the first unless we hear otherwise.
Obviously we need reasonable write access to the remote directory to write files there.
When EFB uploads a file to your server the permissions of that file are controlled by your SFTP server. [On
Unix, by default, it will inherit the “umask” (the file permissions mask) of the root user.] Generally, default
permissions will very probably look something like this (in Unix terms) for an uploaded file:

24
 Restricted

-rw-r----- 1 efbuser efbgroup 15474 Feb 7 2014 test.txt

That is, only the “efbuser” has full read/write privileges on the uploaded file. You will need to ensure that
whatever permissions the user EFB logs in as inherits are suitable for what you intend to do with the files
after upload. Obviously, “efbuser” and “efbgroup” are examples of user name and group name.
It is possible for us to configure EFB to try to change permissions and groups of files after upload, but this
relies on your SFTP server behaving in a “sensible” fashion and implementing all parts of the SFTP protocol
correctly. If it’s a Unix host using OpenSSH to implement SFTP then things will probably work fine. If it’s a
Windows machine implementing SFTP then these directives are often poorly supported, if at all. One
particular Windows SFTP server implementation (WS_FTP) is known for completely not handling permission
modifications.

Push untouched source

Normally files are uploaded with an extension consisting of “.inprogress” followed by a 13 to 15 digit
number (Unix epoch time in seconds and the Unix process ID for those that are interested.) The file is then
renamed to the final filename without this extensions at the conclusion of the upload.
Some third parties actively monitor the open filehandle of the file we upload, and as soon as the filehandle
closes they want to do something with the file. They do not want us uploading with a temporary filename
and renaming at the conclusion of the upload. In this case, it is possible to use a configuration parameter to
ensure EFB simply uploads the file with its original name.
It is still possible to use a temporary transfer directory and EFB will move the file at the conclusion of the
upload. I could not think of a possible use for this but the code does not preclude it.
Files for BT from Third Party
There are two distinct possibilities here. The first is by far the preferred option by BT.

Pull by BT from Third Party

Periodically, at sensible times and agreed times, EFB logs into your server and checks a specific directory
for a filename pattern. Any files it finds there that match the pattern are downloaded by EFB. Obviously,
files placed in this directory should be complete and ready for download as soon as they are visible in this
directory.
As part of the download, EFB will normally rename each file with a .inprogress extension, download it, then
perform whatever action it needs to take at conclusion of a download (delete, archive or rename). This is
because there are actually two EFB servers communicating with the outside world. As both could hit your
SFTP server at the same time, this rename ensures only one copy of the file is downloaded. (EFB
ignores .inprogress files, and in the incredibly unlikely event that both attempt a rename at the same time,
renames are guaranteed atomic by the POSIX standard, so one should fail and skip that file. (This does rely
on your server being POSIX compliant. Very rarely, we encounter SFTP server daemons that fail to respond
with success for one connection and error for the other when two SFTP connections attempt to rename the
same file at the same time. This is problematic, and we would recommend that you do not use SFTP server
software that does not respect the POSIX standard. We have yet to come up with a robust solution to SFTP
servers of this type.)
[The internal EFB still performs this rename, as for many years internal EFB was also load balanced across
multiple internal servers.]
There is one other fact all this leads to: the user EFB logs in as must have sensible read/write/modify
permissions for the directory and files it is to download.

25
 Restricted

EFB does not have any “memory” of the files it has downloaded, so it needs to make sure it does not
download files multiple times. There are three simple solutions to this that we select from:
a) Files are deleted after download. This is the default option.
b) Files are moved to an archive directory. They can also have year, month, day and a random number indicator
added to the filename when this occurs, or any other arbitrary text string. We cannot modify the base name
of the file, only add text to it. As the archive directory can actually be the same as the original directory, this
can be configured to effectively simply rename the file after download.
(Note that is an error condition to ask EFB to overwrite an already existing archive file. If files have already
been presented to EFB at a source location, and pulled and archived, it will be an error if the same original
file is placed at source a second time. In this case, EFB will pull the file, but will leave the .inprogress file in
place so it does not damage the previously archived version. If nothing changes, and the file is presented a
third time at source, it will simply be skipped as EFB cannot rename it to the .inprogress version as that
already exists.
If you wish to redeliver files across a transport where files are archived [perhaps final destination has ‘lost’
the files], the canonical solution is to move the archived version you have preserved back to its original name
and location. Then EFB can safely pull and archive the file again.)
c) Files are renamed by simply adding “.pro” (for “processed”) to the end of the original filename. If this option
is used then care must be taken to ensure that the filename pattern matcher does not pick up these files in
future.
If we are not allowed to delete/archive/rename the files on the remote directory there is one other possible
solution. If a file contains a datetime stamp we can include datetime components in filenames, so we could
run a transfer “today” and pick up “yesterday’s” files, as we can specify file patterns like this example:
data_$DATETIME{'1 day ago','%Y-%m-%d').log.gz
Which, if run on 2014-06-26, would pick up a file called:
data_2014-06-25.log.gz
Note that we would still need write/modify access permissions on the server even in this situation for
renaming the file with a .inprogress postfix and back again after the download (but see pull untouched
source below).
This datetime pattern is generally advised against as if there are any issues with delivery and a transport
does not run for a whole day (e.g. a server is offline), a file will be missed. EFB will not “catch up” missed
files like this as they no longer match the datetime pattern. You can put multiple datetime patterns in if
you wish to pull multiple files. Obviously, if you do this you will need to allow overwrites and handle the
fact that the same file will be transmitted end-to-end more than once. If this is a simple lookup file (for
example) then this could be acceptable. If each is a delta of data then it probably would not be.

Pull untouched source

It is possible to request that EFB does not rename the files with a .inprogress extension. Sometimes we may
not have these permissions, or it is considered unacceptable for us to perform this rename. In this case, it is
possible to configure EFP not to rename the file during download, and download it untouched. This allows
us to pull files when we do not have rename permissions.
Note that as we have no way of knowing we have already pulled the file, this is probably best only used
with transports that do things like pick up “yesterday’s” file once per day. Something that runs with a
regular polling interval will just keep on getting the exact same files.
There are some SFTP servers that monitor our downloads and detect the completion of a download by us.
They do not want us to rename the files during the download. They then delete the file themselves from
their server. Pull untouched source would work well with them.

26
 Restricted

As we have multiple DMZ servers, there is a risk that both may pull the file at the same time. We cannot
limit the pull to a single server - as we usually do - as that is done by the success or failure of the rename
command, which we are not running.

Push to BT by Third Party

We tell you the hostname to use over the internet (either efb.bt.com or btwholesalebigdata.sdeds.bt.com),
which will resolve to one of the multiple addresses in Internet-Facing-address) and provide you with the
username to login with. You provide us with the public key component of the private key you will use. We
add this public key to the authorized keys of the username you will login with.
You tell us the IP address your connection will appear to come from so we can open a hole in our firewall to
allow your incoming connection. Obviously, you may need to open your firewall (if you’re using one)
outbound from your server as well.
We tell you the directory to use, and we agree on the filenames to use.
Your system then initiates an SFTP connections, logs in and moves to the relevant directory.
You then upload the files to directory.
Care needs to be taken to ensure we do not take the file from this directory before the upload is complete.
We are monitoring the folder with a simple file pattern matcher. There are several obvious solutions to this
which can be negotiated:
d) You upload the file with a different name which means it does not meet the filename pattern matcher
criteria. For example, you could give it a “.uploading” postfix (or similar) then rename it after the upload.
Note that we ask you do not use the text “.inprogress” as the postfix as that collides with how EFB
operates. It’s not a significant issue, but EFB reports ‘unexpected’ “.inprogress” files with a warning, so we
would prefer not to see these warnings when they are false positives.
e) Uploads take place at a specific point in time, and we do not check the directory until after this time. For
example, you could upload files at 03:00 each day, and we agree not to check the directory till 04:00
onwards.
f) We provide a temporary upload directory one level down, which you upload into (nearly always called
“temp”). You then move your files one level up after upload.
The uploaded file will also need sensible permissions. In Unix terms, the uploaded file will need read
permissions at the group level. Obviously, the user you login as will have read and write permissions on the
file, but it will be processed at our end by a production user with a different username but in the same user
group. So minimal permissions like:
-rw-r----- 1 3rd_prty_username efb_prod 15474 Feb 7 2014 test.txt
Overwrites and Uniqueness
Generally, EFB is used to move uniquely named files from server to server for prompt processing. Filenames
usually contain sequence numbers or datetime stamps that render them unique. If EFB discovers it is
attempting to overwrite a file that is already there it will fail. This feature can be switched off if required.
Runtime and Delivery Order
Generally EFB polls every 15 minutes during specified time periods (e.g “Mon-Fri 01:00-07:00”).
Runtimes are a little vague, due to EFB being a multi-user, multi-process system. Do not expect a runtime
like “Mon-Fri 01:00-07:00” to mean that nanoseconds after 01:00:00 the EFB transport will run, and
then run again precisely 900 seconds later. In reality, it’s far more likely to run at something like 01:03:30.
The delivery order of files is not guaranteed in any way, it is effectively random. When a transport runs, it
will inspect the folder for files, create a list of files then deliver them end-to-end in random sequence. This

27
 Restricted

is a feature. You cannot rely on a particular order for file delivery. You cannot expect a termination file to be
delivered last end-to-end.
Error Handling
Problems with transfers are carefully logged. By this, we mean explicit problems with SFTP connections or
attempting a file transfer. Commonly seen examples of such issues would be:
 A file server is not available, i.e. it has simply gone down, or perhaps an external FQDN (e.g.
"sftp.acme.com") has moved to a new hosting environment and has acquired a new IP address, but
nobody thought to tell the EFB team and so the firewall is not open to the new host and a
connection cannot be made (yes, this happens a lot).
 EFB can no longer login via the username we have been given. Perhaps the account has expired, or
for some reason our access has been revoked. Note that it is common for systems to be setup so
that if a password expires the account is locked, even though EFB is logging in via public key
authentication and does not use the password.
 A directory EFB must use no longer exists or we no longer have the correct permissions for it.
Perhaps someone was being over-zealous with cleaning up a server.
 The remote server has run out of disk space. This happens surprisingly often given the relative
cheapness of disk space compared to everything else these days.
 An overwrite of a previously delivered file is being attempted and you have not explicitly requested
that overwrites should be allowed. The default state of EFB is that overwrites of files in the remote
delivery directory are not allowed and are an error.
When a problem like this occurs the transport goes into a FAILED state. A list of transports in a failed state
is detected automatically and mailed internally to the support team for investigation at regular intervals
during business hours. The outsourced support team is listed at the start of this document. This team
supports a variety of applications, not just EFB. They then work to try to fix broken transports with "best
endeavours" during office hours. IIRC, this is known as "Bronze" support level, and is considered standard
for EFB. They won't report the issues back to any teams within EE.
Only failed transports which are supported 24x7 will raise P3 incident/callout for investigation. This is
generally known as "Gold" level support.
If an EE/BT team believes there is an issue with their transport (e.g. “why have I not got any files today on
my server?”) they can open an incident with the ops team and they can then investigate it, do whatever is
required to fix it (e.g. clear .inprogress files, check servers are available, re-run transports, etc.) and report
back with their findings.
Note that “lack of files” is not considered an error by EFB. If it is polling a source location for files, and
there are no files there, this is simply BAU and EFB continues on its way. If you expect a set of (for example)
five files by 09:00 every day, and only four files appear before 09:00, this is not considered an error by
EFB: it has transferred all the files that were available.
SPLUNK logs
EFB logs are now sent to the SPLUNK system. Go here to read about SPLUNK access and quick ways to
check your logs. If you're feeling very clever, this allows you to create your own alerts based on the
contents of the SPLUNK logs. This is where you can potentially create alerts for such concepts as “I expect
five specific files before 09:00 and I consider it an error if they have not transferred end-to-end by that
time”.

28