Maneuvering (MAN 1-5) and Payload Functionality (PAY 1-5) : Quick Start Guide

Standard Test Methods for Small Unmanned Aircraft Systems
ASTM International Standards Committee on Homeland Security Applications;

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov
Maneuvering (MAN 1-5) and Payload Functionality (PAY 1-5)

Quick Start Guide
Standard Test Methods for Response Robots
ASTM International Standards Committee for Homeland Security Applications;
Response Robots (E54.09)
WATCH THE VIDEO VERSION WITH

TEST METHOD FLY THROUGHS
https://vimeo.com/354145833
Test Director: Sponsor:

Adam Jacoff Adam Jacoff Phil Mattson
Intelligent Systems DivisionIntelligent Systems Division
Adam Jacoff
Intelligent Systems DivisionScience and Technology Directorate
NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY
National Institute of Standards and Technology
NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY
U.S. Department of Homeland Security
U.S. Department of Commerce
Internet Email
RobotTestMethods.nist.gov RobotTestMethods@nist.gov
Internet Email
RobotTestMethods.nist.gov RobotTestMethods@nist.gov
1
Acknowledgements
Acknowledgements Collaborators
This work was sponsored by Philip Mattson and Kai-Dee Chu from the Andy Olesen, Canadian Explosives Technicians Assoc., Canada
Department of Homeland Security, Science and Technology Directorate, Tom Haus, Los Angeles Fire Dept. & CA-TF1, CA
through an interagency agreement with the National Institute of Tom Prentice, Reveille Peak Ranch, Burnet, TX
Parry Boogard, Valley Regional Fire Authority & WA-TF1, WA
Standards and Technology (NIST). Michael Leo, Fire Department of New York City, NY
Clint Arnett, TEEX/Disaster City & TX-TF1, TX
The NIST Team includes: Katie Thielmeyer, Woodlawn Fire Dept. OH
George Hough, Fire Dept. of New York City & NY-TF1, NY
Adam Jacoff, Kamel Saidi, Raymond Sheh,
Kenny Kimble, and Ann Virts. Jim Ingledue, Virginia Beach Fire Dept. & VA-TF2, VA
Mark Hundley, Virginia Beach Fire Dept. & VA-TF2, WA
Dozens more people have contributed to the development and
validation of these test methods. They include FEMA urban search and Michael O’Shea, U.S. Dept. of Justice, DC
rescue task force teams, firefighters, law enforcement, collaborating
Martin Hutchings, Sacramento Sheriff & IAB, CA
test facilities, other civilian and military organizations, and commercial
manufacturers. There are far too many to mention, but some of the John Delaney, Arlington County Fire, Dept., & IAB, VA
ongoing (non-commercial) collaborators are listed below, roughly in
Mike Marino, Prince George’s County Fire Dept. & IAB, MD
order of their involvement:
Coitt Kessler, Austin Fire Dept., TX
Disclaimer Chris Sadler, York County Fire Dept., VA
Commercial equipment shown in this document are for illustrative
Andy Moore, Southwest Research Institute, San Antonio, TX
purposes only. This does not imply recommendation or endorsement by
the National Institute of Standards and Technology, nor does it imply Al Frazier, Grand Forks County Sheriff’s Dept., ND
that the products identified are necessarily the best available for the
Ben Miller, DFPC Center of Excellence for Aerial Fire Fighting, CO
purpose.
Max Delo, ESF-13, U.S. Marshals Service, DOJ
Measurement Units Bryan Gillespy, ESF-13, U.S. Marshals Service, DOJ
The International System of Units (a.k.a. SI Units) and U.S. Customary
Units (a.k.a. Imperial Units) are used throughout this document. Howie Stockhowe, Virginia Beach Fire Dept, Virginia Beach, VA
Approximate equivalents in each system of units enable use of readily Tony Galladora, Montgomery County Police, MD
available materials in different countries. This avoids excessive
purchasing and fabrication costs. The differences between the stated Satoshi Tadokoro, Tohoku University, Sendai, Japan
unit dimensions are insignificant for comparison of test method results, Tetsuya Kimura, Nagoaka Univ. of Technology, Nagoaka, Japan
so each set of units are considered standard for the purposes of these
test methods.
2
Scalable Test Lane

e 3
lti tud
2X A FLY SMALL SYSTEMS FLY LARGE SYSTEMS
X
Spiral Bucket
Leveling Post
de
A ltitu
X
X 2
1 X WORK THE INTERFACES PRACTICE PROCEDURES
Omni Bucket Leveling Stand
X
X
Launch/Land Platform
X
Lane Marker
FL
IG EVALUATE SENSORS COMPARE PAYLOADS
HT
LIN
E
3
Scalable Test Lane

3
de X
A ltitu
2X
X
Spiral Bucket X
Leveling Post
X
2
de
A ltitu 1 X
X
X X
X
X
Lane Marker Inside each bucket is an inscribed ring to evaluate alignment. LEFT
FL
IG
HT
LIN
is aligned, RIGHT is not quite. Center targets can be letters,
E visual/color/thermal acuity charts, hazmat labels, or other items. 4
Test Lane Layout

• Pilot flight line with lane marker (A-frame)
• Centerline (long measuring tape or flat cones)
• 1X spacing (10ft, 20ft, 30ft or other)
• 4X overall length (40ft, 80ft, 120ft or other)
• The flight altitudes are always 1X and 2X X
e X
A l t i t ud X
2 X
X X
INE
NTER L
CE
t it ude X
X Al
X X
X LE NGT
H
LL
VERA
X 4X O
FL
IG
Inside each bucket is an inscribed ring to evaluate alignment. LEFT
HT is aligned, RIGHT is not quite. Center targets can be letters,
LIN
E visual/color/thermal acuity charts, hazmat labels, or other items. 5
Bucket Details – Align and Identify Visual Acuity Targets

3 NUMBERED BUCKETS (1-3)

Top Vertical MAN 1-5 PAY 1-5
LETTER IDENTIFIERS VISUAL ACUITY TARGETS
33
20 LETTERED BUCKETS (A-T)
Angled 45° and Horizontal S R
O
T
Q N
START TOWARD LANDING A
WITH A VOWEL AND P
WRAP AROUND RIGHTWARD A A
L K
H
22 I NOT A
G J Gap sizes:
M QUITE
D
11 E ALIGNED
e C F
tud
X Alti Align to see the entire Align and identify the
X A inscribed ring inside each acuity target inside each
B
bucket. The letters are bucket with increasingly
bucket identifiers. small concentric C gaps in
one of eight directions.
X
FL
IG
HT
LIN
E
6
Bucket Details – Numbering and Lettering

3 NUMBERED BUCKETS (1-3)

Top Vertical Numbers and letters inside the buckets help guide the pilot.
33
20 LETTERED BUCKETS (A-T) Perch Acuity Targets
Angled 45° and Horizontal S R
O
T
Q
START TOWARD LANDING
WITH A VOWEL AND P 1) SURFACE
WRAP AROUND RIGHTWARD
L K
H
22 I 2) UNDERBODY
G J
M
D
11 E
e C F
tud
X Alti
X A
B
Viewed from the 1) SURFACE
X
FL
IG Perch acuity from the Launch/Land Platform
HT
LIN
E
benefit from accurate landings to apply full zoom capabilities
7
Bucket Details - 2 Gallon (8in Diam) Sticker Files

Waterproof polyester stickers are preferred because they are easy to insert and adjust initially. They also survive the elements.
Stickers can contain more than just numbers, letters, and acuity targets. More operationally significant or just random targets work too.
All Basic Lane Buckets

https://drive.google.com/open?id=1NQrHY3UH98fUeXKyffnQwt6-h5ewoeqU
Letters - INSERT DISCS FOR MAN

https://drive.google.com/open?id=1FoQvoKkQu5jUC4bJJNM7TailCWWs-C3_
Concentric Cs Black - SENSOR PANELS

https://drive.google.com/open?id=1YxY1_26dn1KB0FHfleU4Xna_gxHzw98L
Concentric Cs Color - SCENARIOS

https://drive.google.com/open?id=198sR8TzRB4TKtvZvHcAuXRWdgS35Thzi
Misc Hazmats, Directions, Plates, Images

https://drive.google.com/open?id=1nuHvZS9ARZ6KkF_vJZKbhO6qwEo4UlkM
TEMPORARY EASY TO
INSERT DISCS RANDOMIZE
IN SCENARIOS Xtra Bucket Stands for Scenarios
8IN CARDBOARD https://drive.google.com/open?id=1RklQazk4r8ZyUPJxidjlNpVyF-ZNRCrn
CAKE ROUNDS
8
Bucket Details – 5 Gallon (10in Diam) Inscribed Rings

• 2 gallon and 5 gallon buckets with inscribed rings

and targets inside are essentially interchangeable.
The larger bucket has only a slighter bigger
diameter but at 20+ ft altitude this is negligible.
+
• 5 gallon white buckets with 10 in diameter inside
bottoms require an inscribed ring plus a sticker.
• Use an extra large black marker to make a 5/8in

inscribed ring.
• Tip: Press down using the thickest dimension of

the marker tip and pull toward you in the bottom
corner of the bucket. Then ROTATE THE BUCKET
two revolutions.
9
Choosing An Appropriate Lane Spacing

MAN 1-5 LETTER IDENTIFIERS
Align to see the entire inscribed
ring inside the buckets. The
letters are bucket identifiers.
2X Altitude
NOT QUITE ALIGNED
3
3
PAY 1-5 VISUAL ACUITY TARGETS
Align and identify the visual RE
acuity targets with increasingly CHECK ALIGNMENT AD
A
A
A
small concentric C gaps in one
A
CO
A
of eight directions.
Gap sizes:
NC THIS SHOWS THE ANGLED BUCKETS ARE

EN
TR TOO FAR AWAY FOR THE OPTICS ON THIS AIRCRAFT
IC
C ---- MOVE TO A SHORTER LANE SPACING ----
22
11 X
1
X
X
An appropriate lane spacing is when a
X 2X hover allows reading at least the
FL
outer ring of a concentric C target THIS IS JUST BARELY CLOSE ENOUGH
IG
HT
LIN two stands away. TO BE CERTAIN OF A COMPLETELY INSCRIBED RING
E (ROUGHLY 1 / 10 OF THE DISPLAY OR LARGER) 10
Position Test Procedure

Maneuvering (MAN 1) and Payload Functionality (PAY 1)
MAN 1-5 LETTER IDENTIFIERS SCORING
ring inside the buckets. The FLIGHT PATH MAN PAY .
NOT QUITE ALIGNED START THE TIMER when the drone launches from the platform
PAY 1-5 VISUAL ACUITY TARGETS
Align and identify the visual 1. HOVER at X over Bucket 1 ------------------ Bucket E Bucket 1
A acuity targets with increasingly 180° 2. ROTATE RIGHT 360° -------------------------- Bucket E Bucket 1
A
small concentric C gaps in one
A
Gap sizes:
360° 3. ROTATE LEFT 360° ---------------------------- Bucket E Bucket 1
180°
4. CLIMB to 2X ------------------------------------ Bucket I Bucket 1
5. DESCEND to X --------------------------------- Bucket E Bucket 1
X Alt
itud
e 6. FORWARD over Bucket 2 ------------------- Bucket I Bucket 2
7. BACKWARD over Bucket 1 ----------------- Bucket E Bucket 1
8. FORWARD/ROTATE 180° over Bucket 2- Bucket C Bucket 2
9. FORWARD/ROTATE 180° over Landing - Bucket A Landing
10. LAND CENTERED facing stands ----------- Centered Perch 1
Centered Perch 2
MAN: 20 points, 10 Positions, 18 Alignments and a Landing (2pts)

FL
IG
HT
LIN
PAY: 100 points, 10 Positions, 18 Bucket Targets and 2 Perch Targets
E
*If your training aircraft has only a fixed camera, or limited range of motion, align with as many buckets as possible. Performance is never compared across aircraft anyway. 11
Position Test Form

MAN 1-5 LETTER IDENTIFIERS
Maneuvering (MAN 1) and Payload Functionality (PAY 1)
ring inside the buckets to. The FORM FOR TRACKING YOUR PERFORMANCE OVER TIME
MAN and PAY TEST
NOT QUITE ALIGNED
Check mark the
PAY 1-5 VISUAL ACUITY TARGETS buckets when aligned
Align and identify the visual for both MAN and PAY
A
acuity targets with increasingly
A A
small concentric C gaps in one 180°
A
Gap sizes:
PAY TEST ONLY

360° Write the number
of C’s correctly identified
180°
e RESULTS
itud
X Alt • Total aligned buckets
• Total C’s identified
• Faults
• Elapsed time of trial
METRICS (in order)

1. Score
2. Reliability/Acuity
FL
IG
HT 3. Efficiency
LIN
E
12
Comprehensive Flight Paths in a Single Lane

2X
Altitude/Radius
de
ltitu
e 1X A
ltitud 1X
2X A Altitude/Radius
180°
2 x 360° Flight
Free
e
ltitud
1X A
Position Traverse Orbit Spiral Sustain Speed/Deliver Accurately

MAN 1 / PAY 1 MAN 2 / PAY 2 MAN 3 / PAY 3 MAN 4 / PAY 4 MAN 5 / PAY 5
• Hover position stability • Sideways along a line • Orbit identifications • Free flight inspections • Max speed following a line
• Basic maneuvers • Left and right directions • Left and right directions • Any proximity • 5X distance
• Landing accuracy • Landing accuracy • X and 2X altitudes • Any altitude • X altitude
• 20 tasks in 1 lap • 20 tasks in 2 laps • 20 tasks in 4 laps • 20 tasks in 1 lap • 20X distance per lap
MAN: Align only MAN: Align only MAN: Align only MAN: Align only MAN: Follow paths
PAY: Align and Identify PAY: Align and Identify PAY: Align and Identify PAY: Align and Identify PAY: Deliver Payload Accurately
13
Evaluate System Capabilities or Pilot Proficiency

Position (Version D)
s
diu
/Ra
itu 2X
MAN 1 / PAY 1
de
START POSITION
Alt
s
diu
/Ra
itu 1X
de
1, I NOT QUITE MANEUVERING 1 (MAN 1)

Alt
ALIGNED
e
A
ltitud • 10 positions in 1 lap
2X A
180°
2, I, C
• 18 bucket alignments and 1 landing (2pts)
2 x 360° • 1 point each
Bucket
180°
1, E
Targets • 20 points maximum
e A
Altitud
1X
A PAYOAD FUNCTIONALITY 1 (PAY 1)
A
A
A
A
A
A
• 10 positions in 1 lap
• 18 bucket targets and 2 perch targets
A Gap sizes:
Gap sizes:
• 5 concentric Cs per target

• 100 points maximum per lap
14
lA

Traverse
MAN 2 / PAY 2
START POSITION
DE LAPS L
A L TITU WARD K
1X F T
uLdEe
AANltDitH
R D
TWA 1X
NOT QUITE MANEUVERING 2 (MAN 2)
RIGH
ALIGNED
J
A • 20 alignments in 2 laps (rightward/leftward)
• 18 bucket alignments and 2 landings
• 1 point each
F Bucket
Targets • 20 points maximum
A B

A
A
A
A
A
A
• 20 targets in 2 laps (rightward/leftward)
PERCH
• 18 bucket targets and 2 perch targets
AGap sizes:
Gap sizes:

• 100 points maximum
Alti
tud
15
Alti
2X
e/
t

L 2X
Altitude/Radius
K START POSITION
Orbit NOT QUITE MANEUVERING 3 (MAN 3)
MAN 3 / PAY 3 ALIGNED
L
1X
Altitude/Radius
K
A • 20 alignments in 4 orbits
(rightward & leftward, 1X & 2X)
1, I • Each orbit has 4 buckets toward center
J
Bucket
and 1 downward radius
2, I Targets • 1 point each
J

A A
• 20 targets in 4 orbits
A
A A
A
A
(rightward & leftward, 1X & 2X)

Gap sizes:
Gap sizes:
• Each orbit has 4 buckets toward center

and 1 downward radius
16

Spiral
MAN 4 / PAY 4
START POSITION
NOT QUITE
ALIGNED MANEUVERING 4 (MAN 4)
ree Fligh
t A • 20 bucket alignments in 1 lap
• 1 Point Each
F
• 20 Points Maximum
Bucket
Targets
PAYOAD FUNCTIONALITY 4 (PAY 4)
A • 20 targets in 1 lap
A A
A
A A
A
A

Gap sizes:
Gap sizes:
17

Sustain Speed / 5X
Deliver Accurately 5X
MAN 5 / PAY 5
de
START POSITION
A ltitu
2X
5X NOT QUITE MANEUVERING 5 (MAN 5)
A
Overshoot ALIGNED
Limit
• 5 laps
de • 4 flight path (2 horizontal, 2 vertical)
ltitu
XA • 1 Point Each
PAYOAD FUNCTIONALITY 5 (PAY 5)

• 5 placements or drop from 2X altitude
• 20 points for diameters from 4-20ft
(max weight object)
18

Sustain Speed / 5X
Deliver Accurately 5X
MAN 5 / PAY 5
de
A ltitu
2 X WEIGHTED PAYLOAD LAND or DROP
5X
Overshoot
AIRCRAFT
Limit
de
A ltitu
X
BUCKET AT CENTER
OF DROP ZONE
PLACE PLACE (DETAIL)

19
Circuit Training with Scores

2X
Altitude/Radius
de
ltitu
e 1X A
ltitud 1X
2X A Altitude/Radius
180°
2 x 360° Flight
Free
e
ltitud
1X A
Position Traverse Orbit Spiral Sustain Speed / Deliver Accurately

MAN 1 / PAY 1 MAN 2 / PAY 2 MAN 3 / PAY 3 MAN 4 / PAY 4 MAN 5 / PAY 5
• Hover position stability • Sideways along a line • Orbit to identify objects • Free flight to inspect objects • Max speed following a line
• Basic maneuvers • Left and right directions • Left and right directions • Any proximity (use zooms) • 5X distance
• Landing accuracy • Landing accuracy • X and 2X altitudes • Any altitude • X altitude
• 20 tasks in 1 lap • 20 tasks in 2 laps • 20 tasks in 4 laps • 20 tasks in 1 lap • 20X distance per lap
20

Maneuvering (MAN 1-5) and Payload Functionality (PAY 1-5) : Quick Start Guide

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Maneuvering (MAN 1-5) and Payload Functionality (PAY 1-5) : Quick Start Guide

Uploaded by

Copyright:

Available Formats

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications;

Maneuvering (MAN 1-5) and Payload Functionality (PAY 1-5)

WATCH THE VIDEO VERSION WITH

Test Director: Sponsor:

Scalable Test Lane

Scalable Test Lane

Test Lane Layout

Bucket Details – Align and Identify Visual Acuity Targets

3 NUMBERED BUCKETS (1-3)

Bucket Details – Numbering and Lettering

3 NUMBERED BUCKETS (1-3)

Bucket Details - 2 Gallon (8in Diam) Sticker Files

All Basic Lane Buckets

Letters - INSERT DISCS FOR MAN

Concentric Cs Black - SENSOR PANELS

Concentric Cs Color - SCENARIOS

Misc Hazmats, Directions, Plates, Images

Bucket Details – 5 Gallon (10in Diam) Inscribed Rings

• 2 gallon and 5 gallon buckets with inscribed rings

• Use an extra large black marker to make a 5/8in

• Tip: Press down using the thickest dimension of

Choosing An Appropriate Lane Spacing

NC THIS SHOWS THE ANGLED BUCKETS ARE

Position Test Procedure

MAN: 20 points, 10 Positions, 18 Alignments and a Landing (2pts)

Position Test Form

PAY TEST ONLY

METRICS (in order)

Comprehensive Flight Paths in a Single Lane

Position Traverse Orbit Spiral Sustain Speed/Deliver Accurately

Evaluate System Capabilities or Pilot Proficiency

1, I NOT QUITE MANEUVERING 1 (MAN 1)

• 5 concentric Cs per target

Evaluate System Capabilities or Pilot Proficiency

A PAYOAD FUNCTIONALITY 2 (PAY 2)

• 5 concentric Cs per target

Evaluate System Capabilities or Pilot Proficiency

A PAYOAD FUNCTIONALITY 3 (PAY 3)

(rightward & leftward, 1X & 2X)

• Each orbit has 4 buckets toward center

Evaluate System Capabilities or Pilot Proficiency

• 100 Points Maximum

Evaluate System Capabilities or Pilot Proficiency

PAYOAD FUNCTIONALITY 5 (PAY 5)

Evaluate System Capabilities or Pilot Proficiency

PLACE PLACE (DETAIL)

Circuit Training with Scores

Position Traverse Orbit Spiral Sustain Speed / Deliver Accurately

You might also like