vice.

com

25-32 minutes

Hacking. Disinformation. Surveillance. CYBER is Motherboard's

podcast and reporting on the dark underbelly of the internet.

Andrew Frame was excited.

It was Saturday night two weeks ago, and Frame, the CEO of the
crime and neighborhood watch app Citizen, was on Slack,
whipping himself and his employees into what he'd later call at an
all-hands meeting a "fury of passion" about a wildfire that had
broken out earlier that afternoon in Los Angeles' Pacific Palisades
neighborhood.

Citizen had gotten a tip that the wildfire was started by an arsonist,
and Frame had decided earlier in the night that the fire was a huge
opportunity. Citizen, using a new livestreaming service it had just
launched called OnAir, would catch the suspect live on air, with
thousands of people watching. Frame decided the Citizen user
who provided information that led to the suspect’s arrest would get
$10,000. Frame wanted him. Before midnight. As the night wore
on, Citizen got more information about the supposed suspect.
They obtained a photo of the man, which they kept up on the
livestream for large portions of the night. More information trickled
in through a tips line Citizen had set up.

"first name? What is it?! publish ALL info," Frame told employees
working in a Citizen Slack room who were working on the case.

"FIND THIS FUCK," he told them. "LETS GET THIS GUY

BEFORE MIDNIGHT HES GOING DOWN."

"BREAKING NEWS. this guy is the devil. get him," Frame said. "by
midnight!@#! we hate this guy. GET HIM."

He was growing impatient. He increased the bounty to $20,000.

Thousands of people were watching Citizen's livestream, but the
man still hadn't been caught. Frame asked his staff to send out
another notification, one that would hit all Citizen users in Los
Angeles. The bounty had to go higher.

"Close in on him. 30k Let's get him. No escape. Let's increase.

30k," Frame said. "Notify all of la. Blast to all of la."

"Citizen is OnAir: Arsonist Pursuit Continues," the notification,

which went out to 848,816 Citizen users in Los Angeles, said. "We
are now offering a $30,000 reward for any information directly
leading to his arrest tonight. Tap to join the live search."
Over the course of nearly seven hours, Citizen, under the
increasingly frantic direction of Frame, conducted a citywide, app-
fueled manhunt for a specific suspected arsonist. The employees
went back and forth on how they should frame the manhunt they
had started, who in Los Angeles they should notify via the app,
and how often they should do it.

Do you work at Citizen? Do you have access to internal

Citizen documents? We’d love to hear from you. Using a non-
work phone or computer, you can contact Joseph Cox
securely on Signal on +44 20 8133 5190, Wickr on josephcox,
OTR chat on jfcox@jabber.ccc.de, or
email joseph.cox@vice.com.

In the Slack room with Frame, one staffer brought up a "loophole,"

pointing out that Citizen was violating its own terms of service that
prohibit "posting of specific information that could identify parties
involved in an incident." The staffer who brought up the terms of
service violation was ignored in that specific Slack room, and the
broadcast continued to specifically name the person and share his
photo for hours.

Earlier in the night, soon after news of a fire broke, Frame said he
saw the fire as a chance to catch a suspected arsonist live on the
internet, therefore proving Citizen's utility to users and helping the
app grow.

"The more courage we have, the more signups we will have. go

after bad guys, signups will skyrocket. period ... we should catch a
new bad guy EVERY DAY," Frame said.

At one point, Frame said "these metrics will be great." And they
were. At one point 40,000 people were watching the live feed,
according to the Slack messages. Citizen saw a sharp spike in
signups as the livestream spread. Frame said at a later all-hands
meeting that 1.4 million people engaged with the content,
according to other Slack messages.

Well after midnight, Los Angeles police made an arrest. In a

separate Slack room, employees cautiously began to celebrate:
"cop said its an ongoing investigation, this looks like our guy!!!"
one employee wrote.

It wasn't Citizen's guy. Frame and the entirety of the Citizen

apparatus had spent a whole night putting a bounty on the head of
an innocent man.

(Motherboard is not publishing the name of the person Citizen

falsely accused, though Citizen repeatedly used it both internally
and externally)

Motherboard spoke to eight sources in reporting this story: five

former Citizen employees, two sources with knowledge of the
company's operations, and one person close to the company's
founders. Motherboard also obtained multiple caches of internal
policy documents, Slack messages, and company notes. Our
reporting spells out not only what happened in Pacific Palisades,
but also how workers and Andrew Frame view the incident and
Citizen's role in society. The app pitches itself as a public-safety
tool, but aims to grow its user base and revenue just as much as
any other startup. The Palisades incident was characterized by
Frame as a risk, a test, an experiment, even though it potentially
put the person they named in danger.

Motherboard has learned that:

Users are flooded with notifications in what multiple sources

interpret as an attempt to make users feel anxious enough about
their neighborhoods to buy "Protect," a $19.99 per month service
that allows users to livestream their phone's camera and location
to a Citizen "Protect agent" who monitors it and sends "Instant
emergency response" in case of an emergency.

The return of a missing autistic teen to his family in the Bronx

earlier this month was done by Citizen employees on a "Street
Team" that films and interacts with people while pretending to be
ordinary app users.

Employee performance is measured by how many seconds it

takes workers to input an incident into the app and how many
incidents they cover.

Citizen's grand vision has never been a secret: From its initial
launch as an app called "Vigilante" in 2016, the company pictured
a world in which people were alerted to crime as it happened, and
then app users stepped in to stop it before the police needed to
intervene. In the Vigilante launch advertisement, a criminal stalks
and then attacks a woman in New York City. The app broadcasts
the location of this active crime to Vigilante's users, and a horde of
people descend on the criminal, stopping the crime in progress:
"Can injustice survive transparency?" the ad asks.

Thus far, however, Citizen has essentially been a social network

for reporting crime that operates in around 50 cities. Citizen
workers listen to and summarize police scanner audio as
"incidents," which are then pushed to the app. Users can also post
their own incidents, upload photos and videos, and comment on or
react to incidents with emojis. The app allows users to search
"around you" for incidents, and also sends push alerts to users for
nearby events.

"The whole idea behind Protect is that you could convince people
to pay for the product once you’ve gotten them to the highest point
of anxiety you can possibly get them to," one former employee
said, referring to Citizen's subscription service. "Citizen can’t make
money unless it makes its users believe there are constant, urgent
threats around them at all times," they added. A Citizen
spokesperson denied this in a statement: "It’s actually the
opposite. With user feedback in mind, we have designed the
Citizen home screen so users only see relevant, real-time
information within their immediate surroundings," the
spokesperson said.

The disastrous Palisades fire bounty hunt and the discovery of a

Citizen-branded "private patrol" vehicle driving around Los
Angeles (part of a pilot program in which Citizen envisions offering
a physical private security force to respond to the problems of its
users) hint that Citizen's goals essentially remain the same as
Vigilante's.

Frame seems to imagine Citizen as an all-encompassing crime-

fighting machine that he believes will make the world safer. In
Slack messages viewed by Motherboard, Frame calls ProtectOS,
the system Citizen uses to create incidents and push them out to
users, "the most powerful operating system ever created."

"Our vision is a global safety network of people protecting each

other—a world in which a kidnapping is impossible, because
everyone is looking out for each other, and neighbors are alerted
as soon as a kidnapping attempt is made," a Citizen spokesperson
told Motherboard in a statement.
Vigilante was instantly controversial for a variety of reasons.
Unsurprisingly, police said the app encouraged vigilantism. Critics
worried that the app's users would racially profile Black people as
suspicious, as happened on other safety-focused apps. Apple took
the app out of the App Store because it violated terms of service
that ban apps that risk "physical harm to people." The app
relaunched as "Citizen" in 2017, with Frame saying that the
original name "distracted from our mission" and that people should
not take the law into their own hands. They should use Citizen to
avoid crime rather than fight it.

In practice, Citizen is an app that experts say fuels paranoia and a

fear of one's neighbors and surroundings by reporting "suspicious"
people. Many of the incidents reported on the app are about
people experiencing homelessness, for example.

"It plays into people’s anxieties and fears and magnifies people’s
fears of the other and who and what they think should not exist in
their neighborhood or their area," Chris Gilliard, a research fellow
with the Technology and Social Change Research Project at
Harvard Kennedy School’s Shorenstein Center, said. "As we’ve
seen, that often means people who don’t look like them."

One former Citizen employee told Motherboard that a portion of

the app's user base is "insanely racist, which comes out in
comment sections that are especially vile even by the standards of
internet comment sections." Citizen does moderate comments, but
"two people having an argument about whether or not someone’s
comment is racist drives engagement," the source added. A
hacker recently scraped a wealth of information from Citizen,
including user comments that repeatedly use the N-word,
according to a screenshot provided by the hacker. Some of these
were deleted by Citizen, but racist comments are regularly posted
on incidents.

Citizen incentivizes both its employees and the public to create

incidents because they are the core currency of the app and what
drives user engagement, user retention, and a sense of reliance
on the app itself. The scrape of Citizen data published by the
hacker earlier this week and shared with Motherboard shows at
least 1.7 million incidents in the United States.

incident_graph_final_1.png
The weekly incidents on Citizen, using data scraped by the hacker.
Image: Ishaan Jhaveri, Computational Research Fellow, Tow
Center for Digital Journalism, Columbia University.

Workers have been measured by how many Citizen users see

incidents they covered, how many reports they produce, and how
quickly they do so, multiple former employees told Motherboard.

"It’s basically an anxiety sweatshop," a Citizen source said. "On

days when things are 'slow,' they relax the standards around
incidents because a dip in incident count is really bad," they
added. The company sends congratulatory emails announcing
which analysts reported the highest number of incidents, another
source added.

This results in Citizen warning users about "everything," according

to one former employee. This includes lost dogs, minor car
crashes, unsubstantiated reports of gunshots, and domestic
incidents, they said. This week in Los Angeles, incidents ranged in
severity from "assault" to "gunfire" to "two men brawling" to
"injured bird," "firefighter activity," and "crowd gathered."

“In a healthy society we are typically not incentivized to

sensationalize mundane events and code them as crime. I can’t
help but think it plays into people’s anxieties and fears and
magnifies people’s fears of the other,” Gilliard said. “What’s really
dangerous is the ways they’re starting to serve as infrastructure,
where people start to feel like they have to use them to maintain
society and order.”

A former employee added, "They don’t much care about the

accuracy or the usefulness of the information they put out, they
just want to push as many notifications to create that feeling of
vulnerability that leads people to the subscription services."

Another former employee said that although fear is an aspect of

the app, it isn't the only one: Notifications about fires get a lot of
engagement, even if the danger to other people is not imminent,
"because the videos get crazy." They added that Citizen
sometimes acts as a source of entertainment for users. "People
like to read about and watch videos of incidents around them,"
they said.

Even Frame, the company's CEO, acknowledges Citizen's

bombardment of notifications. "We send so many dumb notifs,"
Frame said in one of the Slack messages obtained by
Motherboard.

andrew_frame.jpeg
Andrew Frame. Image: Steve Jennings/Getty Images for
TechCrunch.

There is a formal process for creating incidents, one of the sources

explained. Medical calls are not reported and domestic violence
incidents are recorded at an intersection rather than an address,
they said. But "the guidelines evolve, and there is definitely
disagreement internally over the guidelines." In the Palisades
incident, Frame said he "overrode the policy we have for Mission
Control" in official notes from a company all-hands meeting.

A Citizen spokesperson said in a statement that "We continue to

work to improve the relevance and frequency of notifications. In
addition, we are focused on reducing the reach of notifications
about violent incidents, and increasing the reach of notifications
about incidents such as missing people or pets being reunited with
their families—we could all use some more good news."

To help with the deluge of incidents that Citizen creates, it has

outsourced work to CloudFactory, a data processing service, three
sources told Motherboard. CloudFactory uses workers based in
Kenya and Nepal, according to CloudFactory's website. Citizen's
use of CloudFactory has not been previously reported.
CloudFactory did not respond to a request for comment.

Citizen hires "central analysts" to listen to police scanners and

then enter information into Citizen so users can receive a
notification about an incident close to them. Now some of those
jobs are being filled by CloudFactory.

Citizen's outsourcing of labor overseas is making U.S. employees

nervous they may lose their jobs, one added. They added that the
U.S. workers train the CloudFactory workers. Because Citizen is
focused on specific cities in the United States, remote workers at
CloudFactory are inherently not on the ground, though Citizen
says its own "analysts" review incident updates.

A Citizen spokesperson told Motherboard in an email that "As the

company grows, we are working with CloudFactory to augment our
team. All incident updates and alerts continue to be reviewed by
our 24/7 team of Citizen analysts."

Citizen believes that the more people who click into the app, the
more users sign up, a source with knowledge of the company's
operations said. So, like many other apps and tech companies,
Citizen experiments to see where the optimal engagement may
be.

But Citizen presents itself as a public-safety service while also

trying to increase engagement. This creates a fundamental friction
that, at best, ends with a barrage of useless push notifications. At
worst, it ends with the CEO putting a bounty on an innocent
person's head.

"It’s basically an anxiety sweatshop."

Ultimately, monetizing its user base has also led Citizen to test a
product where users could order on-demand help from a private
security service. This month, Motherboard reported that a Citizen-
branded vehicle was driving around Los Angeles. Leaked emails
showed the vehicle was part of a pilot working with Los Angeles
Professional Security, a local security company whose CEO wants
the power to arrest people and take them to jail. Citizen told CBS
on Wednesday that the trial with Los Angeles Professional
Security is now over.

The emails also showed Citizen is testing the program with well-
known security firm Securitas, and claimed that high-level
members of the LAPD said the product could be a game changer.
A Citizen spokesperson told Motherboard at the time that the
vehicle was part of a trial to test a service for users that, for
example, may want an escort to walk them home. But details from
internal emails already published by Motherboard explain that
Citizen believes it can help with property crime.

This move explicitly into the private security space is directly linked
to Citizen's potential plans for monetization. A Citizen product
roadmap document obtained by Motherboard lists "Paid Private
Response" under a section titled "$$$." That section also
mentions "Insurance Perks," "Open Hardware Platform," and
"Send First Responders.". One former employee mentioned that
OnAir, the broadcasting service used during the manhunt, may be
turned into a premium content service, and the roadmap also
mentions "Live Video" under the "$$$" section.

A Citizen spokesperson told Motherboard in an email that the

company's monetization plan fits with its vision of being a global
safety network of people protecting each other, but did not specify
what that would involve. "We will not serve ads or sell user data,"
the spokesperson said.

"I never thought Citizen would go this far," one of the sources said.
"I didn’t anticipate the ways they would circumvent the police and
go that much further," referring to the private security force Citizen
piloted in Los Angeles.

While Frame was instructing his team to up the stakes of the

search for the alleged arsonist, the public faces of the manhunt
were two people presenting Citizen's live broadcast. As part of its
OnAir product, Citizen essentially runs a pseudo-cable news TV
show, with presenters reading out tips they've received from the
public, speaking to people on the ground, and, in the case of the
wildfire, reminding users that Citizen was offering a bounty for
information leading to the arrest of the suspect.

"We mobilized the community, people in the Palisades area and

surrounding. People have sent in so many tips saying that this is
the guy," a woman named Kris, the main host of the broadcast,
said. (A Motherboard employee in Los Angeles repeatedly got
push notifications during the stream and took notes at the time.)
"We had a bunch of people send in the photo that this is a known
arsonist in the area. They know this guy, this is where he hangs
out. While it's not official, so many people have thrown out this
guy's name. He goes by the name [redacted by Motherboard] and
it seems like everybody in the community knows this is the guy. So
we are offering a $30,000 reward."

Toward the end of the night, Kris had been joined by Prince Mapp,
Citizen's head of community. He pointed out the stakes: "When
have you seen 860,495 people committed to finding one person?
We have mobilized a city to bring one person to justice," he said.
"Look for [the person's name]. Look for him. Family members of
[the person's name]. He wasn't just brought on this world by
himself, we need your help. We need you to help us contact him
and identify where he is. We need the scent of his clothing. We
need this man off the street so we can stop burning the city of Los
Angeles […] This person is the devil and we need to get him off
the street. We need to get our city back in order."

Throughout the night, these broadcasters were being coached

behind the scenes in another Slack room that Andrew Frame
seemingly didn't participate in much that night: "Kris, keep
repeating the $10k reward. 'LET'S FIND THIS ARSONIST,'" one
employee wrote. "REWARD. MONEY MONEY MONEY," they
added later. "Don't stop mentioning reward for the next 7 minutes."

The model for the Palisades fire response, according to official

notes posted in a Slack room from an all-hands meeting about the
incident, was similar to something that happened in the Bronx
earlier this month, when an autistic boy called Jeremiah went
missing. "We had this really strong moment with Jeremiah," the
notes quote Frame as saying. "This seemed like a perfect
opportunity to use OnAir."

In that case, Citizen users went to Jeremiah's family's house and

started filming, and Citizen itself started one of its OnAir
broadcasts, where, similar to the bounty case, hosts read out
incoming tips and interview users on the ground.

"There had been sightings of him running away from users trying
to approach him," a source with knowledge of Citizen's operations
said.

Eventually, two users found Jeremiah at a Target and he agreed to

go with them. Jeremiah got in their car and the people took him
home.

"They kept the broadcast going and had his grandma and mom on
speakerphone," the source added. "People were also concerned
that he got in a car with random strangers."

After the incident, Citizen uploaded a supercut of the event, with

the Citizen users finding Jeremiah and him being reunited with his
family. The company added hopeful background music to the
footage. A Citizen spokesperson told Motherboard in a statement
that "In the last 30 days, Citizen has shared critical safety
information that has contributed to at least four missing people,
including a 13-year-old autistic boy, reuniting with their families."

What the video didn't make clear is that the people on the ground
filming Jeremiah and speaking to his family were not Citizen users.
They were part of Citizen's "Street Team," who go out to events
and contribute footage to the app while posing as ordinary users.
Two sources confirmed the Street Team's existence; Citizen
doesn't publicly acknowledge the existence of this team, one of the
sources said. The sources added they believe the purpose of the
Street Team is to make regular users think they could get involved
too, so they will start broadcasting their own footage. "Essentially
for engagement," one of the people said.

jeremiah.png
A screenshot of the OnAir broadcast featuring Jeremiah.
Redaction by Motherboard. Image: Citizen.

"These Street Team members were not dressed in Citizen gear

and looked like regular users," the source added. A document
written by the company after the incident says "When Jeremiah
went with Prince and Chris into the car, users did not know that
they were members of the Citizen team. As a user, this scene
could be alarming. Commenters had already shared how kids with
autism might go easily with strangers, and then they watched the
live broadcast as he did just that. Moving forward, how can we
ensure our street team is presented as a safe community
alternative to police?"

Asked about the Street Team, a Citizen spokesperson told

Motherboard in an email that "From time to time, we put temporary
teams in place in some of the cities where Citizen is available to
demonstrate how the platform works, and to show responsible
broadcasting practices—similar to how social media platforms
have paid creators." Slack chats obtained by Motherboard indicate
Citizen may have used a Street Team during the bounty incident
as well.
The Citizen document about the Jeremiah case also reflected on
what could be improved, asked how the app could be changed to
help setup search parties for users to join, or for people to report
missing persons, and acknowledged that there were significant
privacy, safety, and special needs issues with how Citizen handled
the live streaming of a search for an autistic child.

Externally, this was presented as a success.

"If you're watching Citizen, keep watching so we can reunite more

people, and use technology for good," the narrator of the recap
video posted by Citizen said.

"The Jeremiah incident was spun extremely positive despite all red
flags—a missing child was found live. That was how it was
presented. So it was essentially fuel for them to continue
searching for these big incidents that would activate users," a
source with knowledge of the company's operations added.

Shortly after, another such incident appeared: the Palisades

wildfire. In contrast to how the Jeremiah broadcaster said Citizen
was about reuniting people and using technology for good, during
the Palisades manhunt Frame said on Slack he wanted to create a
situation where criminals felt like "'This is tech closing in on you.
Good luck buddy.'"

Publicly, Citizen said it was very sorry for the Palisades fire
incident and for putting a bounty on the head of an innocent man.
The company called the entire incident "a mistake we are taking
very seriously."

Kris and Prince, two of the public faces of the manhunt, both
posted lengthy messages on Slack about what they believe
happened Saturday night. Kris said she didn't know what she was
getting into: "It was not immediately clear that the pursuit/reward
was the angle of the OnAir. we have successfully covered wildfires
before without that aspect," she said. "I will say I agreed to help
before I understood this but that I take full responsibility for not
backing out once it was clear I’d have to say the reward."

Prince said "I went too far and I'm willing to accept the
consequences. We should not be labeled as lynchers or
encourage anyone to capture. I understand that we are not a news
app. We are not the cops either."

While many employees at Citizen felt the Pacific Palisades

incident was a huge mistake, Andrew Frame looked at it differently.
While Frame showed some contrition, he sees the bounty
experiment as a "massive net win," a step on the way for his app
to become a private safety network that is "going into what the
government is failing to do," which is, in the company's mind,
failing to keep people safe, according to his Slack response to
Prince.

wildfire.jpeg
Firefighters battle the Palisades fire in Los Angeles. Image: Eric
Thayer/Bloomberg via Getty Images.

"There is no consequence for taking a risk, safe environment," he

said, not acknowledging the potential danger the company put the
man they falsely accused in. "This product work is the future of the
world ... the team came together on a Saturday and pulled off
something incredibly awesome."

In a later all-hands the Wednesday after the incident, Frame

admitted that the company "got a lot of things wrong Saturday,"
according to official notes posted on Slack and obtained by
Motherboard. But "even though this feels really bad on the outside,
it's not nearly as bad as it may feel. One of our investors wants to
assuage everyone's concerns," Frame said. "The investors have
never been more excited."

"For me, this is a cultural milestone," he said, adding that the team
had devised a new strategy: "The next 50-100 OnAir stories will be
heartwarming ones because we will do this responsibly." When
asked questions about the meeting, a Citizen spokesperson told
Motherboard in a statement that "We don’t comment on anecdotes
from internal meetings that are taken out of context."

Frame said at the all-hands that he is still performing a manhunt

for the person Citizen falsely accused, but this time in order to
apologize.

"We need to find this person and we are actively looking to find
him. We are not done when it comes to this person," notes from
the all-hands say. "Andrew [Frame] said they are working on that
and this has the chance to turn into a very happy moment."