Module 5

Prototyping the Pilot Execution

 Prototype developing Stages

 Deploy real-time UI/UX visualizations

 Methods and metrics to analyze and convey business outcomes

 Feedback and data obtained from execution

Proof of Concept
POC (Proof of Concept) is an approach used to validate the idea or the feasibility of the concept.
The stage comes into existence way before an application development process starts.
A small project is implemented for verifying if a concept can be implemented on the technical capability and business model
grounds.
An application POC is not usually shown to the public or the customers.
The main idea behind it is to simply check if the idea is workable or not.
It is the method that leads the development of an application to a definitive answer regarding the viability of a product. It gives
an answer in the form of a yes or a no. It helps a company put its money on the right product. In case a company wants to
know whether or not it is feasible, the POC concept gives a precise answer. If the answer is no, a company can try out any
other idea to achieve its objectives.
It is build to verify some minor technical assumptions before getting down to development. When you aren’t sure if your idea
can possibly work build a POC not MVP
It is like a small research that gives you green light to go further with the development of the product
POC cover a small part and not the entire system and user will never get to see it because POC is used inside the company
just to clarify which way to go with the development

 Can be used to obtain seed-stage funding

 To check if the idea works, technically
 Intend to share internal knowledge among team
 To assess project or the features’ success
Prototype
While POC offers you a model of just one product’s aspects, a prototype is a working model of several aspects of the product
The development team usually uses prototype discover errors in the system. By building the prototype they test the product’s
design, usability, and functionality. POC verify only single issue. Rapid prototyping allows to better plan for executing on little plan
A prototype simulates the full system or at least a relevant part of it. While a POC shows that a product or feature can be done, a
prototype explains how it will be done.
Prototype focuses entirely on showing how an application flows, how it’d look to the end users, what would be its usability, and
what all features must be added in the application.
A cost-effective and rapid way to bring ideas to life and validate hypotheses.
Can be used to attract investors for product development and prove a faster time-to-market.
Allows for an IoT company to take into consideration client needs, inputs, wishes and quickly demonstrate through mockups and
wireframes what their solution would look like.

 Used to visualize how the app would function

 Requires limited money, material and time to show the application performance to stakeholders
 Requires very limited technical availability
 Gives instant look and feel of the application
 Prove the value of the system to the user
 Allow users to interact with and explore the system
 Enables us to get customer feedback
 Low risk high reward
Piloting/ Minimum Viable Product (MVP)
MVP is a functional application that comes with the prime features that best represent the application. The MVP development
process helps in verifying the app feasibility, teams’ assumptions about the application, and its probable usability along with
the market demand. When you build a minimum viable product, it ultimately helps in developing an application which is
prepared to gather the end-user feedback.
MVPs are essentially prototypes, except they are normally created at a much later stage of the product development journey.
MVPs are built to demonstrate the functions that the final product will possess and are generally distributed to the early-
adopters of the product. These early adopters begin to test the MVP and provide value-added feedback on the functionality,
design and user experience of the product. Based on the feedback, iterations are made to the MVP to evolve it into the market-
ready product that the creators have in mind. MVPs give creators the opportunity to indicate the value their product can
provide to the end-consumers.
A pilot is a productionalized system available for a subset of the whole audience. The reason for doing a pilot is to get a better
understanding of how the product will be used in the field and to refine the product.
MVP represents only the core functionality and the purpose is to get the minimum version of the product to the market. By this
way you will get to know whether it has any value at all.

 Used to show the customers a working app

 Used to make money out of the application
 A bug-free app ready to be used by the world
 higher retention at low cost
 Used to know the market perception to improvise the application
POC – Prototype – MVP

Milestones to Product Market Fit Product Market Fit

Market validates value
proposition and business model

Minimum Viable product

Market Validation
This should be built
This will be viable

Prototype
This is how we will build it
This how it will be used
Proof of Concept
This is the feasible solution
Prototype Developing Stages
Determine Objectives – Develop – Refine – Demonstrate - Test - Implement

Bringing IoT prototypes from idea to reality fast and reliably

Electronic Development
Design circuitry needed and embed it in a PCB.
Understand sensors and new technologies for optional
solution
Mechanical development
Implement all kinds of moving parts and actuators in your
prototype
Web Development
Develop all the dashboards and web pages functionalities
that your prototype may require
App Development
Develop Android, IOS and Windows Apps based on both
Desktop and Mobile devices to grant you prototype
further functionalities
Prototype Development Process
Example Model for Prototyping IoT
Goals

Physical hardware – Host device’s sensors, modules, and

functionality.
A mobile application – That app you open on your mobile that lets you
Goals
read your hardware’s sensor data, allows you to trigger functions,
and helps you interface with your device.
A web application – The cloud-connected and portable display of
information that your device feeds to you.

Web controlled object

Constraints

The user is able to control a

certain physical object.
through a web interface. E.g.
Modules turn on/off motor
Sensor-logging System

The device is used to collect information from the surrounding environment and then
logs the data to a mobile app for the user to view.

Full Stack Prototype

The device interacts with the user, the environment, and the mobile app. A mobile
app can be used as a controller for the device. the data from the device can be
uploaded to the web app for storage.
Constraints
Modules Processor Modules
Arduino
RaspberryPi

Power Modules
Rechargeable LiPo batteries
Rechargeable/replaceable coin cells
USB powered
PoE (Power over ethernet) – often used in “smart homes”, “smart buildings”

User Input/Output Modules

User Interaction: Buttons, Switches, Knobs, LEDs,
Speakers, LCD displays, Touchpads

Sensor Modules
Temperature Sensor, Accelerometer Sensor, Gyroscope Sensor,
Connectivity/Communication Modules Brightness Sensor, Humidity Sensor, Camera

LoRa – wireless Action Modules

BLE, Cellular, Wi-Fi,
Zigbee – wireless Relays
USB – wired
Thread – wireless Motor controllers
Ethernet – wired
Z-Wave – wireless Sound amplifiers
Deploy Real-time UI/UX Visualizations
Mobile App
Data Visualization
Application Layer Application Development
(Creating Dashboard)
Web Application

Network Layer Network Layer

Device Layer Data Collection, Storage, Basic Analytics

User Interface
IoT System
Physical Environment End User
(Physical Set –up) User communicate with the
IoT System via application
(Web and/or mobile)

Type of communication takes place between user and IoT System - Example
 IoT System to User – Data Visualization
 IoT System to User – alerts/notification
 User to IoT System – Remote control of devices
User Interface(UI)
 Interfaces are very different, and their design determines how convenient and pleasant they will be to use
 An important metric for a decent interface can be the inclusion threshold. In case of mobile app development,
it refers to the effort or time it takes to start operating a certain interface.
 UI design must be
 Convenient
 Simple and easy to use
 Understandable by the user

Challenges in Creating UI/UX Design for IoT Applications

 Myriad of emerging IoT products
A year ago the number of IoT devices was 20.4 billion, and it is forecasted to grow to 75 billion, by 2025. The competition is massive. A
large number of newly emerged start-ups as well as software giants, such as Google and Amazon, launch new products and constantly
innovate the old ones.

 High Customer Demands

As competition rises and customers become more sophisticated, demands grow as well. Users choose IoT products that stand out from
the crowd. The challenge is to define the users’ expectations and meet their needs and desires. The design approach requires solid
experience in identifying the underlying needs and preferences of potential users.
 Need for Dynamic Adjustment
The IoT landscape evolves at a rapid pace. The volatile device designs, growing capabilities of the IoT technology, emerging new
features and entire fleets of devices are major realities in today’s IoT market. To make a new IoT product appealing and useful for its end
users, such trends must be taken into account and reflected in the IoT mobile app. Besides, IoT apps must also be able to scale up and
adjust on demand.

 Smooth UX across all Devices

The difference of UX and UI in IoT
As IoT platforms manage a large number of various devices
from lighting to remote vehicle charging, the IoT interface
can be extremely complex and difficult to understand if
designed poorly. A smooth user experience (UX) that
provides appealing and user-friendly way to manage
varying interaction between objects requires a deep
research into users’ behavior patterns in their own
environments. A strong IoT app design enables its user to
move from one part of the system to the next one without
much effort.
Creating UI/UX Design for IoT Applications

 User-centricity and Personalization

Personalization in the IoT landscape is extremely important, it is also very challenging. For instance, in case of large IoT platforms that
manage warehouses or power grids, app designers have to consider multiple groups of end-users who have access to the IoT app and
design app UI and UX in accordance. In such scenario for designing the UI/UX following steps can adopted
 Conduct in-depth research into users’ behavior in order to design an app that will definitely be easy-to-use and attractive to the target
audience.
 Find a group of potential users and interview them in order to gain valuable insights and reveal relevant trends in their evaluation of
the product.
 When the pilot version of an IoT app is ready, test the app design on our target group to identify flaws, fix them, and enhance the
product.

 Alerts and Push-Notifications

The most sensitive and challenging feature in IoT app design is notifications and alerts. Mostly IoT applications provide users with
notifications passively rather than actively in order to not irritate them with unnecessary information, deterring from using the
application. On the other hand, the complete rejection of push notifications is also an absolutely failed strategy. In some cases, for
instance, in an emergency situation or in cases when some anomalous activity is detected, users must be immediately notified. But the
challenge here is to identify which information is to be prioritized, should the notification be sent immediately or it is necessary to
determine the most appropriate for the user to be notified.
 Remote Control
Another important feature of every IoT app is a convenient and user-friendly remote control. For instance, on the way to work a user
suddenly remembers that he or she forgot to lock the front door.

 Data Visualisation
As IoT applications manage huge amounts of data from various devices, sensors, and integrated platforms, they must analyze, select
and present relevant information in an understandable and user-friendly manner . Dashboards, charts, graphs and tables are heavily
applied in IoT interfaces to facilitate understanding of complex processes and information reports. Noteworthy is the fact that selection
and presentation of valuable for users’ content is the most challenging task in IoT data visualization.
Steps for an effective IoT data visualization
 Select an effective data representation model, which will represent the gained data in a way best suited to end-users.
 Apply appropriate filters to sort out relevant information from data that can be omitted.
 Ensure that the omitted information can be easily retrieved and detailed upon user request.
 Enable users to smoothly move from one view to the next one and get relevant information and necessary details on the spot.

 Mobile Capabilities
The designer must also be a visionary. The key is to predict how, when and where people will use the application, what functions will be
important for them, what devices they will be using. Smartphones offer a great number of functionalities. such as voice interface,
location tracking etc. If, for instance, some users prefer a time-saving app with voice control, accordingly, the application should
provide this opportunity. Designers must take into consideration all available capabilities of mobile devices and enhance designed IoT
applications with best UX.
 Consistency
One of the highest priorities in designing an IoT app is consistency. As IoT platforms are mostly extremely complex in terms of
managing various objects, devices, and systems, it is highly important that the design contains unifying elements, making it easy for
users to navigate across diverse functionalities of each integrated item. Consistent design will allow users to smoothly jump from one
part of the IoT application to the next one.

 Scalability
The IoT landscape is a fast-developing and constantly evolving segment. For instance, a Google-owned company Nest started its way in
the smart home market with smart thermostats, but within a short time it began to rapidly grow and today Nest offers overall
management of home devices. Any IoT company must develop a strategy for scaling up end-user tools and create effective IoT design
models for various sets of devices.
UI/UX Design – Smart Home System

Temperature

Living
Master Heater Fan
Room

Kitchen AC

Heater Mode

Auto Manual Sleep Timer

Heater Display

ON Auto OFF

Usage of Electricity (Data Visualisation)

kWh

Day
UI/UX Design – Fitness Wearables

Steps Sleep

Heart
Running
Beat

Active
Time
Methods and Metrics to analyze and convey business outcomes
Business aims to build product and services to fulfill the requirements of customers. To achieve the next
level of success it is necessary to track the business metrics and key performance indicators (KPIs).
Business metrics – tracks all area of business
KPIs – targets only the critical area of business

Business metrics
 Business metrics are used to track, monitor and analyze quantified measures(example: response time, first
response time, profit margin, minimum time to restore) that assess the business process’s success or
failure.
 It helps in effective decision making for business management. It is important to note that business
metrics address the business stakeholders, like; customers, managers, business owners, investors, and
vendors.
 Business metrics give an insight into the business before making a deep dive into the core business data.
 Business metrics address the stakeholders.
 It depends on the type of business, industry, business departments and business size.
 Each business department track, monitor and analyze their specific performance and key metrics.
Business Metrics - Examples
 Sales Metrics
Lead Response Time:
It is the average time your sales rep, marketing rep or customer support rep responds to the lead.

Example: For example, you run an e-commerce store and a website visitor leaves a query on your contact form. You replied the lead
after 15 minutes, which means your lead response time is 15 mins.
 Marketing Metrics
Website Traffic:
It is simply the number of website visitors landing on your website domain.

 Financial Metrics
Profit Margin
 Human Resource Metrics
 Project Management Metrics
Critical Path (CP):
A Critical Path (CP) represents the longest path to complete the entire project. It includes every activity that must be performed to get
the desired project result. A critical refers to the scheduling order that makes a complete project. It helps in pointing out which
activity could delay the project delivery date.
 Product Performance Metrics
 Other Important Business Metrics
Monthly Recurring Revenue (MRR): Total revenue from existing customers in the last month
Metrics for every IoT stage(Start-up, Scale-up and Enterprise)
Start-up Phase
1. Business case development
2. Build vs Buy
3. PoC
4. Prototype
5. Pilot
At this initial stage/start-up phase proposition and technology will be tested. These are both important in different
ways:
Capturing proposition metrics reveals how the device is being used, for example
• Did it get successfully installed?
• Which features are being used, such as which buttons are being pressed?
• Does it get used regularly?
Capturing technology metrics can reveal the underlying function of your devices. Some examples include:
• Battery life and how it varies
• Signal strength – by location, by SIM vendor and other factors
• Application crashes and free memory
Metrics to be considered while shifting from start-up to scale-up phase
To test whether you’re ready to embark on your next stage of growth, decide on a set of quality metrics
such as:
• Less than N% new software bugs reported
• Uptime more than X%
• Customer support calls less than Y% per week
• Battery life more than Z%
Scale-up
At this stage the number of devices deployed will probably be growing to thousands and then tens of thousands.
This is the stage when the development mindset must give way to a day-to-day operations mindset – from
manufacturing to logistics to support to service, creating a well-oiled machine that gets better and bigger every day.
The team must become efficient at constantly scaling itself – which we can define as supporting an exponentially-
growing estate of devices with only linearly-growing costs.
You’ll now find yourself needing to track more numbers such as:
• How many and which sort of devices have been deployed, where?
• Are they working? If not, why not?
• How many are running old software?
• Is the new firmware performing better than the old?
The problem, however, is that once you have a thousand or more devices deployed, manually-kept lists – whether on
paper or a screen – become overwhelming, so various tools are needed to help keep track of devices, analyse their
performance at scale and solve the most pressing issues first.
Enterprise
At this stage, the product will be starting to mature as a category, though still growing fast – you should have good
processes, clear ownership of responsibilities and a better idea of drivers for cost and profitability.
To grow even faster, you may choose to sell to larger enterprise customers or through channel partners, and
increasingly they’ll demand reports proving that you are deploying devices as planned and keeping them running well.
Focuses on service level agreement
- Availability of service
- Hours of service
Example
 Energy delivered
 Parking space minutes
Gather feedback and data obtained from execution of the pilot

IoT Solution Development Stages

Business case Development
Build vs Buy
PoC
Prototype Technology-led implementation rather than Value-led implementation
Piloting/Initial Roll Out
Final roll out

Technology-led implementation – Example Value-led implementation – Example

 Device Hardware and Software  Integrity
 Connectivity  Scalability
 App development  Interoperability

Implementing new technology, value can be amplified

 PoC (Verification of Piloting(Launching in market with specific
Prototype(Developing Model)
Idea and Concept) group of people)

Define Determine Objectives Define Objectives

Develop Develop Develop

Test, Refine and Validate

 System is tested in real-time
Execute Demonstrate  Evaluate the results and compare the KPIs
Stages with the objectives
 Validate the outcomes
Test
Evaluate Implementation
(Testing the system in lab condition)

Refine

Implementation

Pilot Testing/ Evaluation

 Verifies a component of the system or the entire system under a real-time operating condition.
 The purpose of the Pilot Test is to evaluate the feasibility, time, cost, risk, and performance of a research project.
 In Pilot testing, a selected group of end users try the system under test and provide the feedback before the full
deployment of the system.
An evaluation is a systematic assessment of how well a project or program is meeting established goals and
objectives. Evaluations involve collecting and analyzing data to inform specific evaluation questions related to project
impacts and performance. his performance information enables project managers to:
 Report progress and make improvements, as necessary, to ensure the achievement of longer-term impacts
 Assess and communicate the effectiveness of new technologies

Purpose of Pilot testing

Pilot Testing is Important because it helps in many ways like
 debugging software and procedure used for testing
 checking product readiness for full-scale implementation
 better decision on time and resources allocation
 gives opportunity to gauge your target population’s reaction to program
 measurement of success of program
 gives team a chance to practice activities they will use for usability test
 Demonstrate the values to the stakeholder
 Helps in finding the errors/bugs and improving the quality
Data Collection Methods - Example
Information Sources/Method Data Sources Data Collection Considerations

 Field test location/scope

 Data collection period
 Data elements to be collected, including unit of analysis
Field Test  Sensors, devices  Data collection frequency/interval (hourly, daily, etc.)
 Data requirements related to modeling or simulation (if applicable)
 Data management (e.g., storage, quality control)
 Data security (e.g., protecting privacy)

 Target population and sampling procedures

 Participant recruitment/contact procedures
 Survey responses  Expected sample size
Surveys or Interviews
 Interview responses  Methods for encouraging survey response
 Survey administration period
 Key topics to be addressed in survey and/or interview guides

 Information management systems

 Data collection period
 Operations data (e.g., response times, system
 Data elements to be collected, including unit of analysis
Internal Agency Data downtime, maintenance data, BW, network
 Frequency/interval (hourly, daily, etc.)
connectivity), website tracking, reports,
 Accuracy/completeness of internal agency data
documents, etc.