Professional Documents
Culture Documents
Artificial Intelligence in Label Free Microscopy Biological Cell Classification by Time Stretch 1st Edition Ata Mahjoubfar
Artificial Intelligence in Label Free Microscopy Biological Cell Classification by Time Stretch 1st Edition Ata Mahjoubfar
https://textbookfull.com/product/label-free-and-multi-parametric-
monitoring-of-cell-based-assays-with-substrate-embedded-sensors-
oberleitner/
https://textbookfull.com/product/artificial-intelligence-by-
example-develop-machine-intelligence-from-scratch-using-real-
artificial-intelligence-use-cases-denis-rothman/
https://textbookfull.com/product/atomic-force-microscopy-in-
molecular-and-cell-biology-jiye-cai/
https://textbookfull.com/product/real-time-speech-and-music-
classification-by-large-audio-feature-space-extraction-1st-
edition-florian-eyben-auth/
Intelligence is not Artificial A History of Artificial
Intelligence and Why the Singularity is not Coming any
Time Soon Expanded Edition Piero Scaruffi
https://textbookfull.com/product/intelligence-is-not-artificial-
a-history-of-artificial-intelligence-and-why-the-singularity-is-
not-coming-any-time-soon-expanded-edition-piero-scaruffi/
https://textbookfull.com/product/biological-classification-a-
philosophical-introduction-1st-edition-richard-a-richards/
https://textbookfull.com/product/teaching-music-to-students-with-
special-needs-a-label-free-approach-second-edition-hammel/
https://textbookfull.com/product/fluorescence-microscopy-from-
principles-to-biological-applications-2nd-edition-ulrich-
kubitscheck/
https://textbookfull.com/product/artificial-intelligence-in-
healthcare-1st-edition-adam-bohr-editor/
Ata Mahjoubfar · Claire Lifan Chen
Bahram Jalali
Artificial
Intelligence
in Label-free
Microscopy
Biological Cell Classification by Time
Stretch
Artificial Intelligence in Label-free Microscopy
Ata Mahjoubfar • Claire Lifan Chen • Bahram Jalali
Artificial Intelligence
in Label-free Microscopy
Biological Cell Classification by Time Stretch
123
Ata Mahjoubfar Claire Lifan Chen
University of California Los Angeles University of California Los Angeles
Los Angeles, CA, USA Los Angeles, CA, USA
Bahram Jalali
University of California Los Angeles
Los Angeles, CA, USA
High-throughput real-time imaging and vision systems for capture and identification
of fast phenomena are among the most essential tools for scientific, industrial,
military, and most importantly biomedical applications. The key challenge in these
instruments is the fundamental trade-off between speed and sensitivity of the
measurement system due to the limited signal energy collected in each measurement
window. Based on two enabling technologies, namely, photonic time stretch and
optical amplification, several novel high-throughput optical measurement tools are
recently developed for applications such as volumetric scanning, vibrometry, and
flow cytometry. Here, we introduce time stretch imaging, a high-content computer
vision system developed for big data acquisition and analysis of images. Time
stretch imaging is able to capture quantitative optical phase and intensity images
simultaneously, enabling accurate surface inspection, volumetric scans, defect
detection, cell analysis, and cancer diagnostics.
We further describe a complete artificial intelligence pipeline for time stretch
microscopy that performs optical phase measurement, image processing, feature
extraction, and classification. Multiple biophysical features such as morphological
parameters, optical loss characteristics, and protein concentration are measured on
individual biological cells. These biophysical measurements form a hyperdimen-
sional feature space in which supervised learning is performed for cell classification.
The technology is in clinical testing for blood screening and circulating tumor
cell detection, as well as studying lipid-accumulating algal strains for biofuel
production. By integrating machine learning with high-throughput quantitative
imaging, this system achieves record-high accuracy in label-free cellular phenotypic
screening and opens up a new path to data-driven diagnosis.
Furthermore, we explained a real-time image compression technique performed
in the optical domain to solve the big data challenge created by ultrafast mea-
surement systems. Many ultrafast and high-throughput data acquisition equipment,
including time stretch imaging, produce a torrent of data in a short time, e.g.,
several gigabytes per second. Such a data volume and velocity place a burden on
data acquisition, storage, and processing and call for technologies that compress
images in optical domain and in real-time. As a solution, we have experimentally
v
vi Preface
We would like to thank all of our publication coauthors, which our collaborative
works consist many chapters of this book. Chapter 2 is a version of IEEE 49th
Annual Conference on Information Sciences and Systems (CISS) Proceedings, pp.
7086896, 2015 [1]. Chapter 3 is a version of Applied Physics Letters, Vol. 98, No.
10, 101107, 2011 [12]. Chapter 4 is a version of Proceedings of SPIE, Frontiers
in Ultrafast Optics: Biomedical, Scientific, and Industrial Applications XIII, San
Francisco, CA, 2013, 86110N [16]. Chapter 5 is a version of Biomedical Optics
Express, Vol. 4, No. 9, pp. 1618–1625, 2013 [17]. Chapters 6 and 8 are based on
Scientific Reports, Vol. 6, pp. 21471, 2016 [3]. Chapter 7 is a version of Proceedings
of SPIE, High-Speed Biomedical Imaging and Spectroscopy: Toward Big Data
Instrumentation and Management II, San Francisco, CA, 2017, 100760J [2].
Chapter 9 is a version of PLoS ONE, Vol. 10, No. 4, pp. e0125106, 2015 [104].
Finally, Chapter 10 is a version of Scientific Reports, Vol. 5, pp. 17148, 2015 [4].
vii
Contents
ix
x Contents
References .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 123
Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 131
List of Figures
xiii
xiv List of Figures
Fig. 6.5 Quantitative optical phase and loss images of OT-II (blue)
and SW-480 (green box) cells; The optical loss images of
the cells are affected by the attenuation of multiplexed
wavelength components passing through the cells. The
attenuation itself is governed by the absorption of the light
in cells as well as the scattering from the surface of the
cells and from the internal cell organelles. The optical loss
image is derived from the low frequency component of the
pulse interferograms. The optical phase image is extracted
from the analytic form of the high frequency component
of the pulse interferograms using Hilbert Transformation,
followed by a phase unwrapping algorithm. Also,
supplementary Videos 1 and 2 show measurements of
cell-induced optical path length difference by TS-QPI
at four different points along the rainbow for OT-II and
SW-480, respectively . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 59
Fig. 6.6 (a) Pairwise correlation matrix visualized as a heat map.
The map depicts the correlation between all major 16
features extracted from the quantitative images. Diagonal
elements of the matrix represent correlation of each
parameter with itself, i.e., the autocorrelation. The subsets
in box 1, box 2, and box 3 show high correlation because
they are mainly related to morphological, optical phase,
and optical loss feature categories, respectively. (b)
Ranking of biophysical features based on their AUCs in
single-feature classification. Blue bars show performance
of the morphological parameters, which includes
diameter along the interrogation rainbow, diameter
along the flow direction, tight cell area, loose cell area,
perimeter, circularity, major axis length, orientation, and
median radius. As expected, morphology contains most
information, but other biophysical features can contribute
to improved performance of label-free cell classification.
Orange bars show optical phase shift features, i.e., optical
path length differences and refractive index difference.
List of Figures xxi
Those who had cheap horses and donkeys sold them, and the
price of the cheaper class of horse fell very low, till at last beasts
were turned out as worthless, and killed and eaten by the poor.
Meat fell in price, but this did not much help those who had no
money to buy even bread. Large establishments were suddenly
much reduced; the armies of hangers-on, who live on the leavings of
the rich and their attendants, were now thrown upon the streets.
Many of the bakers and butchers closed their shops and fled.
Mule-hire rose to an almost prohibitive price, and it must be
remembered that this, in a country where all transport is by mule and
camel, meant the paralysis of trade. All the animals, save of the very
rich, presented a half-starved appearance. In the waste grounds
near the towns, and by the sides of the high-roads, lay the bodies of
dead and dying mules and horses.
Flour became adulterated, and was ground at home by the
consumers. Grandees, and merchants began to lay in stores of grain
from their villages, disposing of none, although an enormous profit
could be obtained on the contents of their granaries. The Governors
of the towns seized grain, or paid for it at a nominal price, and sealed
it up in the public or Government grain stores. Provender on the
high-roads became unattainable.
Prices, though steadily rising for all descriptions of cereals,
suddenly dropped on the hope of rain, only to rise in a few hours to a
still more serious figure. The lower classes began to pledge and sell
their copper-ware, tools, arms, and clothing. In the post-houses,
where from six to ten horses were generally kept, only two, and at
times none, were seen.
Villagers in quest of food began now to pour into the towns, and
remained herding in starving crowds in the mosques, having neither
the means nor the strength to return to their homes. The charity of
the Persians themselves was nearly exhausted, for each rich man
had to feed his crew of hungry servants and their families. The few
unorganised attempts to feed the poor, resulted in the crushing to
death of several, and the one loaf of bread doled out to each person
on these rare occasions only served to prolong their sufferings.
Children now began to be deserted in the streets, the dead and
dying to be seen frequently, the greater portion of the bazaar to be
closed, typhoid to be rife, and crimes of violence to be frequent.
And now came the first funds from England from the Persian
Relief Committee. In each town the money was husbanded and relief
given in the way most efficient and economical. Money was found to
be the most safe plan, at all events in Shiraz, of which I speak from
experience, for any attempt to buy bread in quantities failed, and
caused an immediate rise in price. Very many applicants were sent
away; relief in the shape of a numbered ticket, entitling the bearer,
whose person was described in a book kept for the purpose, to
weekly relief in money, was given to the utterly destitute. The
difficulty of deciding on the claims of the various applicants was
great, and in many cases which had to be denied permanent relief,
temporary alms were given.
A large house was rented, and in it were placed all the deserted
orphans found in the streets; these were mostly the children of
villagers, though some were those of townspeople. These children
were plainly but comfortably clothed in the ordinary dress of well-to-
do Persian villagers, and well and regularly fed. They were placed
under the care of an intelligent and humane Persian, who really did
his duty to them, and were regularly inspected by the members of
the Relief Committee; also they were frequently seen at unexpected
times. The poor emaciated bundles of rags soon developed into
strong, healthy children, and the regular food, comfortable quarters,
and good clothes did wonders. Most of the staff took one or two into
their service.
Seven years after, one of my two, who were taken as stable-helps,
was getting pay from me at the rate of thirty shillings a month, and
was my head groom, and would anywhere obtain that pay. Two were
taken as markers in the billiard-room, and are now respectable
servants. As the famine ceased, the unclaimed orphans were
apprenticed to good trades, or placed in the houses of wealthy
Persians as servants. No attempt at proselytism was made, but a
Persian priest was engaged to teach the usual rudiments of reading,
writing, and the Koran.
Many villagers came in and claimed their children, and these were
often loath to leave their clean quarters and good food, to return to
hard drudgery and rags in their native villages.
It may be safely said that no deaths from starvation took place in
Shiraz after the arrival of the first instalment of relief money from
England. Of course, the application of the funds was carried out
irrespective of the religion of the applicants; and this application was
easier in Shiraz than in Ispahan. The Armenian community in Shiraz
were very few, and only some four families needed relief; while, on
the contrary, the Jews were many and terribly poor.
As to the labour question, a few of the more able-bodied were set
to the nominal work of picking the stones off the high-road, but no
heavy labour was insisted on. In the winter, too, the snow having
blocked the streets, the poor were employed in removing it for the
general good.
I happened to go to Ispahan, and also assisted in the distribution
there. The Ispahanis are much more provident than the people of
Shiraz, and I do not think the distress would have been so great but
for the influx of villagers. At Koomishah, the third stage from Ispahan
towards Shiraz, the effects of the famine were very severe, and I
was glad to be able to distribute some four hundred kerans of the
Poor Fund, both going and coming, there. Of course this amount did
not go far, and I was besieged in the post-house by the hungry
crowd of women and children; the sum was too small to permit of
giving anything to the men. First we admitted all the aged women,
and gave them a keran and a half each; then each child was given a
keran, and, when they had secreted it, the whole number were
passed out and the gates closed. From the roof of the post-house I
perceived a big burly villager, who was employed in robbing the
children, as they went out, of their slender store, even throwing them
on the ground and taking the coin from their mouths. The other
villagers, of whom there was a large mob, merely laughed, but did
not interfere. But getting down from the back wall of the post-house
by means of horse-ropes, the postmaster, my groom and I
succeeded in catching the fellow, and dragging him into the post-
house, and then the post-boys gave him a good hiding by my order,
and we took the money away. He, of course, complained to the local
Governor, who requested an explanation. I called on him and told
him of the fellow’s misdeeds, and, much to his astonishment, the
man in power gave the ruffian a liberal bastinado.
Terrible stories are extant of what happened in certain places, and
there is no doubt of the truth of many of these. That the people ate
grass and the carrion, that they lived on the blood at the public
slaughter-houses, that they, having sold all, also sold their children,
is within my personal knowledge. Cannibalism, too, was proved. In
fine, had it not been for the exertions of the Persian Relief
Committee in London, the ravages of the famine would have only
ended in the temporary depopulation of the south and centre of
Persia.
Each great personage in Ispahan and Shiraz did his best to
preserve his own dependents from starvation; but there being no
kind of organisation among the Persians, and transit-rates being
prohibitive, and the roads unsafe, small local famines were frequent,
and the ravages of typhoid and diphtheria—the latter previously
unknown in the country—were very great.
Just now an accident to the Prince Zil-es-Sultan took place. He
was out shooting near Shiraz, and having charged one barrel of his
gun twice, the weapon burst, tearing the palm of his hand and the
ball of the thumb. I was called in to attend him, and was fortunate
enough to preserve the hand. For this his Royal Highness was very
grateful, and during the whole of my time in Persia showed me many
kindnesses, besides giving me an extremely liberal fee, even for a
king’s son: he compelled his vizier also to give me one. He even
insisted on decorating me with the star of the Lion and Sun; but as
Englishmen in Government employ are not allowed to accept the
decorations of foreign Governments without special permission, the
honour, much coveted among the Persians, was not of much benefit
to me. I got it in a very public and sudden manner, and as the
occasion of giving it was sufficiently curious, I may as well describe
it.
It is the custom in Persia to send to all governors, royal
personages, and ministers, a yearly present from the king, to show
the royal satisfaction. These presents are all termed kalāats (or
dresses of honour), even though the gift may be in jewellery, or even
specie; a dress or robe of greater or less value, or a jewelled
weapon, being the general kalāat. The withholding of the yearly robe
of honour to a provincial governor is generally the sign of the royal
disfavour, and the despatching of it often the token of the recipient’s
confirmation in office, though at times it is what gilds the bitter pill of
his recall. The kalāat is usually sent from the capital by the hands of
some person of consequence, generally some favoured servant of
the Shah, and this man is sent down that he may receive a present,
generally large in amount, from the recipient, and may bring back the
usual bribe to the Prime Minister for retention in power, or even the
same thing to the king himself.
The New Year’s festival is generally the time of the despatch of the
official dresses of honour from the capital. The bearer, and his two or
more attendants, generally come on post-horses, and the etiquette is
that the recipient goes out to meet the royal gift. The bringer, on
arrival at the last stage, is met by the servants or friends of the
recipient, who send off to announce the arrival. He now takes off his
travel-stained garments, puts on his finery, and starts on horses sent
out for him, bearing the royal bounty at his saddle-bow wrapped in a
Cashmere shawl. The recipient, accompanied by all his friends and
the greater portion of the populace—for the bazaar is closed by
order, and a general illumination commanded for the evening; all the
shops are visited, and severe fines inflicted on any one disobeying—
proceeds to meet the present, and await its arrival. The distance that
is gone is regulated by the position of the recipient—the greater the
personage, the less distance he goes.
One morning the prince sent for me and told me that a kalāat from
the Shah would arrive for him the next morning, and that he wished
his hakim-bashi and myself to ride out with the magnates of the
place, who would accompany him, to meet it. I of course expressed
my readiness to attend his Royal Highness, and I was told by the
hakim-bashi, who was very jubilant, that probably a decoration would
be given to each of us. To have declined would have been to give
mortal offence, and to have lost the favour of the Governor of the
province, whose partiality secured me against annoyance from the
natives of any kind. So the next day I presented myself at nearly
noon and found the prince in great feather, the head astrologer
having appointed two in the afternoon for the enduing of the dress of
honour. Every one was in gala dress, the streets were thronged by a
holiday mob in high good humour. And out we all rode. First came
four yessaouls, or outriders, with silver maces, showing off their
horses by capering in circles; then six running footmen, each with his
silver-headed staff and clad in the royal scarlet, in the ancient
costume of Persia, and with the strange head-dress somewhat like a