Carla L. Cornillon1,*, Yuan Liang2, Jia Li3, Tingyi “Leo” Liu4, Connor M. Choi5, Lei He2, and Chang-Jin “CJ” Kim1,6 1 Department of Mechanical and Aerospace Engineering, 2Department of Electrical and Computer Engineering, 6Department of Bioengineering, University of California, Los Angeles (UCLA), California; 3Microfabrica, Van Nuys, California; 4Department of Mechanical and Industrial Engineering, University of Massachusetts, Amherst, Massachusetts; 5Anderson W. Clark Magnet High School, La Crescenta, California, U.S.A.
ABSTRACT existing contact-angle measurement software is eliminated
We report development of a measurement method for by employing a deep learning model that detects the three- the public to upload a sessile drop image and receive its phase contact regions in a wide variety of droplet images contact angles. Different from all existing practices, the and measures the contact angle using the droplet edge reported method is aimed to process a wide variety of within each region. The reported project can support, for images by introducing a deep learning model for detection example, the electrowetting-on-dielectric (EWOD) cyber- and segmentation of the three-phase contact regions. manufacturing system, which aims to offer an online Currently trained to measure angles between ~10° and platform with design tools and links to chip fabricators for ~65°, the model is continuing training to expand the any users to obtain own EWOD devices and exercise detectable range. The measurements are quick and digital microfluidics [13]. automated, producing contact angles within five seconds of receiving an image without supervision. The measurement EXPERIMENT AND DESIGN method is currently being tested on a local website but will The following method describes the process used to be deployed for public use once the full range is reached. automate the measurement of contact angles produced from the sessile drop experiment. Figure 1 illustrates the INTRODUCTION steps for the new approach. Rather than focusing on the full The contact angle formed by a solid surface and liquid- profile of the droplet, either symmetric or asymmetric, to fluid interface is a fundamental parameter used in many create a global approximation of its shape, the proposed fields to determine the properties of the solid surface and algorithm focuses on detecting the three-phase contact two fluids involved. In MEMS, where interfacial properties become more prominent due to the large surface-to-volume ratio, contact angles are frequently measured for both fabrication processes and devices, such as the characterization of anti-stiction coatings [1], wettability controls [2], micro-lens systems [3], and digital microfluidics [4, 5]. Contact angles have been widely measured for academic research and industrial processes in various disciplines, such as chemistry, biology, and polymer science, to measure surface tension [6], characterize surface energy and roughness [7], probe surface charges [8], find work of adhesion [9], and study functional groups grafting between material interfaces [10], to list just a few. Due to the impact in diverse fields, there are many different methods of measuring the contact angles formed at the three-phase contact points [11, 12]. However, existing methods are either too simple (e.g., circle or ellipse fitting methods) to provide accurate measurements or too inconvenient for the general public to Figure 1: Sample images undergoing measurement steps to utilize because nontrivial amount of learning is required for retrieve contact angles. (A) The two three-phase contact proper usage. The contact-angle measurement software regions (i.e., the green boxes) identified during the object that comes with commercial goniometers are highly detection phase. (B) The three-phase contact points (i.e., optimized for the instrument and not only inadequate for the green dots) and the droplet edges (i.e., the red dotted general usage but also ineffective for the images not lines) determined using the segmented images (i.e., the blue obtained by the instrument. triangles). (C) The contact angles calculated using the Focusing on the sessile drop experiment, here we horizontal line (i.e., the yellow straight line) plotted develop a fast and automatic algorithm for measuring the between the three-phase contact points (i.e., the green and droplet interface contact angle, giving the user the red dots) and the tangential lines (i.e., the green and red simplicity of just uploading an image and retrieving the lines) of the fitted droplet edges at the three-phase contact data they need. The human intervention required for all points. regions. This new approach allows bypassing the difficulty measurement. The fit of the localized edge of the droplet of producing the global model from the droplet shape and determined above is used to find the tangent along the edge avoiding the typical errors of resulting contact angles [11, that lies through the three-phase contact point and produces 12] – common to most existing contact-angle measurement the resulting contact angle measurement of the droplet methods. The algorithm first detects the two three-phase interface on both ends of the droplet. contact regions of the droplet image, as shown in Figure 1(A), and then uses only the segmented image shown in Figure 1(B) to determine both the droplet edge and the location of the contact points of the image. From there, it then plots a horizontal line between the two three-phase contact points to approximate the cross-sectional solid surface as reference for measurement and fits the edges with a separate third-degree (i.e., cubic) polynomial. Finally, the contact angles are measured as the angle between a linear approximation from the slope of the polynomial at the contact points and the plotted reference line, as shown in Figure 1(C). The single shot multibox detector model (SSD) is used to detect the three-phase contact region within a droplet image where the contact point resides. The SSD is a deep Figure 2: Cropped original image that has undergone learning neural network that is used for the detection of segmentation within the three-phase contact region to specified objects by plotting bounding boxes at the areas of include only labeled droplet pixels. (A) The segmented interest in an image. Compared to other neural network image shows the droplet pixels labeled as red within the models, the SSD is easy to train, fast, and straightforward three-phase contact region. (B) The interpolated data is in its integration with other tasks, making it compatible shown, including the three-phase contact points, the with our uses [14]. Our model was trained using a reference line, and the tangential lines of droplet edges multitude of images with droplets of varying contact angles used for contact angle measurement. as well as other varying factors including different droplet positions, backlighting, and background conditions. To RESULTS AND DISCUSSION train the model, about 6000 images were manually labeled To evaluate the success of the measurement method, with bounding boxes specifying the three-phase contact we determined both the accuracy of the model’s ability to regions we are interested in. These boxes are also referred detect the three-phase contact region and compared the to as the “ground truth”, and the labeled data is used to train calculated contact angle results with those produced from the model to recognize the area of interest: the three-phase a standard measurement software. Most measurement contact region. software currently available to the public are for a specific Within the three-phase contact region, important instrument (i.e., for processing a certain type of images) information can be interpolated for the measurement of the and also require human involvement before each reading. contact angle. The key to finding this information is to The detection model was tested using a batch of 400 identify the edge of the droplet. In practice, the quality of manually labeled images that were kept separate from the the edge detection is affected by image resolution, pre- 6000 images used for the training and validation of the processing distortion, background noise, and other factors, model. The images were captured by a do-it-yourself (DIY) limiting the precision and accuracy of the measurement contact angle measurement software with changing [12]. Taking these issues in consideration, we restrict the background light and droplet position. To understand the edge detection to the pixels within the area of the three- results, it is important to explain how the detection model phase contact regions. This produces localized profiles of labels the bounding boxes it produces that identify the the droplet edge within these two regions, rather than a regions of interest. The model generates bounding boxes global approximation of the droplet edge across the image. where it detects the three-phase contact region in the testing Applying the k-means algorithm from CNN-based images. We then compared the detected bounding boxes detection for unsupervised image segmentation, the with the ground truth boxes from manual labeling. The algorithm clusters each pixel within the three-phase contact generated bounding boxes are categorized as either positive regions into two categories: droplet pixel or background or negative. Positive signifies whether the generated pixel, as shown in Figure 2 [15]. This clustering is based bounding box contains the three-phase contact region with on pixel intensities, and it optimizes within cluster an Intersection over Union (IoU) larger than 80%. variances. Once each area is properly labeled so that the Moreover, only one generated bounding box was taken as droplet is identified within the region, the algorithm positive for each ground truth box in order to reduce the determines the edge of the droplet area. This edge is used duplicated box predictions. In specific, IoU is defined as: to determine the pixel that represents the three-phase IoU = area of overlap / area of union (1) contact point as well as fit the localized edge with a third- degree polynomial. where the area of overlap refers to the area of overlap The three-phase contact points and pixels of the between the generated bounding box and the ground truth, droplet are used to plot a reference line that approximates and the area of union is the total area between both boxes. the cross-sectional solid surface for contact angle The precision of the model is directly linked to the rate Table 1: Left (L) and right (R) contact angles (CA) at which it produces false positives; the more false calculated using the proposed measurement method and positives the model generates, the less precise it is. In the ImageJ Drop Analysis. The proposed method is automatic case of our model, false positives would occur as bounding and processed in 0-5 seconds for each image. ImageJ boxes that do not contain the three-phase contact region, requires multiple manual steps from the user and would but are labeled as such. False negatives, in contrast, signify take 45 seconds – 2 minutes per image depending on the the existence of a three-phase contact region the model skills and tools. failed to identify. The sensitivity measures the ratio of its true positives to the total of true positives and false Proposed method Using ImageJ negatives, which indicates the ability of the model to detect Automatic; 0-5 sec Manual; 45 sec – 2 min real contact regions. As the sensitivity of the model Image L CA R CA L CA R CA approaches one, the model produces no false negatives and thus accurately detects all the three-phase contact region. A 11.7 7.7 10.2 8.7 Figure 3 shows the accuracy of the detection model by B 16.1 13.0 15.1 11.6 plotting its free-response receiver operating characteristic (FROC) curve [16]. Our FROC curve approaches C 28.0 18.7 23.8 23.5 maximum sensitivity while having a low number of false D 34.8 29.9 34.1 34.9 positive detection per image, which demonstrates the model is consistently and accurately detecting the three- E 34.6 23.0 27.7 26.5 phase contact region. F 38.5 40.0 38.8 42.4
G 50.4 56.1 51.9 57.2
H 69.1 65.6 69.2 67.7
Figure 3: FROC curve for our detection model. The FROC
curve shows the relationship between the model’s accuracy by comparing its sensitivity to its false positive rate as we change the threshold of the overlap for identifying a positive in our model. For each data point in the FROC plot, we set a threshold value, and used it to determine the true positive detection and false positive detection, and further calculate the model sensitivity and false positive per image. If the threshold value is too high, the model is unable to identify both three-phase contact regions, producing false negatives. In comparison, if the threshold value is too low, the model may label false positives. Figure 4: Images passed through both our measuring method and the ImageJ Drop Analysis software. The Table 1 shows the contact angles calculated from our images selected show droplets with different experimental measurement method compared to contact angles measured conditions and contact angles to demonstrate the from a standard software, ImageJ Drop Analysis capability of our measurement method to process a wide (DropSnake), across the range for which our model range of images – difficult for the software coming with currently works. Figure 4 shows the images used in the commercial goniometers. measurements. The calculated contact angles between the two methods are similar (usually within 5° of each other Figure 5 demonstrates the resulting image generated with the greatest difference in measurement equal to ~7°). for a user to validate the contact angle measurements Different factors affect the difference in measurement, calculated from the original image uploaded to the website, notably how the droplet edge was calculated as well as the at this point, hosted locally that contains our measurement method for calculating the contact angles. Despite these method. If the calculated contact angle measurements seem differences, this suggests our model is capable as a means unreasonable to the user, the model returns the generated of measurement compared to other common software used image to show the user the data it collected to produce its for measuring contact angles. calculations. It is important to note the accessibility of a website to a wide range of users, as well as the quickness of the return of the measurements and image. Running the measurement method with a four-core CPU laptop, the information is returned within 5 seconds. “Ionic-surfactant-mediated electro-dewetting for digital microfluidics,” Nature, vol. 572, pp. 507–510, 2019. [6] K. G. Kabza, J. E. Gestwicki, and J. L. McGrath, “Contact Angle Goniometry as a Tool for Surface Tension Measurements of Solids, Using Zisman Plot Method. A Physical Chemistry Experiment,” J. Chem. Educ., vol. 77, p. 63, 2000, . [7] L. Gao and T. J. McCarthy, “Ionic Liquids Are Useful Contact Angle Probe Fluids,” J. Am. Chem. Soc., vol. 129, pp. 3804–3805, 2007. [8] G. Hurwitz, G. R. Guillen, and E. M. V. Hoek, “Probing polyamide membrane surface charge, zeta potential, wettability, and hydrophilicity with contact angle measurements,” Journal of Membrane Science, vol. 349, pp. 349–357, 2010. [9] K. M. Liechti, S. T. Schnapp, and J. G. Swadener, Figure 5: Sample image uploaded by user and the “Contact Angle and Contact Mechanics of a generated image returned by measurement method. The Glass/Epoxy Interface,” International Journal of generated image (B) is the original image (A) labeled with Fracture, vol. 86, pp. 361–374, 1997. the three-phase contact points, the contact line and the [10] A. Y. Fadeev and T. J. McCarthy, “Trialkylsilane linear approximation of the droplet edge used in the Monolayers Covalently Attached to Silicon Surfaces: contact angle measurement. Wettability Studies Indicating that Molecular Topography Contributes to Contact Angle CONCLUSION Hysteresis,” Langmuir, vol. 15, pp. 3759–3766, 1999. We have developed a fast and automatic measurement [11] A. F. Stalder, G. Kulik, D. Sage, L. Barbieri, and P. method that calculates the contact angles from images of Hoffmann, “A snake-based approach to accurate sessile droplets. Our method uses deep learning to detect determination of both contact points and contact the three-phase contact point; it locally and accurately angles”, Colloids and Surfaces A: Physicochemical measures the contact angles using the three-phase contact and Engineering Aspects, vol. 286, pp. 92-103, 2006. points and the droplet edges. Currently being tested on a [12] S. F. Chini and A. Amirfazli, “A method for measuring contact angle of asymmetric and symmetric drops”, local website for accuracies and robustness, the contact- angle measurement results are quickly returned to the Colloids and Surfaces A: Physicochemical and uploading user along with the original image labeled with Engineering Aspects, vol. 388, pp. 29-37, 2011. three-phase contact points. [13] X. Huang, C.-C. Liang, J. Li, T.-Y. Ho, and C.-J. Kim, “Open-Source Incubation Ecosystem for Digital Microfluidics – Status and Roadmap,” Digest of ACKNOWLEDGEMENTS Technical Papers, 2019 IEEE/ACM Int. Conf. This work has been supported by National Science Computer-Aided Design (ICCAD), Westin Foundation (NSF) Award #1711708 (CK, JL) and Award Westminster, CO, Nov. 2019, 9B1. #1720499 (CK, LH, JL), NSF Research Experiences for [14] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, Undergraduates (REU) program (CC), Volgenau Endowed C. Fu, and A. Berg, “SSD: Single Shot MultiBox Chair in Engineering (CK), and University of Detector”, ArXiv, abs/1512.02325, 2016. Massachusetts Amherst startup package (TL). The authors [15] N. Dhanachandra and Y. J. Chanu, “A survey on thank Qining Wang for his advice on the website. image segmentation methods using clustering techniques,” European Journal of Engineering REFERENCES Research and Science, vol. 2, pp. 15-20, 2017. [1] Y. X. Zhuang and A. Menon, “On the stiction of [16] A. I. Bandos, H. E. Rockette, T. Song, and D. Gur, MEMS materials,” Tribol Lett, vol. 19, pp. 111–117, “Area under the free-response ROC curve (FROC) and 2005. a related summary index”, Biometrics, vol. 65, pp. [2] T. Liu and C.-J. Kim, “Turning a surface superrepellent 247-56, 2009. even to completely wetting liquids,” Science, vol. 346, pp. 1096–1100, 2014. CONTACT [3] F. Krogmann, W. Mönch, and H. Zappe, “A MEMS- * C. Cornillon, tel: +1-408-455-4243; based variable micro-lens system,” J. Opt. A: Pure carlacornillon@ucla.edu Appl. Opt., vol. 8, pp. S330–S336, 2006. [4] S. K. Cho, H. Moon, and C.-J Kim, “Creating, Transporting, Cutting, and Merging Liquid Droplets by Electrowetting-Based Actuation for Digital Microfluidic Circuits”, J. Microelectromechanical Systems, vol. 12, pp. 70-80. [5] J. Li, N. S. Ha, T. Liu, R. M. van Dam, and C.-J. Kim,