You are on page 1of 43
=< 2 a s a a g e S a z (12) INTERNAT (19) World Intellectual Property Organization International Bureau (43) International Publication Date 06 December 2018 (06.12.2018) ONAL APPLICATION PUBLISHED UNDER THE PA’ SS Z O1 PCT NT COOPERATION TREATY (PCT) AUTON UE 00 (10) International Publication Number WO 2018/223038 Al G1) International Patent Classiieat GOSB 23702(2006.01)——-GO6T 19/00 2011.01) GO6F 17730(2006.01) G96 3200(2006.01) GO6T 7/00 2017.01) 6096 5/00 2006.01) 21) International Application Number: PCT/US2018035667 (22) International Filing Date: (01 June 2018 (01.06.2018) 8) 26) G0) Language: English Publication Language: English 1 June 2017 01.06.2017) 31 May 2018 31.05.2018) 31 May 2018 (31.05.2018) Applicant: TESLA, INC. [USVUSI; 3500 Deer Creek Rd, Palo Alto, CA 94304 (US). Inventor: CARDENAS BERNAL, Ivan; 3500 Deer Croek, Ra, Palo Alto, CA 94304 (US). Agent: PARK, Jong Andrew, H.; Van Pelt, ¥i& James LLP, 10080 N. Foothill Blvd. Suite 200, Cupertino, CA 95014 (US), Designated States (unless otherwise indicated, for every ind of national protection available): AE, AG, AL, AM, us us us 15)994,919 15)994,914 am m™ my on we) Publishes ‘AO, AT, AU, AZ, BA, BB, BG CA, CH, CL,CN,CO,CR,CU, DZ, EC, BE, EG, ES, Fl, GB, GD, GE, GH, GM, GT, HN, HR, HU, ID, IL, INR, 1S, 10, JP, KE, KG, KH, KN, KP KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, ME, MG, MK, MN, MW, MX, MY, MZ, NA, NG, NI, NO, NZ, BH, BN, BR, BW, BY, BZ, , DE, DI, DK, DM, DO, (OM, PA, PE, PG, PH, PL, PT, QA, RO, RS, RU, RW, SA, SC.SD, SI (THT, TM, TN, TR, TT, TZ, UA, UG, US, UZ, VC, VN, ZA, ZM, ZW Designated States (unless otherwise indicated, for every kind of regional protection available. ARIPO (BW, G GM, KE, LR, LS, MW, MZ, NA, RW, SD, SL, ST, SZ, TZ, UG, ZM, ZW), Furasian (AM, AZ, BY, KG, KZ, RU, TI, ‘TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, Fl, FR, GB, GR. HR, HU, IE IS, IT, LT, LU, LV, MC, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, SM, ‘TR), OAPI (BF, BI, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG) with international search report (drt. 21(3) (G4) Title: AUGMENTED REALITY APPLICATION FOR MANUFACTURING (87) Abstract: A series ofimages ofa physical environment are obtained. Atleast ‘portion of an object detected inthe series of obtained images is identified. A deviance from a reference property associated with the detected objects detected ae 1 [ene | InView | See | | Featuees in View —+—_, 105 | bisplay Corresponding | | object Data n View — ProcessUser | Interaction With 107 LL object in view. Fig.1 using the series of images, Information associated withthe deviance is provided via an augmented reality device WO 2018/223038 PCT/US2018/035667 AUGMENTED REALITY APPLICATION FOR MANUFACTURING CROSS REFERENCE TO OTHER APPLICATIONS [0001] This aplication claims priority to U S. Provisional Patent Application No (62/513,902 ented AUGMENTED REALITY APPLICATION FOR MANUFACTURING filed Sune 1, 2017 which is incorporated herein by reference fr all puposes BACKGROUND OF THE VENTION {0002} Existing automotive manufacturing techniques are both ime consuming and ru significant manual calibeation and inspection, The positioning and programming of robots for ‘constuetng nd assembling automotive parts, the marking and placement of mechanical joins, the quatic inspection of assembled pats te, rire a worker specifically trained to perform aks that include setup, configuration, calibration, andor inspecting the quality f the work and results. The ‘me requted to portoem the steps is extnsive and increases the time and cost to build a new vehicle For example, a current practice for marking joints and/or inspecting dimensional accuracy of the joints involves overlaying paper or plastic molds over a sheet metal object in order to mark the pat, Similarly, joins may be inspected by manually referencing adjacent features, molds, oF ‘sing coordinate measuring machine (CMM) inspection. Therefore, there exists ned fora process an tots fr increasing the efficiency and decreasing the cost of automotive manufacturing ‘asks. Applying computer vison and ayzmented reality tools tothe manufacturing process can significa increase the speed und efficiency elated to manufacturing and in particular to the manufacturing of automobile pars and vehicles, BRIEF DESCRIPTION OF THE DRAWINGS [0003] Various embodiments of the invention are disclose in the following detailed description and the accompanying dawings. {0004} Figure 1 is» low digram ilstraing an embodiment of «proces for applying stugmented reality to manufacturing [000s] Figure? is a low digram lusting an embodiment ofa process for matching an object of interes to a reference model WO 2018/223038 PCT/US2018/035667 0006] Figure 3 ow diagram itustrating an embodiment of a process fr matching an object of intrest to a reference mode 0007] Figure 4a flow diagram illustrating an embodiment ofa proces for preparing reference daa for an augmented reality manufacturing application 0008} Figure $4 flow diagram illustrating an embodiment of a process fr applying stugmented realy to manufacturing 10009} Figure 6s «flow diagram illustrating en embodiment of process fr applying augmented reality to manufictring (0019) Figure7 system for manufacturing. lock diagram illustrating an embodiment of an augmented reality 0011] Figure 8a diagram ilustating a move of assembled manufactured items for an embodiment ofan augmented reality manufacturing application, 0012} Figure 94 diagram illustrating an embodiment ofa user interface for an augmented reality manufacturing aplication DETAILED DESCRIPTION {0013} The invention canbe implemented in numerous ways, including a a process an apparatus; sytem: a composition of mater computer program prodvt embodied on a computer readable storage motium: andra processor, such as a processor configured to exceue instructions stored on andr provided by « memory couple vo the processor. In his specification, these plementation, or any other frm tha the invention may take, may be refered (ows techniques. In general, the order ofthe steps of disclosed processes may be altered within the scope ofthe invention. Unless tated otherwise, a component such as a processor ora memory described as ‘being configured to perform a task may be implemented as general component hai temporarily configured to perform the tsk ta given time ora specific component that is manufactred to perform the sk. As used herein, the term "processor refers o ne or mare devies, ics, andior processing cores configured to process data, such as computer progam instructions [0014] A deuied description of one or more embodiments ofthe invention is provided below along with accompanying figures tha illustrate the principles ofthe invention, ‘The