Scribd Logo
Upload

Welcome to Scribd!

  • US10939143B2-System and Method For Dynamically Creating and Inserting Immersive Promotional Content in A Multimedia

    Uploaded by

    Santosh Nathan
    12 views18 pages

    Original Title

    US10939143B2-System and method for dynamically creating and inserting immersive promotional content in a multimedia

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    12 views18 pages

    US10939143B2-System and Method For Dynamically Creating and Inserting Immersive Promotional Content in A Multimedia

    Uploaded by

    Santosh Nathan

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    of 18
    2) United States Patent Iyer et al. (54) SYSTEM AND METHOD FOR DYNAMICALLY CREATING AND INSERTING IMMERSIVE PROMOTIONAL CONTENT IN A MULTIMEDIA (71) Applicant: Wipro Limited, Bangalore (IN) (72) Inventors: Ganeshkumar Laxminarayanan Iyer, Chennai (IN); Arvind Sundararajan, (Chennai (IN) (73) Assignee: Wipro Limited, Bangalore (IN) (4) Notice: Suibject to any disclaimer, the term of this patent is extended ot adjusted under 35 USS. 154¢b) by 62 days. en Appl. No.: 16:365,220 (22) Filed: Mar 6, 2019 6s) Prior Publication Data 202010314465 A1 Oct. 1, 2020 G0) Foreign Application Priority Data Mat. 26,2019 (IN) 201941011789 (1) nec, OWN 710 HON 217234 (2006.01) Go1101) (Continued ) US. C1 CPC. MOdN 21723424 (2013.01); Ga6E 16/483 (2019.01); HOMN 212402 (2013.01): (Continued) Field of Classification Search CPC on FIOAN 21/2342; HOMN 21/4858; HON 2125801; HOAN 21/4545; HO8N 21 (58) (Continved) a PLAYER APPLICATION 3B | | || a | | | St: PROF ME DATA a8 TFRCTVE ANS TREAT] |_“nwurmoou 306 | US010935 (10) Patent No.: (4s) Date of Patent: 914382 431 US 10,939,143 B2 Mar. 2, 2021 66) References Cited US. PATENT DOCUMENTS. 6100925 4 * $2000. Rosser o01s 37864 38/160 10091,556 BI* 102018 Liden (Continved) FOREIGN PATENT DOCUMENTS wo aoiwmsins A 22018 (OTHER PUBLICATIONS (CDN Optimization for VR Streaming —The Broadcast Bridge ast retrived from the Internet Mar. 26, 2019: “bps: ww, thebroaeastridge com conten ny 12158 -opimization fr (Continved) Primary Examiner — Michael B. Pierorszio (74) Attorney, Agent or Firm — Finnegan, Henderson, Farabows, Gantt & Dunner, LLP on ABSTRACT Te present invention relates to the field of dynamic content creation and management and more particularly to dynami- cally create and inser immersive promotional content in a ‘multimedia. The multimedia requested by a user is procured from a media server (108) and the ane or more objects present in the multimedia is determined, The content of the ‘multimedia is analyzed, and a decision is takea to immerse the promotional content in the multimedia base on the analysis ofthe multimedia. Further, «promotional content is ‘read in realtime using the components presen i the asset database (106) based on the personalized user penile. The ‘ereated promotional content is overlaid and bleaded with @ at least one object from the one oF more objects in the ‘multimedia, The created promotional content aad the mul- timedia is joined to form & composite media and is steamed to the user device 20 Claims, 6 Drawing 5 eee tL _wooue ze? | US 10,939,143 B2 age 2 (st) Inc 66 References Cited Gia wees (201901) we hw sae (oii US. PATENT DOCUMENTS tiny dase Goll) Bein uns Gatto) 12010616 AL* $2012 Kir vas 212509 own zits (201101) eae eee ae OsN 21485201101) Soigoterse Al &2019 Game msn 218101101) Stowers at $2018 PS (2) Use Soisorssree At S018 Tato st CPE" sos 21728891 201301), HO4N 2178882 (201301) aogn 21/4845 (2013 01). HOES BAMSH (201801) MOAN 2124858 201801) Noa 21022 201301) COTTIER PUBLICATIONS, (68) Field of Classification Search ce TOA 21/4854: HOGN 21/2402; HON 2vsI2 spe: ‘Tasa2 See application file for complete search history ‘We Are VR Ready. Vista Reality Soltons, las retreve fom the fntemet Ma. 26,2019: cht warwsvinke 60 cm ™ * cited by examiner U.S. Patent Mar. 2, 2021 100 ~w 1a woeo __ NETWORK AB, PLAYER 202 | Sheet 1 of 6 | CONTENT | MANAGEMENT | senveRaoa | | Ll FIGURE 1 US 10,939,143 B2 MEDIA SERVER 105 ASSET DATABASE 106 U.S. Patent Mar. 2, 2021 Sheet 2 of 6 US 10,939,143 B2 ‘CONTENT MANAGEMENT SERVER 104 V/O INTERFACES 201, MeMoRY 202 PROCESSOR 203 PERSONALIZED USER PROFILE DATA 205 DATA 204 OTHER DATA 206 MODULES 207 COMMUNICATION. MODULE 208 (OBJECT DETERMINATION, MODULE 209 CONTENT CLASSIFICATION MODULE 210 PROMOTIONAL CONTENT CREATION MODULE 244 OVERLAY MODULE 212 JOIN MODULE 213 OTHER MODULE 214 MEDIA SERVER 105. FIGURE2 ASSET DATABASE 106 U.S. Patent Mar. 2, 2021 Sheet 3 of 6 US 10,939,143 B2 USER DEVICE 102, VIDEO PLAYER 103 POSITION DETECTOR MODULE 301 ] VIDEO PLAYER ANALYTICS AND LOCATION MODULE 302 I PLAYER APPLICATION 303 | USER PROFILE DATA 304 t f t [OTHER INPUT MODULE Ls "AFFECTIVE AND FACIAL VOICE INPUT INPUT MODULE 306 __-MODULE 307 FIGURE 3 U.S. Patent Mar. 2, 2021 Sheet 4 of 6 US 10,939,143 B2 = 400 Procure a multimedia from a media sever based on a user request }- 401 ¥ Determine one or more objects present in the multimedia ~ 402 ¥ Decide to immerse a promotional content in the multimedia 403 ¥. Create the promotional content matching a personalized user profile 404 aN a Overlay the promotional content on the multimedia + 405 ¥. Join the promotional content with the multimedia 406 FIGURE 4 US 10,939,143 B2 Sheet 5 of 6 Mar. 2, 2021 U.S. Patent FIGURE 5 U.S. Patent Mar. 2, 2021 Sheet 6 of 6 US 10,939,143 B2 INPUT DEVICE purl 610 DEVICE sit emore | DEVICES 612 1/0 INTERFACE | sot | | x | | | NETWORK COMMUNICATION PROCESSOR Ly gt nreRFACE NETWORK 609. ~ 603 — STORAGE INTERFACE 604 4 MEMORY 605 USER INTERFACE 606 OPERATING SYSTEM 607 WEB SERVER 608 COMPUTER SYSTEM 600 FIGURE 6 US 10,939,143 B2 1 SYSTEM AND METHOD FOR DYNAMICALLY CREATING AND INSERTING IMMERSIVE PROMOTIONAL, CONTENT IN A MULTIMEDIA, ‘Ths application claims the benefit of Tnaian Patent Appi ‘ation Serial No. 201941011789, fled Mar. 26,2019, which js hereby incorporated by reference in its entirety FIRLD ‘The present disclosure relates to the field of dynamic ‘content creation and manggement, Particularly, but not ‘exclusively; the present disclosure relates to a system and method for dynamically erating and inserting immersive promotional conten in a multimedia, BACKGROUND An existing digital marketing technology knowa as pro- ‘grammatie Advertsing (AD) technology encompasses an ‘array of technologies that automate the buying, placement nd optimization of television programming inventory for an AAD campaign. In this process, supply-and-demand partners utilize automated systems and business rues place ADS in ‘electconically programming inventory and generate an AD. ‘campaign. The programmatic AD technology uses AD for mats such as Video Ad Serving Template (VAST) and Video Player Ad-serving Interface Definition (VPAID) Ads. The VAST specification isa universal XMI. schema for serving ADs to digital video players and provides information about the type of AD to play the time to show up the AD, duration ‘of the AD, whether users are abe to skip it and the like. The ‘PAID is a serpt that instructs a video player on the type of AD to play, the recommended length ofthe AD to be played time 10 surface the AD on the sereen, and positon oa the sereen for placing the AD. ‘An issue with the existing programmatic AD technology js the obstmctive nature othe promotional content blocking the user experience and causing discomfort to the use. For ‘example, the video of the AD occupies the entire fame oF display area of the sereen, the hanners displayed atthe lower section of the multimedia content hiding portions of the multimedia. Depending onthe screen resolution ofa stram- ing device, the revolution of the AD may be chosen for ‘example leaderboard size (728%), skyscraper (120%600), wide skyseraper (160x600), small square (2005200), square (250280), largo rectangle (3365280), rectangle (300%250) ‘and the Tike, ‘Another issue with the existing programmatic AD tech- nology is the placement of promotional content is deter mined by splice point, manifest and Society of Cable Tele- ‘communications Engineers (SCTE) makers. The promotional contents inserted based onthe marker position ‘and Torees the video player to switch between multimedia and the promotional content. ‘An issue with the existing programmatie AD technology {is that the promotional content isnot contextual with respact to the multimedia and the user Further, the $-Dimensional information ofthe multimedia is not captured for placement ‘of the promotional content, For example, displaying the promotional content when news about war between two ‘countries are teloasted or telecasting about marty of sol diets of country. Thus, displaying the promotional content ‘durin telecast of such news would not suit with the context, ‘emotions of the user watching the news, and also probability ‘of alfecting the brand associated with the promotional 0 o 2 content may be high. Another seenario where displays promotional content having cricket players of @ country ‘when cricket team of tht country is losing the match. This ‘would alfet popularity of the cricket players and associated brand value. Also, de existing programmatic AD technology does not take into account the content in the video before isplaying the AD. Often the content in the video may be ‘unmoderated for example adult, violence, gore, war, ter ism, gambling and lke, Displaying the AD along with the ‘unmoderated content may destroy the brad reputation and results ina loss of faith onthe brand, Manval and Machine Jeaming content maderations services provide moderation st the time of video upload or post the video being. lagged by the user. In another similar case, programmatic ads for ephemeral content, where the content could be a short burst live or nonelive content Furber, an issue with the existing programmatic AD technology is that the promotional content displayed is pre-conceived. i. the promotional content is created, stored ina repository, and is slotted in the multimedia based on the schedule. The object asset is retrieved from the repository ‘based on the schedule, stitched with the video and sent to the video player fom the server or client bowser. ‘The quality ofthe video and the promotional content may deteriorate die to insuliient bandwidih ia the network. Che existing programmatic AD technology’ does not provide any ‘mechanism Tor upgrading the bandwidh to ensure a sooth playout ofthe AD. Incase of a 360-degree video, the content fand promotional content adaptation is done based on the fiold of view and bandwidth upgrade is done without ren- ering the whole content as done with the existing program- ‘matie AD technology. "The information disclosed in this background of the disclosure section is only for enhancement of understanding ofthe wenerl background ofthe invention and should not be taken asan scknovsledgement or any’ form of suggestion that {his information forms the prior art already known to a person skilled in the at SUMMARY, (One oF more shortcomings ofthe prior art are overcome, ‘and ational advantages are provided through the provi- sion of method of the present disclosure, ‘Additional features and advantages are realized through the techniques ofthe present disclosure. Other embodiments ‘and aspects of the disclosure are deseribed in detail herein ‘nd are considered part of the claimed disclosure. Disclose herein isa method for dynamically ereting and inserting immersive promotional content in @ multimedia The method includes procuring, by a content management server, the multimedia from a media server based on w user roquest to stream the multimedia to a user deve. Further the method includes determining one or more objects pres- ent in the moltimedia. Furthermore, the method inelides classifying a content present in the mulimedia to decide ‘immersion of the promotional content on at least one object from the one or more objects. Thereafter, the method includes creating, in real time, the promotional content ‘matching @ personalized user profile based on a decision of ‘immersing the promotionsl content into the a least one ‘object. Subsequently, the method includes overlaying the promotional content onto the atleast one object. Finally, the ‘method includes joining the promotional content with the ‘multimedia to abn composite multimedia, The composite multimedia may stream W the user device, US 10,939,143 B2 3 Further, the present disclosure includes a content man ‘agement server for dynamically cresting and inserting Jmmersive promotional content ina multimedia, The content management server includes a processor and memory ‘communicatively coupled to the processor. The memory stores the processor-exccutable instructions, which, on ‘execution, cause the processor to procure the multimedia from a media server based on a user request to stream the rnitimesia to a user deviee. Funter, the processor may be ‘configured to determine one or more objects preseat in the ‘content. Furthermore, the processor may be configured (0 classify content present in the multimedia t0 decide ‘immersion ofthe promotional content on at least one object fom the one oF more objects. Thereafter, the processor may be configured to create, in realtime, the promotional content matching a personalized user profile based ona decision of ‘immersing the promotional content into the at least one ‘object. Subsequently, the processor may be configured t0 ‘overlay the promotional coatet onto the atleast one object Finally, the processor may be configured to join the promo- tional content with the multimedia to obtain composite sultimedia, The processor may be configured to stream the ‘composite multimedia to the user device. Furthermore, the present disclosure includes non-tran- sitory computer reidable medium including instructions stored thereon that when processed by atleast one processor ‘cause «device to perform procuring the multimedia from a media server (108) based on a user request fo seam the multimedia to a user device (101). Further, the processor ‘causes the device to perform determining one or more ‘objects present in the multimedia, Furthermore, the proces- sor causes the device to perform classifying content present in the multimedia to decide immersion of the pro- ‘motional content on at Feast one object from the one or more ‘objects. Therwafler, the processor causes the device to per- orm creating, in real time, the promotional content mateh- ing a personalized user profile based on a decision of immersing the promotional content into the at least one ‘object. Subsequently, the processor causes the device to perform overlaying of the promotional content onto the at Jeast one object Finaly, the processor causes the device 10 Perform joining of the promotional eontent withthe malti- media t obtain composite multimedia, wherein the com- posite multimedia is streamed to the user device (101), The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the iustative aspects, embodiments, and features described above, futher aspects, embodiments, and features may become apparent by reference 10 the drawings and the following detailed description BRIEF DESCRIPTION OF THE DRAWINGS “The novel Fetures and characteristic ofthe disclosure are set forth in the appended claims. The disclosure itself however, as well as a preferred mode of use, further objec” tives and advantages thereof, may best be understood by reference to the following detailed description of an ilus- trative embodiment when read in conjunction with the ‘accompanying drawings. The accompanying drawings, ‘which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together ‘with the description, serve to explain the disclosed prin ciples. In the igure, the leemost digit(s) of a reference fimber identifies the figure in which the eference number first appears. One or more embodiments are now described, 0 o 4 by way of example only, with reference to the accompanying sigures wherein lke reference numerals represent like ele- sents and in whieh FIG. I shows an exemplary environment for dynamically creating and inserting immersive promotional content in a ‘multimedia in accordance with some embodiments of the present disclosure; FIG. 2 shows a dotailed block diagram of the content ‘management server for dynamically creating and inserting ‘immersive promotional content in a multimedia in accor dance with some embodiments ofthe present disclose FIG. 3 shows a detailed block disgram ofa user device for ‘creating personalized user profile in accordance with som tembodinient ofthe present disclosure FIG. 4 shows a fowehar illustrating method of dynami- cally creating and inserting immersive promotional content in # multimedia in accordance with some embodiments of the present disclosure; FIG. § shows an exemplary multimedia immersed with a promotional content in accondance with some embodiments of the present diselosure: and FIG, 6 shows a general-purpose computer system to ‘deatfy the ebjectionable eontett present in a mltimesia content before upload in accordance with embodiments of the present disclosure, (should be appreciated by dhose skilled in Use at that say block diagrams herein represent conceptual views of illus trative systems embodying the principles of the present subject matter. Similarly, it may be appreciated that any floss charts, flow diagrams, state trusition diagrams, pseudo ode, and the like represent various processes which may be substantially represented in computer readable medium and ‘executed by a computer or processor, whether or not such fcomputer or procestor is explicitly show, DETAILED DESCRIPTION In the present document, the word “exemplary” is nsed herein to mean “serving a5 an example, instance, oF ills- tration.” Any embodiment or implementation of the present stbjoct mater described herein ns “exemplary” is not nec- essaily to be construed as preferred or advantageous over cer embodiments ‘While the disclosure is susceptible 10 various modifies. tions and alterative forms, specific embodiment Theeot as been shown by way of example in the drawings and may be described in detail below. It should he understood, however ‘hat it snot intended to limit the disclosure to the particular Torms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternative falling ‘within the scape of the disclosure, The terms “comprises”, “includes” “comprising” “including” or any other variations thereof, are intended 10 cover non-exclusive inclusion, such that «setup, deviee or snc that comprises a Fst of components or steps docs not ‘include only those components of steps but may include other components or steps not expressly listed or inherent to sch setup or device or method. In other words, one or more clements' in a system or apparatus proceeded by eomprises or “includes... does not, without more constants, preclude the existence of other elements or ‘additional elements in the system or apparatus, Embodiments of the present disclosure relates toa system and method for dynamically creating and inserting immer- sive promotional content in a multimedia. An Ambient AD fF 8 promotional content may he an uirobstnictive adver tisemtent (AD) which may be inserted within a video content US 10,939,143 B2 5 Without disrupting the end user experience, This means the AD ox the promotional content may not be a banner pro- motional content or a display promotional content that appears on top of a video as half page, full age or occupies portion of the sereen. Embodiment of the disclosure pro- poses t display the promotional content on the content or ia ‘other words, blended with content. Further, the disclosure proposes maichine leaming based algorithms to recognize ‘objects present in the multimedia and super impose the promotional content on an object matched with a database or metadata in the video. Thereafter, the present disclosure provides a system and method for implementing the fadvanced ambient AD model. The Ambient AD model ‘enubles rich, intervtve video Ads, for a owo-dimensional video oF a 360-degree video, using multiple input sources (Other Input Nodes, Voice and Affective) across multiple streaming and JoT deviees only ihe content of the video is moderated Inn embodiment, a model may be proposed for creating ‘8 promotional contest base on the user viewing direction ‘and field of view to render an AD, in an un-obsirvtive ‘manner. In case of a two-dimensional video, where one oF ‘more cameras are used, it has been proposed in the following sisclosure that advertisements be slotted within the coatent rather than as an overlay blocking user experience. In case ‘of a 36Qdogree video, ads are slotted only in the user Viewing direction and the ADs do not appear as separate banners rather are embeded or lend within the content. ‘The promotional content getting inserted in a 3 plane may be created using various factors that include, but are not Timited t, edge detection, object recognition, animation motion timing. blending, or compositing, created in real time, overlaid and positioned contextually on top of an ‘existing content Further, in an embodiment, the ambient AD model takes ‘inputs from affective intelligence further validated by voice alerts and other input nodes, to detect the emotions of the user and choose a relevant promotional content based on the ‘deste emotion, The svertisement (AD), in the proposed ‘vention may he crated in rel time and embedded in the video content. The ambient AD model may have a repository (of 3D objects which are extracted, branded, overlaid, com: posited and blended on the asset, The ADs ereated may be ‘aligned to the viewing direction ofthe wer andthe object a ‘which the promotional content may be embedded, Partcu- Jarly, ADs on network may have intelligence to understand the video content and enable compression aligned tothe user ‘device and accordingly the Content Service Providers may ‘charge broadcasters or Over the top players on pay per use model for network utilization, Punter, the Ad may be ‘ereated by choosing a 3 model from an object asset ‘database, The object asset database may include 3D models ‘none ormore shapes and sizes, The 3D modelsin the object asset database may not be skinned or colored. The AD asset may be matched to the size and shape of the a least one ‘object inthe multimedia and a pre-defined animation may be ‘’tiosen from one or more animations that may be performed by the 3D model In the following detailed description of the embodiments ‘of the disclosure, reference is made to the accompanying ‘drawings that form a part heroof, and in which are shown by ‘way of illustration specific embodiments in which the dis ‘losure may be practiced. These embovdiments are described jn sufficient detail to enable those skilled in the aft 10 practice the disclosure, ad it sto he wnderstood that other ‘embodiments may be utilized ad that changes may be made 0 o 6 without departing from the scope ofthe present disclosure Te following description is, therefore, nt to be taken in a Timiting sense. FIG. 1 shows an exemplary environment for dynamically creating and inserting immersive promotional content in a ‘multimedia in accordance with some embodiments of the present disclosure Ta an embodiment, a wser device (101) may include & video player (102) capable of steaming, multimedia based fom a user request. The user device (101) may be connected to the content management server (104) trough a network (103), In some embodiments, the network (103) may be at Teast one ofa wired comminication nework and a wireless ‘communication network. The content management server (104) may be connected to a media server (105) and an asset atabase (106). The user device (101) may send the request to the content management server (104) for streaming. @ multimedia, The content management server (104) may procure the requested multimedia from the media server (105) and determines one or more objects present in the ‘multimedia for immersing a promotional content. Furor, the content management server (104) may analyze the content of the multimedia and based on the moderation level decided 10 immerse the promotional content in the multi ‘media. Upon deciding to immerse the promotional content a the multimedia, the content management server (104) based on the personalized user profile may retrieve the object asses. AS an example, the objet assets may be a 3D ‘model of logo, and animation elect required to create a sromotional content from the asset database (106). The promotional content may be ercated in realtime using the retrieved assets and overlaid on the multimedia by matching the retrieved abject assets to the size and co-ordinates ofthe predefined object in the multimedia. Further, the content ‘management server (104) may join the erested promotional ‘content and the procured multimedia form the media server (105) forming a composite media. The composite media may be streamed to the user device (101) through the setwork (103). The user device (101) displays the compesite ‘multimestia on the video player (102), In an embodiment, the user device (101) may include any {device apable of streaming a multimedia that may inelwde, but is not limited to, a smartphone, internet connected {elevision sets, desktops, tublets, entertainment systems, poruble communication devices, eatersinment systems ‘nfotainment systems, wearables such as ngs, bands, "watches and the ike, Augmented Reality and Virtual Reality devices, spectacles, head gear, Out of Home, digital signage equipment, gaming devices, and newer form of devices that ‘may connect fo internet and steam or interact with content and interact with end use, ‘In an embodiment, the content management server (104) may be configured in at least oe of a Muli-Access Edge Computing (MEC) network, a cloud network, a remote server, ona 3D platform, Detaled explanation on the content ‘management server (104) has been provided in subsequent description In an embodiment, the asset database (106) may’ inchude ‘brand interested to sell an advertisement (AD) and brand asset. As an example, the brand asset may be a patter, Jogo with URL to piek the brand asset, 2D and 3D models ‘of multiple brands in different shapes and sizes, a set of sre-defined snimation which the 3P model may perform, Funker, the asset database (106) may be connected to sidechain based blockchain AD network and a zero-kntwl- tedge proof-hased model for AD transaction, The sidechain nay a1 real time model wherein aa ad buyer may raise & US 10,939,143 B2 1 request 19 buy a promotional content object asset and the system forwards the request to sellers for exhibiting their ‘corresponding assets for sale, For example, a seller may ‘exhibit a shirt of celebrity and the interested buyer may ‘hioose to display the promotional content on the shi TIG. 2 shows a detailed block diagram of the content management server for dynamically creating. and inserting Jmmersive promotional content in a multimedia in accor ‘dance with Some embodiments ofthe present disclosure The content management server (104) may include Cen- teal Processing Unit CPU" of “processor") (203) and memory (202) storing instructions executable by the pro- ‘essor (203). The processor (203) may include at least one data processor for executing program components for ‘executing user or systen-generated requests. The memory (202) may be communicatively coupled to the processor (203), “The content’ management server (104) further includes an IapuOutput (UO) interfce (201), The UO Jmerface (201) is coupled with the processor (203) through Which an input signal orfand an output signal may be ‘communicated. In one embodiment, the content manage ment server (104) may receive a user request for streaming 1 multimedia through the LO interface (201). Tn some implementations, the content management server (104) may include data (204) and modules (207). As an ‘example, the data (204) and modules (207) are stored inthe ‘memory (202) configured in the content management server (104) ss shown in the FIG. 2. none embuliment, the data (204) may include, for example, personalized user profile ‘data (208) and other data (206). Inthe illustrated FIG. 2, ‘modules (207) are described herein in detail In Some embodiments, data (204) may be stored in the sicmory (202) in form of various data strictures, Addition- ally, the data (204) may be organized using data models, ‘uch ae relational or hierarchical data models, The other data (206) may stone data, including temporary data and tempo- rary files, generated by the modiles (207) for performing the various fanetions of the content management server (104). In some embodiments, the data (204) stored in the memory (202) may be processed by the modules (207) of the ‘content management server (104). The modules (207) may be stored within the memory (202). In an example, the ‘modules (207) communicatively coupled to the processor (203) coafigured ia the content management server 104, ‘ay also be present ouside the memory (202) as shown ia FIG. 2A and implemented as hardware. As used herein, the teem modules (207) may refer to an application specific imeprated circuit (ASIC), a FPGA (Field Programmable Gate Array), an electronic cireit, a processor (shared, dicated, oF group) and memory tat execute one or more sollware or firmware programs, « combinational logic ci ‘cuit, andor other suitable components that provide the ‘described finetionality. In some other embodiments, the modules (207) may be implemented using at least one of ASICS and FPGAS, ‘In an embodiment, the personalized user profile data (205) may include data retrieved from the user profile present on the user deviee (101), As an example, the data may be a facial information, extracted user's face, the personal desis of te user, the information in tems of roll, Pitch and yaw forthe viewing direction ofthe multimedia, ser buying habits, demographic information of the user user responses corresponding to different types of multime- sia content and the like. Further, the content management server (104) may use viola jones or Histogram of Oriented Gradients detection to detect facial information of the user from the user images retrieved from the user deviee (101). 0 o 8 The landmark estimation may be used t Wrap the wser images and one or more measurements generated by a ncial network may be used as feature se to identi the face of the use. A linear Suppor Vector Machine (SVM) classifier and random forest machine learning algorithms are used t0 perform the classification of the face sing the feature sct senerated by the neural network. Ta one implementation, the modules (207) may include, for example, @ communication module (208), an object determination module (209), content classification module (210), a promotional content creation module (211), aa overlay module (212), a join module (213), und other mexlufe (214). It may be appreciated that such aforemen- tioned modules may be represented as a single module or a combination of different modules. Ta.an embodiment, the communication module (208) may be used to communicate with the user device (101) for receiving requests regarding the multimedia, receiving the user profile information, and sending the composite multi ‘media, Further, the communication module (208) may be responsible for procuring the user requested multimedia {orm the media server (108) and extracting the promotional content assets, animation eflecs and the timing details from the asset database (106) for ereating the promotional eon- ‘ent ‘In an embodiment, the objet determination module (209) ‘may be responsible for extracting metadata from the multi media and identifying one or more objects present in the ‘multimedia wsing an Arfcial Intelligence (Al based meta- ata extractor and classifier. A person skilled inthe art would ‘understand that, any other technighe or tool, not mentioned explicitly, may also be used as the metadata extractor and classifier in the present disclosure. The identified one or more objects are stored in a repository ora database. ‘nan embodiment, the content classification module (210) ‘may use an AL based classifiers 1 identify type of content based on the extracted metadata and the content of the ‘multimedia, As an example, the multimedia may be las fied into an appropriate oF inappropriate type by the AI based classifiers, In one embodiment, content, for example, related to the political, elgious and adult type may be classified as ‘inappropriate content type and content for example, related to sports, news, kids, and nature may be classified as appropriate content. If the multimedia is classified as ppeo- priate content, then a decision may be taken to immerse the ‘promotional content. Ifthe multimedia is classified as inap- propriate, then a decision may be taken not to immerse the promotional content. nan embodiment, the promotional content eration mod- ule (211) may be responsible for creating the promotional ‘content in realtime. The promotional content may be ereated ‘matching to the personalized user profile. In one embed ‘eal, creation of the promotional content may include etocting the promotional content object asset, retrieving corresponding 3D or 2D shape ofthe object asst, painting the 2D of 3D model with eolor ofthe brand and generating required animation effect of the brand from the asset data ‘base (106). Further, the promotional content may be created {o match the size and shape of the at least one of object the multimedia ‘In an embodiment the created promotional content may include an advertisement, trailers of movies, and like. ‘In an embsxliment, the overlay module (212) may be responsible for detecting edge ofthe atleast one of object ‘nd obtain est matehing model to overiay the promotional content. The hest matching model provides the details regarding the object segmentation and the chordiogra US 10,939,143 B2 9 distance of the object in the multimedia, Further, a best ‘atch of the brand logos may be detenmined {rom the asset ‘database (106) based on the resolution and the backround, ‘color ofthe abject asset, Further, the co-ordinates ofthe at Teast one of object are identified to overlay the promotional content object asset. The promotional object asset may be ‘overlaid on the at leat one of object by tracking motion of the at least one of object using # discriminative correlation fier and rendering the promotional content 1 a specific 0 the location on the multimedia. A person skilled in the art would understand that, any other techniques, not mentioned ‘explicily, may also be used to wack motion of the atleast ‘one of object inthe present disclosure Further, the overlay module (212) may use convolution rnewal networks 10 shape, blend and mask the promotional content 2D or 3D object asset. The object asset. may be adjusted to match the size and coordinates of the at least one ‘object by performing a shrink, rotate and flip operations on the object assot. In an embodiment, an alge detection algorithm may be used wo adjust the object asset to match the size and coordinates of the at least one of object. The direction of the at Teast one object may be determined through one or more meastireients generated by «neural network and may be used 10 uniquely identify the at last ‘one object and correspondingly rotate te at least one object, The overlay may be placed on the video aad in certain cases, mask may be inserted to maintain the contrast In an embodiment, the join module (213) may sick the rmukimedia with the promotional content resulting in @ ‘composite multimedia, The join module (213) may use ‘markers o¢ join points ina multimedia stream to signal the start or end ofthe promotional content to be immersed in the ‘multimedia. Purther, an overlay filter may be sed to etch the promotional content in the multimedia, The origin of the promotional content may accurately splice the multimedia at the exact timestamp signaled by the eu. TFIG.3 shows a detailed block diagram ofa user device for ‘creating personalized user peofite in accordance with some ‘embodiment of the present disclosure The user device (101) displays the composite multimedia ‘on the screen present on the user device using a video player (102). The video player (102) may include positon detec- tor module (301), a video player analytics and location module (302) a player application (303), 2 user profile data (04), other input module (308), an affective and facial input ‘module (306), a voice input modiale (307), “The video player (102) may inelude a User Interface (UD, ‘4-media engine, anda decoder. The UT inchides a video playerskin, the (I logie for example a mechanism to display advertising, adjust content to streaming and ToT device specificities, authentication management and the like, The media engine may handle the functionalities relating multimedia playback logie for example « manifest parsing 8 segment retrieval, an adaptive bitrate ries and switching The decoder performs decoding and rendering of a video segment on the seen andthe DRM manager manages the decryption In an embodiment, the video player (102) provides a mechanism to provision additional bandit For the pro- motional content. The video player (102) ealeultes the required bandwidth to stream the composite mulkimedtia, 1 the required bandwidth i insafficent to stream the compos- ite multimedia the video player temporarily upgrades the ‘composite multimedia to the AD bandwidth, The AD band= width may be total bandwidth required to stream the com- posite multimedia. Punter, the video player (102) through the user deviee (101) requests an additional bandwidth t0 0 o 10 Policy and Charging Enforcement Function (PCE) inked to Policy and Changing Rules Function (PCRE) or Online Charging System (OCS). In an embodiment, the video player (102) may receive a request from the content man- ‘agement server (104) for an addtional Bandwidth upon ‘tection of insuficient hand for streaming the com posite multimedia to the user device (101). The video player analytics an location module (302) may capture the Quality of Service (QoS), Quality of Experience (QoE) metrics ofthe video player (102) and user informa- ‘ion for example a Time of visit of a webpage, the number of pages visited, time spent on each page of the webpages referring site details, a location details such as Latte and Longitude, address of the user device (101) ia temns of county, state, locality and street, a type of web browser, @ {ype of operating system (OS), a Flash version, JavaScript support, sereen resolution of the user device (101), and seren colour processing ability, network location and IP addres of the user deviee (101). ‘The position detector module (301) may capture user information for example user viewing angles on the video player (102) and pass eontrl directives to instruct the video player (102) to contol the position of the multimedia being splayed, orientation ofthe sereen, and focus of multime- «a's pte, roll and yaw seting, The other input module (308) may eapture the user inputs for example an Blectroencephalogram, a fingertip-based ‘monitors, 2 pulse monitors and the lke and stores it in the ‘ser profile data (304). Further, th other input module (305) ‘may be used to determine the valenee and arousal score The valence score may be the degree of attrction oF aversion that the user feels toward a specific object or event in the ‘uitimedia. The aroasal score may be a physiological state of being awake or reactive to the evens inthe multimedia The affective and facial input module (306) may capture the affective inteigence of the user through an Artficial Intelligence (AT based automated facial anelysis performed ‘on one oF more video frames. Based on the facial action Selected a variety of parameters for example allect valence, spontaneous resetions and facil action units may be deter ‘ined. The detemnined parameters may be used to predict the emotional and mental tates ofthe user. Further the facial inputs captured atthe video player (102) may be sent to @ personalised user profile and the know your customer (KYC) detsils may be validated “The voice input module (307) may capture the inputs, for ‘example voice-based inputs, from the user received from a ‘wired and a wireless medium eonnected to the user device (104) that allo users to interact with the user device (104). The voice-based inputs include the ability to move the smuliimedia on a 2-Dimension of a 3-Dimension plane, ability 10 zoom into the promotional content and control the promotional conte, In some implementations standard Al algorithms slong with token parser, semantic parsor Object based databases, learning structures using neuronal retworks, pattem matehers and neural network based answering systems may be used to process the voice-based inputs. The user profile data (804) may include Al based taining classifiers «© recognize the Tice of user, the user viewing

    You might also like