Depixelizing Pixel Art
Johannes Kopf Microsoft ResearchDani LischinskiThe Hebrew UniversityNearest-neighbor result (original:
pixels) Our result
Na¨ ıve upsampling of pixel art images leads to unsatisfactory results. Our algorithm extracts a smooth, resolution-independent vector representation from the image, which is suitable for high-resolution display devices. (Input image
Nintendo Co., Ltd.).
We describe a novel algorithm for extracting a resolution-independent vector representation from
images, which en-ables magnifying the results by an arbitrary amount without im-age degradation. Our algorithm resolves pixel-scale features in theinput and converts them into regions with smoothly varying shad-ing that are crisply separated by piecewise-smooth contour curves.In the original image, pixels are represented on a square pixel lat-tice, where diagonal neighbors are only connected through a singlepoint. This causes thin features to become visually disconnectedunder magniﬁcation by conventional means, and creates ambigui-ties in the connectedness and separation of diagonal neighbors. Thekey to our algorithm is in resolving these ambiguities. This enablesus to reshape the pixel cells so that neighboring pixels belongingto the same feature are connected through edges, thereby preserv-ing the feature connectivity under magniﬁcation. We reduce pixelaliasing artifacts and improve smoothness by ﬁtting spline curvesto contours in the image and optimizing their control points.
pixel art, upscaling, vectorization
Pixel art is a form of digital art where the details in the image arerepresented at the pixel level. The graphics in practically all com-puter and video games before the mid-1990s consist mostly of pixelart. Other examples include icons in older desktop environments,as well as in small-screen devices, such as mobile phones. Becauseof the hardware constraints at the time, artists where forced to work with only a small indexed palette of colors and meticulously ar-range
by hand, rather than mechanically downscalinghigher resolution artwork. For this reason, classical pixel art is usu-ally marked by an economy of means, minimalism, and inherentmodesty, which some say is lost in modern computer graphics. Thebest pixel art from the golden age of video games are masterpieces,many of which have become cultural icons that are instantly recog-nized by a whole generation, e.g. “Space Invaders” or the 3-colorSuper Mario Bros. sprite. These video games continue to be en- joyed today, thanks to numerous emulators that were developed toreplace hardware that has long become extinct.In this paper, we examine an interesting challenge: is it possible totake a small sprite extracted from an old video game, or an entireoutput frame from an emulator, and convert it into a resolution-independent vector representation? The fact that every pixel wasmanually placed causes pixel art to carry a maximum of expressionand meaning per pixel. This allows us to infer enough informationfromthespritestoproducevectorartthatissuitableevenforsigniﬁ-cant magniﬁcation. While the quantized nature of pixel art providesfor a certain aesthetic in its own right, we believe that our methodproduces compelling vector art that manages to capture some of thecharm of the original (see Figure 1).Previous vectorization techniques were designed for natural imagesand are based on segmentation and edge detection ﬁlters that do notresolve well the tiny features present in pixel art. These methodstypically group many pixels into regions, and convert the regions’boundaries into smooth curves. However, in pixel art, every singlepixel can be a feature on its own or carry important meaning. As aresult, previous vectorization algorithms typically suffer from detailloss when applied to pixel art inputs (see Figure 2).A number of specialized pixel art upscaling methods have been de-velopedinthepreviousdecade, whichwereviewinthenextsection.These techniques are often able to produce commendable results.However, due to their local nature, the results suffer from staircas-ingartifacts, andthealgorithmsareoftenunabletocorrectlyresolvelocally-ambiguous pixel conﬁgurations. Furthermore, the magniﬁ-cation factor in all these methods is ﬁxed to 2
, or 4
.In this work, we introduce a novel approach that is well suited forpixel art graphics with features at the scale of a single pixel. Weﬁrst resolve all separation/connectedness ambiguities of the origi-nal pixel grid, and then reshape the pixel cells, such that connectedneighboring pixels (whether in cardinal or diagonal direction) sharean edge. We then ﬁt spline curves to visually signiﬁcant edges andoptimize their control points to maximize smoothness and reducestaircasing artifacts. The resulting vector representation can be ren-dered at any resolution.