Image Processing and Understanding for the Analysis of Master Drawings

and Paintings


David G. Stork
Chief Scientist
Ricoh Innovations
2882 Sand Hill Road Suite 115
Menlo Park, CA 94025-7054 USA
http://www.rii.ricoh.com/~stork/index.html


Overview

This one-day tutorial will apply methods from image processing, computer vision and image analysis to problems in the history and understanding of master drawings and paintings. Some of these analysis techniques are built upon methods used in forensic image analysis of photographs but are tailored to specific contingencies of painting. Questions addressed include: How do we judge the sizes and positions of objects depicted and the geometry of structures such as architecture? Was the image created using a mechanical or optical aid? What were the sources of illumination and their color? What form of perspective was used? What is revealed by shadows and reflections depicted within a painting? Some of the analysis techniques require nothing more than a tutored and perceptive eye; others merely a ruler and pencil; yet others require advanced statistical estimation procedures and computer analysis. This course will provide an entr¨¦e into the methods and literature, ending with a number of outstanding problems amenable to attack by rigorous techniques. After completing the course, students should be able to collaborate with scholars in art history to address a range of problems in art history.

¡¡


Prerequisites/expected background

Students must have basic background to the level of an introductory course in image processing or computer vision (digital filtering, affine transformations, geometric analysis, etc.). It is highly desirable that students have at least a basic knowledge of periods of representational art: Medieval, Renaissance, Baroque, realism and photorealism and at least a passing acquaintance with representational artists from a wide range of periods.

¡¡


Outline

  • Analysis of geometrical perspective

          Single point and multiple point perspective, cross ratios, Desargue¡¯s theorem, detecting perspective inconsistencies,  

          inferring viewing position and relative depth, inherent scale

  • Anamorphic art

         Slant, conical and cylindrical anamorphic art; distorted perspective; the mathematics of perspective distortions

  • Metrology from images

    Estimating size, location from uncalibrated and from single ¡°calibrated¡± images (i.e., ones containing objects of known size or angular size and/or position, e.g., sun, moon, reference objects)
    ¡¡

  • Analysis of shadows and scattered light

    Cast shadows (cast onto another object) and form shadows (on the object itself). Inferring the number, position and color of illuminants. Detecting lighting inconsistencies. Estimating time of day or geographical latitude from shadow analysis.

  • Analysis of depictions of mirrors

    Reflections in plane, convex, concave mirrors; reconstructing the tableau using multiple viewpoints provided by mirrors.

  • Analysis of luminance

    Computation of brightness in projected images; atmospheric perspective

  • Analysis of color

    Additive color mixing, subtractive color mixing, colored shadows, simultaneous color contrast, simulation of "aging" paintings

  • Elementary optical systems and camera model

    Elementary image forming systems (concave mirror, converging lens); the lens and mirror equation; focal length, angle of view, depth of field/depth of focus

  • Introduction to historical drawing and copying aids

    Camera obscura, camera lucida, pantograph, compasso da reduzione, Claude mirror, Alberti¡¯s screen, Albrecht D¨¹rer¡¯s drawing machines, and their effects in drawings and paintings.

  • Analysis of common visual illusions in paintings

    Qualitative and quantitative analysis of the Poggendorff illusion, Ponzo illusion, lights contrast, ¡­

  • Outstanding problems in the image analysis of master paintings



References

Books

  • Antonio Criminisi, "Accurate visual metrology from single and multiple uncalibrated images" (Springer 2001)

  • David Falk, Dieter Brill and David G. Stork, "Seeing the Light: Optics in nature, photography, color, vision and holography" (Wiley, 1986)

  • Ernst Gombrich, "Art and Illusion: A study in the psychology of pictorial representation" (Princeton U. Press, 1961)

  • Richard Hartley and Andrew Zisserman, "Multiple view geometry in computer vision" (Cambridge U. Press, 2004)

  • Martin Kemp, "The Science of Art" (Yale U. Press, 1997)


Papers

  • Antonio Criminisi, ¡°The virtual Trinity¡±, Proc. Workshop on Art, Science and Techniques of Drafting in the Renaissance, Florence, Italy (May, 2001)

  • Antonio Criminisi, Martin Kemp and Andrew Zisserman, ¡°Bringing Pictorial Space to Life: Computer Techniques for the Analysis of Paintings¡±, CHArt Annual Conference 2002: Digital Art History? Exploring Practice in a Network Society (November, 2002)

  •  Antonio Criminisi and David G. Stork, "Did the great masters use optical projections while painting? Perspective comparison of paintings and photographs of Renaissance chandeliers", in J.Kittler, M. Petrou and M. S. Nixon (eds.), Proceedings of the 17th International Conference on Pattern Recognition, Volume IV, pp. 645-648 (2004)

  • Antonio Criminisi, Martin Kemp and Sing-Bing Kang, ¡°Reflectionsof reality in Jan van Eyck and Robert Campin¡±, Proc. Measuring Art: A Scientific Revolution in Art History, Paris (May-June, 2003)

  • David Hockney and Charles M. Falco, ¡°Optical perspectives on Renaissance art¡±, Optics and Photonics News, 11:52 (2000)

  • David G. Stork, "Color and illumination in the Hockney theory: A critical evaluation", Proceedings of the Color Imaging Conference (CIC11), Scottsdale AZ, pp. 11-15 (November, 2003)

  •  David G. Stork, "Were optical projections used in early Renaissance painting? A geometric vision analysis of Jan van Eyck's 'Arnolfini portrait' and Robert Campin's 'M¨¦rode Altarpiece' ", SPIE Electronic Imaging, Vision Geometry XII, L. J. Latecki, D. M. Mount and A. Y. Wu (eds), pp.23-30 (2004)

  • David G. Stork, "Did Jan van Eyck build the first 'photocopier' in 1432?", SPIE Electronic Imaging Color Imaging IX: Processing, Hardcopy, and Applications, R. Eschbach and G. G. Marcu (eds.) pp. 50-56 (2004)

  • David G. Stork, "Optics and the old masters revisited", Optics and Photonics News, 15 (3):30-37 (March, 2004)

  • David G. Stork, "Did Hans Memling employ optical projections when painting Flower still-life?", Leonardo, 38(2) 57-62 (2005)

  • David G. Stork, "Asymmetry in 'Lotto carpets' and implications for Hockney's optical projection theory", SPIE Electronic Imaging, Bernice E. Rogowitz, Thrasyvoulos N. Pappas and Scott J. Daly (eds.), volume 5666, pp. 337-343 (2005)

  • David G. Stork, "Optics and realism in Renaissance art", Scientific American, 291(6):76-84 (December, 2004)

  • Christopher W. Tyler, ¡°Rosetta Stoned? Hockney, Falco and the sources of ¡®opticality¡¯ in Renaissance art¡±,  Leonardo,
    37(5):397-401 (2004)

¡¡

Biography

Dr. David G. Stork is Chief Scientist of Ricoh Innovations. He has published in optics and art for over two decades, including "Seeing the Light: Optics in nature, photography, color, vision and holography" (Wiley) the leading textbook on optics in the arts (now in its 21st printing). A graduate in physics of the Massachusetts Institute of Technology and the University of Maryland at College Park, he also studied art history at Wellesley College and was Artist-in-Residence through the New York State Council of the Arts. His anamorphic photographs and graphics (based on late Renaissance methods) have appeared in small art journals as well as /Optics and Photonics News/ and /Scientific American /magazine. He has taught courses such as "Light, color and visual phenomena," "The physics of aesthetics and perception," and "Optics, perspective and Renaissance painting" over the last quarter century variously at leading liberal arts and research universities such as Wellesley College, Swarthmore College, Clark University and Stanford University. He has published over a hundred technical papers on human and machine learning and perception of patterns, physiological optics, image understanding, concurrency theory, theoretical mechanics, and five books, including "Pattern Classification" (2nd ed.), the world's all-time best-selling textbook in the field, used in courses in over 200 universities worldwide. He sits on the editorial boards of two international journals and has delivered over 40 plenary and invited lectures, and at major museums such as the Metropolitan Museum of Art and the National Gallery London. He created the PBS television documentary "2001: HAL's Legacy," based on his book "HAL's Legacy: 2001's computer as dream and reality" (MIT). He was one of four scientists invited to the December 2001 "Art and Optics" Symposium at the New York Institute for the Humanities to comment on David Hockney's theory that Renaissance painters traced projected images, and one of two scientists invited to present a lecture in the subject at the Optical Society of America's Annual Meeting in Rochester, NY, October 2004.


Dr. David G. Stork <mailto:stork@rii.ricoh.com>
Chief Scientist <http://www.rii.ricoh.com/%7Estork/>
Ricoh Innovations <http://www.rii.ricoh.com/>
2882 Sand Hill Road Suite 115
Menlo Park, CA 94025-7022 USA
stork@rii.ricoh.com
www.rii.ricoh.com/~stork
650-496-5720 (w)
650-854-8740 (fax)
¡¡